Coding and decoding method, decoder, encoder and computer readable storage medium

Abstract

The embodiment of the invention provides a coding and decoding method, a decoder, an encoder and a computer readable storage medium. The coding and decoding performance can be improved. The method comprises the following steps: analyzing a code stream, determining coding information corresponding to a second scale point cloud, and determining a first scale point cloud; performing local density prediction based on the first scale point cloud, determining a local density corresponding to a first scale voxel in the first scale point cloud, performing occupancy probability prediction on a second scale voxel, and determining an occupancy probability corresponding to the second scale voxel; the second scale voxel is an up-sampling voxel corresponding to the first scale voxel; the local density represents the number of occupied second scale voxels in the second scale voxels corresponding to the first scale voxels; and based on the occupancy probability corresponding to the second scale voxel and the local density corresponding to the first scale voxel, decoding reconstruction is performed on the coding information corresponding to the second scale point cloud, and reconstruction geometric information corresponding to the second scale point cloud is determined.

Description

Coding and decoding method, decoder, encoder and computer-readable storage medium Technical Field

[0001] The present application relates to point cloud compression coding and decoding technology, and in particular to a coding and decoding method, a decoder, an encoder and a computer-readable storage medium. Background Art

[0002] A point cloud is a collection of points that stores the geometric position and associated attribute information of each point, thereby accurately describing objects in space. Point cloud data is massive; a single point cloud frame can contain millions of points. This poses significant challenges to its efficient storage and transmission. Therefore, compression technology is used to reduce redundant information in point cloud storage, facilitating subsequent processing.

[0003] At present, representative point cloud compression algorithms include: Video-based Point Cloud Compression (V-PCC) and Geometry-based Point Cloud Compression (G-PCC). The geometric compression in G-PCC is mainly implemented through octree models and / or triangular surface models. V-PCC is mainly implemented through three-dimensional to two-dimensional projection and video compression. In the above two compression algorithms, the structural information of the real environment expressed by the point cloud is restored by reconstructing the geometric information of the point cloud. Generally, the process of reconstructing the geometric information of the point cloud includes: using a sparse convolutional neural network, taking the voxelized geometric information of the low-scale point cloud as input, and predicting the occupancy probability of each high-scale voxel in the high-scale point cloud. Then, based on the occupancy probability of each high-scale voxel, the occupancy symbol of each high-scale voxel is determined, and the geometric information of the high-scale point cloud is reconstructed based on the occupancy symbol to represent the occupied high-scale voxels.

[0004] However, the method of geometric reconstruction based on occupancy probability prediction is prone to inaccurate occupancy symbol determination when the occupancy probabilities of multiple high-scale voxels are relatively close, or the occupancy probability threshold is set unreasonably, thereby reducing the quality of geometric reconstruction and further reducing the encoding and decoding performance.

[0005] Summary of the Invention

[0006] The embodiments of the present application provide a coding and decoding method, a decoder, an encoder, and a computer-readable storage medium, which can improve the quality of geometric reconstruction of point cloud coding and decoding, thereby improving coding and decoding performance.

[0007] The technical solution of this application is achieved as follows:

[0008] This embodiment of the present application provides a decoding method, including:

[0009] Parsing the bitstream to determine encoding information corresponding to the second-scale point cloud and determining a first-scale point cloud; the first-scale point cloud is the previously decoded point cloud data corresponding to the second-scale point cloud;

[0010] performing local density prediction based on the first-scale point cloud to determine the local density corresponding to first-scale voxels in the first-scale point cloud, and performing occupancy probability prediction on second-scale voxels to determine the occupancy probability corresponding to the second-scale voxels; the second-scale voxels are upsampled voxels corresponding to the first-scale voxels; the local density represents the number of occupied second-scale voxels in the second-scale voxels corresponding to the first-scale voxels;

[0011] Based on the occupancy probability corresponding to the second-scale voxel and the local density corresponding to the first-scale voxel, the encoded information corresponding to the second-scale point cloud is decoded and reconstructed to determine the reconstructed geometric information corresponding to the second-scale point cloud.

[0012] This embodiment of the present application provides an encoding method, including:

[0013] Downsampling the second-scale point cloud to determine a first-scale point cloud, and upsampling the first-scale voxels in the first-scale point cloud to the second scale to determine the second-scale voxels corresponding to the first-scale voxels;

[0014] performing a local density prediction based on the first-scale point cloud to determine a local density corresponding to the first-scale voxel, and performing an occupancy probability prediction on the second-scale voxel to determine the occupancy probability corresponding to the second-scale voxel; the local density represents the number of occupied second-scale voxels in the second-scale voxel corresponding to the first-scale voxel;

[0015] determining, based on the local density corresponding to the first-scale voxels and the occupancy probability corresponding to the second-scale voxels, reconstructed geometric information corresponding to the second-scale point cloud;

[0016] Encoding is performed based on the reconstructed geometric information corresponding to the second-scale point cloud, encoding information corresponding to the second-scale point cloud is determined, and the encoding information is written into a bitstream.

[0017] An embodiment of the present application provides a decoder, including:

[0018] A parsing part is configured to parse the code stream and determine the encoding information corresponding to the second-scale point cloud;

[0019] The determining part is configured to determine a first-scale point cloud; the first-scale point cloud is the previously decoded point cloud data corresponding to the second-scale point cloud;

[0020] a local density prediction part, configured to perform local density prediction based on the first-scale point cloud, and determine a local density corresponding to a first-scale voxel in the first-scale point cloud;

[0021] an occupancy probability prediction section configured to perform occupancy probability prediction on a second-scale voxel based on the first-scale point cloud, and determine the occupancy probability corresponding to the second-scale voxel; the second-scale voxel is an upsampled voxel corresponding to the first-scale voxel;

[0022] The decoding and reconstruction part is configured to decode and reconstruct the encoded information corresponding to the second-scale point cloud based on the occupancy probability corresponding to the second-scale voxel and the local density corresponding to the first-scale voxel, and determine the reconstructed geometric information corresponding to the second-scale point cloud.

[0023] An embodiment of the present application provides an encoder, including:

[0024] A downsampling portion configured to perform voxel downsampling on the second-scale point cloud to determine a first-scale point cloud;

[0025] a local density prediction part, configured to perform local density prediction based on the first-scale point cloud and determine the local density corresponding to the first-scale voxel;

[0026] an occupancy probability prediction section configured to upsample first-scale voxels in the first-scale point cloud to a second scale, determine second-scale voxels corresponding to the first-scale voxels, and perform occupancy probability prediction on the second-scale voxels to determine the occupancy probability corresponding to the second-scale voxels;

[0027] a reconstruction part configured to determine reconstructed geometric information corresponding to the second-scale point cloud based on the local density corresponding to the first-scale voxel and the occupancy probability corresponding to the second-scale voxel;

[0028] The encoding part is configured to perform encoding based on the reconstructed geometric information corresponding to the second-scale point cloud, determine encoding information corresponding to the second-scale point cloud, and write the encoding information into a bitstream.

[0029] This embodiment of the present application provides a code stream, including:

[0030] The code stream is generated by bit encoding according to the encoding information; wherein the encoding information at least includes: encoding information corresponding to the second-scale point cloud.

[0031] An embodiment of the present application provides a decoder, including:

[0032] a first memory configured to store executable instructions;

[0033] The first processor is configured to implement any one of the above decoding methods when executing the executable instructions stored in the first memory.

[0034] An embodiment of the present application provides an encoder, including:

[0035] a second memory configured to store executable instructions;

[0036] The second processor is configured to implement any one of the encoding methods described above when executing the executable instructions stored in the second memory.

[0037] An embodiment of the present application provides a computer-readable storage medium storing executable instructions for causing a first processor to execute the above-mentioned decoding method, or for causing a second processor to execute the above-mentioned encoding method.

[0038] An embodiment of the present application provides a computer program product, including a computer program or instructions. When the computer program or instructions are executed by a first processor, the decoding method provided by the embodiment of the present application is implemented; or, when the computer program or instructions are executed by a second processor, the encoding method provided by the embodiment of the present application is implemented.

[0039] Embodiments of the present application provide a coding and decoding method, a decoder, an encoder, and a computer-readable storage medium. The decoder can determine the number of occupied second-scale voxels in the second-scale voxels obtained by sampling each first-scale voxel by predicting the local density. In this way, the occupancy probability corresponding to the second-scale voxels can be screened in combination with the local density to determine the occupancy of the second-scale voxels. The second-scale point cloud is reconstructed based on the occupancy of the second-scale voxels, and the reconstructed geometric information of the second-scale point cloud is determined. In this way, the occupancy of the second-scale voxels can be determined more accurately, improving the accuracy of the reconstructed geometric information of the second-scale point cloud, thereby improving the quality of the reconstructed geometry of the decoder and improving decoding performance. Furthermore, in the encoder, the occupancy probability corresponding to the second-scale voxels is screened based on the local density corresponding to the first-scale voxels. This can improve the accuracy of determining the occupancy of the second-scale voxels, thereby improving the accuracy of encoding the reconstructed geometric information of the second-scale point cloud determined based on the occupancy of the second-scale voxels, thereby improving encoding performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] FIG1 is a flow chart of G-PCC coding;

[0041] FIG2 is a flow chart of G-PCC decoding;

[0042] FIG3 is a schematic diagram of an optional flow chart of a decoding method provided in an embodiment of the present application;

[0043] FIG4 is a schematic diagram of an optional process of voxel upsampling provided in an embodiment of the present application;

[0044] FIG5 is a schematic diagram of an optional flow chart of a decoding method provided in an embodiment of the present application;

[0045] FIG6 is a schematic diagram of an optional flow chart of the occupancy probability prediction and local density prediction processes provided in an embodiment of the present application;

[0046] FIG7 is a schematic diagram of an optional flow chart of the occupancy probability prediction and local density prediction processes provided in an embodiment of the present application;

[0047] FIG8 is a schematic diagram of an optional flow chart of the occupancy probability prediction and local density prediction processes provided in an embodiment of the present application;

[0048] FIG9 is a schematic diagram of an optional structure of a local density prediction network provided in an embodiment of the present application;

[0049] FIG10 is a schematic diagram of an optional structure of a feature extraction network provided in an embodiment of the present application;

[0050] FIG11 is a schematic diagram of an optional structure of a residual layer in a feature extraction network provided in an embodiment of the present application;

[0051] FIG12 is a schematic diagram of an optional structure of an occupancy probability prediction network provided in an embodiment of the present application;

[0052] FIG13 is a schematic diagram of occupancy probabilities corresponding to voxels at a second scale provided by an embodiment of the present application;

[0053] FIG14 is a schematic diagram of an optional flow chart of a decoding method provided in an embodiment of the present application

[0054] FIG15 is a schematic diagram of an optional process for reconstructing point cloud geometric information based on occupancy probability and local density according to an embodiment of the present application;

[0055] FIG16 is a schematic diagram of an optional flow chart of an encoding method provided in an embodiment of the present application;

[0056] FIG17 is a schematic diagram of an optional process of voxel downsampling provided in an embodiment of the present application;

[0057] FIG18 is a schematic diagram of an occupancy symbol obtained by voxel downsampling according to an embodiment of the present application;

[0058] FIG19 is a schematic diagram of an optional process of applying the decoding method provided in an embodiment of the present application to a practical scenario;

[0059] FIG20 is a schematic diagram of an optional structure of a decoder provided in an embodiment of the present application;

[0060] FIG21 is a schematic diagram of an optional structure of an encoder provided in an embodiment of the present application;

[0061] FIG22 is a schematic diagram of an optional structure of a decoder provided in an embodiment of the present application;

[0062] Figure 23 is a schematic diagram of an optional structure of the encoder provided in an embodiment of the present application. DETAILED DESCRIPTION

[0063] In order to make the purpose, technical solutions and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings. The described embodiments should not be regarded as limiting this application. All other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0064] In the following description, reference is made to “some embodiments”, which describes a subset of all possible embodiments, but it will be understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0065] In the following description, the terms "first\second\third" involved are merely used to distinguish similar objects and do not represent a specific ordering of the objects. It can be understood that "first\second\third" can be interchanged with a specific order or sequence where permitted, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein.

[0066] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which this application pertains. The terms used herein are for the purpose of describing the embodiments of this application only and are not intended to limit this application.

[0067] Before further describing the embodiments of the present application in detail, the nouns and terms involved in the embodiments of the present application are explained. The nouns and terms involved in the embodiments of the present application are subject to the following interpretations.

[0068] 1) Voxel: Voxel, short for volume element, is the smallest unit of digital data used in three-dimensional space. Voxels allow for the gridding of 3D space and the assignment of characteristics to each grid. For example, a voxel is a fixed-size cube in three-dimensional space. Voxels are widely used in fields such as 3D imaging, scientific data, and medical imaging.

[0069] Point cloud compression algorithms include video-based point cloud compression (V-PCC) and geometry-based point cloud compression (G-PCC). G-PCC uses an octree model and / or a triangular surface model for geometric compression. V-PCC uses 3D-to-2D projection and video compression.

[0070] With the development of artificial intelligence (AI) technology, neural networks have been applied to geometry-based point cloud compression. Neural network-based point cloud geometry compression can be broadly categorized into lossy and lossless compression. Lossless compression algorithms primarily focus on designing prediction models for voxel occupancy probabilities. Voxel data representations typically utilize octree models, volumetric models, sparse tensor representations, and other methods. For lossless geometry compression, the encoder typically uses surrounding context, such as parent nodes and neighboring nodes, as input. After processing through neural network layers (e.g., convolutional and fully connected), the encoder outputs the occupancy probability of each voxel in the point cloud's geometric data. An entropy encoder then converts the voxel occupancy symbol corresponding to each voxel's occupancy probability into a bitstream. Correspondingly, the decoder uses the same process to predict the occupancy probability of each voxel. Based on the predicted occupancy probability, an entropy decoder is used to decode the voxel occupancy symbol from the bitstream to reconstruct the point cloud's geometric data.

[0071] It can be seen that determining the occupancy symbol of a voxel based solely on occupancy probability can easily lead to inaccurate occupancy symbol determination when the occupancy probabilities of multiple adjacent voxels are close, or when the occupancy probability threshold is set improperly. In particular, when the point cloud density is unevenly distributed, the occupancy symbol representing the number of occupied voxels can easily be inconsistent with the actual number of occupied voxels, resulting in more or less points in the reconstructed geometric information. This reduces the quality of geometric reconstruction and, in turn, the encoding and decoding performance.

[0072] The embodiments of the present application provide a coding and decoding method, a decoder, an encoder and a computer-readable storage medium, which can improve coding and decoding efficiency and improve coding and decoding performance. In order to facilitate the understanding of the technical solution provided by the embodiments of the present application, a flow chart of G-PCC encoding and a flow chart of G-PCC decoding are first provided. It should be noted that the flow chart of G-PCC encoding and the flow chart of G-PCC decoding described in the embodiments of the present application are only for more clearly illustrating the technical solution of the embodiments of the present application, and do not constitute a limitation on the technical solution provided by the embodiments of the present application. It is known to those skilled in the art that with the evolution of point cloud compression technology and the emergence of new business scenarios, the technical solution provided by the embodiments of the present application is also applicable to point cloud coding and decoding architectures similar to G-PCC. The point cloud compressed by the embodiments of the present application can be a point cloud in a video, but is not limited to this.

[0073] In the point cloud G-PCC encoder framework, the point cloud of the input 3D image model is sliced ​​and each slice is encoded independently.

[0074] As shown in the G-PCC encoding flow diagram in Figure 1, the encoder first divides the point cloud data to be encoded into multiple slices using striping. Within each slice, the point cloud's geometric and attribute information are encoded separately. During the geometric encoding process, the geometric information undergoes coordinate transformation so that the entire point cloud is contained within a bounding box. Quantization then occurs. Quantization primarily serves a scaling purpose. Due to quantization rounding, the geometric information of some point clouds remains the same, allowing for parameter-based decision-making regarding whether to remove duplicate points. This process of quantization and removing duplicate points is also known as voxelization. The bounding box is then partitioned into an octree. In the octree-based geometric information encoding process, the bounding box is divided into eight equal parts. The sub-cubes that are not empty (contain points in the point cloud) are further divided into eight equal parts until the resulting leaf nodes are 1x1x1 unit cubes. The points in the leaf nodes are then arithmetic-coded to generate a binary geometric bitstream, or geometry codestream. In the process of geometric information encoding based on triangle soup (trisoup), octree partitioning must also be performed first. However, unlike the geometric information encoding based on octree, the trisoup does not need to divide the point cloud into unit cubes with a side length of 1x1x1 step by step. Instead, the division stops when the sub-block (block) has a side length of W. Based on the surface formed by the distribution of the point cloud in each block, the surface and the twelve edges of the block are obtained. The vertices are arithmetically encoded (surface fitting is performed based on the intersections) to generate a binary geometric bit stream, i.e., a geometric code stream. Vertex is also used to implement the process of geometric reconstruction, and the reconstructed geometric information is used when encoding the attributes of the point cloud.

[0075] During the attribute encoding process, color conversion is performed to convert the color information (i.e., attribute information) from the RGB color space to the YUV color space. The reconstructed geometric information is then used to recolor the point cloud so that the unencoded attribute information corresponds to the reconstructed geometric information. There are two main transformation methods for color information encoding: a distance-based lifting transform that relies on level of detail (LOD) partitioning, and a direct region-adaptive hierarchical transform (RAHT) transformation. Both methods convert color information from the spatial domain to the frequency domain, obtaining high-frequency and low-frequency coefficients. These coefficients are then quantized (i.e., quantized coefficients). Finally, the geometrically encoded data (octree partitioning and surface fitting) is sliced ​​and synthesized with the attribute encoded data (quantized coefficients). The vertex coordinates of each block are then encoded (i.e., arithmetic encoding) to generate a binary attribute bitstream, i.e., the attribute codestream.

[0076] The flowchart of G-PCC decoding shown in Figure 2 is applied to the decoder. The decoder obtains the binary code stream and independently decodes the geometric bit stream (i.e., geometric code stream) and attribute bit stream in the binary code stream. When decoding the geometric bit stream, the geometric information of the point cloud is obtained through arithmetic decoding-octree synthesis-surface fitting-reconstruction geometry-inverse coordinate transformation; when decoding the attribute bit stream, the attribute information of the point cloud is obtained through arithmetic decoding-inverse quantization-LOD-based inverse lifting or RAHT-based inverse transformation-inverse color conversion, and the three-dimensional image model of the point cloud data to be encoded is restored based on the geometric information and attribute information.

[0077] The encoding method of the embodiment of the present application can be applied to the geometric information encoding process of the G-PCC as shown in Figure 1. After voxelization is completed, the geometric encoding process in Figure 1 is performed based on the voxelized second-scale point cloud to obtain a geometric bitstream; in the geometric reconstruction process of the geometric encoding, the voxelized second-scale point cloud is downsampled to determine the first-scale point cloud, and the first-scale voxels in the first-scale point cloud are upsampled to the second scale to determine the second-scale voxels corresponding to the first-scale voxels; local density prediction is performed based on the first-scale point cloud to determine the local density corresponding to the first-scale voxels, and occupancy probability prediction is performed on the second-scale voxels to determine the occupancy probability corresponding to the second-scale voxels; the local density represents the number of occupied second-scale voxels in the second-scale voxels corresponding to the first-scale voxels; based on the local density corresponding to the first-scale voxels and the occupancy probability corresponding to the second-scale voxels, the reconstructed geometric information corresponding to the second-scale point cloud is determined; in the attribute encoding process, the attribute bitstream is obtained based on the reconstructed geometric information corresponding to the second-scale point cloud.

[0078] The decoding method of the embodiment of the present application can be applied to the geometric information decoding process of the G-PCC as shown in Figure 2. By parsing the bitstream, the encoding information corresponding to the second-scale point cloud is determined, and the first-scale point cloud is determined; the first-scale point cloud is the previously decoded point cloud data corresponding to the second-scale point cloud. During the geometric decoding process, the encoding information corresponding to the second-scale point cloud is arithmetic decoding, octree synthesis, and surface fitting are performed. During the geometric reconstruction process of the geometric decoding, local density prediction is performed based on the first-scale point cloud to determine the local density corresponding to the first-scale voxel in the first-scale point cloud, and occupancy probability prediction is performed on the second-scale voxel to determine the occupancy probability corresponding to the second-scale voxel; the second-scale voxel is the upsampled voxel corresponding to the first-scale voxel; the local density represents the number of occupied second-scale voxels in the second-scale voxel corresponding to the first-scale voxel; based on the occupancy probability corresponding to the second-scale voxel and the local density corresponding to the first-scale voxel, the encoding information corresponding to the second-scale point cloud is decoded and reconstructed to determine the reconstructed geometric information corresponding to the second-scale point cloud. During the attribute decoding process, the reconstructed geometric information corresponding to the second-scale point cloud is used to perform LOD-based inverse lifting or RAHT-based inverse transformation-inverse color conversion to obtain the attribute information of the second-scale point cloud. The three-dimensional image model of the second-scale point cloud is restored based on the reconstructed geometric information and attribute information.

[0079] It should be noted that the encoding method and decoding method of the embodiments of the present application can also be used in other point cloud encoding and decoding processes besides G-PCC.

[0080] The following describes a decoding method applied to a decoder provided in an embodiment of the present application.

[0081] Refer to Figure 3, which is an optional flow chart of the decoding method provided in an embodiment of the present application, and will be explained in conjunction with the steps shown in Figure 3.

[0082] S101: Parse the code stream to determine the coding information corresponding to the second-scale point cloud and determine the first-scale point cloud.

[0083] In the embodiment of the present application, the decoder parses the received code stream to obtain the encoding information corresponding to the second-scale point cloud and determines the first-scale point cloud, where the first-scale point cloud is the previously decoded point cloud data corresponding to the second-scale point cloud.

[0084] In the embodiments of the present application, the bitstream typically includes encoded information corresponding to at least one scale of point cloud sent by the encoder. The decoder decodes in order from low to high scales. That is, before decoding the encoded information corresponding to the second-scale point cloud, the decoder has already decoded the encoded information of the previous scale point cloud, namely the first-scale point cloud, and determined the decoded first-scale point cloud data, namely the first-scale point cloud. In this way, the decoder can use the decoded low-scale first-scale point cloud data to decode the encoded information corresponding to the high-scale second-scale point cloud and reconstruct the geometric information.

[0085] S102 : performing local density prediction based on the first-scale point cloud to determine the local density corresponding to the first-scale voxels in the first-scale point cloud, and performing occupancy probability prediction on the second-scale voxels to determine the occupancy probability corresponding to the second-scale voxels.

[0086] In the embodiment of the present application, the geometric information of the first-scale point cloud and the second-scale point cloud has undergone a voxelization process of the encoder and can be represented in the form of a voxel grid.

[0087] In an embodiment of the present application, for the voxelization process, a point in the point cloud can correspond to an occupied voxel (i.e., a non-empty voxel), and an unoccupied voxel (i.e., an empty voxel) indicates that there is no point in the point cloud at that voxel position. In some embodiments, occupied voxels can be marked as 1, and unoccupied voxels can be marked as 0. In this way, the voxelized point cloud can represent the geometric data of the point cloud by the occupancy symbols of the voxels at each position in the voxel grid.

[0088] In this embodiment of the present application, the scale of the point cloud corresponds to the scale of the voxels in the point cloud. That is, the voxels contained in the first-scale point cloud are first-scale voxels, and the voxels contained in the second-scale point cloud are second-scale voxels. The second-scale voxels are upsampled voxels corresponding to the first-scale voxels. The decoder can perform voxel upsampling on the first-scale voxels in the first-scale point cloud. For example, the decoder can upsample the first-scale voxels representing occupied points in the first-scale point cloud to obtain multiple second-scale voxels corresponding to each first-scale voxel.

[0089] In some embodiments, the decoder can implement voxel upsampling through pooling, such as using a maximum pooling layer with a stride of 2×2×2 to divide one first-scale voxel in the first-scale point cloud into eight second-scale voxels. Each upsampling evenly divides the size of the first-scale voxel in three dimensions, that is, the size of the second-scale voxel in three dimensions is half of the first-scale voxel. Upsampling each first-scale voxel in the first-scale point cloud completes upsampling from a low-scale point cloud with known geometric information to a high-scale point cloud, thereby obtaining a second-scale point cloud with geometric information to be reconstructed.

[0090] Please refer to Figure 4, which shows a first-scale point cloud containing 2×2×1 first-scale voxels. After a single voxel upsampling, a second-scale point cloud containing 4×4×2 second-scale voxels is obtained. The first-scale point cloud is the decoded point cloud data. In Figure 4, occupied first-scale voxels are represented by solid cubes, representing the locations of points in the point cloud. After the first-scale voxels are upsampled to second-scale voxels, whether the second-scale voxels are occupied needs to be further determined through a geometric reconstruction process, which is represented by empty cubes in Figure 4. However, the point cloud in Figure 4 is merely exemplary, and an actual point cloud may include more voxels.

[0091] In this embodiment of the present application, local density represents the number of occupied second-scale voxels within the second-scale voxel corresponding to a first-scale voxel. Therefore, the decoder performs local density prediction based on the first-scale point cloud and determines the local density corresponding to the first-scale voxels in the first-scale point cloud. This can then determine the number of occupied second-scale voxels within the multiple second-scale voxels corresponding to each first-scale voxel. This can then be combined with the multiple occupancy probabilities corresponding to the multiple second-scale voxels to more accurately determine the number of occupied second-scale voxels within the multiple second-scale voxels.

[0092] In the embodiment of the present application, the decoder predicts the occupancy probability of the second-scale voxels obtained by upsampling the first-scale voxels, and determines multiple occupancy probabilities corresponding to multiple second-scale voxels corresponding to the first-scale voxels.

[0093] In some embodiments, based on FIG. 3 , as shown in FIG. 5 , S102 may be implemented by executing the processes of S201 - S203 as follows:

[0094] S201 : Extract features from geometric information of a first-scale point cloud to determine features of the first-scale point cloud.

[0095] S202: Upsampling the first-scale point cloud features to the second scale to determine initial second-scale point cloud features, performing feature extraction on the initial second-scale point cloud features to determine second-scale point cloud features, and performing occupancy probability prediction based on the second-scale point cloud features to determine occupancy probabilities corresponding to second-scale voxels.

[0096] In embodiments of the present application, the geometric information of the first-scale point cloud may include the occupancy status of each voxel in the first-scale point cloud and the position information of each voxel. In some embodiments, the occupancy status may be a preset flag such as 0 or 1, and the position information may be the three-dimensional coordinates of the voxel. In some embodiments, the geometric information of the first-scale point cloud may be obtained by decoding and reconstructing the encoded information of the first-scale point cloud. In other words, the geometric information of the first-scale point cloud may be equivalent to the reconstructed geometric information of the first-scale point cloud.

[0097] In an embodiment of the present application, the decoder extracts features from the geometric information of the first-scale point cloud, maps the geometric information of the first-scale point cloud to a preset low-scale feature space, and determines features of the first-scale point cloud. In some embodiments, the features of the first-scale point cloud may include implicit features extracted from the geometric information of the first-scale point cloud.

[0098] In an embodiment of the present application, the decoder upsamples the first-scale point cloud features to the second scale to determine the initial second-scale point cloud features. The decoder performs feature extraction on the initial second-scale point cloud features to determine the second-scale point cloud features. Based on the second-scale point cloud features, the decoder predicts the occupancy probability of the second-scale voxels, predicting the probability that the voxel position corresponding to each second-scale voxel falls within the midpoint of the point cloud, that is, determining the occupancy probability corresponding to the second-scale voxel. It can be understood that the second-scale point cloud features are obtained by performing two feature extractions on the geometric information of the first-scale point cloud. Therefore, the second-scale point cloud features have better feature expression and can improve the accuracy of the decoder's occupancy probability prediction using the second-scale point cloud features.

[0099] S203 : Perform local density prediction based on the first-scale point cloud features to determine the local density corresponding to the first-scale voxels.

[0100] In this embodiment of the present application, the decoder performs local density prediction based on first-scale point cloud features. The decoder predicts the number of occupied second-scale voxels among the multiple second-scale voxels corresponding to each first-scale voxel in the first-scale point cloud, using this number as the local density corresponding to each first-scale voxel. It is understood that the local density is a numerical value that can be an integer greater than or equal to 1 and less than or equal to the total number of second-scale voxels corresponding to a first-scale voxel.

[0101] It should be noted that the execution order of the feature upsampling, feature extraction of the initial second-scale point cloud features, and occupancy probability prediction processes in S202, and the local density prediction process in S203 is not limited to that shown in Figure 5. In practical applications, they can also be executed in any order or simultaneously. The specific selection is made according to the actual situation and is not limited in the embodiments of the present application.

[0102] For example, the above-mentioned process of S201-S203 can be shown in Figure 6. The decoder extracts features from the geometric information of the first-scale point cloud through the third feature extraction network to determine the first-scale point cloud features; the first-scale point cloud features are respectively input into the first branch comprising the upsampling network, the fourth feature extraction network, and the occupancy probability prediction network, and the second branch comprising the local density prediction network; the occupancy probability corresponding to each second-scale voxel is output through the first branch, and the local density corresponding to each first-scale voxel is output through the second branch. In some embodiments, the upsampling network can be implemented by a transposed convolutional network; the third feature extraction network is used to extract features from the geometric information of the first-scale point cloud, and the fourth feature extraction network is used to extract features from the initial second-scale point cloud features; the occupancy probability prediction network and the local density prediction network can be implemented using pre-trained neural networks.

[0103] In some embodiments, S102 in FIG. 3 may also be implemented by executing the process of S301 to S304 as follows:

[0104] S301 : Perform feature extraction on geometric information of a first-scale point cloud through a first feature extraction network to determine first point cloud features of the first scale.

[0105] S302 : Upsampling the first point cloud features at the first scale to the second scale to determine the second-scale point cloud features, and performing occupancy probability prediction based on the second-scale point cloud features to determine the occupancy probability corresponding to the second-scale voxel.

[0106] S303 : Perform feature extraction on the geometric data of the first-scale point cloud through a second feature extraction network to determine second point cloud features of the first scale.

[0107] S304 : Perform local density prediction based on the second point cloud features at the first scale to determine the local density corresponding to the voxels at the first scale.

[0108] In an embodiment of the present application, the decoder can also extract different first-scale point cloud features, that is, first point cloud features and second point cloud features, for the occupancy probability prediction and local density prediction processes through different feature extraction networks, which are used for occupancy probability prediction and local density prediction, respectively.

[0109] In some embodiments, the first feature extraction network can be obtained by jointly learning or training with the neural network for occupancy probability prediction, and the second feature extraction network can be obtained by jointly learning or training with the local density prediction network. In this way, the point cloud geometric features used to predict occupancy probability and the point cloud geometric features used to predict local density can be extracted more specifically, thereby improving the accuracy of local density prediction and occupancy probability prediction.

[0110] Likewise, the embodiment of the present application does not limit the execution order of the S301 - S302 process and the S303 - S304 process.

[0111] For example, the above-described steps S301-S304 can be illustrated in Figure 7. The first branch in Figure 7 includes a first feature extraction network, an upsampling network, and an occupancy probability prediction network; the second branch includes a second feature extraction network and a local density prediction network. The decoder inputs the geometric information of the first-scale point cloud into the first and second branches, respectively, and performs feature extraction and network prediction on the geometric information of the first-scale point cloud through the first and second branches, respectively. The first branch outputs the occupancy probability corresponding to each second-scale voxel, and the second branch outputs the local density corresponding to each first-scale voxel.

[0112] It should be noted that in some embodiments, based on FIG7 , after the upsampling network and before the occupancy probability prediction network, further feature extraction can be performed on the second-scale point cloud features output by the upsampling network. The extracted second-scale point cloud features are then input into the occupancy probability prediction network for occupancy probability prediction, thereby enhancing feature expression and thereby improving the accuracy of occupancy probability prediction. The specific selection is based on actual circumstances and is not limited in this embodiment of the present application.

[0113] In some embodiments, S102 in FIG. 3 may also be implemented by executing the process of S401 to S403 as follows:

[0114] S401 : Extract features from geometric information of the first-scale point cloud to determine features of the first-scale point cloud.

[0115] S402 : Upsampling the first-scale point cloud features to a second scale to determine the second-scale point cloud features, and performing occupancy probability prediction based on the second-scale point cloud features to determine the occupancy probability corresponding to the second-scale voxels.

[0116] S403 : Perform local density prediction based on the first-scale point cloud features to determine the local density corresponding to the first-scale voxels.

[0117] In the embodiment of the present application, occupancy probability prediction and local density prediction can also be performed separately based on the first-scale point cloud features determined by feature extraction of the geometric information of the first-scale point cloud. In this way, feature information can be reused, module complexity can be reduced, and decoder processing efficiency can be improved.

[0118] Likewise, the embodiment of the present application does not limit the execution order of S402 and S403.

[0119] For example, the above-mentioned S401-S403 process can be shown in Figure 8. The decoder extracts features from the geometric information of the first-scale point cloud through the fifth feature extraction network to determine the first-scale point cloud features. The first-scale point cloud features are respectively input into the first branch comprising the upsampling network and the occupancy probability prediction network, and the second branch comprising the local density prediction network. The first branch outputs the occupancy probability corresponding to each second-scale voxel, and the second branch outputs the local density corresponding to each first-scale voxel. Here, the fifth feature extraction network in Figure 8 and the third feature extraction network in Figure 6 can be the same or different feature extraction networks.

[0120] In some embodiments, for the local density prediction process in S203, S304, and S403 above, a local density prediction network can be used to perform local density prediction based on the first-scale point cloud features to determine the local density corresponding to the first-scale voxel. The local density prediction network may include: a first sparse convolution layer, a first activation function layer, a second sparse convolution layer, and a second activation function layer. Exemplarily, the local density prediction network may be shown in FIG9 , including: a first-layer sparse convolution layer (i.e., a first sparse convolution layer), a second-layer ReLu activation function layer (i.e., a first activation function layer), a third-layer sparse convolution layer (i.e., a second sparse convolution layer), and a fourth-layer Sigmoid function layer (i.e., a second activation function layer). The local density prediction network is used to output local density based on implicit features of the point cloud.

[0121] In some embodiments, the feature extraction process in S201, S202, S301, S303, S401, and S402 can be implemented using the feature extraction network shown in FIG10 , including: a first sparse convolution layer, a second activation function layer (e.g., a ReLu activation function layer), a third residual layer, and a fourth sparse convolution layer. In some embodiments, the network structure of the third residual layer can be shown in FIG11 .

[0122] In some embodiments, the occupancy probability prediction process in S202, S302, and S402 can be implemented using an occupancy probability prediction network as shown in FIG12, including: a sparse convolution layer in the first layer, an activation function ReLu layer in the second layer, a sparse convolution layer in the third layer, an activation function ReLu layer in the fourth layer, a sparse convolution layer in the fifth layer, and a Sigmoid function layer in the sixth layer. The occupancy probability prediction network is used to perform occupancy prediction on each second-scale voxel obtained by upsampling each first-scale voxel based on the implicit features of the second-scale point cloud, and obtain the occupancy probability corresponding to each second-scale voxel. The occupancy probability prediction network in FIG12 is only an example. In actual applications, it can also be a convolutional neural network (CNN) with other hierarchical structures. The specific selection is made according to the actual situation and is not limited in the embodiments of the present application.

[0123] In the embodiment of the present application, during the processing of the occupancy probability prediction network, for the unoccupied first-scale voxels in the low-scale point cloud (i.e., the first-scale point cloud), the multiple high-scale second-scale voxels obtained by upsampling are also unoccupied, and no probability prediction is required. However, the multiple second-scale voxels obtained by upsampling the occupied first-scale voxels in the first-scale point cloud need to be predicted. For example, the occupancy probability corresponding to the second-scale voxels facing the side of the paper in the second-scale point cloud in Figure 4 can be shown in Figure 13. The occupancy probability in Figure 13 is only for convenience of explanation and cannot be understood as the result of actual calculation. It can be seen that if the occupancy probability predicted for the actually occupied second-scale voxels is closer to 1, and the occupancy probability predicted for the actually unoccupied second-scale voxels is closer to 0, the more accurate the prediction is, and the more accurate the reconstruction quality of the point cloud geometric information is.

[0124] S103 : Decode and reconstruct the encoded information corresponding to the second-scale point cloud based on the occupancy probability corresponding to the second-scale voxel and the local density corresponding to the first-scale voxel, and determine the reconstructed geometric information corresponding to the second-scale point cloud.

[0125] In an embodiment of the present application, the decoder uses the predicted local density corresponding to each first-scale voxel in the first-scale point cloud and the occupancy probability corresponding to each second-scale voxel obtained by upsampling each first-scale voxel to determine whether each second-scale voxel is occupied. Then, based on whether each second-scale voxel is occupied, the decoder decodes and reconstructs the encoded information corresponding to the second-scale point cloud, decodes the position information corresponding to each occupied second-scale voxel in the second-scale point cloud, and determines the reconstructed geometric information corresponding to the second-scale point cloud based on the position information corresponding to each occupied second-scale voxel in the second-scale point cloud.

[0126] In some embodiments, based on FIG. 3 or FIG. 5 , as shown in FIG. 14 , S103 may be implemented by executing the process of S1031 - S1032 as follows:

[0127] S1031 . For each first-scale voxel in the first-scale point cloud, determine second-scale voxels with a high local density of occupation probability among multiple second-scale voxels corresponding to each first-scale voxel as occupied second-scale voxels.

[0128] In the embodiment of the present application, since the local density represents the number of occupied second-scale voxels in the second-scale voxels corresponding to the first-scale voxel, the multiple occupation probabilities of the multiple second-scale voxels corresponding to each first-scale voxel can be screened according to the local density corresponding to each first-scale voxel, and the local density of the second-scale voxels with high occupation probability can be determined as the occupied second-scale voxels.

[0129] For example, taking the example of upsampling one first-scale voxel to obtain eight second-scale voxels, the local density corresponding to the first-scale voxel can be any value from 1 to 8. Taking the local density of 4 as an example, the decoder determines the four second-scale voxels with high occupancy probabilities among the eight second-scale voxels as occupied second-scale voxels, and the remaining second-scale voxels as unoccupied second-scale voxels. As shown in Figure 15, a first-scale voxel in the first-scale point cloud is upsampled to obtain 8 second-scale voxels, and the local density corresponding to the first-scale voxel is 4; the occupancy probabilities corresponding to the 8 second-scale voxels are [0.9, 0.8, 0.7, 0.6, 0.5, 0.4, 0.3, 0.2] from high to low. Based on the local density, the second-scale voxels with the occupancy probabilities [0.9, 0.8, 0.7, 0.6] can be determined as occupied second-scale voxels; and the other second-scale voxels corresponding to [0.5, 0.4, 0.3, 0.2] are determined as unoccupied second-scale voxels.

[0130] S1032 : Decode and reconstruct the encoded information corresponding to the second-scale point cloud based on the occupied second-scale voxels corresponding to each first-scale voxel, and determine the reconstructed geometric information corresponding to the second-scale point cloud.

[0131] In this embodiment of the present application, occupied second-scale voxels represent points within the point cloud that fall within the second-scale voxels obtained by upsampling the first-scale voxels. The decoder can decode and reconstruct the encoded information corresponding to the second-scale point cloud based on the occupied second-scale voxels corresponding to each first-scale voxel, thereby determining the reconstructed geometric information corresponding to the second-scale point cloud.

[0132] In some embodiments, the decoder may distinguish between occupied and unoccupied second-scale voxels using different occupancy markers based on the determination result of S1031. Based on the occupancy markers representing each occupied second-scale voxel, the decoder may decode and reconstruct the encoded information corresponding to the second-scale point cloud to determine reconstructed geometric information corresponding to the second-scale point cloud.

[0133] Here, when performing geometric reconstruction, the decoder primarily decodes and reconstructs the geometric encoding information within the encoding information corresponding to the second-scale point cloud. After determining the reconstructed geometric information corresponding to the second-scale point cloud, the decoder can also decode the attribute encoding information corresponding to the second-scale point cloud based on the reconstructed geometric information corresponding to the second-scale point cloud through an attribute decoding process to determine the attribute information corresponding to the second-scale point cloud. This process then combines the reconstructed geometric information and the attribute information corresponding to the second-scale point cloud to restore the 3D image model of the second-scale point cloud.

[0134] It will be appreciated that in embodiments of the present application, the decoder can determine the number of occupied second-scale voxels in the second-scale voxels sampled from each first-scale voxel by predicting local density. In this way, the occupancy probabilities corresponding to the second-scale voxels can be screened in combination with the local density to determine the occupancy of the second-scale voxels. Based on the occupancy of the second-scale voxels, a second-scale point cloud can be reconstructed to determine the reconstructed geometric information of the second-scale point cloud. This can make the determined occupancy of the second-scale voxels more accurate, improve the accuracy of the reconstructed geometric information of the second-scale point cloud, and thus improve the quality of the decoder's reconstructed geometry and decoding performance.

[0135] In some embodiments, after S103, the decoder may continue to parse the bitstream to determine the encoded information corresponding to the i-th scale point cloud, where i is an integer greater than or equal to 3. Using the decoding method described in the embodiments of the present application, the decoder may perform a local density prediction of the i-th scale voxel based on the reconstructed geometric information of the i-1th scale point cloud to determine the local density corresponding to the i-1th scale voxel, and perform an occupancy probability prediction of the i-th scale voxel corresponding to the i-1th scale voxel to determine the occupancy probability corresponding to the i-th scale voxel. The i-th scale is obtained by upsampling the i-1th scale. Based on the occupancy probability corresponding to the i-th scale voxel and the local density corresponding to the i-1th scale voxel, the decoder may decode and reconstruct the encoded information corresponding to the i-th scale point cloud to determine the reconstructed geometric information corresponding to the i-th scale point cloud. That is, after decoding and reconstructing the second-scale point cloud, the decoder may also utilize the decoding method described in the embodiments of the present application to decode and reconstruct the encoded information of a higher-scale point cloud in the bitstream, such as a third-scale point cloud, based on the reconstructed geometric information of the second-scale point cloud, to obtain the reconstructed geometric information corresponding to the third-scale point cloud. In this way, when the decoder decodes adjacent scales, it can use the known geometric data of the previously decoded low-scale point cloud to decode and reconstruct the encoded information of the high-scale point cloud and determine the reconstructed geometric information of the high-scale point cloud; by decoding scale by scale, until the reconstructed geometric information of the target scale point cloud is restored, the target scale can be determined according to the preset decoding accuracy of the decoder.

[0136] In some embodiments, after S103, the decoder may further perform the above-mentioned decoding method in the embodiments of the present application to predict the local density of the n-th scale voxel and the occupancy probability of the n+1-th scale voxel based on the reconstructed geometric data corresponding to the n-th scale point cloud, and determine the local density corresponding to the n-th scale voxel and the occupancy probability corresponding to the n+1-th scale voxel; n is a positive integer greater than or equal to 2; the n+1-th scale is obtained by upsampling the n-th scale; based on the local density corresponding to the n-th scale voxel and the occupancy probability corresponding to the n+1-th scale voxel, determine the reconstructed geometric data corresponding to the n+1-th scale point cloud.

[0137] For example, when a bitstream contains coded information of point clouds at multiple non-adjacent scales, such as coded information corresponding to a first-scale point cloud, a third-scale point cloud, and a fifth-scale point cloud, the decoder can, based on the decoded and reconstructed first-scale point cloud, perform voxel upsampling based on first-scale voxels in the first-scale point cloud to obtain second-scale voxels. Furthermore, the decoder can extract features from the geometric information of the first-scale point cloud, predict local density of the first-scale voxels, and predict occupancy probability of the second-scale voxels. Based on the determined local density of the first-scale voxels and the occupancy probability of the second-scale voxels, the decoder can reconstruct geometric data corresponding to the second-scale point cloud, combining the second-scale voxel position information determined by upsampling the first-scale voxel position information. The decoder can then use the reconstructed geometric data corresponding to the second-scale point cloud to decode and reconstruct the coded information corresponding to the third-scale point cloud, and so on.

[0138] It should be noted that the decoding method in the embodiments of this application can be applied to scalable encoding and decoding methods. That is, for multiple encoded information of point clouds at multiple scales sent by the encoder, the decoder can decode and reconstruct the point cloud at any scale, in a sequence from low to high scale, based on the actual decoding accuracy requirements. For example, the encoder writes and transmits encoded information for a first-scale point cloud, a second-scale point cloud, and a fifth-scale point cloud in the bitstream. The decoder can, based on preset accuracy requirements and using the decoding method provided in the embodiments of this application, decode from the first-scale point cloud to a third-scale point cloud, reconstruct the geometric data of the third-scale point cloud, and restore the 3D image model of the third-scale point cloud before ending the decoding process and discontinuing the decoding of the encoded information corresponding to the fifth-scale point cloud. The specific method should be selected based on actual circumstances and is not limited by the embodiments of this application.

[0139] It is understandable that the decoding method provided in the embodiment of the present application can be repeatedly applied between multiple adjacent scales, and the decoding between each group of adjacent scales is independent of each other, so scale-scalable decoding can be flexibly implemented.

[0140] It should be noted that each decoding process of the above decoder uses the decoded low-scale point cloud as known information to decode the encoded information of the high-scale point cloud. For the first decoding process of the decoder, its known information can be a preset number of unencoded point cloud information sent by the encoder side. The encoder can use the preset number of point cloud information, such as the coordinates of 100 points in the point cloud, as the first known information and send it directly to the decoding end in an unencoded manner, so that the decoder does not need to decode the first known information and can directly use the position information of the preset number of points sent by the encoder to reconstruct the point cloud of the corresponding scale to continue the subsequent decoding process.

[0141] The following describes an encoding method applied to an encoder provided in an embodiment of the present application.

[0142] See Figure 16, which is an optional flow chart of the encoding method provided in an embodiment of the present application, and will be explained in conjunction with the steps shown in Figure 16.

[0143] S501 : Down-sample the second-scale point cloud to determine a first-scale point cloud, and up-sample the first-scale voxels in the first-scale point cloud to the second scale to determine the second-scale voxels corresponding to the first-scale voxels.

[0144] In the embodiment of the present application, the encoder voxelizes the raw point cloud data of the second-scale point cloud. The voxelized point cloud can represent the geometric data of the point cloud by the occupancy symbols of the voxels at each position in the voxel grid. The encoder performs voxel downsampling on the voxelized second-scale point cloud to determine the first-scale point cloud.

[0145] In some embodiments, the encoder can implement voxel downsampling by pooling. As shown in FIG17 , a maximum pooling layer with a step size of 2×2×2 is used to merge 8 second-scale voxels into 1 first-scale voxel. After voxel downsampling, 3 of the 4 first-scale voxels corresponding to the first-scale point cloud are occupied, and 1 first-scale voxel is not occupied. The size of the first-scale voxel in three dimensions is twice that of the second-scale voxel. The encoder uses an occupancy symbol to mark the occupancy of the voxel. For example, the occupancy symbol corresponding to the second-scale voxel on the side of the second-scale point cloud facing the paper is obtained by voxel upsampling, and the process of obtaining the occupancy symbol corresponding to the first-scale voxel on the side of the first-scale point cloud facing the paper can be shown in FIG18 . In this way, by voxel downsampling, the geometric data of the first-scale point cloud of relatively low scale is obtained.

[0146] In the embodiment of the present application, the encoder then upsamples the first-scale point cloud to the second scale, determines multiple second-scale voxels corresponding to each first-scale voxel, and implements a lossless encoding process by predicting the occupancy of the second-scale voxels.

[0147] S502 : Perform local density prediction based on the first-scale point cloud to determine the local density corresponding to the first-scale voxel, and perform occupancy probability prediction on the second-scale voxel to determine the occupancy probability corresponding to the second-scale voxel.

[0148] In the embodiment of the present application, local density represents the number of occupied second-scale voxels in the second-scale voxels corresponding to the first-scale voxels. The encoder performs local density prediction based on the first-scale point cloud, determines the local density corresponding to the first-scale voxels, and predicts the occupancy probability of the second-scale voxels. The process of determining the occupancy probability corresponding to the second-scale voxels is the same as the same processing method in the decoder, and will not be repeated here.

[0149] In some embodiments, the process of S502 may include: performing feature extraction on the geometric information of the first-scale point cloud to determine the first-scale point cloud features; upsampling the first-scale point cloud features to the second scale to determine the initial second-scale point cloud features; performing feature extraction on the initial second-scale point cloud features to determine the second-scale point cloud features, and performing occupancy probability prediction based on the second-scale point cloud features to determine the occupancy probability corresponding to the second-scale voxel; performing local density prediction based on the first-scale point cloud features to determine the local density corresponding to the first-scale voxel.

[0150] Alternatively, in some embodiments, the process of S502 may also include: performing feature extraction on the geometric information of the first-scale point cloud through a first feature extraction network to determine the first point cloud features of the first scale; upsampling the first point cloud features of the first scale to a second scale to determine the second-scale point cloud features, and performing occupancy probability prediction based on the second-scale point cloud features to determine the occupancy probability corresponding to the second-scale voxel; performing feature extraction on the geometric data of the first-scale point cloud through a second feature extraction network to determine the second point cloud features of the first scale; performing local density prediction based on the second point cloud features of the first scale to determine the local density corresponding to the first-scale voxel.

[0151] Alternatively, in some embodiments, the process of S502 may also include: performing feature extraction on the geometric information of the first-scale point cloud to determine the first-scale point cloud features; upsampling the first-scale point cloud features to the second scale to determine the second-scale point cloud features, and performing occupancy probability prediction based on the second-scale point cloud features to determine the occupancy probability corresponding to the second-scale voxel; performing local density prediction based on the first-scale point cloud features to determine the local density corresponding to the first-scale voxel.

[0152] In the above-mentioned local density prediction process, the encoder can use the local density prediction network to perform local density prediction based on the first-scale point cloud features to determine the local density corresponding to the first-scale voxel; wherein the local density prediction network includes: a first sparse convolution layer, a first activation function layer, a second sparse convolution layer and a second activation function layer.

[0153] The above processing is consistent with the description of the same processing of the decoder and will not be repeated here.

[0154] S503 : Determine reconstructed geometric information corresponding to the second-scale point cloud according to the local density corresponding to the first-scale voxel and the occupancy probability corresponding to the second-scale voxel.

[0155] In the embodiment of the present application, the encoder determines the reconstructed geometric information corresponding to the second-scale point cloud based on the local density corresponding to the first-scale voxel and the occupancy probability corresponding to the second-scale voxel, which is consistent with the description of the same processing process of the decoder and is not repeated here.

[0156] In some embodiments, S503 may be implemented by executing the process of S5031-S5032 as follows:

[0157] S5031. For each first-scale voxel in the first-scale point cloud, determine the second-scale voxels with the highest local density of occupation probability among the multiple second-scale voxels corresponding to each first-scale voxel as occupied second-scale voxels.

[0158] S5032: Determine reconstructed geometric information corresponding to the second-scale point cloud based on the occupied second-scale voxels corresponding to each first-scale voxel.

[0159] Here, the process of S5031-S5032 is consistent with the process description of S1031-S1032 above, and will not be repeated here.

[0160] S504 : Encode based on the reconstructed geometric information corresponding to the second-scale point cloud, determine encoding information corresponding to the second-scale point cloud, and write the encoding information into a bitstream.

[0161] In an embodiment of the present application, the encoder can perform recoloring based on the reconstructed geometric information corresponding to the second-scale point cloud to obtain colored point cloud data, perform color information encoding based on the colored point cloud data, and determine the attribute information encoding corresponding to the second-scale point cloud; perform geometric information encoding on the point cloud data of the second-scale point cloud to determine the geometric information encoding corresponding to the second-scale point cloud; and combine the geometric information encoding and the attribute information encoding to determine the encoding information corresponding to the second-scale point cloud.

[0162] In some embodiments, entropy coding may adopt a context-based adaptive binary arithmetic coding (CABAC) algorithm, but is not limited thereto. According to the principle of entropy coding, the more accurate the prediction of voxel occupancy, the smaller the information entropy, and the more actual bit rate and bandwidth are saved. In this way, the encoder writes the code corresponding to the second-scale point cloud into the bitstream and sends it to the decoder. The decoder parses the coding information corresponding to the second-scale point cloud. Using the decoding method provided in the embodiment of the present application, based on the geometric data of the previously decoded low-scale point cloud (such as the first-scale point cloud), it is sent to the entropy decoder, and the geometric data of the lossless second-scale point cloud can be reconstructed, that is, the reconstructed geometric information corresponding to the second-scale point cloud is determined, and then the geometric decoding and attribute decoding of the coding information of the second-scale point cloud are realized based on the reconstructed geometric information corresponding to the second-scale point cloud, and the three-dimensional image model of the second-scale point cloud is restored.

[0163] It can be understood that in the embodiment of the present application, the occupancy probability corresponding to the second-scale voxels is screened based on the local density corresponding to the first-scale voxels, which can improve the accuracy of determining the occupancy status of the second-scale voxels, and further improve the accuracy of encoding the reconstructed geometric information of the second-scale point cloud determined based on the occupancy status of the second-scale voxels, thereby improving the encoding performance.

[0164] In some embodiments, the encoder can also perform at least one voxel downsampling based on the first-scale point cloud through the same encoding process to determine a point cloud at a lower scale than the first-scale point cloud, and encode the point clouds at multiple scales one by one. The encoder writes the encoded information of the point clouds at multiple scales into the bitstream and sends it to the decoder.

[0165] It is understandable that the encoding method provided in the embodiment of the present application can be repeatedly applied between multiple adjacent scales, and the encoding between each group of adjacent scales is independent of each other, so scale-scalable encoding can be flexibly implemented.

[0166] Below, in conjunction with Figure 19, the application of the decoding method provided in an embodiment of the present application in a practical scenario is explained.

[0167] As shown in Figure 19, the encoder performs G-PCC encoding on the raw point cloud data of the first-scale point cloud, obtaining encoded information corresponding to the first-scale point cloud and sending it to the decoder via a bitstream. The decoder performs G-PCC decoding on the encoded information corresponding to the first-scale point cloud data and determines the reconstructed geometric information of the first-scale point cloud through the geometric decoding process in G-PCC decoding. Using the geometric decoding process in G-PCC decoding as an example, the decoder performs feature extraction on the reconstructed geometric information of the first-scale point cloud to obtain first-scale point cloud features. The decoder then upsamples the first-scale point cloud features to the second scale using a 2×2×2 transposed convolution to determine initial second-scale point cloud features. The decoder then performs feature extraction on the initial second-scale point cloud features to determine second-scale point cloud features. Based on the second-scale point cloud features, the decoder predicts occupancy probabilities for multiple second-scale voxels obtained by upsampling each first-scale voxel in the first-scale point cloud, obtaining multiple occupancy probabilities corresponding to the multiple second-scale voxels. Furthermore, based on the first-scale point cloud features, the decoder predicts the local density of each first-scale voxel to obtain the local density corresponding to each first-scale voxel. During the reconstruction process, the decoder filters the multiple occupancy probabilities corresponding to multiple second-scale voxels sampled from each first-scale voxel based on the local density corresponding to that first-scale voxel. For example, the decoder sorts the occupancy probabilities from high to low, determines the local density occupancy probabilities with the highest occupancy probabilities, and then determines the local density occupancy probabilities corresponding to the local density occupancy probabilities with the highest occupancy probabilities as the local density occupancy probabilities of the second-scale voxels. The reconstructed geometric information of the second-scale point cloud is determined based on the local density occupancy probabilities of the second-scale voxels corresponding to each first-scale voxel in the first-scale point cloud. Based on the reconstructed geometric information of the second-scale point cloud, the same process is used to reconstruct the reconstructed geometric information of the third-scale point cloud.

[0168] It can be understood that the embodiments of the present application have good geometric reconstruction quality and better encoding and decoding performance than the traditional G-PCC method. In some embodiments, the applicant conducted a comparative encoding and decoding test on multiple point cloud data sets using the encoding and decoding method of the embodiments of the present application and the traditional G-PCC method. The results are shown in Table 1 as follows:

[0169] Table 1

[0170] Point cloud dataset: The rate-distortion performance gain of the embodiment of the present application compared to the traditional G-PCC is: facade_00009_vox12 -25.91% house_without_roof_00057_vox12 -43.11% boxer_viewdep_vox12 -42.76% soldier_viewdep_vox12 -45.60% Average gain -39.35%

[0171] In Table 1, facade_00009_vox12, house_without_roof_00057_vox12, boxer_viewdep_vox12, and soldier_viewdep_vox12 are different point cloud data sets, and BD-rate gain over. The rate-distortion (Bjontegaard-Delta, BD-rate) performance gain value is negatively correlated with the codec performance. The smaller the gain, such as the smaller the value represented by the negative value, the better the codec performance. It can be seen that the embodiment of the present application has a higher BD-rate gain than the traditional G-PCC codec method, with a maximum increase of 45.60% and an average increase of 39.35. This data illustrates the improvement in codec performance.

[0172] The embodiment of the present application provides a decoder 1, as shown in FIG20 , including:

[0173] The parsing part 11 is configured to parse the code stream and determine the coding information corresponding to the second-scale point cloud;

[0174] The determining part 12 is configured to determine a first-scale point cloud; the first-scale point cloud is the previously decoded point cloud data corresponding to the second-scale point cloud;

[0175] The first prediction unit 13 is configured to perform local density prediction based on the first-scale point cloud to determine the local density corresponding to the first-scale voxels in the first-scale point cloud, and perform occupancy probability prediction on the second-scale voxels to determine the occupancy probability corresponding to the second-scale voxels; the second-scale voxels are upsampled voxels corresponding to the first-scale voxels; the local density represents the number of occupied second-scale voxels in the second-scale voxels corresponding to the first-scale voxels;

[0176] The decoding and reconstruction part 14 is configured to decode and reconstruct the encoded information corresponding to the second-scale point cloud based on the occupancy probability corresponding to the second-scale voxel and the local density corresponding to the first-scale voxel, and determine the reconstructed geometric information corresponding to the second-scale point cloud.

[0177] In some embodiments, the decoding and reconstruction part 14 is further configured to, for each first-scale voxel in the first-scale point cloud, determine the second-scale voxels with a high local density of occupation probability among the multiple second-scale voxels corresponding to each first-scale voxel as occupied second-scale voxels; based on the occupied second-scale voxels corresponding to each first-scale voxel, decode and reconstruct the encoded information corresponding to the second-scale point cloud to determine the reconstructed geometric information corresponding to the second-scale point cloud.

[0178] In some embodiments, the first prediction part 13 is further configured to perform feature extraction on the geometric information of the first-scale point cloud to determine first-scale point cloud features; upsample the first-scale point cloud features to a second scale to determine initial second-scale point cloud features, perform feature extraction on the initial second-scale point cloud features to determine second-scale point cloud features, and perform occupancy probability prediction based on the second-scale point cloud features to determine the occupancy probability corresponding to the second-scale voxel; and perform local density prediction based on the first-scale point cloud features to determine the local density corresponding to the first-scale voxel.

[0179] In some embodiments, the first prediction part 13 is further configured to perform feature extraction on the geometric information of the first-scale point cloud through a first feature extraction network to determine the first point cloud features of the first scale; upsample the first point cloud features of the first scale to a second scale to determine the second-scale point cloud features, and perform occupancy probability prediction based on the second-scale point cloud features to determine the occupancy probability corresponding to the second-scale voxel; perform feature extraction on the geometric data of the first-scale point cloud through a second feature extraction network to determine the second point cloud features of the first scale; perform local density prediction based on the second point cloud features of the first scale to determine the local density corresponding to the first-scale voxel.

[0180] In some embodiments, the first prediction part 13 is further configured to perform feature extraction on the geometric information of the first-scale point cloud to determine the first-scale point cloud features; upsample the first-scale point cloud features to the second scale to determine the second-scale point cloud features, and perform occupancy probability prediction based on the second-scale point cloud features to determine the occupancy probability corresponding to the second-scale voxel; and perform local density prediction based on the first-scale point cloud features to determine the local density corresponding to the first-scale voxel.

[0181] In some embodiments, the first prediction part 13 is further configured to use a local density prediction network to perform local density prediction based on the first-scale point cloud features to determine the local density corresponding to the first-scale voxel.

[0182] In some embodiments, the local density prediction network includes:

[0183] The first sparse convolution layer, the first activation function layer, the second sparse convolution layer and the second activation function layer.

[0184] In some embodiments, the parsing part 11 is further configured to parse the code stream and determine the encoding information corresponding to the i-th scale point cloud; i is an integer greater than or equal to 3; the first prediction part 13 is further configured to perform local density prediction of the i-1th scale voxel based on the reconstructed geometric information of the i-1th scale point cloud, determine the local density corresponding to the i-1 scale voxel, and perform occupancy probability prediction on the i-th scale voxel corresponding to the i-1 scale voxel, and determine the occupancy probability corresponding to the i-th scale voxel; the i-th scale is obtained by upsampling the i-1th scale; the decoding and reconstruction part 14 is further configured to decode and reconstruct the encoding information corresponding to the i-th scale point cloud based on the occupancy probability corresponding to the i-th scale voxel and the local density corresponding to the i-1th scale voxel, and determine the reconstructed geometric information corresponding to the i-th scale point cloud.

[0185] In some embodiments, the first prediction part 13 is further configured to perform local density prediction of the n-th scale voxel and occupancy probability prediction of the n+1-th scale voxel based on the reconstructed geometric data corresponding to the n-th scale point cloud, and determine the local density corresponding to the n-th scale voxel and the occupancy probability corresponding to the n+1-th scale voxel; n is a positive integer greater than or equal to 2; the n+1-th scale is obtained by upsampling the n-th scale; the decoding and reconstruction part 14 is further configured to determine the reconstructed geometric data corresponding to the n+1-th scale point cloud based on the local density corresponding to the n-th scale voxel and the occupancy probability corresponding to the n+1-th scale voxel.

[0186] The embodiment of the present application provides an encoder 2, as shown in FIG21 , including:

[0187] a downsampling portion 21 configured to perform voxel downsampling on the second-scale point cloud to determine a first-scale point cloud;

[0188] The second prediction unit 22 is configured to perform local density prediction based on the first-scale point cloud to determine the local density corresponding to the first-scale voxel, and perform occupancy probability prediction on the second-scale voxel to determine the occupancy probability corresponding to the second-scale voxel; the local density represents the number of occupied second-scale voxels in the second-scale voxel corresponding to the first-scale voxel;

[0189] The reconstruction part 23 is configured to determine the reconstructed geometric information corresponding to the second-scale point cloud according to the local density corresponding to the first-scale voxel and the occupancy probability corresponding to the second-scale voxel;

[0190] The encoding part 24 is configured to perform encoding based on the reconstructed geometric information corresponding to the second-scale point cloud, determine encoding information corresponding to the second-scale point cloud, and write the encoding information into a bitstream.

[0191] In some embodiments, the reconstruction part 23 is further configured to, for each first-scale voxel in the first-scale point cloud, determine the second-scale voxels with a high local density of occupation probability among the multiple second-scale voxels corresponding to each first-scale voxel as occupied second-scale voxels; and determine the reconstructed geometric information corresponding to the second-scale point cloud based on the occupied second-scale voxels corresponding to each first-scale voxel.

[0192] In some embodiments, the second prediction part 22 is further configured to perform feature extraction on the geometric information of the first-scale point cloud to determine the first-scale point cloud features; upsample the first-scale point cloud features to the second scale to determine the initial second-scale point cloud features, perform feature extraction on the initial second-scale point cloud features to determine the second-scale point cloud features, and perform occupancy probability prediction based on the second-scale point cloud features to determine the occupancy probability corresponding to the second-scale voxel; perform local density prediction based on the first-scale point cloud features to determine the local density corresponding to the first-scale voxel.

[0193] In some embodiments, the second prediction part 22 is further configured to perform feature extraction on the geometric information of the first-scale point cloud through a first feature extraction network to determine the first point cloud features of the first scale; upsample the first point cloud features of the first scale to a second scale to determine the second-scale point cloud features, and perform occupancy probability prediction based on the second-scale point cloud features to determine the occupancy probability corresponding to the second-scale voxel; perform feature extraction on the geometric data of the first-scale point cloud through a second feature extraction network to determine the second point cloud features of the first scale; perform local density prediction based on the second point cloud features of the first scale to determine the local density corresponding to the first-scale voxel.

[0194] In some embodiments, the second prediction part 22 is further configured to perform feature extraction on the geometric information of the first-scale point cloud to determine the first-scale point cloud features; upsample the first-scale point cloud features to the second scale to determine the second-scale point cloud features, and perform occupancy probability prediction based on the second-scale point cloud features to determine the occupancy probability corresponding to the second-scale voxel; and perform local density prediction based on the first-scale point cloud features to determine the local density corresponding to the first-scale voxel.

[0195] In some embodiments, the second prediction part 22 is further configured to use a local density prediction network to perform local density prediction based on the first-scale point cloud features to determine the local density corresponding to the first-scale voxel; wherein the local density prediction network includes: a first sparse convolution layer, a first activation function layer, a second sparse convolution layer and a second activation function layer.

[0196] In some embodiments, the encoding part 24 is further configured to perform recoloring based on the reconstructed geometric information corresponding to the second-scale point cloud to obtain colored point cloud data, perform color information encoding based on the colored point cloud data, and determine the attribute information encoding corresponding to the second-scale point cloud; perform geometric information encoding on the point cloud data of the second-scale point cloud to determine the geometric information encoding corresponding to the second-scale point cloud; and combine the geometric information encoding and the attribute information encoding to determine the encoding information corresponding to the second-scale point cloud.

[0197] It should be noted that the description of the above device embodiment is similar to the description of the above method embodiment and has similar beneficial effects as the method embodiment. For technical details not disclosed in the device embodiment of this application, please refer to the description of the method embodiment of this application for understanding.

[0198] In some embodiments, the present application also provides a decoder. Figure 22 is a schematic diagram of an optional structure of a decoder 3 provided in the present application. As shown in Figure 22, the decoder 3 includes a first memory 32 and a first processor 33. The first memory 32 and the first processor 33 are connected via a first communication bus 34. The first memory 32 is used to store executable instructions. The first processor 33 is used to implement the decoding method provided in the present application when executing the executable instructions stored in the first memory 32.

[0199] In some embodiments, the present application also provides an encoder. Figure 23 is a schematic diagram of an optional structure of an encoder 4 provided in the present application. As shown in Figure 23, encoder 4 includes a second memory 42 and a second processor 43. The second memory 42 and the second processor 43 are connected via a second communication bus 44. The second memory 42 is used to store executable instructions. The second processor 43 is used to implement the encoding method provided in the present application when executing the executable instructions stored in the second memory 42.

[0200] An embodiment of the present application provides a computer-readable storage medium storing executable instructions, wherein the executable instructions are stored. When the executable instructions are executed by a first processor, the first processor will be caused to execute any one of the decoding methods provided in the embodiments of the present application; or, when the executable instructions are executed by a second processor, the second processor will be caused to execute any one of the encoding methods provided in the embodiments of the present application.

[0201] In some embodiments, the computer-readable storage medium may be a memory such as FRAM, ROM, PROM, EPROM, EEPROM, flash memory, magnetic surface storage, optical disk, or CD-ROM; or various devices including one or any combination of the above memories.

[0202] In some embodiments, executable instructions may be in the form of a program, software, software module, script, or code, written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and may be deployed in any form, including as a stand-alone program or as a module, component, subroutine, or other unit suitable for use in a computing environment.

[0203] As an example, executable instructions may, but need not, correspond to a file in a file system, may be stored as part of a file that stores other programs or data, such as in one or more scripts in a HyperText Markup Language (HTML) document, in a single file dedicated to the program in question, or in multiple coordinating files (e.g., files storing one or more modules, subroutines, or code portions).

[0204] By way of example, executable instructions may be deployed to be executed on one computing device, or on multiple computing devices at one site, or on multiple computing devices distributed across multiple sites and interconnected by a communication network.

[0205] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of hardware embodiments, software embodiments, or embodiments combining software and hardware. Furthermore, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage and optical storage, etc.) containing computer-usable program code.

[0206] The present application is described with reference to the flow chart and / or block diagram of the method, device (system), and computer program product according to the embodiment of the present application. It should be understood that each flow process and / or box in the flow chart and / or block diagram and the combination of the flow process and / or box in the flow chart and / or block diagram can be realized by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processing machine or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device produce a device for realizing the function specified in one flow chart flow or multiple flows and / or one box or multiple boxes of the block diagram.

[0207] These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce a product including an instruction device that implements the functions specified in one or more processes in the flowchart and / or one or more boxes in the block diagram.

[0208] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, so that the instructions executed on the computer or other programmable device provide steps for implementing the functions specified in one or more processes in the flowchart and / or one or more boxes in the block diagram.

[0209] The above description is only a preferred embodiment of the present application and is not intended to limit the scope of protection of the present application. Any modifications, equivalent replacements, and improvements made within the spirit and scope of the present application are included in the scope of protection of the present application. Industrial Applicability

[0210] Embodiments of the present application provide a coding and decoding method, a decoder, an encoder, and a computer-readable storage medium. The decoder can determine the number of occupied second-scale voxels in the second-scale voxels obtained by sampling each first-scale voxel by predicting the local density. In this way, the occupancy probability corresponding to the second-scale voxels can be screened in combination with the local density to determine the occupancy of the second-scale voxels. A second-scale point cloud is reconstructed based on the occupancy of the second-scale voxels, and the reconstructed geometric information of the second-scale point cloud is determined. In this way, the occupancy of the second-scale voxels can be determined more accurately, improving the accuracy of the reconstructed geometric information of the second-scale point cloud, improving the quality of the reconstructed geometry of the decoder, and thus improving decoding performance. Furthermore, in the encoder, the occupancy probability corresponding to the second-scale voxels is screened based on the local density corresponding to the first-scale voxels. This can improve the accuracy of determining the occupancy of the second-scale voxels, thereby improving the accuracy of encoding the reconstructed geometric information of the second-scale point cloud determined based on the occupancy of the second-scale voxels, thereby improving encoding performance.

Claims

1. A decoding method, comprising: Parse the code stream to determine the encoding information corresponding to the second-scale point cloud and determine the first-scale point cloud; The first-scale point cloud is previously decoded point cloud data corresponding to the second-scale point cloud; performing local density prediction based on the first-scale point cloud to determine the local density corresponding to the first-scale voxel in the first-scale point cloud, and performing occupancy probability prediction on the second-scale voxel to determine the occupancy probability corresponding to the second-scale voxel; the second-scale voxel is an upsampled voxel corresponding to the first-scale voxel; The local density represents the number of occupied second-scale voxels in the second-scale voxels corresponding to the first-scale voxels; Based on the occupancy probability corresponding to the second-scale voxel and the local density corresponding to the first-scale voxel, the encoded information corresponding to the second-scale point cloud is decoded and reconstructed to determine the reconstructed geometric information corresponding to the second-scale point cloud.

2. The method according to claim 1, wherein The decoding and reconstructing of the encoded information corresponding to the second-scale point cloud based on the occupancy probability corresponding to the second-scale voxel and the local density corresponding to the first-scale voxel to determine the reconstructed geometric information corresponding to the second-scale point cloud includes: For each first-scale voxel in the first-scale point cloud, determine second-scale voxels with a high local density of occupation probability among the multiple second-scale voxels corresponding to each first-scale voxel as occupied second-scale voxels; Based on the occupied second-scale voxels corresponding to each first-scale voxel, the encoded information corresponding to the second-scale point cloud is decoded and reconstructed to determine the reconstructed geometric information corresponding to the second-scale point cloud.

3. The method according to claim 1, wherein The performing local density prediction based on the first-scale point cloud to determine the local density corresponding to the first-scale voxel in the first-scale point cloud, and performing occupancy probability prediction on the second-scale voxel to determine the occupancy probability corresponding to the second-scale voxel, includes: performing feature extraction on the geometric information of the first-scale point cloud to determine first-scale point cloud features; Upsampling the first-scale point cloud features to a second scale to determine initial second-scale point cloud features, performing feature extraction on the initial second-scale point cloud features to determine second-scale point cloud features, and performing occupancy probability prediction based on the second-scale point cloud features to determine occupancy probabilities corresponding to voxels at the second scale; Local density prediction is performed based on the first-scale point cloud features to determine the local density corresponding to the first-scale voxels.

4. The method according to claim 1, wherein The performing local density prediction based on the first-scale point cloud to determine the local density corresponding to the first-scale voxel in the first-scale point cloud, and performing occupancy probability prediction on the second-scale voxel to determine the occupancy probability corresponding to the second-scale voxel, includes: Performing feature extraction on the geometric information of the first-scale point cloud through a first feature extraction network to determine first point cloud features at the first scale; Upsampling the first point cloud features at the first scale to a second scale to determine the second-scale point cloud features, and performing occupancy probability prediction based on the second-scale point cloud features to determine the occupancy probability corresponding to the second-scale voxel; performing feature extraction on the geometric data of the first-scale point cloud through a second feature extraction network to determine features of a second point cloud at the first scale; Local density prediction is performed based on the second point cloud features at the first scale to determine the local density corresponding to the voxel at the first scale.

5. The method according to claim 1, wherein The performing local density prediction based on the first-scale point cloud to determine the local density corresponding to the first-scale voxel in the first-scale point cloud, and performing occupancy probability prediction on the second-scale voxel to determine the occupancy probability corresponding to the second-scale voxel, includes: performing feature extraction on the geometric information of the first-scale point cloud to determine first-scale point cloud features; Upsampling the first-scale point cloud features to a second scale to determine the second-scale point cloud features, and performing occupancy probability prediction based on the second-scale point cloud features to determine the occupancy probability corresponding to the second-scale voxel; Local density prediction is performed based on the first-scale point cloud features to determine the local density corresponding to the first-scale voxels.

6. The method according to any one of claims 3 to 5, wherein: The method further comprises: A local density prediction network is used to perform local density prediction based on the first-scale point cloud features to determine the local density corresponding to the first-scale voxel.

7. The method according to claim 6, wherein: The local density prediction network includes: The first sparse convolution layer, the first activation function layer, the second sparse convolution layer and the second activation function layer.

8. The method according to any one of claims 1 to 5 or claim 7, wherein: The method further comprises: Parse the code stream to determine the coding information corresponding to the i-th scale point cloud; i is an integer greater than or equal to 3; Based on the reconstructed geometric information of the i-1th scale point cloud, a local density prediction of the i-1th scale voxel is performed to determine the local density corresponding to the i-1th scale voxel, and an occupancy probability prediction is performed on the i-th scale voxel corresponding to the i-1th scale voxel to determine the occupancy probability corresponding to the i-th scale voxel; the i-th scale is obtained by upsampling the i-1th scale; Based on the occupancy probability corresponding to the i-th scale voxel and the local density corresponding to the i-1-th scale voxel, the encoded information corresponding to the i-th scale point cloud is decoded and reconstructed to determine the reconstructed geometric information corresponding to the i-th scale point cloud.

9. The method according to any one of claims 1 to 5 or claim 7, wherein: The method further comprises: Based on the reconstructed geometric data corresponding to the n-th scale point cloud, a local density prediction of the n-th scale voxel and an occupancy probability prediction of the n+1-th scale voxel are performed to determine the local density corresponding to the n-th scale voxel and the occupancy probability corresponding to the n+1-th scale voxel; n is a positive integer greater than or equal to 2; the n+1-th scale is obtained by upsampling the n-th scale; Based on the local density corresponding to the n-th scale voxel and the occupancy probability corresponding to the n+1-th scale voxel, reconstructed geometric data corresponding to the n+1-th scale point cloud is determined.

10. A coding method comprising: Downsampling the second-scale point cloud to determine a first-scale point cloud, and upsampling the first-scale voxels in the first-scale point cloud to the second scale to determine the second-scale voxels corresponding to the first-scale voxels; performing local density prediction based on the first-scale point cloud to determine the local density corresponding to the first-scale voxel, and performing occupancy probability prediction on the second-scale voxel to determine the occupancy probability corresponding to the second-scale voxel; the local density represents the number of occupied second-scale voxels in the second-scale voxel corresponding to the first-scale voxel; determining, based on the local density corresponding to the first-scale voxels and the occupancy probability corresponding to the second-scale voxels, reconstructed geometric information corresponding to the second-scale point cloud; Encoding is performed based on the reconstructed geometric information corresponding to the second-scale point cloud, encoding information corresponding to the second-scale point cloud is determined, and the encoding information corresponding to the second-scale point cloud is written into a bitstream.

11. The method according to claim 10, wherein: The determining, based on the local density corresponding to the first-scale voxel and the occupancy probability corresponding to the second-scale voxel, reconstructing geometric information corresponding to the second-scale point cloud includes: For each first-scale voxel in the first-scale point cloud, determine second-scale voxels with a high local density of occupation probability among the multiple second-scale voxels corresponding to each first-scale voxel as occupied second-scale voxels; Based on the occupied second-scale voxels corresponding to each first-scale voxel, reconstructed geometric information corresponding to the second-scale point cloud is determined.

12. The method according to claim 10, wherein: The performing local density prediction based on the first-scale point cloud to determine the local density corresponding to the first-scale voxel, and performing occupancy probability prediction on the second-scale voxel to determine the occupancy probability corresponding to the second-scale voxel, includes: performing feature extraction on the geometric information of the first-scale point cloud to determine first-scale point cloud features; Upsampling the first-scale point cloud features to a second scale to determine initial second-scale point cloud features, performing feature extraction on the initial second-scale point cloud features to determine second-scale point cloud features, and performing occupancy probability prediction based on the second-scale point cloud features to determine occupancy probabilities corresponding to voxels at the second scale; Local density prediction is performed based on the first-scale point cloud features to determine the local density corresponding to the first-scale voxels.

13. The method according to claim 10, wherein: The performing local density prediction based on the first-scale point cloud to determine the local density corresponding to the first-scale voxel, and performing occupancy probability prediction on the second-scale voxel to determine the occupancy probability corresponding to the second-scale voxel, includes: Performing feature extraction on the geometric information of the first-scale point cloud through a first feature extraction network to determine first point cloud features at the first scale; Upsampling the first point cloud features at the first scale to a second scale to determine the second-scale point cloud features, and performing occupancy probability prediction based on the second-scale point cloud features to determine the occupancy probability corresponding to the second-scale voxel; performing feature extraction on the geometric data of the first-scale point cloud through a second feature extraction network to determine features of a second point cloud at the first scale; Local density prediction is performed based on the second point cloud features at the first scale to determine the local density corresponding to the voxel at the first scale.

14. The method according to claim 10, wherein: The performing local density prediction based on the first-scale point cloud to determine the local density corresponding to the first-scale voxel, and performing occupancy probability prediction on the second-scale voxel to determine the occupancy probability corresponding to the second-scale voxel, includes: performing feature extraction on the geometric information of the first-scale point cloud to determine first-scale point cloud features; Upsampling the first-scale point cloud features to a second scale to determine the second-scale point cloud features, and performing occupancy probability prediction based on the second-scale point cloud features to determine the occupancy probability corresponding to the second-scale voxel; Local density prediction is performed based on the first-scale point cloud features to determine the local density corresponding to the first-scale voxels.

15. The method according to any one of claims 12 to 14, wherein: The method further comprises: Using a local density prediction network, perform local density prediction based on the first-scale point cloud features to determine the local density corresponding to the first-scale voxel; Wherein, the local density prediction network includes: The first sparse convolution layer, the first activation function layer, the second sparse convolution layer and the second activation function layer.

16. The method according to any one of claims 10 to 14, wherein: The encoding based on the reconstructed geometric information corresponding to the second-scale point cloud, determining the encoding information corresponding to the second-scale point cloud, and writing the encoding information into the bitstream includes: recoloring the reconstructed geometric information corresponding to the second-scale point cloud to obtain colored point cloud data, performing color information encoding based on the colored point cloud data, and determining attribute information encoding corresponding to the second-scale point cloud; performing geometric information encoding on the point cloud data of the second-scale point cloud to determine the geometric information encoding corresponding to the second-scale point cloud; The geometric information encoding and the attribute information encoding are determined as encoding information corresponding to the second-scale point cloud.

17. A decoder comprising: A parsing part is configured to parse the code stream and determine the encoding information corresponding to the second-scale point cloud; a determining part, configured to determine a first-scale point cloud; The first-scale point cloud is previously decoded point cloud data corresponding to the second-scale point cloud; The prediction part is configured to perform local density prediction based on the first-scale point cloud to determine the local density corresponding to the first-scale voxel in the first-scale point cloud, and perform occupancy probability prediction on the second-scale voxel to determine the occupancy probability corresponding to the second-scale voxel; the second-scale voxel is an upsampled voxel corresponding to the first-scale voxel; The local density represents the number of occupied second-scale voxels in the second-scale voxels corresponding to the first-scale voxels; The decoding and reconstruction part is configured to decode and reconstruct the encoded information corresponding to the second-scale point cloud based on the occupancy probability corresponding to the second-scale voxel and the local density corresponding to the first-scale voxel, and determine the reconstructed geometric information corresponding to the second-scale point cloud.

18. An encoder comprising: A downsampling portion configured to perform voxel downsampling on the second-scale point cloud to determine a first-scale point cloud; a local density prediction part, configured to perform local density prediction based on the first-scale point cloud and determine the local density corresponding to the first-scale voxel; an occupancy probability prediction portion configured to upsample first-scale voxels in the first-scale point cloud to a second scale and determine second-scale voxels corresponding to the first-scale voxels; and performing occupation probability prediction on the second-scale voxels to determine the occupation probability corresponding to the second-scale voxels; a reconstruction part configured to determine reconstructed geometric information corresponding to the second-scale point cloud based on the local density corresponding to the first-scale voxel and the occupancy probability corresponding to the second-scale voxel; The encoding part is configured to perform encoding based on the reconstructed geometric information corresponding to the second-scale point cloud, determine encoding information corresponding to the second-scale point cloud, and write the encoding information into a bitstream.

19. A code stream comprising: The code stream is generated by bit encoding according to the encoding information; wherein the encoding information at least includes: encoding information corresponding to the second-scale point cloud.

20. A decoder comprising: a first memory configured to store executable instructions; The first processor is configured to implement the method according to any one of claims 1 to 9 when executing the executable instructions stored in the first memory.

21. An encoder comprising: a second memory configured to store executable instructions; The second processor is configured to implement the method according to any one of claims 10 to 16 when executing the executable instructions stored in the second memory.

22. A computer-readable storage medium storing executable instructions for causing a first processor to execute the instructions to implement the method according to any one of claims 1 to 9, or for causing a second processor to execute the instructions to implement the method according to any one of claims 10 to 16.

Images