Coding and decoding method, decoder, encoder and computer readable storage medium

Abstract

The embodiment of the invention provides a coding and decoding method, a decoder, a coder and a computer readable storage medium, which can improve the coding and decoding efficiency and compression performance, thereby improving the coding and decoding performance. The method comprises the following steps: analyzing a code stream, and determining occupation indication information corresponding to a first scale point cloud; determining a second scale point cloud, performing up-sampling and division processing on the second scale point cloud, and determining m groups of up-sampling voxels; wherein the second scale point cloud is the previous decoded point cloud data corresponding to the first scale point cloud; m is an integer greater than or equal to 1; performing occupancy probability prediction on the m groups of up-sampling voxels in sequence, determining m groups of occupancy probabilities, decoding the occupancy indication information according to the m groups of occupancy probabilities, and determining respective occupancy symbols of the m groups; and based on the m groups of occupancy symbols, determining reconstruction geometric data corresponding to the first scale point cloud.

Description

Coding and decoding method, decoder, encoder and computer-readable storage medium Technical Field

[0001] The present application relates to point cloud compression coding and decoding technology, and in particular to a coding and decoding method, a decoder, an encoder and a computer-readable storage medium. Background Art

[0002] A point cloud is a collection of points that stores the geometric position and associated attribute information of each point, thereby accurately describing objects in space. Point cloud data is massive; a single point cloud frame can contain millions of points. This poses significant challenges to its efficient storage and transmission. Therefore, compression technology is used to reduce redundant information in point cloud storage, facilitating subsequent processing.

[0003] At present, point cloud geometry compression technology based on neural networks is mainly divided into geometric lossy compression and geometric lossless compression. For geometric lossless compression, on the encoder side, it is often necessary to take the surrounding context such as parent nodes and neighbor nodes as input, and after processing through the neural network (such as convolution, fully connected) layer, output the occupancy probability of each voxel in the geometric data of the point cloud, and then use the entropy encoder to convert the voxel occupancy symbol corresponding to the occupancy probability of each voxel into a bitstream. Correspondingly, on the decoder side, the occupancy probability of each voxel is predicted according to the same process, and based on the predicted occupancy probability, the entropy decoder is used to decode the voxel occupancy symbol from the bitstream to reconstruct the geometric data of the point cloud.

[0004] It can be seen that the method of predicting the occupancy probability of each voxel in the point cloud and then encoding and decoding based on the occupancy probability will lead to a long encoding and decoding time, thereby reducing the encoding and decoding efficiency and thus the encoding and decoding performance.

[0005] Summary of the Invention

[0006] The embodiments of the present application provide a coding and decoding method, a decoder, an encoder, and a computer-readable storage medium, which can improve coding and decoding efficiency and compression performance, thereby improving coding and decoding performance.

[0007] The technical solution of this application is achieved as follows:

[0008] This embodiment of the present application provides a decoding method, including:

[0009] Parse the code stream to determine occupancy indication information corresponding to the first-scale point cloud;

[0010] Determine a second-scale point cloud, and perform upsampling and partitioning processing on the second-scale point cloud to determine m groups of upsampled voxels; wherein the second-scale point cloud is the previously decoded point cloud data corresponding to the first-scale point cloud; and m is an integer greater than or equal to 1;

[0011] performing occupancy probability prediction on the m groups of upsampled voxels in sequence to determine the m groups of occupancy probabilities, and decoding the occupancy indication information according to the m groups of occupancy probabilities to determine occupancy symbols for the m groups;

[0012] Based on the m groups of occupancy symbols, reconstructed geometric data corresponding to the first-scale point cloud is determined.

[0013] This embodiment of the present application provides an encoding method, including:

[0014] Perform voxel downsampling on the first-scale point cloud to determine the second-scale point cloud;

[0015] Performing upsampling and partitioning processing on the second-scale point cloud to determine m groups of upsampled voxels, where m is an integer greater than or equal to 1;

[0016] performing occupancy probability prediction on the m groups of upsampled voxels in sequence to determine the m groups of occupancy probabilities, and determining occupancy symbols for the m groups according to the m groups of occupancy probabilities;

[0017] Encode the occupancy symbols of the m groups, determine occupancy indication information corresponding to the first-scale point cloud, and write the occupancy indication information into a bitstream.

[0018] An embodiment of the present application provides a decoder, including:

[0019] A parsing portion configured to parse the code stream and determine occupancy indication information corresponding to the first-scale point cloud;

[0020] The first grouping portion is configured to determine a second-scale point cloud, and perform upsampling and partitioning processing on the second-scale point cloud to determine m groups of upsampled voxels; wherein the second-scale point cloud is the previously decoded point cloud data corresponding to the first-scale point cloud; and m is an integer greater than or equal to 1;

[0021] a first prediction part configured to perform occupancy probability prediction on the m groups of upsampled voxels in sequence to determine the m groups of occupancy probabilities, and decode the occupancy indication information according to the m groups of occupancy probabilities to determine occupancy symbols for the m groups;

[0022] The decoding part is configured to determine the reconstructed geometric data corresponding to the first-scale point cloud based on the m groups of occupancy symbols.

[0023] An embodiment of the present application provides an encoder, including:

[0024] A downsampling portion is configured to perform voxel downsampling on the first-scale point cloud to determine a second-scale point cloud;

[0025] The second grouping part is configured to perform upsampling and division processing on the second-scale point cloud to determine m groups of upsampled voxels, where m is an integer greater than or equal to 1;

[0026] a second prediction part configured to perform occupancy probability prediction on the m groups of upsampled voxels in sequence, determine the m groups of occupancy probabilities, and determine the occupancy symbols of the m groups according to the m groups of occupancy probabilities;

[0027] The encoding part is configured to encode the occupancy symbols of the m groups, determine the occupancy indication information corresponding to the first-scale point cloud, and write the occupancy indication information into a code stream.

[0028] This embodiment of the present application provides a code stream, including:

[0029] The code stream is generated by bit encoding according to information to be encoded; wherein the information to be encoded includes at least: occupancy indication information corresponding to the first-scale point cloud.

[0030] An embodiment of the present application provides a decoder, including:

[0031] a first memory configured to store executable instructions;

[0032] The first processor is configured to implement any one of the above decoding methods when executing the executable instructions stored in the first memory.

[0033] An embodiment of the present application provides an encoder, including:

[0034] a second memory configured to store executable instructions;

[0035] The second processor is configured to implement any one of the encoding methods described above when executing the executable instructions stored in the second memory.

[0036] An embodiment of the present application provides a computer-readable storage medium storing executable instructions for causing a first processor to execute the above-mentioned decoding method, or for causing a second processor to execute the above-mentioned encoding method.

[0037] An embodiment of the present application provides a computer program product, including a computer program or instructions. When the computer program or instructions are executed by a first processor, the decoding method provided by the embodiment of the present application is implemented; or, when the computer program or instructions are executed by a second processor, the encoding method provided by the embodiment of the present application is implemented.

[0038] The embodiments of the present application provide a coding and decoding method, a decoder, an encoder, and a computer-readable storage medium. When parsing the code stream to obtain occupancy indication information corresponding to a first-scale point cloud, that is, the encoded information representing the occupancy status of each voxel in the first-scale point cloud, the previously decoded second-scale point cloud corresponding to the first-scale point cloud can be upsampled and divided, the second-scale point cloud can be upsampled to the first scale, and the upsampled voxels are divided into m groups. The occupancy probability prediction is performed on the groups in sequence to obtain m groups of occupancy probabilities. By grouping the prediction, the prediction speed of the occupancy probability is accelerated and the processing efficiency of the prediction process is improved. In this way, the occupancy indication information corresponding to the first-scale point cloud is decoded using the m groups of occupancy probabilities to obtain the decoded m groups of occupancy symbols, reconstruct the geometric data of the first-scale point cloud, and complete the decoding of the first-scale point cloud, thereby improving the decoding efficiency and decoding performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] FIG1 is a flow chart of G-PCC coding;

[0040] FIG2 is a flow chart of G-PCC decoding;

[0041] FIG3 is a schematic diagram of an optional flow chart of an encoding method provided in an embodiment of the present application;

[0042] FIG4 is a schematic diagram of an optional process of voxel downsampling provided in an embodiment of the present application;

[0043] FIG5A is a schematic diagram showing an optional effect of occupancy symbols of voxels in a point cloud of scale p provided by an embodiment of the present application;

[0044] FIG5B is a schematic diagram showing an optional effect of occupancy symbols of voxels in a point cloud at a scale of p-1 according to an embodiment of the present application;

[0045] FIG6 is a schematic diagram of an optional flow chart of an encoding method provided in an embodiment of the present application;

[0046] FIG7 is a schematic diagram of an optional flow chart of an encoding method provided in an embodiment of the present application;

[0047] FIG8 is a schematic diagram of an optional process of upsampling and partitioning provided in an embodiment of the present application;

[0048] FIG9 is a schematic diagram of an optional flow chart of an encoding method provided in an embodiment of the present application;

[0049] FIG10 is an optional schematic diagram of a group-by-group occupancy probability prediction process according to an embodiment of the present application;

[0050] FIG11 is a schematic diagram of an optional effect of the occupancy probability of voxels in a point cloud provided by an embodiment of the present application;

[0051] FIG12 is a schematic diagram of an optional flow chart of a decoding method provided in an embodiment of the present application;

[0052] FIG13 is a schematic diagram of an optional flow chart of a decoding method provided in an embodiment of the present application;

[0053] FIG14 is a schematic diagram of an optional process of applying the encoding and decoding method provided in an embodiment of the present application to an actual scenario;

[0054] FIG15 is a schematic diagram of an optional structure of a decoder provided in an embodiment of the present application;

[0055] FIG16 is a schematic diagram of an optional structure of an encoder provided in an embodiment of the present application;

[0056] FIG17 is a schematic diagram of an optional structure of a decoder provided in an embodiment of the present application;

[0057] FIG18 is a schematic diagram of an optional structure of an encoder provided in an embodiment of the present application. DETAILED DESCRIPTION

[0058] In order to make the purpose, technical solutions and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings. The described embodiments should not be regarded as limiting this application. All other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0059] In the following description, reference is made to “some embodiments”, which describes a subset of all possible embodiments, but it will be understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0060] In the following description, the terms "first\second\third" involved are merely used to distinguish similar objects and do not represent a specific ordering of the objects. It can be understood that "first\second\third" can be interchanged with a specific order or sequence where permitted, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein.

[0061] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which this application pertains. The terms used herein are for the purpose of describing the embodiments of this application only and are not intended to limit this application.

[0062] Before further describing the embodiments of the present application in detail, the nouns and terms involved in the embodiments of the present application are explained. The nouns and terms involved in the embodiments of the present application are subject to the following interpretations.

[0063] Voxel: Short for volume element, voxel is the smallest unit of digital data used in three-dimensional space. Voxels allow for the gridding of 3D space and the assignment of characteristics to each grid. For example, a voxel is a fixed-size cube in three-dimensional space. Voxels are widely used in fields such as 3D imaging, scientific data, and medical imaging.

[0064] Point cloud compression algorithms include geometry-based point cloud compression (G-PCC). Geometry compression in G-PCC is mainly implemented through octree models and / or triangle surface models.

[0065] With the development of artificial intelligence (AI) technology, neural networks have been applied to geometry-based point cloud compression. Neural network-based point cloud geometry compression can be broadly categorized into lossy and lossless compression. Lossless compression algorithms primarily focus on designing prediction models for voxel occupancy probabilities. Voxel data representations typically utilize octree models, volumetric models, sparse tensor representations, and other methods. For lossless geometry compression, the encoder typically uses surrounding context, such as parent nodes and neighboring nodes, as input. After processing through neural network layers (e.g., convolutional and fully connected), the encoder outputs the occupancy probability of each voxel in the point cloud's geometric data. An entropy encoder then converts the voxel occupancy symbol corresponding to each voxel's occupancy probability into a bitstream. Correspondingly, the decoder uses the same process to predict the occupancy probability of each voxel. Based on the predicted occupancy probability, an entropy decoder is used to decode the voxel occupancy symbol from the bitstream to reconstruct the point cloud's geometric data.

[0066] It can be seen that methods that predict the occupancy probability of each voxel in the point cloud and then perform encoding and decoding based on the occupancy probability often have performance or complexity defects: for example, insufficient surrounding context of the predicted voxel location leads to inaccurate predictions, which in turn leads to poor encoding and compression performance; or excessive use of context information in predicting the occupancy probability of each voxel causes bit rate redundancy, resulting in slow encoding and decoding times and low encoding and decoding efficiency. In summary, this reduces encoding and decoding performance.

[0067] The embodiments of the present application provide a coding and decoding method, a decoder, an encoder and a computer-readable storage medium, which can improve coding and decoding efficiency and improve coding and decoding performance. In order to facilitate the understanding of the technical solution provided by the embodiments of the present application, a flow chart of G-PCC encoding and a flow chart of G-PCC decoding are first provided. It should be noted that the flow chart of G-PCC encoding and the flow chart of G-PCC decoding described in the embodiments of the present application are only for more clearly illustrating the technical solution of the embodiments of the present application, and do not constitute a limitation on the technical solution provided by the embodiments of the present application. It is known to those skilled in the art that with the evolution of point cloud compression technology and the emergence of new business scenarios, the technical solution provided by the embodiments of the present application is also applicable to point cloud coding and decoding architectures similar to G-PCC. The point cloud compressed by the embodiments of the present application can be a point cloud in a video, but is not limited to this.

[0068] In the point cloud G-PCC encoder framework, the point cloud of the input 3D image model is sliced ​​and each slice is encoded independently.

[0069] As shown in the G-PCC encoding flow diagram in Figure 1, the encoder first divides the point cloud data to be encoded into multiple slices using striping. Within each slice, the point cloud's geometric and attribute information are encoded separately. During the geometric encoding process, the geometric information undergoes coordinate transformation so that the entire point cloud is contained within a bounding box. Quantization then occurs. Quantization primarily serves a scaling purpose. Due to quantization rounding, the geometric information of some point clouds remains the same, allowing for parameter-based decision-making regarding whether to remove duplicate points. This process of quantization and removing duplicate points is also known as voxelization. The bounding box is then partitioned into an octree. In the octree-based geometric information encoding process, the bounding box is divided into eight equal parts. The sub-cubes that are not empty (contain points in the point cloud) are further divided into eight equal parts until the resulting leaf nodes are 1x1x1 unit cubes. The points in the leaf nodes are then arithmetic-coded to generate a binary geometric bitstream, or geometry codestream. In the process of geometric information encoding based on triangle soup (trisoup), octree partitioning must also be performed first. However, unlike the geometric information encoding based on octree, the trisoup does not need to divide the point cloud into unit cubes with a side length of 1x1x1 step by step. Instead, the division stops when the sub-block (block) has a side length of W. Based on the surface formed by the distribution of the point cloud in each block, the surface and the twelve edges of the block are obtained. The vertices are arithmetically encoded (surface fitting is performed based on the intersections) to generate a binary geometric bit stream, that is, a geometric code stream. Vertex is also used to implement the process of geometric reconstruction, and the reconstructed geometric information is used when encoding the attributes of the point cloud.

[0070] During the attribute encoding process, color conversion is performed to convert the color information (i.e., attribute information) from the RGB color space to the YUV color space. The reconstructed geometric information is then used to recolor the point cloud so that the unencoded attribute information corresponds to the reconstructed geometric information. There are two main transformation methods for color information encoding: a distance-based lifting transform that relies on level of detail (LOD) partitioning, and a direct region-adaptive hierarchical transform (RAHT) transformation. Both methods convert color information from the spatial domain to the frequency domain, obtaining high-frequency and low-frequency coefficients. These coefficients are then quantized (i.e., quantized coefficients). Finally, the geometrically encoded data (octree partitioning and surface fitting) is sliced ​​and synthesized with the attribute encoded data (quantized coefficients). The vertex coordinates of each block are then encoded (i.e., arithmetic encoding) to generate a binary attribute bitstream, i.e., the attribute codestream.

[0071] The flowchart of G-PCC decoding shown in Figure 2 is applied to the decoder. The decoder obtains the binary code stream and independently decodes the geometric bit stream (i.e., geometric code stream) and attribute bit stream in the binary code stream. When decoding the geometric bit stream, the geometric information of the point cloud is obtained through arithmetic decoding-octree synthesis-surface fitting-reconstruction geometry-inverse coordinate transformation; when decoding the attribute bit stream, the attribute information of the point cloud is obtained through arithmetic decoding-inverse quantization-LOD-based inverse lifting or RAHT-based inverse transformation-inverse color conversion, and the three-dimensional image model of the point cloud data to be encoded is restored based on the geometric information and attribute information.

[0072] The encoding method of the embodiment of the present application can be applied to the geometric information encoding process of G-PCC as shown in Figure 1. After voxelization is completed, the voxelized first-scale point cloud is down-sampled to determine the second-scale point cloud; the second-scale point cloud is up-sampled and divided to determine m groups of up-sampled voxels; the occupancy probability of the m groups of up-sampled voxels is predicted in turn to determine the occupancy probability of the m groups, and the occupancy symbols of the m groups are determined according to the occupancy probabilities of the m groups as the data that needs to be entropy encoded, thereby replacing the octree allocation and surface fitting processing in Figure 1. The encoding process of the embodiment of the present application can adopt the arithmetic coding method in Figure 1, such as entropy coding. The occupancy symbols of the m groups are encoded through the arithmetic coding process to determine the occupancy indication information corresponding to the first-scale point cloud and write it into the geometric bit stream (code stream).

[0073] The decoding method of the embodiment of the present application can be applied to the geometric information decoding process of G-PCC as shown in Figure 2. By parsing the geometric bit stream, the occupancy indication information corresponding to the first-scale point cloud is determined; the second-scale point cloud is determined, and the second-scale point cloud is upsampled and divided to determine m groups of upsampled voxels; wherein the second-scale point cloud is the previously decoded point cloud data corresponding to the first-scale point cloud; the occupancy probability of the m groups of upsampled voxels is predicted in turn to determine the m groups of occupancy probabilities, and the occupancy indication information is decoded by the m groups of occupancy probabilities to determine the occupancy symbols of each of the m groups. The encoding process of the embodiment of the present application can adopt the arithmetic decoding method in Figure 2, such as entropy decoding. After arithmetic decoding, the octree synthesis and surface fitting processing in Figure 2 are not required. In the process of reconstructing the geometry, the reconstructed geometry data corresponding to the first-scale point cloud is determined directly based on the m groups of occupancy symbols.

[0074] It should be noted that the encoding method and decoding method of the embodiments of the present application can also be used in other point cloud encoding and decoding processes besides G-PCC.

[0075] The following describes an encoding method applied to an encoder provided in an embodiment of the present application.

[0076] Referring to Figure 3, which is a schematic diagram of an optional flow chart of an encoding method provided in an embodiment of the present application, the steps shown in Figure 3 will be described. Since the encoding method provided in an embodiment of the present application is applied to the process of generating a geometric bitstream in Figure 1, the first-scale point cloud, the second-scale point cloud, etc. in the following embodiments specifically refer to the geometric information of the first-scale point cloud, the geometric information of the second-scale point cloud, etc.

[0077] S101 : Perform voxel downsampling on the first-scale point cloud to determine a second-scale point cloud.

[0078] In the embodiment of the present application, before the encoder performs voxel downsampling on the first-scale point cloud, it is necessary to complete voxelization of the first-scale point cloud to represent the first-scale point cloud in the form of a voxel grid.

[0079] In an embodiment of the present application, for the voxelization process, a point in the point cloud can correspond to an occupied voxel (i.e., a non-empty voxel), and an unoccupied voxel (i.e., an empty voxel) indicates that there is no point in the point cloud at that voxel position. In some embodiments, occupied voxels can be marked as 1, and unoccupied voxels can be marked as 0. In this way, the voxelized point cloud can represent the geometric data of the point cloud by the occupancy symbols of the voxels at each position in the voxel grid.

[0080] In an embodiment of the present application, the encoder performs voxel downsampling on the voxelized first-scale point cloud to obtain a second-scale point cloud. In some embodiments, the encoder can achieve voxel downsampling through pooling, such as using a maximum pooling layer with a step size of 2×2×2 to merge 8 voxels of the first-scale point cloud into 1 voxel in the second-scale point cloud. Each downsampling reduces the size of the point cloud in three dimensions to half of the original size. In this way, the first-scale point cloud can be called a high-scale point cloud relative to the second-scale point cloud, and the second-scale point cloud can be called a low-scale point cloud relative to the first-scale point cloud.

[0081] Please refer to Figure 4. As an example of a first-scale point cloud, the scale p point cloud comprises 4×4×2 voxels. After a single voxel downsampling, the resulting second-scale point cloud, the scale p-1 point cloud, comprises 2×2×1 voxels. Occupied voxels in the point cloud are represented by solid cubes, representing the locations of points in the point cloud. Unoccupied voxels in the point cloud are represented by empty cubes, representing locations where no points are located in the point cloud. As shown in Figure 4, the scale p point cloud has six occupied voxels on the side facing the paper, with the corresponding occupancy symbols shown in Figure 5A. A voxel occupancy symbol of 1 indicates that the voxel is occupied, and a symbol of 0 indicates that the voxel is unoccupied. After voxel downsampling, three of the four voxels corresponding to the scale p-1 point cloud are occupied, and one voxel is unoccupied. The corresponding occupancy symbol is shown in Figure 5B. However, the point cloud in Figure 4 is merely illustrative; actual point clouds may include many more voxels.

[0082] S102 : Perform upsampling and division processing on the second-scale point cloud to determine m groups of upsampled voxels.

[0083] In an embodiment of the present application, the encoder performs upsampling processing on the second-scale point cloud and upsamples the second-scale point cloud to the first scale. It can be understood that since the second scale is lower than the first scale, when the second-scale point cloud is upsampled, multiple upsampled voxels corresponding to a voxel of the second scale will be obtained. Whether each of the multiple upsampled voxels contains a point in the point cloud, that is, whether it is occupied, needs to be predicted through the subsequent occupancy probability prediction process. Here, the encoder divides the upsampled voxels obtained by upsampling, determines m groups of upsampled voxels, and performs subsequent prediction processing in a group-based manner. Wherein, m is an integer greater than or equal to 1.

[0084] In the embodiment of the present application, after the encoder performs upsampling processing on the second-scale point cloud, the upsampled voxels of the first scale obtained by upsampling are numbered and grouped, so as to implement division processing according to the numbering and obtain m groups of upsampled voxels.

[0085] In some embodiments, based on FIG. 3 , as shown in FIG. 6 , S102 may be implemented by executing S1021 - S1022 as follows:

[0086] S1021. Perform voxel upsampling on the second-scale point cloud to determine n upsampled voxels of the first scale; n is an integer greater than 1.

[0087] S1022 : Number each of the n up-sampled voxels, and group them based on the number corresponding to each up-sampled voxel to determine m groups of up-sampled voxels.

[0088] In some embodiments, based on FIG. 6 , as shown in FIG. 7 , S1022 may be implemented through S201 - S202 as follows:

[0089] S201 : Number each up-sampled voxel in n up-sampled voxels, and determine a number corresponding to each up-sampled voxel.

[0090] In an embodiment of the present application, the encoder can determine a numbering range based on the number of voxels corresponding to a voxel in a second-scale point cloud after upsampling; using the numbering range, among the n upsampled voxels obtained by upsampling the second-scale point cloud, the multiple upsampled voxels corresponding to each voxel in the second-scale point cloud are numbered in the same manner to obtain the number corresponding to each upsampled voxel.

[0091] S202 : Group the n up-sampled voxels according to the number of each up-sampled voxel to determine m groups of up-sampled voxels.

[0092] In some embodiments, among the n upsampled voxels, the encoder may group the upsampled voxels with the same number into a group according to the number of each upsampled voxel, and determine m groups of upsampled voxels.

[0093] In some embodiments, the encoder may also determine at least two consecutive numbers among the numbers used to mark upsampled voxels; and according to the number of each upsampled voxel, take the upsampled voxels corresponding to the at least two consecutive numbers in the n upsampled voxels as a group to determine m groups of upsampled voxels.

[0094] In some embodiments, as shown in FIG8 , the scale p-1 point cloud, i.e., the second scale point cloud, is upsampled by 2×2×2 to obtain n (n=24) upsampled voxels of scale p, i.e., n upsampled voxels of the first scale. It can be seen that one voxel of the scale p-1 point cloud corresponds to 8 voxels of scale p after upsampling, and the numbering range can be determined to be 1-8. For a voxel in the scale p-1 point cloud, the 8 upsampled voxels corresponding to the voxel in the scale p point cloud are numbered using numbers 1-8 respectively; for three voxels in the scale p-1 point cloud, the 8 upsampled voxels corresponding to the voxel in each p-1 scale point cloud are numbered using the same numbering method, resulting in 3 upsampled voxels numbered 1, 3 upsampled voxels numbered 2, ..., to 3 upsampled voxels numbered 8. The encoder groups the upsampled voxels according to the number of each upsampled voxel. FIG8 shows three optional grouping methods, as follows:

[0095] The first grouping method is to group upsampled voxels with the same number into one group. For example, each upsampled voxel numbered 1 is grouped into one group, each upsampled voxel numbered 2 is grouped into one group, and so on, to obtain 8 groups of upsampled voxels as m groups of upsampled voxels.

[0096] The second grouping method: determine that numbers 1 and 2 are consecutive numbers, numbers 3 and 4 are consecutive numbers, and numbers 5 to 8 are consecutive numbers; the up-sampled voxels corresponding to numbers 1 and 2 are grouped as one group, the up-sampled voxels corresponding to numbers 3 and 4 are grouped as one group, and the up-sampled voxels corresponding to numbers 5 to 8 are grouped as one group, to obtain 3 groups of up-sampled voxels as m groups of up-sampled voxels.

[0097] The third grouping method is to group all n upsampled voxels as one group, which is equivalent to determining the numbers 1-8 as consecutive numbers, and obtaining one group of upsampled voxels as m groups of upsampled voxels.

[0098] S103 , performing occupation probability prediction on the m groups of upsampled voxels in sequence to determine the m groups of occupancy probabilities, and determining the occupancy symbols of the m groups according to the m groups of occupancy probabilities.

[0099] In an embodiment of the present application, the encoder may perform occupancy probability prediction for each of the m groups of upsampled voxels in a progressive manner based on the determined m groups of upsampled voxels. When predicting the occupancy probability of the current group of upsampled voxels, the encoder utilizes the prediction results (e.g., occupancy symbols corresponding to the occupancy probabilities) of one or more previously predicted groups of upsampled voxels to predict the occupancy probability of the current group of upsampled voxels and determine the occupancy probability corresponding to the current group of upsampled voxels. This continues until m groups of occupancy probabilities are determined, and then occupancy symbols are determined for each of the m groups based on the m groups of occupancy probabilities.

[0100] In some embodiments, based on any one of FIG. 3 , FIG. 6 , or FIG. 7 , as shown in FIG. 9 , S103 may be implemented by executing S1031 - S1034 as follows:

[0101] S1031 . Predict occupancy probabilities for the first group of upsampled voxels in the m groups of upsampled voxels to determine a first group of occupancy probabilities, and use a preset occupancy symbol to represent the first group of occupancy probabilities to determine a first group of occupancy symbols.

[0102] In this embodiment of the present application, the first group of upsampled voxels does not have any previously predicted prediction results for other groups of upsampled voxels. Therefore, the encoder can directly predict the occupancy probabilities for the first group of upsampled voxels among the m groups of upsampled voxels to determine the first group of occupancy probabilities. Furthermore, the first group of occupancy probabilities is represented by a preset occupancy symbol, i.e., the first group of occupancy probabilities is labeled with an occupancy symbol to distinguish whether the corresponding voxels represented by the first group of occupancy probabilities are occupied, thereby determining the first group of occupancy symbols.

[0103] S1032. When i is greater than 1 and less than or equal to m, use at least k groups of occupancy symbols corresponding to at least k groups of upsampled voxels represented by completed probability predictions and occupancy symbols to predict the occupancy probability of the i-th group of upsampled voxels, and determine the occupancy probability of the i-th group; where i is an integer and k is an integer greater than or equal to 1 and less than i.

[0104] In the embodiment of the present application, when i is greater than 1 and less than or equal to m, that is, for the second to m-th groups of upsampled voxels in the m groups of upsampled voxels, when the encoder performs occupancy probability prediction on the i-th group of upsampled voxels in the second to m-th groups of upsampled voxels, it uses at least k groups of occupancy symbols corresponding to at least k groups of upsampled voxels whose probability predictions have been completed and represented by occupancy symbols to perform occupancy probability prediction on the i-th group of upsampled voxels and determine the occupancy probability of the i-th group.

[0105] In some embodiments, the encoder may perform feature extraction on the i-th group of upsampled voxels to determine the features of the i-th group of voxels; determine that the ik-th group of occupancy symbols to the i-1-th group of occupancy symbols for which probability prediction and occupancy symbol representation have been completed are at least k groups of occupancy symbols; based on at least k groups of occupancy symbols and in combination with the features of the i-th group of voxels, perform occupancy probability prediction on each upsampled voxel in the i-th group of upsampled voxels, and determine that the occupancy probability corresponding to each upsampled voxel in the i-th group of upsampled voxels is the i-th group of occupancy probability.

[0106] In some embodiments, the encoder may use the occupancy symbols corresponding to all upsampled voxel groups that have completed probability prediction and occupancy symbol representation before the i-th group of upsampled voxels as at least k groups of occupancy symbols; or it may be at least k groups of occupancy symbols represented by the most recent completed probability prediction and occupancy symbol before the i-th group of upsampled voxels. The specific selection is made according to actual conditions and is not limited in the embodiments of the present application.

[0107] S1033. Use a preset occupancy symbol to represent the occupancy probability of the i-th group, and determine the occupancy symbol of the i-th group.

[0108] In an embodiment of the present application, the encoder uses the first occupancy symbol in the preset occupancy symbol to represent the occupancy probability of the i-th group that is greater than or equal to the preset probability threshold based on the obtained occupancy probability of the i-th group; and uses the second occupancy symbol in the preset occupancy symbol to represent the occupancy probability of the i-th group that is less than the preset probability threshold, thereby determining the i-th group of occupancy symbols.

[0109] For example, the preset probability threshold may be 90%, and the preset occupancy code may include 0 and 1. If the occupancy probability of an upsampled voxel in the i-th group of occupancy probabilities is greater than or equal to the preset probability threshold, indicating that the upsampled voxel has a high probability of containing a point in the point cloud, the occupancy probability is represented by 1, and the preset occupancy code 1 is set to correspond to the upsampled voxel corresponding to the occupancy probability.

[0110] It should be noted that the process of using the preset occupancy symbol to represent the first group of occupancy probabilities and determining the first group of occupancy symbols in S1031 is consistent with the method of using the preset occupancy symbol to represent the i-th group of occupancy probabilities and determining the i-th group of occupancy symbols in S1033.

[0111] S1034 , using at least k groups of occupancy symbols including the i-th group of occupancy symbols, continue to perform occupancy probability prediction and occupancy symbol representation on the (i+1)-th group of upsampled voxels until the occupancy symbols of each of the m groups are determined.

[0112] In the embodiment of the present application, during the occupancy probability prediction and occupancy symbol representation process, the encoder utilizes at least k groups of occupancy symbols including the i-th group of occupancy symbols. For example, after obtaining the first to i-th groups of occupancy symbols, the encoder continues to perform occupancy probability prediction and occupancy symbol representation on the i+1-th group of upsampled voxels, obtaining the i+1-th group of occupancy symbols corresponding to the i+1-th group of upsampled voxels. The occupancy probability prediction and occupancy symbol representation are continued sequentially and progressively in this manner until the occupancy symbols for each of the m groups are determined.

[0113] In some embodiments, the above-mentioned processes of S1021-S1022, and S1031-S1034 can be implemented by a probability prediction model. Here, the probability prediction model is a trained deep learning network for performing feature extraction, upsampling and probability prediction processing. For example, taking the example of the second-scale point cloud (scale p-1 point cloud) shown in FIG4 , the sequence of occupancy symbols shown in FIG5B can be input into the probability prediction model for feature extraction, upsampling and occupancy probability prediction. For unoccupied voxels in the low-scale point cloud (i.e., the second-scale point cloud) (such as the voxels marked as 0 in FIG5B ), the multiple upsampled voxels of the high scale (i.e., the first scale) obtained by upsampling are also unoccupied and do not require probability prediction. However, the multiple high-scale voxels obtained by upsampling the occupied voxels in the low-scale point cloud need to be predicted. Taking 2×2×2 upsampling as an example, the scale p-1 voxel in Figure 5B will be decomposed into 8 upsampled voxels of scale p. Whether these 8 upsampled voxels are occupied is uncertain, and occupancy probability prediction is required. The process of occupancy probability prediction can be shown in Figure 10 as follows:

[0114] In Figure 10, the scale p-1 point cloud represents the second scale point cloud. The probability prediction model upsamples the scale p-1 point cloud through a convolutional neural network (CNN) to obtain n (n=24) upsampled voxels of the first scale, that is, scale p. Among them, each voxel of scale p-1 corresponds to 8 upsampled voxels of scale p. Among the 24 upsampled voxels, the encoder uses numbers 1-8 to number the 8 upsampled voxels obtained by decomposing each scale p-1 voxel one by one, and obtains the number corresponding to each upsampled voxel in the 24 upsampled voxels. The probability prediction model groups the upsampled voxels with the same number as a group, and obtains 8 groups of upsampled voxels, which enter the occupancy probability prediction process.

[0115] In the first stage of the occupancy probability prediction process, the probability prediction model uses a CNN network to predict the occupancy probability corresponding to each upsampled voxel in the first group of upsampled voxels, and obtains the occupancy symbol corresponding to each upsampled voxel in the first group of upsampled voxels. The probability prediction model then represents the occupancy probability corresponding to each upsampled voxel in the first group of upsampled voxels with an occupancy symbol, and obtains the occupancy symbol corresponding to each upsampled voxel in the first group of upsampled voxels.

[0116] As shown in Figure 10, in the second stage of the occupancy probability prediction process, the occupancy symbols corresponding to each upsampled voxel in the first group of upsampled voxels obtained in the first stage are combined to perform occupancy probability prediction and occupancy symbol representation on the second group of upsampled voxels, thereby obtaining the occupancy symbols corresponding to each upsampled voxel in the second group of upsampled voxels. In the third stage of the occupancy probability prediction process, the occupancy symbols corresponding to each upsampled voxel in the first group of upsampled voxels obtained in the first stage and the occupancy symbols corresponding to each upsampled voxel in the second group of upsampled voxels obtained in the second stage are combined to perform occupancy probability prediction and occupancy symbol representation on the third group of upsampled voxels, thereby obtaining the occupancy symbols corresponding to each upsampled voxel in the third group of upsampled voxels. This process is repeated in this way until the occupancy symbols corresponding to each upsampled voxel in the eighth group of upsampled voxels are obtained. In this way, the geometric information of the first-scale point cloud, i.e., the scale p point cloud, can be obtained using the occupancy symbols corresponding to each upsampled voxel in the eight groups of upsampled voxels. In this way, by sampling from low scale to high scale, the voxels in the high-scale point cloud are restored, and then through grouping and occupancy probability prediction, the occupancy probability of each high-scale voxel containing the points in the point cloud is predicted, and each high-scale voxel is marked with a preset occupancy symbol according to the occupancy probability, thereby predicting the geometric data of the high-scale point cloud.

[0117] In some embodiments, the occupancy probability corresponding to each of the 24 upsampled voxels predicted through the above 8 stages can be shown in Figure 11, which shows the occupancy probability corresponding to each of the upsampled voxels facing the side of the paper in the 24 upsampled voxels of scale p. The predicted probabilities in Figure 11 are only for convenience of explanation and cannot be understood as the results of actual calculations. It can be seen that if the occupancy probability predicted for the actually occupied voxels is closer to 1, and the occupancy probability predicted for the actually unoccupied voxels is closer to 0, the more accurate the prediction. The more accurate the prediction, the less encoded data is obtained when encoding the occupancy symbol determined according to the occupancy probability of the voxels in the high-scale point cloud, that is, the more accurate the probability prediction, the better the compression performance of the point cloud geometric information.

[0118] S104: Encode the occupancy symbols of the m groups, determine occupancy indication information corresponding to the first-scale point cloud, and write the occupancy indication information into the bitstream.

[0119] In an embodiment of the present application, the encoder may perform entropy coding on each of the m groups of occupancy symbols, and determine that the m groups of occupancy indication information obtained by coding are the occupancy indication information corresponding to the first-scale point cloud.

[0120] In the embodiment of the present application, whether a voxel in the high-scale point cloud is occupied can be indicated by an occupancy symbol. For voxels for which no occupancy probability prediction is performed, the occupancy symbol can be directly set to 0. The encoder encodes m groups of occupancy symbols, such as a sequence of m groups of occupancy symbols, to obtain first encoded data corresponding to the first-scale point cloud, i.e., occupancy indication information corresponding to the first-scale point cloud, thereby achieving lossless compression of the geometric data of the high-scale point cloud.

[0121] In some embodiments, entropy coding may adopt a context-based adaptive binary arithmetic coding (CABAC: Context-based Adaptive Binary Arithmetic Coding) algorithm, but is not limited thereto. According to the principle of entropy coding, the more accurate the prediction of the occupancy probability, the smaller the information entropy, and the more the actual bit rate and bandwidth are saved. In this way, the encoder writes the occupancy indication information corresponding to the first-scale point cloud into the bitstream and sends it to the decoder. The decoder extracts the occupancy indication information corresponding to the first-scale point cloud, combines it with the geometric data of the previously decoded low-scale point cloud (such as the second-scale point cloud), and sends it to the entropy decoder. It can then reconstruct the geometric data of the lossless high-scale point cloud, that is, reconstruct the geometric data corresponding to the first-scale point cloud.

[0122] It can be understood that the group prediction and encoding method in the embodiment of the present application, compared with the method of predicting and encoding the occupancy probability of each voxel based on the parent node and neighbor nodes of each voxel, on the one hand greatly reduces the encoding complexity and improves the encoding speed and compression performance; on the other hand, as the encoding proceeds, the occupancy symbols of all upsampled voxel groups that have previously completed occupancy probability prediction can be used as context information and input into the convolutional network to help predict the occupancy probability of the current upsampled voxel group, thereby improving the accuracy of the occupancy prediction probability prediction, and then improving the encoding accuracy, that is, improving the encoding performance.

[0123] In some embodiments, the encoder may encode the occupied symbols of each of the m groups in a parallel manner, thereby further accelerating the encoding speed and improving the encoding efficiency and encoding performance.

[0124] It will be appreciated that in the embodiments of the present application, a second-scale point cloud is obtained by downsampling the first-scale point cloud, and the second-scale point cloud is then upsampled and partitioned. The second-scale point cloud is then upsampled to the first scale, and the upsampled voxels are divided into m groups. Occupancy probability prediction is then performed on each group in sequence to obtain m groups of occupancy probabilities. Thus, grouped prediction accelerates the prediction speed of occupancy probabilities, improving the processing efficiency and coding efficiency of the prediction process. Thus, m groups of occupancy symbols corresponding to the m groups of occupancy probabilities are encoded, and the occupancy indication information corresponding to the first-scale point cloud is determined and written into the bitstream, completing the encoding of the first-scale point cloud, thereby improving coding efficiency and performance.

[0125] In some embodiments, Figures 4 and 10 illustrate a process of downsampling, upsampling, partitioning, and occupancy probability prediction for a first-scale point cloud, such as a point cloud at scale p. In actual compression, the first-scale point cloud may be downsampled multiple times, for example, two, three, four, or more times, to obtain point cloud geometric data at multiple lower scales. Based on the multiple lower-scale point cloud geometric data, upsampling, partitioning, occupancy probability prediction, occupancy symbol representation, and encoding are performed to obtain occupancy indication information corresponding to multiple scales.

[0126] In some embodiments, the encoding method provided by the embodiments of the present application also includes: performing at least S voxel downsampling based on the second-scale point cloud to determine the third-scale point cloud to the S+2-th scale point cloud; S is an integer greater than or equal to 1; performing voxel upsampling and division processing on the third-scale point cloud to the S+2-th scale point cloud, respectively, and performing occupancy probability prediction and encoding on the divided groups in turn to determine the occupancy indication information of the second-scale point cloud to the occupancy indication information of the S+1-th scale point cloud.

[0127] In the above embodiment, when the encoder performs voxel downsampling on the first-scale point cloud two or more times, the two point clouds before and after each voxel downsampling can be used as the high-scale point cloud and the low-scale point cloud, respectively, and divided, processed, and entropy encoded in a similar manner to obtain occupancy indication information of the second-scale point cloud to the occupancy indication information of the S+1-th scale point cloud; the encoder writes the occupancy indication information of the first-scale point cloud and the occupancy indication information of the second-scale point cloud to the occupancy indication information of the S+1-th scale point cloud into the bitstream and sends it to the decoder.

[0128] It is understandable that the encoding method provided in the embodiment of the present application can be repeatedly applied between multiple adjacent scales, and the encoding between each group of adjacent scales is independent of each other, so scale-scalable encoding can be flexibly implemented.

[0129] The following describes a decoding method applied to a decoder provided in an embodiment of the present application.

[0130] Referring to Figure 12, Figure 12 is an optional flowchart of a decoding method provided in an embodiment of the present application, which will be described in conjunction with the steps shown in Figure 12. Since the decoding method provided in an embodiment of the present application is applied to the process of decoding the geometric bitstream in Figure 2, the first-scale point cloud, the second-scale point cloud, etc. in the following embodiments specifically refer to the geometric information of the first-scale point cloud, the geometric information of the second-scale point cloud, etc.

[0131] S401: parse the code stream to determine occupancy indication information corresponding to the first-scale point cloud.

[0132] In the embodiment of the present application, the decoder parses the received code stream to obtain first encoded data corresponding to the first-scale point cloud, that is, occupancy indication information corresponding to the first-scale point cloud.

[0133] S402: Determine a second-scale point cloud, and perform upsampling and partitioning processing on the second-scale point cloud to determine m groups of upsampled voxels; wherein the second-scale point cloud is the previously decoded point cloud data corresponding to the first-scale point cloud; and m is an integer greater than or equal to 1.

[0134] In this embodiment of the present application, the bitstream contains occupancy indication information for multiple scale point clouds. The decoder decodes the data in descending order. When decoding the occupancy indication information corresponding to a first-scale point cloud, the decoder determines that the previously decoded point cloud data before the first-scale point cloud is a second-scale point cloud. The decoder then uses the second-scale point cloud as the decoded low-scale point cloud geometry data for decoding, prediction, and reconstruction of the first-scale point cloud.

[0135] In some embodiments, S402 may be implemented by executing S4021-S4022 as follows:

[0136] S4021. Perform voxel upsampling on the second-scale point cloud to determine n upsampled voxels of the first scale; n is an integer greater than 1.

[0137] S4022 : Number each of the n up-sampled voxels, and group the up-sampled voxels based on the number corresponding to each up-sampled voxel to determine m groups of up-sampled voxels.

[0138] Here, the processes of S4021 and S4022 are consistent with the process descriptions of S1021 and S1022 in the encoding method, and will not be repeated here.

[0139] In some embodiments, S4022 may be implemented by executing S501-S502 as follows:

[0140] S501 : Number each up-sampled voxel in n up-sampled voxels, and determine a number corresponding to each up-sampled voxel.

[0141] S502 : Group the n up-sampled voxels according to the number of each up-sampled voxel to determine m groups of up-sampled voxels.

[0142] In some embodiments, the decoder may group upsampled voxels with the same number among the n upsampled voxels based on the number of each upsampled voxel, thereby determining m groups of upsampled voxels. Alternatively, at least two consecutive numbers may be determined; and based on the number of each upsampled voxel, the upsampled voxels with the at least two consecutive numbers corresponding to the n upsampled voxels may be grouped together to determine m groups of upsampled voxels. The specific selection is based on actual circumstances and is not limited in this embodiment of the present application.

[0143] Here, the processes of S501 and S502 are consistent with the description of the processes of S201 and S202 in the encoding method, and are not repeated here.

[0144] It can be seen that in the embodiment of the present application, the process of upsampling and dividing processing in the decoder is the same as the process of upsampling and dividing processing in the encoder.

[0145] S403 , performing occupation probability prediction on the m groups of upsampled voxels in sequence to determine the m groups of occupancy probabilities, and decoding the occupancy indication information according to the m groups of occupancy probabilities to determine the occupancy symbols of the m groups.

[0146] In this embodiment of the present application, the decoder performs occupancy probability prediction on the obtained m groups of upsampled voxels in a progressive manner, group by group, and determine the m groups of occupancy probabilities. When predicting the occupancy probability of the current group of upsampled voxels, the decoder uses the prediction results of one or more previously predicted groups of upsampled voxels (such as the occupancy symbols corresponding to the occupancy probabilities) to predict the occupancy probability of the current group of upsampled voxels and determine the occupancy probability corresponding to the current group of upsampled voxels. This continues until m groups of occupancy probabilities are determined, and then the occupancy symbols of each of the m groups are determined based on the m groups of occupancy probabilities.

[0147] In some embodiments, based on FIG. 12 , as shown in FIG. 13 , S403 may be implemented by executing S4031 - S4033 as follows:

[0148] S4031 . Predict occupancy probabilities for the first group of upsampled voxels in the m groups of upsampled voxels to determine a first group of occupancy probabilities, and decode occupancy indication information according to the first group of occupancy probabilities to determine a first group of occupancy symbols.

[0149] In step S4031, occupancy probabilities are predicted for the first group of upsampled voxels. The process for determining the first group of occupancy probabilities is consistent with the corresponding process described in step S1031 of the encoding method and is not further described here. The decoder decodes the occupancy indication information based on the first group of occupancy probabilities to determine the first group of occupancy symbols.

[0150] S4032. When i is greater than 1 and less than or equal to m, use at least k groups of decoded occupancy symbols to predict the occupancy probability of the i-th group of upsampled voxels to determine the i-th group occupancy probability, and decode the i-th group of occupancy indication information in the occupancy indication information based on the i-th group occupancy probability to determine the i-th group of occupancy symbols; where i is an integer and k is an integer greater than or equal to 1 and less than i.

[0151] In S4032, when i is greater than 1 and less than or equal to m, that is, for the second to m-th groups of upsampled voxels in the m groups of upsampled voxels, when the decoder predicts the occupancy probability of the i-th group of upsampled voxels in the second to m-th groups of upsampled voxels, it uses at least k groups of occupancy symbols corresponding to at least k groups of upsampled voxels that have been decoded to predict the occupancy probability of the i-th group of upsampled voxels and determine the occupancy probability of the i-th group.

[0152] Here, the decoder uses at least k groups of occupancy symbols corresponding to at least k groups of decoded upsampled voxels to predict the occupancy probability of the i-th group of upsampled voxels. The process of determining the occupancy probability of the i-th group is consistent with the description of the corresponding probability prediction process in S1032 and will not be repeated here.

[0153] In some embodiments, the encoder may perform feature extraction on the i-th group of upsampled voxels to determine the features of the i-th group of voxels; determine that the decoded ik-th group of occupancy symbols to the i-1-th group of occupancy symbols are at least k groups of occupancy symbols; based on at least k groups of occupancy symbols and in combination with the features of the i-th group of voxels, perform occupancy probability prediction on each upsampled voxel in the i-th group of upsampled voxels, and determine that the predicted occupancy probability corresponding to each upsampled voxel in the i-th group of upsampled voxels is the i-th group of occupancy probability.

[0154] In some embodiments, the decoder may perform entropy decoding on each occupancy indication information in the i-th group of occupancy indication information based on the i-th group of occupancy probabilities to determine an occupancy symbol corresponding to each occupancy indication information in the i-th group of occupancy indication information, thereby determining the i-th group of occupancy symbols. The occupancy symbols include a first occupancy symbol and a second occupancy symbol. The first occupancy symbol indicates that the corresponding voxel is occupied, while the second occupancy symbol indicates that the corresponding voxel is unoccupied. For example, the first occupancy symbol may be 1, and the second occupancy symbol may be 0. The specific selection depends on actual circumstances and is not limited in this embodiment of the present application.

[0155] S4033. Continue decoding the next group based on the i-th group of occupied symbols until the occupied symbols of each of the m groups are determined.

[0156] In this embodiment of the present application, the decoder may continue decoding the next group based on the i-th group of occupancy symbols in the same manner as described above, until the occupancy symbols of each of the m groups are determined. The decoder may use at least k groups of occupancy symbols including the i-th group of occupancy symbols to predict the occupancy probability of the i+1-th group of upsampled voxels, and decode the i+1-th group of occupancy indication information in the occupancy indication information based on the i+1-th group of occupancy probabilities to determine the i+1-th group of occupancy symbols, until the decoding of the occupancy indication information is completed and the occupancy symbols of each of the m groups are determined.

[0157] S404 : Determine reconstructed geometric data corresponding to the first-scale point cloud based on the m groups of occupancy symbols.

[0158] In an embodiment of the present application, when the decoder completes the occupancy probability prediction and decoding of m groups of upsampled voxels and obtains m groups of occupancy symbols, it can determine the reconstructed geometric data corresponding to the first-scale point cloud based on the m groups of occupancy symbols and reconstruct the geometric information of the first-scale point cloud.

[0159] It is understood that in the embodiments of the present application, the previously decoded second-scale point cloud corresponding to the first-scale point cloud is upsampled and divided, the second-scale point cloud is upsampled to the first scale, and the upsampled voxels are divided into m groups. Occupancy probability prediction is performed sequentially for each group to obtain m groups of occupancy probabilities. Grouped prediction speeds up the prediction of occupancy probabilities and improves the processing efficiency of the prediction process. In this way, the occupancy indication information corresponding to the first-scale point cloud is decoded using the m groups of occupancy probabilities, obtaining m groups of decoded occupancy symbols, reconstructing the geometric data of the first-scale point cloud, and completing the decoding of the first-scale point cloud, thereby improving decoding efficiency and performance.

[0160] Furthermore, the group prediction and encoding method in the embodiment of the present application, compared with the method of predicting and decoding the occupancy probability of each voxel based on the parent node and neighbor nodes of each voxel, on the one hand greatly reduces the decoding complexity and improves the decoding speed; on the other hand, as the decoding proceeds, the occupancy symbols of all previously decoded upsampled voxel groups can be used as context information and input into the convolutional network to help predict the occupancy probability of the current upsampled voxel group, thereby improving the accuracy of the occupancy prediction probability prediction, and then improving the decoding accuracy, that is, improving the decoding performance.

[0161] In some embodiments, before determining the second-scale point cloud, the embodiment of the present application may further perform the process of S601-S605 to decode and obtain the second-scale point cloud as follows:

[0162] S601 , parsing a bitstream to determine a T-th scale point cloud and T-1th occupancy indication information corresponding to a T-1th scale point cloud.

[0163] In S601, the decoder can obtain the occupancy indication information corresponding to the T-th scale point cloud, i.e., the T-th occupancy indication information, by parsing the bitstream; and determine the previous decoded low-scale point cloud of the T-1-th scale point cloud, i.e., the T-th scale point cloud, so as to use the T-th scale point cloud to decode the T-1-th occupancy indication information. Wherein, j = T, T-1, ..., 3; T is an integer greater than or equal to 3;

[0164] For example, the decoder can parse the bitstream to obtain occupancy indication information corresponding to the second-scale point cloud, i.e., the second occupancy indication information; and determine the decoded third-scale point cloud prior to the second-scale point cloud. The decoded third-scale point cloud can then be used to decode the second occupancy indication information corresponding to the second-scale point cloud. Alternatively, the decoder can also start decoding from the third occupancy indication information corresponding to the third-scale point cloud until the second-scale point cloud is decoded: the third occupancy indication information is obtained from the bitstream, the decoded fourth-scale point cloud is determined, and the third occupancy indication information is first decoded using the fourth-scale point cloud to obtain the decoded third-scale point cloud; and the third-scale point cloud is then used to decode the second occupancy indication information parsed from the bitstream to obtain the second-scale point cloud.

[0165] Here, for each decoding process in the above process, starting from j=T, the following processing can be performed on each value of j in descending order:

[0166] S602 : Perform voxel upsampling and segmentation processing based on the j-th scale point cloud, and perform occupancy probability prediction on the divided groups in sequence to determine m groups of occupancy probabilities corresponding to the m groups in the j-1-th scale point cloud.

[0167] In S602, the decoder performs voxel upsampling and division processing based on the decoded j-th scale point cloud through a process similar to that in S402-S403 above, and predicts the occupancy probability of the divided groups in turn to determine the m groups of occupancy probabilities corresponding to the m groups in the j-1-th scale point cloud.

[0168] S603 : Perform entropy decoding on the j-1 th occupancy indication information according to the m groups of occupancy probabilities corresponding to the m groups in the j-1 th scale point cloud, and determine the m groups of occupancy symbols corresponding to the j-1 th occupancy indication information.

[0169] S604 : Reconstruct the j-1th scale point cloud based on the m groups of occupancy symbols corresponding to the j-1th occupancy indication information.

[0170] In S603-S604, the decoder performs entropy decoding on the j-1th occupancy indication information based on the predicted m groups of occupancy probabilities corresponding to the m groups in the j-1th scale point cloud, and determines the m groups of occupancy symbols corresponding to the j-1th occupancy indication information. The decoder then performs geometric data reconstruction based on the m groups of occupancy symbols corresponding to the j-1th scale point cloud, thereby reconstructing the j-1th scale point cloud.

[0171] S605 , continue the next processing according to the next value of j until j=3, and reconstruct the second-scale point cloud.

[0172] In S605, the decoder processes the next value of j in descending order using the same upsampling and division, grouping, and occupancy probability prediction and decoding process, and reconstructs the next scale (higher scale) point cloud until j = 3, at which point cloud of the second scale is reconstructed.

[0173] In this way, when the decoder decodes adjacent scales, it can use the known geometric data of the previously decoded low-scale point cloud to decode the occupancy indication information of the high-scale point cloud to be decoded, and reconstruct the high-scale point cloud until the second-scale point cloud is reconstructed.

[0174] It should be noted that the decoding method in the embodiments of the present application can be applied to scalable encoding and decoding methods. That is, for multiple occupancy indications for multiple scale point clouds sent by the encoder, the decoder can decode and reconstruct the point cloud to any scale in a sequence from low to high scales, based on the actual decoding accuracy requirements. For example, the encoder writes and transmits occupancy indications for a first-scale point cloud, a second-scale point cloud, and an S+1-scale point cloud in the bitstream. The decoder can then, based on preset accuracy requirements and using the decoding method provided in the embodiments of the present application, decode from the S+1-scale point cloud to a third-scale point cloud, reconstruct the geometric data of the third-scale point cloud, and terminate decoding without further decoding the second-scale occupancy indications and the occupancy indications corresponding to the first-scale point cloud. The specific selection depends on the actual situation and is not limited by the embodiments of the present application.

[0175] It is understandable that the decoding method provided in the embodiment of the present application can be repeatedly applied between multiple adjacent scales, and the decoding between each group of adjacent scales is independent of each other, so scale-scalable decoding can be flexibly implemented.

[0176] It should be noted that each decoding process of the above decoder uses the decoded low-scale point cloud as known information to decode the occupancy indication information of the high-scale point cloud. For the first decoding process of the decoder, its known information can be a preset number of unencoded point cloud information sent by the encoder side. The encoder can use the preset number of point cloud information, such as the coordinates of 100 points in the point cloud, as the first known information and send it directly to the decoding end in an unencoded manner, so that the decoder does not need to decode the first known information and can directly use the position information of the preset number of points sent by the encoder to reconstruct the point cloud of the corresponding scale to continue the subsequent decoding process.

[0177] Below, in conjunction with Figure 14, the application of the encoding and decoding method provided in the embodiment of the present application in a practical scenario is explained.

[0178] As shown in Figure 14, the point cloud image A is first voxelized to obtain a scale p+1 point cloud. During the encoder's downsampling process, the scale p+1 point cloud is voxel downsampled to obtain a scale p point cloud. During the encoder's upsampling process, the scale p point cloud is upsampled through the CNN network, and the n upsampled voxels of scale p+1 obtained by upsampling are numbered and grouped. For example, they are divided into 8 groups. From group 1 to group 8, the CNN network predicts the occupancy probability (occupancy probability) of each group of upsampled voxels. Here, the occupancy probability prediction process can be referred to Figure 10.

[0179] For example, in the first prediction phase, the first group of upsampled voxels among the eight groups of upsampled voxels undergoes CNN processing, outputting the occupancy probabilities of the first group of upsampled voxels. Based on the occupancy probabilities of the first group of upsampled voxels, the first group of occupancy symbols corresponding to the first group of upsampled voxels is then determined. As encoding progresses, in the second prediction phase through the eighth prediction phase, the input data for each prediction phase includes, in addition to the current group of upsampled voxels, the occupancy symbols of all other groups of upsampled voxels obtained in the previous prediction phase. These occupancy symbols serve as contextual information for the current group of upsampled voxels and are collectively input into the CNN to help predict the occupancy probabilities of the current group of upsampled voxels and convert these occupancy probabilities into occupancy symbols. In this way, the encoder obtains occupancy symbols for each group of upsampled voxels based on the predicted occupancy probabilities. Arithmetic coding is performed on the occupancy symbols of each group of upsampled voxels, resulting in scale p+1 encoded data for the scale p+1 point cloud, which is then written into the bitstream. The scale p+1 encoded data includes eight groups of codes corresponding to the eight groups of occupancy symbols. The encoder performs the next downsampling based on the scale p point cloud to obtain the scale p-1 point cloud. Using the same process as above, the encoder performs prediction and encoding based on the scale p-1 point cloud, obtaining the scale p encoded data of the scale p point cloud and writing it into the bitstream. Similarly, the encoder obtains a bitstream containing the scale p-1 encoded data of the scale p-1 point cloud, the scale p encoded data, and the scale p+1 encoded data, and sends the bitstream to the decoder.

[0180] In the decoder, according to the decoding method provided in the embodiment of the present application, decoding is performed from low scale to high scale. Exemplarily, the scale p-1 coded data is decoded to reconstruct the scale p-1 point cloud. In the upsampling process of the decoder, the scale p-1 point cloud is upsampled to obtain n upsampled voxels of scale p. The n upsampled voxels of scale p are numbered and grouped in the same manner, and the occupancy probability prediction is performed on the groups in sequence to obtain the occupancy probability corresponding to each group of upsampled voxels of scale p. The occupancy probability corresponding to each group of upsampled voxels of scale p is used to perform arithmetic decoding on the scale p coded data obtained by parsing the bitstream to obtain the occupancy symbol corresponding to each upsampled voxel of the n upsampled voxels of scale p, thereby reconstructing the geometric data of the scale p point cloud and completing the decoding of the scale p coded data. Thereafter, based on the decoded scale p point cloud, the scale p+1 coded data is decoded using the same process to reconstruct the geometric data of the scale p+1 point cloud.

[0181] It can be understood that the embodiment of the present application has good compression performance and parallelism compared to the traditional G-PCC method, while being able to maintain a faster encoding and decoding speed and improve encoding and decoding efficiency. The applicant conducted a comparative test on the encoding and decoding method of the embodiment of the present application and the traditional G-PCC method, and the results are shown in Table 1, as follows:

[0182] Table 1

[0183] G-PCC1-stage3-stage8-stageBitrate (bpp)1.0291.0860.7030.633Gain -+5.5% -31.7% -38.5%Encoding time5.80.71.11.9Decoding time3.30.71.01.8

[0184] In Table 1, 1-stage indicates that a single-group grouping scheme is used in the grouping process, and occupancy probability prediction and encoding and decoding are performed on one group of upsampled voxels; 3-stage indicates that a three-group grouping scheme is used in the grouping process, and occupancy probability prediction and encoding and decoding are performed sequentially on the three groups of upsampled voxels; and 8-stage indicates that an eight-group grouping scheme is used in the grouping process, and occupancy probability prediction and encoding and decoding are performed sequentially on the eight groups of upsampled voxels. It can be seen that compared to the traditional G-PCC method, the encoding and decoding times of the embodiments of the present application under the three grouping prediction schemes are significantly shorter than those of the traditional G-PCC method. Furthermore, the bitrate (bits per pixel, bpp) under the 3-stage and 8-stage methods are also lower than that of the traditional G-PCC method, resulting in gains of -31.7% and -38.5% under the 3-stage and 8-stage methods, respectively. Here, the gain value is negatively correlated with performance. A smaller gain, such as a smaller negative value, indicates better compression performance. This data demonstrates improved encoding and decoding performance and saves transmission bandwidth.

[0185] The embodiment of the present application provides a decoder 1, as shown in FIG15 , comprising:

[0186] The parsing part 11 is configured to parse the code stream and determine the occupancy indication information corresponding to the first scale point cloud;

[0187] The first grouping portion 12 is configured to determine a second-scale point cloud, and perform upsampling and segmentation processing on the second-scale point cloud to determine m groups of upsampled voxels; wherein the second-scale point cloud is the previously decoded point cloud data corresponding to the first-scale point cloud;

[0188] The first prediction part 13 is configured to perform occupancy probability prediction on the m groups of upsampled voxels in sequence to determine the m groups of occupancy probabilities, and decode the occupancy indication information according to the m groups of occupancy probabilities to determine occupancy symbols for each of the m groups; m is an integer greater than or equal to 1;

[0189] The decoding part 14 is configured to determine the reconstructed geometric data corresponding to the first-scale point cloud based on the m groups of occupancy symbols.

[0190] In some embodiments, the first grouping part 12 is further configured to perform voxel upsampling on the second-scale point cloud to determine n upsampled voxels of the first scale; n is an integer greater than 1; each of the n upsampled voxels is numbered, and grouped based on the number corresponding to each upsampled voxel to determine the m groups of upsampled voxels.

[0191] In some embodiments, the first prediction part 13 is further configured to perform occupancy probability prediction on the first group of upsampled voxels in the m groups of upsampled voxels to determine the first group of occupancy probabilities, and decode the occupancy indication information based on the first group of occupancy probabilities to determine the first group of occupancy symbols; when i is greater than 1 and less than or equal to m, use at least k groups of occupancy symbols that have been decoded to predict the occupancy probability of the i-th group of upsampled voxels to determine the i-th group of occupancy probabilities, and decode the i-th group of occupancy indication information in the occupancy indication information based on the i-th group of occupancy probabilities to determine the i-th group of occupancy symbols; wherein i is an integer and k is an integer greater than or equal to 1 and less than i; and continue decoding the next group based on the i-th group of occupancy symbols until the occupancy symbols of each of the m groups are determined.

[0192] In some embodiments, the first grouping part 12 is further configured to number each of the n upsampled voxels to determine the number corresponding to each upsampled voxel; and group the n upsampled voxels according to the number of each upsampled voxel to determine the m groups of upsampled voxels.

[0193] In some embodiments, the first grouping portion 12 is further configured to group upsampled voxels with the same number among the n upsampled voxels according to the number of each upsampled voxel, to determine the m groups of upsampled voxels.

[0194] In some embodiments, the first grouping part 12 is further configured to determine at least two consecutive numbers; according to the number of each upsampled voxel, the upsampled voxels corresponding to the at least two consecutive numbers in the n upsampled voxels are grouped as a group to determine the m groups of upsampled voxels.

[0195] In some embodiments, the first prediction part 13 is further configured to perform feature extraction on the i-th group of upsampled voxels to determine the i-th group of voxel features; determine that the decoded ik-th group of occupancy symbols to the i-1-th group of occupancy symbols are the at least k groups of occupancy symbols; based on the at least k groups of occupancy symbols and in combination with the i-th group of voxel features, perform occupancy probability prediction on each upsampled voxel in the i-th group of upsampled voxels, and determine that the predicted occupancy probability corresponding to each upsampled voxel in the i-th group of upsampled voxels is the i-th group of occupancy probability.

[0196] In some embodiments, the parsing part 11, the first grouping part 12, the first prediction part 13 and the decoding part 14 are further configured to parse the code stream before determining the second scale point cloud, determine the T-th scale point cloud, and the T-1th occupancy indication information corresponding to the T-1th scale point cloud; wherein j = T, T-1, ..., 3; T is an integer greater than or equal to 3; starting from j = T, in descending order, perform the following processing on each value of j: perform voxel upsampling and division processing based on the j-th scale point cloud, and perform the division on the divided point cloud. The j-1-th scale point cloud is reconstructed based on the m groups of occupancy symbols corresponding to the m groups in the j-1-th scale point cloud. The next processing is continued according to the next value of j until j=3, and the second-scale point cloud is reconstructed.

[0197] In some embodiments, the decoding portion 14 is further configured to perform entropy decoding on each occupancy indication information in the i-th group of occupancy indication information based on the i-th group of occupancy probability, determine an occupancy symbol corresponding to each occupancy indication information in the i-th group of occupancy indication information, and thereby determine the i-th group of occupancy symbols; wherein the occupancy symbols include: a first occupancy symbol and a second occupancy symbol; the first occupancy symbol indicates that the corresponding voxel is occupied; and the second occupancy symbol indicates that the corresponding voxel is unoccupied.

[0198] In some embodiments, the decoding portion 14 is further configured to continue decoding the next group based on the i-th group of occupancy symbols until the occupancy symbols of each of the m groups are determined, including: using at least k groups of occupancy symbols including the i-th group of occupancy symbols to predict the occupancy probability of the i+1-th group of upsampled voxels, and decoding the i+1-th group of occupancy indication information in the occupancy indication information according to the i+1-th group occupancy probability to determine the i+1-th group of occupancy symbols, until the decoding of the occupancy indication information is completed and the occupancy symbols of each of the m groups are determined.

[0199] In some embodiments, the present application provides an encoder 2, as shown in FIG16 , including:

[0200] A downsampling part 21 is configured to perform voxel downsampling on the first-scale point cloud to determine a second-scale point cloud;

[0201] The second grouping part 22 is configured to perform upsampling and division processing on the second-scale point cloud to determine m groups of upsampled voxels; m is an integer greater than or equal to 1;

[0202] The second prediction part 23 is configured to perform occupation probability prediction on the m groups of upsampled voxels in sequence, determine the m groups of occupancy probabilities, and determine the occupancy symbols of the m groups according to the m groups of occupancy probabilities;

[0203] The encoding part 24 is configured to encode the occupancy symbols of the m groups, determine the occupancy indication information corresponding to the first-scale point cloud, and write the occupancy indication information into the code stream.

[0204] In some embodiments, the second grouping portion 22 is further configured to perform voxel upsampling on the second-scale point cloud to determine n upsampled voxels of the first scale; n is an integer greater than 1; each of the n upsampled voxels is numbered, and grouped based on the number corresponding to each upsampled voxel to determine m groups of upsampled voxels.

[0205] In some embodiments, the second prediction portion 23 is further configured to perform occupancy probability prediction on the first group of upsampled voxels in the m groups of upsampled voxels to determine a first group of occupancy probabilities, and use a preset occupancy symbol to represent the first group of occupancy probabilities and determine a first group of occupancy symbols; when i is greater than 1 and less than or equal to m, perform occupancy probability prediction on the i-th group of upsampled voxels using at least k groups of occupancy symbols corresponding to at least k groups of upsampled voxels for which probability predictions have been completed and represented by occupancy symbols to determine the i-th group of occupancy probabilities; wherein i is an integer and k is an integer greater than or equal to 1 and less than i; use the preset occupancy symbol to represent the i-th group of occupancy probabilities and determine the i-th group of occupancy symbols; and continue to perform occupancy probability prediction and occupancy symbol representation on the i+1-th group of upsampled voxels using at least k groups of occupancy symbols including the i-th group of occupancy symbols until the occupancy symbols of each of the m groups are determined.

[0206] In some embodiments, the second grouping portion 22 is further configured to number each of the n upsampled voxels and determine a number corresponding to each upsampled voxel;

[0207] The n up-sampled voxels are grouped according to the number of each up-sampled voxel to determine the m groups of up-sampled voxels.

[0208] In some embodiments, the second grouping portion 22 is further configured to group the upsampled voxels with the same number among the n upsampled voxels according to the number of each upsampled voxel, to determine the m groups of upsampled voxels.

[0209] In some embodiments, the second grouping part 22 is further configured to determine at least two consecutive numbers; according to the number of each upsampled voxel, the upsampled voxels corresponding to the at least two consecutive numbers in the n upsampled voxels are grouped as a group to determine the m groups of upsampled voxels.

[0210] In some embodiments, the second prediction part 23 is further configured to perform feature extraction on the i-th group of upsampled voxels to determine the i-th group of voxel features; determine the ik-th group of occupancy symbols to the i-1-th group of occupancy symbols for which probability prediction and occupancy symbol representation have been completed as the at least k groups of occupancy symbols; based on the at least k groups of occupancy symbols and in combination with the i-th group of voxel features, perform occupancy probability prediction on each upsampled voxel in the i-th group of upsampled voxels, and determine the occupancy probability corresponding to each upsampled voxel in the i-th group of upsampled voxels as the i-th group of occupancy probability.

[0211] In some embodiments, the second prediction part 23 is further configured to use the first occupancy symbol in the preset occupancy symbol to represent the occupancy probability in the i-th group of occupancy probabilities that is greater than or equal to the preset probability threshold; and use the second occupancy symbol in the preset occupancy symbol to represent the occupancy probability in the i-th group of occupancy probabilities that is less than the preset probability threshold, to determine the i-th group of occupancy symbols.

[0212] In some embodiments, the encoding part 24 is further configured to perform entropy encoding on each of the m groups of occupancy symbols, and determine that the m groups of occupancy indication information obtained by encoding are the occupancy indication information corresponding to the first-scale point cloud.

[0213] In some embodiments, the downsampling part 21, the second grouping part 22, the second prediction part 23, and the encoding part 24 are further configured to perform at least S voxel downsampling based on the second-scale point cloud to determine a third-scale point cloud to an S+2-th scale point cloud; S is an integer greater than or equal to 1; perform voxel upsampling and grouping processing on the third-scale point cloud to the S+2-th scale point cloud, respectively, and perform occupancy probability prediction and encoding on the divided groups in sequence to determine occupancy indication information of the second-scale point cloud to the S+1-th scale point cloud; and write the occupancy indication information of the first-scale point cloud and the occupancy indication information of the second-scale point cloud to the S+1-th scale point cloud into the bitstream.

[0214] It should be noted that the description of the above device embodiment is similar to the description of the above method embodiment and has similar beneficial effects as the method embodiment. For technical details not disclosed in the device embodiment of this application, please refer to the description of the method embodiment of this application for understanding.

[0215] In some embodiments, the present application also provides a decoder. Figure 17 is a schematic diagram of an optional structure of a decoder 3 provided in the present application. As shown in Figure 17, the decoder 3 includes a first memory 32 and a first processor 33. The first memory 32 and the first processor 33 are connected via a first communication bus 34. The first memory 32 is used to store executable instructions. The first processor 33 is used to implement the decoding method provided in the present application when executing the executable instructions stored in the first memory 32.

[0216] In some embodiments, the present application also provides an encoder. Figure 17 is a schematic diagram of an optional structure of an encoder 4 provided in the present application. As shown in Figure 18, encoder 4 includes a second memory 42 and a second processor 43. The second memory 42 and the second processor 43 are connected via a second communication bus 44. The second memory 42 is used to store executable instructions. The second processor 43 is used to implement the encoding method provided in the present application when executing the executable instructions stored in the second memory 42.

[0217] An embodiment of the present application provides a computer-readable storage medium storing executable instructions, wherein the executable instructions are stored. When the executable instructions are executed by a first processor, the first processor will be caused to execute any one of the decoding methods provided in the embodiments of the present application; or, when the executable instructions are executed by a second processor, the second processor will be caused to execute any one of the encoding methods provided in the embodiments of the present application.

[0218] In some embodiments, the computer-readable storage medium may be a memory such as FRAM, ROM, PROM, EPROM, EEPROM, flash memory, magnetic surface storage, optical disk, or CD-ROM; or various devices including one or any combination of the above memories.

[0219] In some embodiments, executable instructions may be in the form of a program, software, software module, script, or code, written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and may be deployed in any form, including as a stand-alone program or as a module, component, subroutine, or other unit suitable for use in a computing environment.

[0220] As an example, executable instructions may, but need not, correspond to a file in a file system, may be stored as part of a file that stores other programs or data, such as in one or more scripts in a HyperText Markup Language (HTML) document, in a single file dedicated to the program in question, or in multiple coordinating files (e.g., files storing one or more modules, subroutines, or code portions).

[0221] By way of example, executable instructions may be deployed to be executed on one computing device, or on multiple computing devices at one site, or on multiple computing devices distributed across multiple sites and interconnected by a communication network.

[0222] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of hardware embodiments, software embodiments, or embodiments combining software and hardware. Furthermore, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage and optical storage, etc.) containing computer-usable program code.

[0223] The present application is described with reference to the flow chart and / or block diagram of the method, device (system), and computer program product according to the embodiment of the present application. It should be understood that each flow process and / or box in the flow chart and / or block diagram and the combination of the flow process and / or box in the flow chart and / or block diagram can be realized by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processing machine or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device produce a device for realizing the function specified in one flow chart flow or multiple flows and / or one box or multiple boxes of the block diagram.

[0224] These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce a product including an instruction device that implements the functions specified in one or more processes in the flowchart and / or one or more boxes in the block diagram.

[0225] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, so that the instructions executed on the computer or other programmable device provide steps for implementing the functions specified in one or more processes in the flowchart and / or one or more boxes in the block diagram.

[0226] The above description is only a preferred embodiment of the present application and is not intended to limit the scope of protection of the present application. Any modifications, equivalent replacements, and improvements made within the spirit and scope of the present application are included in the scope of protection of the present application. Industrial Applicability

[0227] In the embodiment of the present application, the second-scale point cloud is upsampled and divided, upsampled to the first scale, and the upsampled voxels are divided into m groups. Occupancy probability prediction is performed sequentially on each group to obtain m groups of occupancy probabilities. In this way, group prediction accelerates the prediction speed of occupancy probabilities, improving the processing efficiency and coding efficiency of the prediction process. Furthermore, the encoder uses m groups of occupancy symbols corresponding to the m groups of occupancy probabilities to encode the occupancy indication information corresponding to the first-scale point cloud and write it into the bitstream; alternatively, the decoder uses the m groups of occupancy probabilities to decode the occupancy indication information corresponding to the first-scale point cloud, thereby improving encoding and decoding efficiency and performance.

[0228] Furthermore, the group prediction and encoding / decoding method in the embodiment of the present application, compared to the method of performing occupancy probability prediction and encoding / decoding on a voxel-by-voxel basis based on the parent node and neighbor nodes of each voxel, on the one hand greatly reduces the encoding / decoding complexity and improves compression performance and encoding / decoding speed; on the other hand, as encoding / decoding proceeds, the occupancy symbols of all previously upsampled voxel groups that have completed occupancy probability prediction can be used as context information and input into the convolutional network to help predict the occupancy probability of the current upsampled voxel group, thereby improving the accuracy of the occupancy prediction probability prediction, and further improving the encoding / decoding accuracy, that is, improving the encoding performance. Furthermore, the encoder can encode the occupancy symbols of each of the m groups in parallel, thereby further accelerating the encoding speed and improving the encoding efficiency and encoding performance.

[0229] Furthermore, the encoding and decoding method provided in the embodiment of the present application can be repeatedly applied between multiple adjacent scales, and the encoding and decoding between each group of adjacent scales are independent of each other, so scale-scalable encoding and decoding can be flexibly implemented.

Claims

1. A decoding method, comprising: Parse the code stream to determine occupancy indication information corresponding to the first-scale point cloud; Determine a second-scale point cloud, and perform upsampling and partitioning processing on the second-scale point cloud to determine m groups of upsampled voxels; wherein the second-scale point cloud is the previously decoded point cloud data corresponding to the first-scale point cloud; and m is an integer greater than or equal to 1; performing occupancy probability prediction on the m groups of upsampled voxels in sequence to determine the m groups of occupancy probabilities, and decoding the occupancy indication information according to the m groups of occupancy probabilities to determine occupancy symbols for the m groups; Based on the m groups of occupancy symbols, reconstructed geometric data corresponding to the first-scale point cloud is determined.

2. The method according to claim 1, wherein The upsampling and dividing the second-scale point cloud to determine m groups of upsampled voxels includes: Perform voxel upsampling on the second-scale point cloud to determine n upsampled voxels of the first scale; n is an integer greater than 1; Each up-sampled voxel in the n up-sampled voxels is numbered, and grouped based on the number corresponding to each up-sampled voxel to determine the m groups of up-sampled voxels.

3. The method according to claim 1 or 2, wherein: The predicting occupancy probabilities of the m groups of upsampled voxels in sequence to determine the occupancy probabilities of the m groups, and decoding the occupancy indication information according to the m groups of occupancy probabilities to determine occupancy symbols of the m groups includes: performing occupancy probability prediction on a first group of upsampled voxels in the m groups of upsampled voxels to determine a first group of occupancy probabilities, and decoding the occupancy indication information according to the first group of occupancy probabilities to determine a first group of occupancy symbols; When i is greater than 1 and less than or equal to m, using at least k groups of decoded occupancy symbols, predicting occupancy probabilities for the i-th group of upsampled voxels to determine the i-th group occupancy probability, and decoding the i-th group of occupancy indication information in the occupancy indication information based on the i-th group occupancy probability to determine the i-th group of occupancy symbols; wherein i is an integer and k is an integer greater than or equal to 1 and less than i; The next group of decoding is continued based on the i-th group of occupied symbols until the occupied symbols of each of the m groups are determined.

4. The method according to claim 2, wherein: The step of numbering each of the n up-sampled voxels and grouping the up-sampled voxels based on the number corresponding to each up-sampled voxel to determine the m groups of up-sampled voxels includes: Numbering each of the n up-sampled voxels to determine a number corresponding to each up-sampled voxel; The n up-sampled voxels are grouped according to the number of each up-sampled voxel to determine the m groups of up-sampled voxels.

5. The method according to claim 4, wherein The step of grouping the n up-sampled voxels according to the number of each up-sampled voxel to determine the m groups of up-sampled voxels includes: Among the n up-sampled voxels, up-sampled voxels with the same number are divided into a group according to the number of each up-sampled voxel, and the m groups of up-sampled voxels are determined.

6. The method according to claim 4, wherein: The step of grouping the n up-sampled voxels according to the number of each up-sampled voxel to determine the m groups of up-sampled voxels includes: Identify at least two consecutive numbers; According to the number of each up-sampled voxel, the up-sampled voxels corresponding to the at least two consecutive numbers in the n up-sampled voxels are taken as a group to determine the m groups of up-sampled voxels.

7. The method according to claim 3, wherein: The method of predicting the occupancy probability of the i-th group of upsampled voxels using the decoded at least k groups of occupancy symbols to determine the i-th group of occupancy probability includes: Performing feature extraction on the i-th group of upsampled voxels to determine the i-th group of voxel features; Determining that the decoded ikth group of occupied symbols to the i-1th group of occupied symbols are the at least k groups of occupied symbols; Based on the at least k groups of occupancy symbols and in combination with the i-th group of voxel features, an occupancy probability prediction is performed for each upsampled voxel in the i-th group of upsampled voxels, and the predicted occupancy probability corresponding to each upsampled voxel in the i-th group of upsampled voxels is determined as the i-th group occupancy probability.

8. The method according to any one of claims 1, 2, 4-7, wherein: Before determining the second-scale point cloud, the method further includes: Parsing the code stream to determine the T-th scale point cloud and the T-1th occupancy indication information corresponding to the T-1th scale point cloud; wherein j=T, T-1, ..., 3; T is an integer greater than or equal to 3; Starting from j = T, perform the following processing on each value of j in descending order: Perform voxel upsampling and segmentation based on the j-th scale point cloud, and predict the occupancy probability of the divided groups in turn to determine the m groups of occupancy probabilities corresponding to the m groups in the j-1-th scale point cloud; performing entropy decoding on the j-1th occupancy indication information according to m groups of occupancy probabilities corresponding to m groups in the j-1th scale point cloud, to determine m groups of occupancy symbols corresponding to the j-1th occupancy indication information; Reconstructing the j-1th scale point cloud based on the m groups of occupancy symbols corresponding to the j-1th occupancy indication information; The next processing is continued according to the next value of j until j=3, and the second-scale point cloud is reconstructed.

9. The method according to claim 3, wherein: The decoding, according to the i-th group occupancy probability, the i-th group occupancy indication information in the occupancy indication information to determine the i-th group occupancy symbol includes: According to the i-th group occupancy probability, entropy decoding is performed on each occupancy indication information in the i-th group occupancy indication information to determine the occupancy symbol corresponding to each occupancy indication information in the i-th group occupancy indication information, thereby determining the i-th group occupancy symbol; wherein, The occupancy symbol includes: a first occupancy symbol and a second occupancy symbol; the first occupancy symbol indicates that the corresponding voxel is occupied; the second occupancy symbol indicates that the corresponding voxel is not occupied.

10. The method according to claim 3, wherein: The step of continuing decoding a next group based on the i-th group of occupied symbols until determining the occupied symbols of the m groups comprises: Using at least k groups of occupancy symbols including the i-th group of occupancy symbols, occupancy probabilities are predicted for the i+1-th group of upsampled voxels, and the i+1-th group of occupancy indication information in the occupancy indication information is decoded according to the i+1-th group of occupancy probabilities to determine the i+1-th group of occupancy symbols, until the occupancy indication information is decoded and the occupancy symbols of each of the m groups are determined.

11. A coding method comprising: Perform voxel downsampling on the first-scale point cloud to determine the second-scale point cloud; Performing upsampling and partitioning processing on the second-scale point cloud to determine m groups of upsampled voxels, where m is an integer greater than or equal to 1; performing occupancy probability prediction on the m groups of upsampled voxels in sequence to determine the m groups of occupancy probabilities, and determining occupancy symbols for the m groups according to the m groups of occupancy probabilities; Encode the occupancy symbols of the m groups, determine occupancy indication information corresponding to the first-scale point cloud, and write the occupancy indication information into a bitstream.

12. The method according to claim 11, wherein The upsampling and dividing the second-scale point cloud to determine m groups of upsampled voxels includes: Perform voxel upsampling on the second-scale point cloud to determine n upsampled voxels of the first scale; n is an integer greater than 1; Each of the n up-sampled voxels is numbered, and the up-sampled voxels are grouped based on the number corresponding to each up-sampled voxel to determine m groups of up-sampled voxels.

13. The method according to claim 11, wherein The predicting occupancy probabilities of the m groups of upsampled voxels in sequence to determine the occupancy probabilities of the m groups, and determining the occupancy symbols of the m groups according to the occupancy probabilities of the m groups, includes: Performing occupancy probability prediction on a first group of upsampled voxels in the m groups of upsampled voxels to determine a first group of occupancy probabilities, and using a preset occupancy symbol to represent the first group of occupancy probabilities to determine a first group of occupancy symbols; When i is greater than 1 and less than or equal to m, performing occupancy probability prediction for the i-th group of upsampled voxels using at least k groups of occupancy symbols corresponding to the at least k groups of upsampled voxels represented by the completed probability predictions and occupancy symbols to determine the i-th group occupancy probability; wherein i is an integer and k is an integer greater than or equal to 1 and less than i; Determine the i-th group occupancy symbol by using the preset occupancy symbol to represent the i-th group occupancy probability; Using at least k groups of occupancy symbols including the i-th group of occupancy symbols, continue to perform occupancy probability prediction and occupancy symbol representation on the (i+1)-th group of upsampled voxels until the occupancy symbols of the m groups are determined.

14. The method according to claim 12, wherein: The step of numbering each of the n up-sampled voxels and grouping the up-sampled voxels based on the number corresponding to each up-sampled voxel to determine m groups of up-sampled voxels includes: Numbering each of the n up-sampled voxels to determine a number corresponding to each up-sampled voxel; The n up-sampled voxels are grouped according to the number of each up-sampled voxel to determine the m groups of up-sampled voxels.

15. The method according to claim 14, wherein The step of grouping the n up-sampled voxels according to the number of each up-sampled voxel to determine the m groups of up-sampled voxels includes: Among the n up-sampled voxels, up-sampled voxels with the same number are divided into a group according to the number of each up-sampled voxel, and the m groups of up-sampled voxels are determined.

16. The method according to claim 14, wherein The step of grouping the n up-sampled voxels according to the number of each up-sampled voxel to determine the m groups of up-sampled voxels includes: Identify at least two consecutive numbers; According to the number of each up-sampled voxel, the up-sampled voxels corresponding to the at least two consecutive numbers in the n up-sampled voxels are taken as a group to determine the m groups of up-sampled voxels.

17. The method according to claim 13, wherein: The method of performing occupancy probability prediction on the i-th group of upsampled voxels using at least k groups of occupancy symbols corresponding to at least k groups of upsampled voxels represented by completed probability predictions and occupancy symbols to determine the i-th group occupancy probability includes: Performing feature extraction on the i-th group of upsampled voxels to determine the i-th group of voxel features; Determining the ikth group of occupied symbols to the i-1th group of occupied symbols for which probability prediction and occupied symbol representation have been completed as the at least k groups of occupied symbols; Based on the at least k groups of occupancy symbols and in combination with the i-th group of voxel features, an occupancy probability prediction is performed for each upsampled voxel in the i-th group of upsampled voxels, and the occupancy probability corresponding to each upsampled voxel in the i-th group of upsampled voxels is determined as the i-th group occupancy probability.

18. The method according to claim 13, wherein The using the preset occupancy symbol to represent the ith group occupancy probability and determining the ith group occupancy symbol includes: The i-th group of occupancy symbols is determined by using a first occupancy symbol in the preset occupancy symbol to represent an occupancy probability greater than or equal to a preset probability threshold in the i-th group of occupancy probabilities; and using a second occupancy symbol in the preset occupancy symbol to represent an occupancy probability less than the preset probability threshold in the i-th group of occupancy probabilities.

19. The method according to any one of claims 11 to 18, wherein: The encoding of the occupancy symbols of the m groups to determine the occupancy indication information corresponding to the first-scale point cloud includes: Entropy coding is performed on each of the m groups of occupancy symbols, and the m groups of occupancy indication information obtained by coding are determined to be the occupancy indication information corresponding to the first-scale point cloud.

20. The method according to any one of claims 11 to 18, wherein: The method further comprises: Perform at least S voxel downsampling based on the second-scale point cloud to determine a third-scale point cloud to an S+2-th scale point cloud, where S is an integer greater than or equal to 1; Performing voxel upsampling and segmentation processing on the third-scale point cloud to the S+2-th scale point cloud, and sequentially predicting and encoding occupancy probabilities on the divided groups, to determine occupancy indication information of the second-scale point cloud to the S+1-th scale point cloud; The occupancy indication information of the first-scale point cloud, the occupancy indication information of the second-scale point cloud, and the occupancy indication information of the S+1-th scale point cloud are written into a bitstream.

21. A decoder comprising: A parsing portion configured to parse the code stream and determine occupancy indication information corresponding to the first-scale point cloud; The first grouping portion is configured to determine a second-scale point cloud, and perform upsampling and partitioning processing on the second-scale point cloud to determine m groups of upsampled voxels; wherein the second-scale point cloud is the previously decoded point cloud data corresponding to the first-scale point cloud; and m is an integer greater than or equal to 1; a first prediction part configured to perform occupancy probability prediction on the m groups of upsampled voxels in sequence to determine the m groups of occupancy probabilities, and decode the occupancy indication information according to the m groups of occupancy probabilities to determine occupancy symbols for the m groups; The decoding part is configured to determine the reconstructed geometric data corresponding to the first-scale point cloud based on the m groups of occupancy symbols.

22. An encoder comprising: A downsampling portion is configured to perform voxel downsampling on the first-scale point cloud to determine a second-scale point cloud; The second grouping part is configured to perform upsampling and division processing on the second-scale point cloud to determine m groups of upsampled voxels, where m is an integer greater than or equal to 1; a second prediction part configured to perform occupancy probability prediction on the m groups of upsampled voxels in sequence, determine the m groups of occupancy probabilities, and determine the occupancy symbols of the m groups according to the m groups of occupancy probabilities; The encoding part is configured to encode the occupancy symbols of the m groups, determine the occupancy indication information corresponding to the first-scale point cloud, and write the occupancy indication information into a code stream.

23. A code stream comprising: The code stream is generated by bit encoding according to information to be encoded; wherein the information to be encoded includes at least: occupancy indication information corresponding to the first-scale point cloud.

24. A decoder comprising: a first memory configured to store executable instructions; The first processor is configured to implement the method according to any one of claims 1 to 10 when executing the executable instructions stored in the first memory.

25. An encoder comprising: a second memory configured to store executable instructions; The second processor is configured to implement the method according to any one of claims 11 to 20 when executing the executable instructions stored in the second memory.

26. A computer-readable storage medium storing executable instructions for causing a first processor to execute the method according to any one of claims 1 to 10, or for causing a second processor to execute the method according to any one of claims 11 to 20.

Images