A point cloud geometry coding method based on hybrid context entropy model

Abstract

The application relates to a point cloud geometry coding method based on a hybrid context entropy model, wherein a node with a depth of n-i in a tree structure point cloud is converted into a voxel block to obtain hybrid structure point cloud data; context information of the node in the tree structure point cloud and the voxel block in the hybrid structure point cloud is extracted; the context information of the voxel block is spliced with the context information of the node with the same size; the spliced result is mixed through a selection unit to obtain the hybrid context information of each voxel block; the probability of each voxel block is estimated according to the hybrid context information to obtain a prediction probability; and the voxel block is coded according to the prediction probability to obtain a coding result. The application eliminates a large amount of redundant information existing in the tree structure, and realizes fine modeling of the voxel context through bidirectional context information extraction and context information mixing, thereby providing more accurate prediction probability for voxel coding, improving the compression rate, and reducing data loss in the coding process.

Description

Technical Field

[0001] This invention relates to a point cloud geometric encoding method based on a hybrid context entropy model, belonging to the field of point cloud data compression. Background Technology

[0002] Point clouds are a form of 3D data representation, consisting of a large collection of discrete points in 3D space. Each point has geometric information (i.e., 3D position coordinates x, y, z) and attribute information (such as color, reflectance intensity, etc.). Point clouds have efficient 3D information representation capabilities and have been widely used in fields such as virtual / augmented / mixed reality (AR / VR / MR), mobile maps, historical artifact scanning, smart cities, robotics, and autonomous driving. However, point cloud data is extremely large, and storage and transmission costs are high, necessitating effective compression of point cloud data.

[0003] Based on whether information is lost during the encoding process, 3D point cloud encoding can be divided into lossy compression encoding and lossless compression encoding. While lossy compression encoding can compress data to a great extent, due to information loss, the decoder cannot reconstruct the original image. Lossy encoding is unsuitable when data integrity is required. Lossless compression encoding, on the other hand, does not lose information and can accurately reconstruct the original point cloud at the decoder. However, current lossless compression encoding methods generally suffer from insufficient compression ratios.

[0004] Patent CN113284203A, "Point Cloud Compression and Decompression Method Based on Octree Coding and Voxel Context", leverages the advantages of voxel and octree methods by introducing local voxel context into the deep entropy model to better compress octree structured data. However, it still suffers from problems such as high computational cost and complex modeling.

[0005] In addition, existing technologies improve the indexing efficiency of point cloud spatial indexing methods by constructing hybrid structures for point cloud data, such as hybrid structures of octrees and kd-trees, three-dimensional R-tree indexing methods, and hybrid structures of octrees and three-dimensional R-trees. For details, please refer to "Hybrid Point Cloud Data Storage Structure Based on Octree and KD-tree" by Liao Liqiong, Bai Junsong, and Luo De'an. Summary of the Invention

[0006] To overcome the problems existing in the prior art, this invention designs a point cloud geometric encoding method based on a hybrid context entropy model. It adopts a hybrid structure representation based on octree and block division, which eliminates a large amount of redundant information in the tree structure. Through bidirectional context information extraction and context information mixing, it realizes fine modeling of voxel context, provides more accurate prediction probability for voxel encoding, and reduces data loss in the encoding process while improving the compression ratio.

[0007] To achieve the above objectives, the present invention adopts the following technical solution:

[0008] A point cloud geometric encoding method based on a hybrid context entropy model includes the following steps:

[0009] Acquire point cloud data;

[0010] The point cloud data is partitioned ni times to obtain a tree-structured point cloud;

[0011] Nodes of depth ni in the tree-structured point cloud are converted into voxel blocks to obtain hybrid structured point cloud data;

[0012] In the tree-structured point cloud, the node at depth ni is subdivided several times to obtain several nodes of different sizes; context information is extracted from the nodes in the tree-structured point cloud.

[0013] In the hybrid structure point cloud, the voxel blocks are compressed several times to obtain several voxel blocks of different sizes; context information is extracted from the voxel blocks in the hybrid structure point cloud.

[0014] The context information of each voxel block is concatenated with the context information of nodes of the same size to obtain several concatenation results; the several concatenation results are mixed by a selection unit to obtain several mixed context information corresponding to each voxel block.

[0015] Based on the hybrid context information, probability estimation is performed on each voxel block to obtain the predicted probability;

[0016] Encoding is performed based on the predicted probability and voxel blocks to obtain the encoding result.

[0017] Furthermore, the specific steps for extracting context information from nodes in the tree-structured point cloud are as follows:

[0018] Context extraction is performed on each node in the tree-structured point cloud with a depth greater than or equal to ni. Then, the context extraction result of each node is progressively aggregated with the context extraction result of its parent node to obtain several context information corresponding to each node.

[0019] Furthermore, the compression of the voxel block specifically involves performing a max pooling operation on the voxel block.

[0020] Furthermore, the selected unit includes a sequentially connected splicing layer, an average pooling layer, a multilayer perceptron, and a convolutional layer.

[0021] Furthermore, the selection unit processes the input data as follows:

[0022] Input data F1 and data F2 are fed into the selection unit; the concatenation layer concatenates data F1 and data F2 to obtain the concatenated result Fcat; the average pooling layer performs average pooling on the concatenated result Fcat to obtain data Zc used to describe global features; the multilayer perceptron transforms data Zc into attention data r; through channel multiplication, the attention data r is used to select the concatenated result Fcat, which is then mixed by the convolutional layer to obtain the mixed result F. mix1 By channel addition, the mixed result F is adjusted using attention data r. mix Perform selective mixing to obtain and output the mixing result F. mix2 .

[0023] Furthermore, assuming there are three splicing results, the splicing results are blended by selecting units, including the following steps:

[0024] Obtain the first stitching result, the second stitching result, and the third stitching result;

[0025] The first splicing result and the second splicing result are input into the selection unit and mixed to obtain the first mixed result;

[0026] The first blending result and the third splicing result are input into the selection unit for blending to obtain the second blending result; the second blending result is the blending context information of the third voxel block;

[0027] The second and first mixing results are input into the selection unit for mixing to obtain a third mixing result; the third mixing result is the mixing context information of the second voxel block;

[0028] The third blending result and the first splicing result are input into the selection unit for blending to obtain the fourth blending result; the fourth blending result is the blending context information of the first voxel block.

[0029] Furthermore, it also includes decoding the encoded result, with the following specific steps:

[0030] The context entropy model of each voxel block is recovered using an autoregressive network;

[0031] The probability of each voxel is estimated by using the context entropy model to obtain the predicted probability of each voxel block.

[0032] Point cloud data is obtained by decoding the encoded result using an arithmetic decoder based on the predicted probability.

[0033] Compared with the prior art, the present invention has the following features and beneficial effects:

[0034] 1. This invention uses a hybrid structure based on octree and block to represent point cloud data, eliminating a large amount of redundant information in the tree structure. Through bidirectional context information extraction and context information mixing, it achieves fine modeling of voxel context, providing more accurate prediction probabilities for voxel encoding, and reducing data loss during the encoding process while improving compression ratio.

[0035] 2. The present invention slices and selects the context information of voxel blocks by selecting unit S, so that the mixed context information of each voxel block includes not only its own information but also the information of adjacent voxel blocks, thereby improving the fineness of the voxel context. Attached Figure Description

[0036] Figure 1 This is a flowchart of the coding process of the present invention;

[0037] Figure 2 This is a schematic diagram of an octree with a depth of 2;

[0038] Figure 3 This is a schematic diagram of an octree of depth ni in Example 1;

[0039] Figure 4 for Figure 3 Schematic diagram of the partitioning process at node C;

[0040] Figure 5 for Figure 3 A schematic diagram of the compression process of the voxel block obtained by the transformation of node C in the middle;

[0041] Figure 6 This is a schematic diagram of the context information splicing process in Example 1;

[0042] Figure 7 A schematic diagram of the structure of unit S;

[0043] Figure 8 This is a schematic diagram of the context information splicing result mixing process in Example 1. Detailed Implementation

[0044] The present invention will now be described in more detail with reference to the embodiments.

[0045] Example 1

[0046] like Figure 1 As shown, a point cloud geometric encoding method based on a hybrid context entropy model includes the following steps:

[0047] S1. Obtain point cloud data.

[0048] S2, in 2 n ×2 n ×2 nWithin the grid, the point cloud data is partitioned ni times to obtain a tree-structured point cloud of depth ni, such as... Figure 3 As shown. In this embodiment, an octree structure is preferred, and i = 6 is taken, so the size of each leaf node of the octree is 2. 6 ×2 6 ×2 6 .

[0049] S3, Usage size is 2 i ×2 i ×2 i The remaining depth of the point cloud data (i.e., the point cloud data in each leaf node of the octree) in the non-empty binary block voxel tree structure point cloud is used to obtain hybrid structure point cloud data. The size of each voxel block in the hybrid structure point cloud data is 2. 6 ×2 6 ×2 6 .

[0050] S4. Continue to subdivide the point cloud data downwards several times in the tree-structured point cloud to obtain several nodes of different sizes; extract the context information of each node from top to bottom, as follows:

[0051] like Figure 4 As shown, the point cloud data in the tree structure is further subdivided three times to obtain several child nodes of the original leaf nodes. The size of the child nodes is 2. 5 ×2 5 ×2 5 2 4 ×2 4 ×2 4 2 3 ×2 3 ×2 3 That is, the node side lengths are 32, 16, and 8 respectively; the depth of the tree is changed to n-3.

[0052] Context extraction is performed on the point cloud data within each node, extracting information such as the parent node's index, spatial location, and child node occupancy. In this embodiment, context extraction is performed on nodes with side lengths of 64, 32, 16, and 8 in the tree-structured point cloud.

[0053] Then, a multilayer perceptron (MLP) is used to progressively aggregate the context extraction results of each node with the context extraction results of its parent node to obtain several context information corresponding to each node.

[0054] S4. Extract bottom-up contextual information from the hybrid structured point cloud data, as follows:

[0055] like Figure 5As shown, each voxel block undergoes three max-pooling operations to obtain several voxel blocks of different sizes. The voxel block sizes are 2... 5 ×2 5 ×2 5 2 4 ×2 4 ×2 4 2 3 ×2 3 ×2 3 .

[0056] Context extraction is performed on the point cloud data within each voxel block using a ResNet network to obtain several context information corresponding to each voxel block. In this embodiment, context extraction is performed on voxel blocks with side lengths of 64, 32, 16, and 8 in the hybrid structure point cloud data.

[0057] S5, such as Figure 6 As shown, context information for nodes and voxel blocks of the same size is concatenated; for example... Figure 8 As shown, by selecting unit S to blend the splicing results, several blending context information corresponding to each voxel block are obtained, as follows:

[0058] By concatenating the context information of all nodes and voxel blocks of size 8, information F is obtained. l ;

[0059] By concatenating the context information of all nodes and voxel blocks of size 16, information F is obtained. l+1 ;

[0060] By concatenating the context information of all nodes and voxel blocks of size 32, information F is obtained. l+2 ;

[0061] By concatenating the context information of all nodes and voxel blocks of size 64, information F is obtained. l+3 ;

[0062] Information F l And information F l+1 The input selection unit is mixed to obtain the first mixed result;

[0063] Combine the first mixture result and information F l+2 The input selection unit is mixed to obtain a second mixed result;

[0064] The second mixed result and information F l+3 The input selection unit is blended to obtain a third blending result; the third blending result is the blending context information of a voxel block with a size of 64.

[0065] The third and second mixing results are input into the selection unit for mixing to obtain the fourth mixing result; the fourth mixing result is the mixing context information of the voxel block with a size of 32.

[0066] The fourth and first blending results are input into the selection unit and blended to obtain the fifth blending result; the fifth blending result is the blending context information of a voxel block with a size of 16.

[0067] Combine the fifth mixture result and information F l The input selection unit is blended to obtain the sixth blending result; the sixth blending result is the blending context information of the voxel block with a size of 8.

[0068] S6. Estimate the probability of each voxel block:

[0069] Based on the mixed context information, context modeling is performed on each voxel block to obtain several context entropy models; an autoregressive network is used to estimate the probability of the voxel block, and the context entropy models are used to improve the probability accuracy of the autoregressive network for voxels, so as to obtain the predicted probability P.

[0070] S7. Perform context-based arithmetic encoding on the point cloud:

[0071] Construct an arithmetic encoder; input each voxel block and its corresponding prediction probability P into the arithmetic encoder for encoding, and the resulting bitstream output is the encoding result.

[0072] Example 2

[0073] Select unit as follows Figure 5 As shown, it includes a splicing layer, an average pooling layer, a multilayer perceptron (a first fully connected layer, an activation layer using the ReLU function, a second fully connected layer, an activation layer using the Softmax function), and a 1x1 convolutional layer.

[0074] The process of mixing input data using the selection unit is as follows:

[0075] Input data F1 and data F2 are fed into the selection unit S; the concatenation layer concatenates data F1 and data F2 to obtain the concatenated result Fcat; the average pooling layer performs average pooling on the concatenated result Fcat to obtain data Zc used to describe global features; the multilayer perceptron transforms data Zc into attention data r; through channel multiplication, the attention data r is used to select the concatenated result Fcat, which is then mixed by a 1x1 convolutional layer to obtain the mixed result F. mix1 By channel addition, the mixed result F is adjusted using attention data r. mix Perform selective mixing to obtain and output the mixing result F. mix2 .

[0076] Example 3

[0077] Decoding the encoding result obtained in Example 1 specifically includes the following steps:

[0078] Step M1: Input the encoding results into the autoregressive network and use the autoregressive network to recover the context modeling of each voxel block.

[0079] Step M2: Probability estimation of each voxel block is performed using the recovered context entropy model to obtain the predicted probability of each voxel block.

[0080] Step M3: Based on the predicted probability, decode the point cloud data from the encoding result using an arithmetic decoder.

[0081] Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without inventive effort are within the scope of protection of the present invention.

Claims

1. A point cloud geometric encoding method based on a hybrid context entropy model, characterized in that, Includes the following steps: Acquire point cloud data; The point cloud data is partitioned ni times to obtain a tree-structured point cloud; Nodes of depth ni in the tree-structured point cloud are converted into voxel blocks to obtain hybrid structured point cloud data; In the tree-structured point cloud, the node at depth ni is subdivided several times to obtain several nodes of different sizes; context information is extracted from the nodes in the tree-structured point cloud. In the hybrid structure point cloud, the voxel block is compressed several times to obtain several voxel blocks of different sizes. Extracting contextual information from voxel blocks in hybrid structured point clouds; The context information of each voxel block is concatenated with the context information of nodes of the same size to obtain several concatenation results; the several concatenation results are mixed by a selection unit to obtain several mixed context information corresponding to each voxel block. Based on the hybrid context information, probability estimation is performed on each voxel block to obtain the predicted probability; Encoding is performed based on the predicted probability and voxel blocks to obtain the encoding result; The selection unit includes a splicing layer, an average pooling layer, a multilayer perceptron, and a convolutional layer. The selection unit processes the input data as follows: Input data F1 and data F2 are fed into the selection unit; the concatenation layer concatenates data F1 and data F2 to obtain the concatenated result Fcat; the average pooling layer performs average pooling on the concatenated result Fcat to obtain data Zc used to describe global features; the multilayer perceptron transforms data Zc into attention data r; through channel multiplication, the attention data r is used to select the concatenated result Fcat, which is then mixed by the convolutional layer to obtain the mixed result F. mix1 By channel addition, the mixed result F is adjusted using attention data r. mix Perform selective mixing to obtain and output the mixing result F. mix2 ; The process of blending several splicing results by selecting units includes the following steps: Obtain the first stitching result, the second stitching result, and the third stitching result; The first splicing result and the second splicing result are input into the selection unit and mixed to obtain the first mixed result; The first blending result and the third splicing result are input into the selection unit for blending to obtain the second blending result; the second blending result is the blending context information of the third voxel block; The second and first mixing results are input into the selection unit for mixing to obtain a third mixing result; the third mixing result is the mixing context information of the second voxel block; The third blending result and the first splicing result are input into the selection unit for blending to obtain the fourth blending result; the fourth blending result is the blending context information of the first voxel block.

2. The point cloud geometric encoding method based on a hybrid context entropy model according to claim 1, characterized in that, The specific steps for extracting context information from nodes in a tree-structured point cloud are as follows: Context extraction is performed on each node in the tree-structured point cloud with a depth greater than or equal to ni. Then, the context extraction result of each node is progressively aggregated with the context extraction result of its parent node to obtain several context information corresponding to each node.

3. The point cloud geometric encoding method based on a hybrid context entropy model according to claim 1, characterized in that, The compression of the voxel block specifically involves performing a maximum pooling operation on the voxel block.

4. The point cloud geometric encoding method based on a hybrid context entropy model according to claim 1, characterized in that, It also includes decoding the encoded result, with the following specific steps: The context entropy model of each voxel block is recovered using an autoregressive network; The probability of each voxel is estimated by using the context entropy model to obtain the predicted probability of each voxel block. Point cloud data is obtained by decoding the encoded result using an arithmetic decoder based on the predicted probability.

Images