A method for compression and decompression of point cloud

Abstract

The present application provides a method for compressing and decompressing a point cloud, wherein the compression method of the point cloud includes: binary encoding the coordinates of the point cloud through a tree structure, converting the binary code into a decimal code as the node occupancy code corresponding to each node in the tree structure; moving a preset window for the nodes in the tree structure, and determining the uncompressed nodes in the preset window as the current node to be compressed; inputting the context information of the current node to be compressed into a pre-trained attention neural network model to predict the probability distribution of the information to be compressed corresponding to the current node to be compressed; according to the probability distribution of the information to be compressed corresponding to the node to be compressed and the information to be compressed, inputting into an arithmetic encoder for entropy coding to obtain a point cloud code stream, and using the point cloud code stream as the compression result of the point cloud. The present application solves the technical problems of large amount of calculation and low compression efficiency in the prior art, and achieves the technical effect of improving the compression effect and reducing the amount of calculation.

Description

Technical Field

[0001] The present application relates to the technical field of point cloud data compression and decompression, and in particular to a point cloud compression and decompression method. Background Art

[0002] Point cloud data is an important data structure for three-dimensional representation, so effective compression technology is very necessary for the storage and transmission of 3D point clouds. At present, the point cloud is first voxelized and an octree structure is established. The context information of the voxel structure corresponding to each node of the octree structure is obtained, and then the 3D convolutional neural network is used to learn the features of the binary code, and then the occupancy status of the eight child nodes of the octree node is obtained, thereby achieving lossless compression of the octree.

[0003] However, using the context information of the voxel structure can easily lead to the technical problem of excessive computational complexity. In order to reduce the computational complexity, less context information of the voxel structure is necessarily required. However, using less context information of the voxel structure can easily lead to technical problems of inaccurate prediction and poor compression effect. Summary of the invention

[0004] In view of this, the purpose of the present application is to at least provide a point cloud compression and decompression method, which performs compression and decompression processing through the node occupancy code and attribute information of the node to be compressed, and when compressing and decompressing, the node occupancy code and position information of the same-layer node and father node corresponding to the node to be compressed are used as the context information of the node to be compressed, which solves the technical problems of large calculation amount and low compression efficiency in the prior art, and achieves the technical effect of improving compression effect and reducing calculation amount.

[0005] This application mainly includes the following aspects:

[0006] In a first aspect, an embodiment of the present application provides a point cloud compression method, wherein the compression method includes: binary encoding the coordinates of the points in the point cloud through a tree structure, converting the binary code into a decimal code as a node occupancy code corresponding to each node in the tree structure; moving a preset window for the nodes in the tree structure, and determining the uncompressed nodes in the preset window as the current nodes to be compressed; inputting the context information of the current node to be compressed into a pre-trained attention neural network model to predict the probability distribution of the information to be compressed corresponding to the current node to be compressed; according to the probability distribution of the information to be compressed corresponding to the node to be compressed and the information to be compressed, inputting them into an arithmetic encoder for entropy encoding to obtain a point cloud code stream, and using the point cloud code stream as the compression result of the point cloud.

[0007] Optionally, the context information of the current node to be compressed includes: the multi-layer father node of the same-layer node corresponding to the current node to be compressed, the node occupancy code and position information of the previous same-layer node, and the position information of the node to be compressed; the multi-layer father node of the same-layer node corresponding to the current node to be compressed, the node occupancy code and position information of the previous same-layer node, and the position information of the current node to be compressed are used as the context information corresponding to the node to be compressed, including: the node occupancy code and position information of the previous same-layer node of the current node to be compressed and the multi-layer father node corresponding to the previous same-layer node are used as the first context information corresponding to the current node to be compressed; the position information of the current node to be compressed, the node occupancy code of the previous compressed same-layer node of the current node to be compressed, and the node occupancy code and position information of the multi-layer father node corresponding to the current node to be compressed are used as the second context information corresponding to the current node to be compressed, and the first context information and the second context information are used as the context information corresponding to the current node to be compressed; the information to be compressed corresponding to the current node to be compressed includes: the node occupancy code of the current node to be compressed.

[0008] Optionally, the context information of the current node to be compressed also includes: the node occupancy code and position information of the multi-layer father node of the same-layer node corresponding to the current node to be compressed, the node occupancy code and position information of the same-layer node, and the node attributes and / or node attribute residuals corresponding to the previous same-layer node; the information to be compressed corresponding to the current node to be compressed also includes: the node attributes and / or node attribute residuals of the current node to be compressed.

[0009] Optionally, the context information of the current node to be compressed is input into a pre-trained attention neural network model to predict the probability distribution of the information to be compressed corresponding to the current node to be compressed, including: inputting the context information of the current node to be compressed into the first-layer attention operation network of the attention neural network model to obtain a first weighted context matrix of the current node to be compressed; adding the first weighted context matrix and the context information, and inputting the result of the addition into the first-layer multi-layer perceptron network to obtain a second weighted context matrix; then inputting the second weighted context matrix into the second-layer attention operation network to obtain a third weighted context matrix; adding the third weighted context matrix to the second weighted context matrix, and inputting the result of the addition into the second-layer multi-layer perceptron network to obtain a fourth weighted context matrix; inputting the fourth weighted context matrix into the third-layer multi-layer perceptron network to obtain a fifth weighted context matrix; and using the fifth weighted context matrix through a softmax function to predict the probability distribution of the information to be compressed corresponding to the current node to be compressed.

[0010] Optionally, the context information of the current node to be compressed is input into the first layer of the attention operation network of the attention neural network model to obtain the first weighted context matrix of the current node to be compressed, including: inputting the context information of the current node to be compressed into the first multi-layer perceptron to obtain the first output matrix; inputting the context information of the current node to be compressed into the second multi-layer perceptron to obtain the second output matrix; inputting the context information of the current node to be compressed into the third multi-layer perceptron to obtain the third output matrix; multiplying the transposed matrix of the second output matrix by the first output matrix to obtain the matrix inner product; adding the matrix inner product to the mask matrix, and inputting the result of the addition into the softmax function to obtain the attention matrix; multiplying the attention matrix by the third output matrix to obtain the first weighted context matrix of the current node to be compressed.

[0011] Optionally, each element in the attention matrix is ​​an attention value; the attention value of the attention matrix is ​​calculated by the following formula:

[0012]

[0013] In formula (1), refers to the attention value between the jth node and the kth node context. The jth node is the jth node in the same layer and is the current node to be compressed. j is the context information of the jth node, f k is the context information of the kth node, and the kth node is the previous node of the jth node; Represents the similarity value between the context of the jth node and the kth node, the denominator It refers to the sum of similarity values ​​between the j-th node and the contexts from the 1st node to the j-th node; MLP2(f j ) refers to inputting the context information of the j-th node into the second multilayer perceptron to obtain the second output matrix corresponding to the j-th node; It refers to inputting the context information of the k-th node into the first multi-layer perceptron to obtain the transposed matrix of the first output matrix corresponding to the k-th node.

[0014] Optionally, the context information of the current node to be compressed is input into a pre-trained attention neural network model to predict the probability distribution of the information to be compressed corresponding to the current node to be compressed, including: expanding the context information of the current node to be compressed to generate expanded context information; inputting the expanded context information into a pre-trained attention neural network model to predict the probability distribution of the information to be compressed corresponding to the current node to be compressed.

[0015] In the second aspect, an embodiment of the present application also provides a point cloud decompression method, which includes: moving a preset window on a tree structure, determining an undecompressed node within the preset window each time as a current node to be decompressed, inputting the context information of the current node to be decompressed into a pre-trained attention neural network model, and predicting the probability distribution of the information to be compressed corresponding to the current node to be decompressed; inputting the probability distribution of the information to be decompressed corresponding to the current node to be decompressed and the point cloud code stream into an arithmetic decoder for entropy decoding to obtain the information to be decompressed corresponding to the node to be decompressed; constructing a tree structure for the information to be decompressed, obtaining a point cloud from the tree structure, and then obtaining a decompressed point cloud through inverse quantization.

[0016] In a third aspect, an embodiment of the present application further provides an electronic device, comprising: a processor, a memory and a bus, the memory storing machine-readable instructions executable by the processor, and when the electronic device is running, the processor and the memory communicate through the bus, and the machine-readable instructions are executed by the processor to execute the steps of the point cloud compression method in the first aspect or any possible implementation of the first aspect, and / or the steps of the point cloud decompression method in the second aspect.

[0017] In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the point cloud compression method in the above-mentioned first aspect or any possible implementation scheme of the first aspect are executed, and / or the steps of the point cloud decompression method in the above-mentioned second aspect are executed.

[0018] The present application provides a method for compressing and decompressing a point cloud. The method for compressing a point cloud is as follows: binary-encode the coordinates of the point cloud through a tree structure, convert the binary code into a decimal code, and use it as the node occupancy code corresponding to each node in the tree structure; move a preset window for the nodes in the tree structure, and determine the uncompressed nodes in the preset window as the current node to be compressed; input the context information of the current node to be compressed into a pre-trained attention neural network model to predict the probability distribution of the information to be compressed corresponding to the current node to be compressed; input the probability distribution of the information to be compressed corresponding to the node to be compressed and the information to be compressed into an arithmetic encoder for entropy coding to obtain a point cloud code stream, and use the point cloud code stream as the compression result of the point cloud. The present application abandons the context information of the voxel structure, and uses the node occupancy code and position information of the same-layer nodes and the father node corresponding to the current node to be compressed as the context information of the current node to be compressed. When compressing and decompressing the node to be compressed, the same-layer nodes and the father node of the current node to be compressed are considered, which solves the technical problems of large calculation amount and low compression efficiency in the prior art, and achieves the technical effect of improving the compression effect and reducing the calculation amount.

[0019] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, preferred embodiments are specifically cited below and described in detail with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0021] Figure 1 A flow chart of a point cloud compression method provided in an embodiment of the present application is shown.

[0022] Figure 2 A flow chart of a point cloud geometric compression method provided in an embodiment of the present application is shown.

[0023] Figure 3 A flow chart of a point cloud attribute compression method provided in an embodiment of the present application is shown.

[0024] Figure 4 A flow chart of a point cloud decompression method provided in an embodiment of the present application is shown.

[0025] Figure 5 A flow chart of a point cloud geometric decompression method provided in an embodiment of the present application is shown.

[0026] Figure 6 A flow chart of a point cloud attribute decompression method provided in an embodiment of the present application is shown.

[0027] Figure 7 A schematic diagram of attribute completion provided in an embodiment of the present application is shown.

[0028] Figure 8 A schematic diagram of a preset window provided in an embodiment of the present application is shown.

[0029] Fig. 9 A schematic diagram of the attention neural network model provided in an embodiment of the present application is shown.

[0030] Fig.10 A schematic diagram of compression and decompression of a point cloud provided in an embodiment of the present application is shown.

[0031] Fig.11 A schematic diagram of a point cloud compression device provided in an embodiment of the present application is shown.

[0032] Fig.12A schematic diagram of a point cloud decompression device provided in an embodiment of the present application is shown.

[0033] Fig.13 A schematic structural diagram of an electronic device provided in an embodiment of the present application is shown. DETAILED DESCRIPTION

[0034] To make the purpose, technical scheme and advantages of the embodiments of the present application clearer, the technical scheme in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. It should be understood that the drawings in the present application only serve the purpose of explanation and description and are not used to limit the scope of protection of the present application. In addition, it should be understood that the schematic drawings are not drawn in real proportion. The flowchart used in this application shows the operations implemented according to some embodiments of the present application. It should be understood that the operations of the flowchart can be implemented out of sequence, and the steps without logical context can be reversed in order or implemented simultaneously. In addition, those skilled in the art, under the guidance of the content of the present application, can add one or more other operations to the flowchart, or remove one or more operations from the flowchart.

[0035] In addition, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. The components of the embodiments of the present application described and shown in the drawings here can be arranged and designed in various configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the application claimed for protection, but merely represents the selected embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without making creative work belong to the scope of protection of the present application.

[0036] In the prior art, when compressing and decompressing point cloud data, the voxel structure corresponding to the node to be compressed is used as context information, resulting in technical problems such as excessive calculation and inaccurate prediction results. Alternatively, the nodes at the same level of the node to be compressed are not used as context information of the node to be compressed, resulting in inaccurate prediction results, which in turn leads to a technical problem of poor compression effect.

[0037] Based on this, the embodiment of the present application provides a point cloud compression and decompression method. When compressing and decompressing the node to be compressed, the node occupancy code and position information of the same-layer node and the father node corresponding to the node to be compressed are used as the context information of the node to be compressed, which solves the technical problems of large calculation amount and low compression efficiency in the prior art, and achieves the technical effect of improving the compression effect and reducing the calculation amount. The specific method is as follows:

[0038] See also Figure 1 , Figure 1This is a flow chart of a point cloud compression method provided in an embodiment of the present application. Figure 1 As shown, a point cloud compression method provided in an embodiment of the present application includes the following steps:

[0039] S101. Binary encode the coordinates of the points in the point cloud through a tree structure, and convert the binary code into a decimal code as a node occupancy code corresponding to each node in the tree structure.

[0040] Exemplarily, before binary encoding the coordinates of the points in the point cloud through a tree structure and converting the binary code into a decimal code as a node occupancy code corresponding to each node in the tree structure, the method includes quantizing the point cloud. Quantizing the point cloud can be understood as approximating a large number of possible discrete values ​​of coordinates or attributes of the point cloud to fewer discrete values. So that the quantized point cloud can be subsequently constructed into a tree structure. In an embodiment of lossless compression, quantization is not performed. In one embodiment provided in the present application, the tree structure is an octree structure.

[0041] When quantizing a point cloud, it is necessary to perform geometric quantization and attribute quantization on the point cloud. When performing geometric quantization (i.e., coordinate quantization), determine the minimum 3D coordinate value in the point cloud (i.e., the 3D coordinate value of the point in the lower right corner, see the attached Figure 7 ), subtract the 3D coordinate values ​​of all points in the point cloud from the minimum 3D coordinate value (i.e., translate the point cloud data so that the point in the lower right corner of the translated point cloud data is located at the origin), then compare the difference with the quantization step size and round it off, and use the rounded ratio as the 3D coordinate value of each point in the point cloud data after geometric quantization. When performing attribute quantization (i.e., color (RGB value or YUV value) and / or reflectivity quantization), compare the original point cloud attribute with the quantization step size and round it off, and use the rounded ratio as the node attribute of each point in the point cloud after attribute quantization.

[0042] The quantization step size (qs) is a user-adjustable parameter, and its value depends on the desired compression level and the acceptable distortion. The larger the quantization step size, the greater the distortion of the point cloud data, but the lower the bit rate. This also means that our model can be applied to compression with variable bit rates (the same model can achieve different compression sizes and distortions). The quantization step sizes of geometric quantization and attribute quantization can be different.

[0043] The node occupancy code refers to the arrangement position of each point in the point cloud, and the eight-bit binary code of the arrangement position of each point is converted into a decimal code, and the decimal code is used as the node occupancy code of this node.

[0044] For example, see Figure 7 , Figure 7 A schematic diagram of attribute completion provided in an embodiment of the present application is shown. Figure 7The voxel referred to by the node corresponding to the node occupancy code 73 includes eight sub-voxels, and the sub-voxels are numbered from number 0 to number 7. Among them, the sub-voxels of number 1, number 4 and number 7 have points, and their corresponding binary codes are 01001001 (corresponding to Figure 7 0a00b00c in the figure is converted to decimal 73, so 73 is the node occupancy code of this node.

[0045] S102: Move a preset window for nodes in the tree structure, and determine an uncompressed node in the preset window as a current node to be compressed.

[0046] The compression of point cloud is divided into geometric compression and attribute compression. When geometric compression is performed, the corresponding window is the geometric preset window.

[0047] For example, see Figure 8 , Figure 8 A schematic diagram of a preset window provided in an embodiment of the present application is shown. Figure 8 In the “x i "Node refers to the current node to be compressed, N1 refers to the number of nodes at the same level corresponding to the node to be compressed in the geometry preset window; N2 refers to the number of nodes at the same level corresponding to the node to be compressed in the attribute preset window; W1 refers to the geometry preset window in which the number of nodes at the same level (N1) of the current node to be compressed is 8 and the layer height is 4; W2 refers to the attribute preset window in which the number of nodes at the same level (N2) of the current node to be compressed is 7 and the layer height is 1.

[0048] S103: Input the context information of the current node to be compressed into a pre-trained attention neural network model to predict the probability distribution of the information to be compressed corresponding to the current node to be compressed.

[0049] Among them, the preset window is moved according to the breadth-first principle.

[0050] When moving the preset window, the nodes of the tree structure are arranged into a breadth-first sequence according to the breadth-first principle, and the front nodes in the breadth-first sequence are used as the same-layer nodes corresponding to the node to be compressed, until the number of the same-layer nodes of the preset window meets the preset number (N1 or N2). When the number of the same-layer nodes of the preset window and / or the number of multi-layer parent nodes of the same-layer nodes cannot meet the preset number (that is, there are no more nodes in the tree structure), the nodes in the preset window are supplemented with default nodes. This situation occurs in some layers close to the root of the tree structure.

[0051] The peer nodes corresponding to the current node to be compressed include the current node to be compressed, the preceding peer nodes of the current node to be compressed, and / or the succeeding peer nodes of the current node to be compressed.

[0052] In a preferred embodiment, when geometric compression is performed, the context information of the current node to be compressed (i.e., the geometric context information of the current compressed node) includes: multi-layer parent nodes of the same-layer nodes corresponding to the current node to be compressed, node occupancy codes and position information of the previous same-layer nodes, and the position information of the node to be compressed.

[0053] The context information corresponding to the current node to be compressed does not include the node occupancy code of the current node to be compressed. This is because the purpose of decompression is to decode the node occupancy code of the current node to be compressed. That is, the node occupancy code of the current node to be compressed is unknown during decompression. Therefore, during compression, its own node occupancy code is not considered.

[0054] The node occupancy code and position information of the preceding same-layer node of the current node to be compressed and the multi-layer father node corresponding to the preceding same-layer node are used as the first context information corresponding to the node to be compressed; the position information of the node to be compressed, the node occupancy code of the previous compressed same-layer node of the node to be compressed, and the node occupancy code and position information of the multi-layer father node corresponding to the node to be compressed are used as the second context information corresponding to the node to be compressed; the first context information and the second context information are used as the context information corresponding to the node to be compressed.

[0055] The location information includes: node index, and / or node depth, and / or node bounding box coordinates. The node index refers to the position of this node in its sibling nodes with the same parent node, and the node index ranges from 0 to 7; the node depth refers to the depth of the node in the octree structure, and the depth of the root node in the octree structure is 1. The bounding box refers to the cube space of the voxel corresponding to the node, and the bounding box coordinates refer to the coordinates of the two vertices of the diagonal line of the cube space. The diagonal directions of multiple bounding boxes corresponding to the same point cloud are the same, such as Figure 8 As shown, (x0, y0, z0, x1, y1, z1) are the bounding box coordinates of the voxel corresponding to the node occupancy code 73.

[0056] For example, return Figure 8 , Figure 8 The node with the middle node occupation code 9 has a node depth of 1, a node index of 7, and a node bounding box coordinate of (0,0,0,7,7,7).

[0057] When performing attribute compression, the context information of the current node to be compressed also includes: the node occupancy code and position information of the multi-layer parent node of the same-layer node corresponding to the current node to be compressed, the node occupancy code and position information of the same-layer node, and the node attributes and / or node attribute residuals corresponding to the previous same-layer node.

[0058] In a preferred embodiment, when performing attribute compression, the context information of the current node to be compressed (i.e., the node attribute (node ​​attribute residual) context information of the current compressed node) includes: multi-layer parent nodes of the same-layer nodes corresponding to the current node to be compressed, the preceding same-layer nodes, the succeeding same-layer nodes, the node occupancy code and position information of the node to be compressed, and the node attributes and / or node attribute residuals corresponding to the preceding same-layer nodes.

[0059] Specifically, the node occupancy code and position information of the preceding same-layer node of the current node to be compressed and the multi-layer father node corresponding to the preceding same-layer node are used as the first context information corresponding to the node to be compressed; the node occupancy code and position information of the succeeding same-layer node of the current node to be compressed and the multi-layer father node corresponding to the succeeding same-layer node are used as the second context information corresponding to the node to be compressed; the node occupancy code and position information of the node to be compressed, and the node occupancy code and position information of the multi-layer father node corresponding to the node to be compressed are used as the third context information corresponding to the node to be compressed; the node attributes and / or node attribute residuals corresponding to the preceding same-layer node of the current node to be compressed are used as the fourth context information corresponding to the node to be compressed; the first context information, the second context information, the third context information and the fourth context information are used as the context information corresponding to the node to be compressed.

[0060] When performing attribute compression, the node at the same level corresponding to the node to be compressed is the deepest leaf node in the tree structure, that is, a node without child nodes in the tree structure. Only leaf nodes have node attributes and / or node attribute residuals.

[0061] The node attribute refers to the 3-bit RGB value or YUV value or reflectivity or other attributes of the node. The node attribute residual refers to the difference in attribute encoding between this node and the previous node after the node attributes are completed. Among them, the value range of the three primary color channels of the 3-bit RGB value is 0-255, that is, the node attribute is a 3-bit RGB value. Attribute completion is to facilitate the construction of context and the calculation of neural networks. The supplemented attribute value (attribute value residual) does not enter entropy coding.

[0062] For an example of how to complete the attributes of a node, see Figure 7 , Figure 7 FIG. 1 shows a schematic diagram of attribute completion provided by an embodiment of the present application. Figure 7As shown, for example, for the R attribute in the color, in the node corresponding to the node occupancy code 73, the node attribute value corresponding to the sub-voxel numbered 1 is a, the node attribute value corresponding to the sub-voxel numbered 4 is b, and the node attribute value corresponding to the sub-voxel numbered 7 is c. The attribute code corresponding to the node occupancy code 73 is 0a00b00c, where 0 represents that there is no point cloud point in the sub-voxel. The attribute code is cyclically shifted left to complete. The first cycle of left shift completion is aa0bb0cc. If the attribute code after the first attribute completion still has the attribute code of zero, the second cycle of left shift completion is aabbbccc. If there is no zero code in the attribute code after the second attribute completion, the attribute completion is completed, and the attribute code after the attribute completion is aabbbccc. If the attribute code after the attribute completion of the previous node is dddeeedd, the node attribute residual corresponding to the node with the node occupancy code 73 is: (da)(da)(db)(eb)(eb)(ec)(dc)(dc).

[0063] When performing geometric compression, the information to be compressed corresponding to the current node to be compressed includes: the node occupancy code of the current node to be compressed; when performing attribute compression, the information to be compressed corresponding to the current node to be compressed includes: the node attribute and / or node attribute residual of the current node to be compressed.

[0064] When geometric compression is performed, the i-th node to be compressed is determined as the current node to be compressed, and the context information of the i-th node to be compressed is input into the pre-trained attention neural network model to predict the probability distribution q of the node occupancy code corresponding to the i-th node to be compressed. i (x), that is, q i (x) is the probability distribution of the node occupation code of the i-th node to be compressed being 1-255. For example, the probability distribution corresponding to the current node to be compressed is (0.01, 0.2, 0.056, ..., 0.36), which means that the probability of the node occupation code of the current node to be compressed being 1 is 0.01, the probability of being 2 is 0.2, the probability of being 3 is 0.056, ..., the probability of being 255 is 0.36), and the sum of the probability distribution is 1.

[0065] When performing attribute compression, predict the node attribute and / or node attribute residual probability distribution corresponding to the current node to be compressed Right now, is the nth node of the ith node to be compressed A The probability distribution of node attributes and / or node attribute residuals.

[0066] The steps of training the attention neural network model are as follows: Step 1, construct a tree structure from the sample point cloud, and determine the context information of the sample node and the information to be compressed of the node (wherein the sample point cloud is different from the test point cloud, but the sample point cloud is similar to the test point cloud, for example, different frames of the same sequence point cloud or similar point clouds); Step 2, input the context information of the sample node into the first layer of the attention operation network of the attention neural network model to obtain the first weighted context matrix of the sample node; Step 3, add the first weighted context matrix to the context information, and add the result to the weighted matrix. The result is input into the first layer of multi-layer perceptron network to obtain the second weighted context matrix; step 4, the second weighted context matrix is ​​input into the second layer of attention operation network to obtain the third weighted context matrix; step 5, the third weighted context matrix is ​​added to the second weighted context matrix, and the result of the addition is input into the second layer of multi-layer perceptron network to obtain the probability distribution of the information to be compressed corresponding to the node to be compressed; step 6: the probability distribution of the information to be compressed corresponding to multiple sample nodes and the information to be compressed are brought into the loss function to calculate the loss value; wherein, the loss function of geometric compression is:

[0067] loss1=-∑ i log2 q i (x i |f;w1) (2)

[0068] In formula (2), loss1 refers to the loss function of geometric compression, x i refers to the current node to be compressed (the i-th node to be compressed), f refers to the context information of the i-th node to be compressed after dimension expansion, w1 refers to the weight of the geometric compression attention neural network, q i Refers to the probability distribution of the node occupation code corresponding to the i-th node to be compressed, i refers to the range of all nodes to be compressed, ∑ i log2 q i (x i |f; w1) refers to the sum of the estimated bit rates corresponding to all nodes to be compressed; the loss function of attribute compression is designed to only calculate the point cloud points where the attribute exists The attribute residual (i.e., J = {1, 4, 7} of the attribute 0a00b00c before attribute completion in the implementation of this application), the loss function of attribute compression is:

[0069]

[0070] In formula (3), loss2 refers to the loss function of attribute compression, w2 refers to the weight of the attribute compression attention neural network, is the weight of each attribute in the attribute code stream, n ARefers to the nth A attributes (i.e., the attribute number in the RGB value, for example, R is the first attribute in the RGB value), N A Refers to the total number of node attributes (for example, RGB values ​​are 3 attributes), and the range referred to by i is all nodes to be compressed (that is, all leaf nodes when attributes are compressed). Step 7: Use a deep learning-based optimization algorithm for backpropagation to optimize the loss value and update the weights w1 and / or w2 of the attention neural network model. Execute steps 1 to 7 multiple times. When the rate of change of the loss value output by the loss function reaches a preset threshold (that is, the output value of the loss function converges), the trained attention neural network model is obtained.

[0071] The context information of the current node to be compressed is input into a pre-trained attention neural network model to predict the probability distribution of the information to be compressed corresponding to the current node to be compressed, including: expanding the dimension of the context information of the current node to be compressed to generate expanded-dimensional context information; inputting the expanded-dimensional context information into a pre-trained attention neural network model to predict the probability distribution of the information to be compressed corresponding to the current node to be compressed.

[0072] In the present application, the context information of the current node to be compressed is first expanded through a one-hot code operation to generate a one-hot code of the context information, and then the one-hot code of the context information is embedded to generate expanded context information.

[0073] One-hot code is a code in which there are as many bits as there are states, and only one bit is 1, while the others are all 0. For example, if the node index of a node is 1, its one-hot code is (0,1,0,0,0,0,0,0), which expands the one-dimensional data into 8 dimensions. Embedding is a linear mapping of the result of one-hot code operation to the preset dimension.

[0074] Specifically, the context information of the current node to be compressed is input into a pre-trained attention neural network model to predict the probability distribution of the information to be compressed corresponding to the current node to be compressed, including:

[0075] The context information of the current node to be compressed is input into the first layer of attention operation network of the attention neural network model to obtain the first weighted context matrix of the current node to be compressed; the first weighted context matrix and the context information are added, and the result of the addition is input into the first layer of multi-layer perceptron network to obtain the second weighted context matrix; the second weighted context matrix is ​​input into the second layer of attention operation network to obtain the third weighted context matrix; the third weighted context matrix is ​​added to the second weighted context matrix, and the result of the addition is input into the second layer of multi-layer perceptron network to obtain the third weighted context matrix; the third weighted context matrix is ​​input into the third layer of multi-layer perceptron network to obtain the fourth weighted context matrix; the fourth weighted context matrix is ​​input into the third layer of multi-layer perceptron network to obtain the fifth weighted context matrix; the fifth weighted context matrix is ​​used through the softmax function to predict the probability distribution of the information to be compressed corresponding to the current node to be compressed.

[0076] For a preferred embodiment, please refer to Fig. 9 A schematic diagram of the attention neural network model provided in an embodiment of the present application is shown. Fig. 9 In the above example, N refers to the number of nodes on the same layer corresponding to the node to be compressed (N1 for the geometry preset window and N2 for the attribute preset window. By default, N1 = N2 = N when performing attribute compression); H refers to the layer height of the preset window (K for the geometry preset window and 1 for the attribute preset window), and C refers to the dimension of the context information (C for the geometry compression). G , the dimension corresponding to attribute compression is C A +K×C G ); H×C refers to the dimension of the context information in the preset window; N0 refers to the number of nodes to be compressed; C0 refers to the dimension of the probability distribution corresponding to the nodes to be compressed (the dimension corresponding to geometric compression is 255, and the dimension of attribute compression such as RGB is 3×256×8 (each leaf node has 8 bits of attribute encoding)).

[0077] The context information (N, H, C) of the current node to be compressed is input into the first layer of attention operation network to obtain the first weighted context matrix (N, H×C); the first weighted context matrix is ​​added to the context information, and the result of the addition is input into the first layer of multi-layer perceptron to obtain the second weighted context matrix (N, H×C); the second weighted context matrix is ​​input into the second layer of attention operation network to obtain the third weighted context matrix (N, H×C); the third weighted context matrix is ​​added to the second weighted context matrix, and the result of the addition is input into the second layer of multi-layer perceptron network to obtain the fourth weighted context matrix (N, H×C); the fourth weighted context matrix is ​​input into the third layer of multi-layer perceptron network to obtain the fifth weighted context matrix (N, C0); the fifth weighted context matrix is ​​used through the softmax function to predict the probability distribution (N0, C0) of the information to be compressed corresponding to the current node to be compressed.

[0078] The context information of the current node to be compressed is input into the first layer of the attention operation network of the attention neural network model to obtain the first weighted context matrix of the current node to be compressed, including:

[0079] Input the context information of the current node to be compressed into the first multilayer perceptron to obtain the first output matrix; input the context information of the current node to be compressed into the second multilayer perceptron to obtain the second output matrix; input the context information of the current node to be compressed into the third multilayer perceptron to obtain the third output matrix; multiply the transposed matrix of the second output matrix by the first output matrix to obtain the matrix inner product; add the matrix inner product to the mask matrix, and input the result of the addition into the softmax function to obtain the attention matrix; multiply the attention matrix by the third output matrix to obtain the first weighted context matrix of the current node to be compressed.

[0080] Among them, the mask matrix (the matrix with -∞ in the upper right corner and 0 in the lower left corner) is used to predict the probability distribution corresponding to the current node to be compressed, and only the compressed nodes before the current node to be compressed are considered, which leads to the denominator of the attention value The length of the sum is different. For example, when the jth node is the current node to be compressed, the context information of the nodes after j+1 will not exist in the result of the jth node. Therefore, in one calculation, the N output results can be truncated to obtain the last N0 results as the probability distribution of the last N0 nodes to be compressed, so as to reduce the number of network transmissions and speed up the compression.

[0081] Among them, each element in the attention matrix is ​​the attention value, and the attention value of the attention matrix is ​​calculated by the following formula:

[0082]

[0083] In formula (1), refers to the attention value between the jth node and the kth node context. The jth node is the jth node in the same layer and is the current node to be compressed. j is the context information of the jth node, f k is the context information of the kth node, and the kth node is the previous node of the jth node; Represents the similarity value between the context of the jth node and the kth node, the denominator It refers to the sum of similarity values ​​between the j-th node and the contexts from the 1st node to the j-th node; MLP2(f j ) refers to inputting the context information of the j-th node into the second multilayer perceptron to obtain the second output matrix corresponding to the j-th node; It refers to inputting the context information of the k-th node into the first multi-layer perceptron to obtain the transposed matrix of the first output matrix corresponding to the k-th node.

[0084] That is to say, the kth node is located between the 1st node and the jth node.

[0085] S104, according to the probability distribution of the information to be compressed corresponding to the node to be compressed and the information to be compressed, input it into the arithmetic encoder for entropy coding to obtain a point cloud code stream, and use the point cloud code stream as the compression result of the point cloud.

[0086] That is, the probability distribution of the information to be compressed and the information to be compressed corresponding to all the nodes to be compressed are input into the arithmetic encoder for entropy coding to obtain the point cloud code stream, and the point cloud code stream is used as the compression result of the point cloud. According to the object of point cloud compression, the point cloud code stream is divided into a geometric code stream and an attribute code stream, that is, when geometric compression is performed, the compression result is a geometric code stream; when attribute compression is performed, the compression result is an attribute code stream.

[0087] The arithmetic encoder converts the probability distribution and information to be compressed corresponding to the input nodes to be compressed into a decimal less than 1. This decimal is saved in binary, which is the generated point cloud code stream. The point cloud code stream is a compressed binary bit stream. For example, a point cloud file with an original size of 1.94MB (2038480 bytes) can be compressed into a file with a size of 55.5KB (56890 bytes), saving 97.2% of storage space.

[0088] For a preferred embodiment, please refer to Figure 2 , Figure 2 The flowchart of a point cloud geometric compression method provided by an embodiment of the present application is shown. The steps of a point cloud geometric compression method are as follows:

[0089] S201. Obtain geometric context information corresponding to the current node to be compressed, and obtain N1×K×3-dimensional geometric context information.

[0090] Get the geometric context information corresponding to the current node to be compressed, and obtain the geometric context information of N1×K×3 dimensions. Among them, N1 nodes refer to the number of nodes in the same layer of the current node to be compressed in the geometric preset window, K refers to the height of the geometric preset window, and 3 refers to the node occupancy code, node depth, and node index. In other words, N1 nodes include the current node to be compressed.

[0091] S202, the N1×K×3-dimensional geometric context information is expanded by one-hot encoding and embedding operations to generate expanded-dimensional geometric context information, which is N1×K×C G The extended geometric context information of the dimension.

[0092] In the embodiment of the present application, the range of the node occupancy code is 1-255, and the node occupancy code is 255-dimensional after the one-hot code operation; the range of the node depth is 1-16, and the node index is 16-dimensional after the one-hot code operation; the range of the node index is 0-7, and the node depth is 8-dimensional after the one-hot code operation. Then, the 255-dimensional node occupancy code is converted to 128 dimensions through the embedding operation, the 16-dimensional node depth is converted to 6 dimensions through the embedding operation, and the 8-dimensional node index is converted to 4 dimensions through the embedding operation. Therefore, after the dimension expansion through the one-hot code operation and the embedding operation, the 3-dimensional data of the geometric context information is converted into C G Dimension (a total of 138 dimensions, which is the sum of 128 dimensions, 6 dimensions, and 4 dimensions).

[0093] S203, N1×K×C G The expanded-dimensional geometric context information of the dimensional image is input into the trained attention neural network model to predict the probability distribution of the node occupancy code corresponding to the current node to be compressed.

[0094] S204: Determine whether the probability distribution of the node occupancy codes corresponding to all the nodes to be compressed has been obtained.

[0095] If the probability distribution of the node occupancy codes corresponding to all the nodes to be compressed has not been obtained, return to step S201; if the probability distribution of the node occupancy codes corresponding to all the nodes to be compressed have been obtained, the probability distribution of the node occupancy codes corresponding to all the nodes to be compressed and the node occupancy codes of the nodes to be compressed are input into the arithmetic encoder for entropy coding to obtain a geometric code stream, and the geometric code stream is used as the geometric compression result of the point cloud.

[0096] For a preferred embodiment, please refer to Figure 3 , Figure 3The flowchart of a point cloud attribute compression method provided by an embodiment of the present application is shown. The steps of a point cloud attribute compression method are as follows:

[0097] S301, obtain the node attribute (node ​​attribute residual) context information corresponding to the current node to be compressed, and obtain N2×8×N A Dimensional node attribute (node ​​attribute residual) context information.

[0098] Get the node attribute context information corresponding to the current node to be compressed, and get N2×8×N A The attribute context information of the dimension. Among them, N2 nodes refer to the number of nodes in the same layer corresponding to the current node to be compressed in the attribute preset window, 8 refers to the number of bits of the completed attribute code included in each node (that is, the attribute code corresponding to the node occupancy code 73 mentioned above is completed as aabbbccc), N A Refers to the number of node attributes (i.e., 3-bit RGB values ​​or 1-bit reflectance value).

[0099] S302, N2×8×N A The node attribute (node ​​attribute residual) context information of the dimension is expanded through the one-hot code operation and the embedding operation to generate the expanded node attribute (node ​​attribute residual) context information, which is N2×C A The context information of the expanded node attributes (node ​​attribute residuals) of the dimension.

[0100] The node attributes in the embodiment of the present application adopt 3-bit RGB values, so N A The dimensions correspond to the red value, blue value, and green value. The value ranges of red value, blue value, and green value are all 0-255. Therefore, the red value is converted to 256 dimensions after the one-hot encoding operation, the blue value is converted to 256 dimensions after the one-hot encoding operation, and the green value is converted to 256 dimensions after the one-hot encoding operation. Then, the 256-dimensional red value is converted to 128 dimensions through the embedding operation, the 256-dimensional blue value is converted to 128 dimensions through the embedding operation, and the 256-dimensional green value is converted to 128 dimensions through the embedding operation. Therefore, after the dimension expansion through the one-hot encoding operation and the embedding operation, the N of the attribute context information is A dimensional data into C A Dimension (a total of 384 dimensions, which is the sum of 128 dimensions, 128 dimensions, and 128 dimensions).

[0101] S303, N2×C A The expanded node attributes (node ​​attribute residuals) context information and N2×K×C G The expanded-dimensional geometric context information of the node is input into the trained attention neural network model to predict the probability distribution of the node attributes (node ​​attribute residuals) corresponding to the current node to be compressed.

[0102] That is to say, geometric context information is also required during attribute compression, and the geometric window corresponding to the geometric context information required during attribute compression may be different from the geometric window corresponding to the geometric compression.

[0103] S304: Determine whether the probability distributions of the node attributes (node ​​attribute residuals) corresponding to all the nodes to be compressed have been obtained.

[0104] If the probability distribution of the node attributes (node ​​attribute residuals) corresponding to all the nodes to be compressed has not been obtained, return to step S301; if the probability distribution of the node attributes (node ​​attribute residuals) corresponding to all the nodes to be compressed have been obtained, the probability distribution of the node attributes (node ​​attribute residuals) corresponding to all the nodes to be compressed and the node attributes (node ​​attribute residuals) of the nodes to be compressed are input into the arithmetic encoder for entropy coding to obtain an attribute code stream, and the attribute code stream is used as the attribute compression result of the point cloud.

[0105] Based on the same application concept, the present application also provides a point cloud decompression method corresponding to the point cloud compression method provided in the above embodiment. Since the point cloud decompression method in the present application solves the problem in a similar principle to the point cloud compression method in the above embodiment, the implementation of the point cloud decompression method can refer to the implementation of the point cloud compression method, and the repeated parts will not be repeated. Fig.10 , Fig.10 A schematic diagram of compression and decompression of a point cloud provided in an embodiment of the present application is shown. Fig.10 The geometric prediction network and attribute prediction network in are the attention neural networks in the embodiments of the present application.

[0106] See also Figure 4 , Figure 4 A flowchart of a point cloud decompression method provided in an embodiment of the present application is shown. The steps of a point cloud decompression method are as follows:

[0107] S401: Move a preset window for a tree structure, and determine an undecompressed node in the preset window each time as a current node to be decompressed.

[0108] Point cloud decompression is divided into geometric decompression and attribute decompression.

[0109] S402: Input the context information of the current node to be decompressed into a pre-trained attention neural network model to predict the probability distribution of the information to be decompressed corresponding to the current node to be decompressed.

[0110] When performing geometric decompression, the multi-layer parent nodes of the same-layer nodes corresponding to the current node to be decompressed, the node occupancy codes and position information of the previous same-layer nodes, and the position information of the current node to be decompressed are used as the context information corresponding to the current node to be decompressed.

[0111] When decompressing attributes, the node occupancy code and position information of the multi-layer parent node of the same-layer node corresponding to the current node to be compressed, the node occupancy code and position information of the same-layer node, the node attributes and / or node attribute residuals corresponding to the previous same-layer node are used as the context information corresponding to the node to be decompressed.

[0112] The attention neural network for decompressing the point cloud is the same as the attention neural network for compressing the point cloud, and the specific method of generating the attention neural network will not be repeated here.

[0113] It should be noted that since the attention neural network models for compression and decompression are the same, the input context information is also the same, and thus the probability distribution corresponding to the predicted nodes to be decompressed obtained during compression is the same as the probability distribution corresponding to the predicted nodes to be decompressed obtained during decompression.

[0114] S403, inputting the probability distribution of the information to be decompressed corresponding to the current node to be decompressed and the point cloud code stream into the arithmetic decoder for entropy decoding to obtain the information to be decompressed corresponding to the current node to be decompressed.

[0115] When performing geometric decompression, the information to be decompressed refers to the node occupancy code, and the point cloud code stream refers to the geometric code stream; when performing attribute decompression, the information to be decompressed refers to the node attributes and / or node attribute residuals, and the attribute code stream refers to the attribute code stream.

[0116] S404: construct a tree structure with the information to be decompressed, obtain a point cloud from the tree structure, and then obtain a decompressed point cloud through inverse quantization.

[0117] The information to be decompressed is gradually decompressed to construct a tree structure, the point cloud is obtained from the tree structure, and then the decompressed point cloud is obtained through inverse quantization.

[0118] For a preferred embodiment, please refer to Figure 5 , Figure 5 The flowchart of a point cloud geometric decompression method provided in an embodiment of the present application is shown. The steps of a point cloud geometric decompression method are as follows:

[0119] S501. Obtain geometric context information corresponding to the current node to be decompressed, and obtain N1×K×3-dimensional geometric context information.

[0120] S502: Expand the N1×K×3-dimensional geometric context information through a one-hot encoding operation and an embedding operation to generate expanded-dimensional geometric context information, which is N1×K×C G The extended geometric context information of the dimension.

[0121] S503, N1×K×C G The expanded-dimensional geometric context information of the current node to be decompressed is input into the trained attention neural network model to predict the probability distribution of the node occupancy code.

[0122] S504: Input the probability distribution of the node occupancy code of the current node to be decompressed and the geometric code stream into the arithmetic decoder for entropy decoding to decode the node occupancy code of the current node to be decompressed.

[0123] S505: Determine whether the node occupancy codes of all nodes to be decompressed have been obtained.

[0124] If the node occupation codes of all nodes to be decompressed have not been obtained, the process returns to step S501.

[0125] If the node occupancy codes of all nodes to be decompressed are obtained (i.e., all nodes to be decompressed are decompressed), the node occupancy codes obtained by step-by-step decoding are constructed into a tree structure. The quantized point cloud is obtained from the constructed complete tree structure, and the quantized point cloud is dequantized to obtain the coordinates of the decompressed point cloud.

[0126] For a preferred embodiment, please refer to Figure 6 , Figure 6 The flowchart of a point cloud attribute decompression method provided in an embodiment of the present application is shown. The steps of a point cloud attribute decompression method are as follows:

[0127] S601, obtain the node attribute (node ​​attribute residual) context information corresponding to the current node to be decompressed, and obtain N2×8×N A Dimensional node attribute (node ​​attribute residual) context information.

[0128] S602, N2×8×N A The node attribute (node ​​attribute residual) context information of the dimension is expanded through the one-hot code operation and the embedding operation to generate the expanded node attribute (node ​​attribute residual) context information, which is N2×C A The context information of the expanded node attributes (node ​​attribute residuals) of the dimension.

[0129] S603, N2×C A The expanded node attributes (node ​​attribute residuals) context information and N2×K×C GThe expanded-dimensional geometric context information of the node is input into the trained attention neural network model to predict the probability distribution of the node attributes (node ​​attribute residuals) corresponding to the current node to be decompressed.

[0130] S604: Input the probability distribution of the node attribute (node ​​attribute residual) and the attribute code stream of the current node to be decompressed into the arithmetic decoder for entropy decoding, and decode the node attribute (node ​​attribute residual) of the node to be decompressed.

[0131] S605: Determine whether the node attributes (node ​​attribute residuals) of all nodes to be decompressed have been acquired.

[0132] If the node attributes (node ​​attribute residuals) of all nodes to be decompressed have not been obtained, the process returns to step S601.

[0133] If the node attributes (node ​​attribute residuals) of all nodes to be decompressed are obtained, the node attributes (node ​​attribute residuals) of the nodes to be decompressed are gradually decompressed to construct the node attributes (node ​​attribute residuals) of the leaf nodes of the tree structure. The node attributes (node ​​attribute residuals) are dequantized to reconstruct the point cloud attributes.

[0134] The attention neural network mentioned in this embodiment is a framework of deep learning. Fig.10 The operation described in formula (1) is neutralized. Similar networks named "Transformer" all belong to the category of attention neural network described in the present invention.

[0135] The embodiments of the present application also provide a point cloud compression device corresponding to the point cloud compression method provided in the above embodiments. Since the principle of solving the problem by the point cloud compression device in the embodiments of the present application is similar to that of the point cloud compression method in the above embodiments of the present application, the implementation of the point cloud compression device can refer to the implementation of the point cloud compression method, and the repeated parts will not be repeated.

[0136] See also Fig.11 , Fig.11A schematic diagram of a point cloud compression device provided by an embodiment of the present application is shown. The point cloud compression device 100 includes a first quantization module 101, a first determination module 102, a first prediction module 103 and a first compression module 104. Among them, the first quantization module 101 is used to binary encode the coordinates of the points of the point cloud through a tree structure, and convert the binary code into a decimal code as a node occupancy code corresponding to each node in the tree structure; the first determination module 102 is used to move a preset window to the nodes in the tree structure, and determine the uncompressed nodes in the preset window as the current nodes to be compressed; the first prediction module 103 is used to input the context information of the current node to be compressed into a pre-trained attention neural network model, and predict the probability distribution of the information to be compressed corresponding to the current node to be compressed; the first compression module 104 is used to input the probability distribution of the information to be compressed corresponding to the node to be compressed and the information to be compressed into the arithmetic encoder for entropy coding to obtain a point cloud code stream, and use the point cloud code stream as the compression result of the point cloud.

[0137] The embodiment of the present application also provides a point cloud decompression device corresponding to the point cloud decompression method provided in the above embodiment. Since the principle of solving the problem by the point cloud decompression device in the embodiment of the present application is similar to the point cloud decompression method in the above embodiment of the present application, the implementation of the point cloud decompression device can refer to the implementation of the point cloud decompression method, and the repeated parts will not be repeated.

[0138] Fig.12 A schematic diagram of a point cloud decompression device provided in an embodiment of the present application is shown. Fig.12 . The point cloud decompression device 200 includes a second determination module 201, a second prediction module 202, a first decompression module 203 and a second quantization module 204. Among them, the second determination module 201 is used to move the preset window of the tree structure, and determine the undecompressed node in the preset window each time as the current node to be decompressed; the second prediction module 202 is used to input the context information of the current node to be decompressed into the pre-trained attention neural network model to predict the probability distribution of the information to be decompressed corresponding to the current node to be decompressed; the first decompression module 203 is used to input the probability distribution of the information to be decompressed corresponding to the current node to be decompressed and the point cloud code stream to the arithmetic decoder for entropy decoding to obtain the information to be decompressed corresponding to the current node to be decompressed; the second quantization module 204 is used to construct a tree structure of the information to be decompressed, obtain the point cloud from the tree structure, and then obtain the decompressed point cloud through inverse quantization.

[0139] See also Fig.13 , Fig.13A schematic diagram of the structure of an electronic device 300 provided in an embodiment of the present application is shown, including: a processor 310, a memory 320 and a bus 330, the memory 320 stores machine-readable instructions executable by the processor 310, when the electronic device 300 is running, the processor 310 and the memory 320 communicate through the bus 330, and the machine-readable instructions are executed by the processor 310 when running, such as the steps of the point cloud compression method in the above-mentioned embodiment, and / or the steps of the point cloud decompression method.

[0140] Based on the same application concept, the embodiment of the present application also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by the processor, the steps of the point cloud compression method provided in the above embodiment and / or the steps of the point cloud decompression method are executed. Specifically, the storage medium can be a general storage medium, such as a mobile disk, a hard disk, etc. When the computer program on the storage medium is executed, the steps of the point cloud compression method and / or the steps of the point cloud decompression method can be executed. By using the node occupancy code and position information of the same-layer nodes and the father node corresponding to the node to be compressed as the context information of the node to be compressed, and considering the same-layer nodes and the father node of the node to be compressed, the technical problems of large amount of calculation and low compression efficiency in the prior art are solved, and the technical effect of improving the compression effect and reducing the amount of calculation is achieved.

[0141] Those skilled in the art can clearly understand that, for the convenience and simplicity of description, the specific working process of the system and device described above can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here. In the several embodiments provided in the present application, it should be understood that the disclosed system, device and method can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some communication interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0142] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0143] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0144] If the function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a non-volatile computer-readable storage medium that is executable by a processor. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk, etc., various media that can store program codes.

[0145] The above are only specific implementations of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A point cloud compression method, characterized in that: The method comprises: Binary code the coordinates of the points in the point cloud through the tree structure, and convert the binary code into a decimal code as a node occupancy code corresponding to each node in the tree structure; Moving a preset window for nodes in the tree structure, and determining an uncompressed node in the preset window as a current node to be compressed; Inputting the context information of the current node to be compressed into a pre-trained attention neural network model to predict the probability distribution of the information to be compressed corresponding to the current node to be compressed; According to the probability distribution of the information to be compressed corresponding to the node to be compressed and the information to be compressed, the information is input into the arithmetic encoder for entropy coding to obtain a point cloud code stream, and the point cloud code stream is used as the compression result of the point cloud; The context information of the current node to be compressed includes: the multi-layer father node of the node at the same layer corresponding to the current node to be compressed, the node occupancy code and position information of the previous node at the same layer, and the position information of the current node to be compressed; The multi-layer parent nodes of the same-layer nodes corresponding to the current node to be compressed, the node occupancy codes and position information of the previous nodes in the same layer, and the position information of the current node to be compressed are used as the context information corresponding to the node to be compressed, including: Using the node occupancy code and position information of the previous peer node of the current node to be compressed and the multi-layer parent node corresponding to the previous peer node as the first context information corresponding to the current node to be compressed; The position information of the current node to be compressed, the node occupancy code of the previous compressed node of the same layer of the current node to be compressed, and the node occupancy code and position information of the multi-layer father node corresponding to the current node to be compressed are used as the second context information corresponding to the current node to be compressed, Using the first context information and the second context information as context information corresponding to the current node to be compressed; The information to be compressed corresponding to the current node to be compressed includes: a node occupancy code of the current node to be compressed; The context information of the current node to be compressed also includes: node occupancy codes and position information of multi-layer father nodes of the same-layer nodes corresponding to the current node to be compressed, node occupancy codes and position information of the same-layer nodes, and node attributes and / or node attribute residuals corresponding to the previous same-layer nodes; The information to be compressed corresponding to the current node to be compressed further includes: node attributes and / or node attribute residuals of the current node to be compressed.

2. The compression method according to claim 1, characterized in that: The step of inputting the context information of the current node to be compressed into a pre-trained attention neural network model to predict the probability distribution of the information to be compressed corresponding to the current node to be compressed includes: Inputting the context information of the current node to be compressed into the first layer of the attention operation network of the attention neural network model to obtain a first weighted context matrix of the current node to be compressed; Adding the first weighted context matrix and the context information, and inputting the addition result into the first layer of the multi-layer perceptron network to obtain a second weighted context matrix; Then, the second weighted context matrix is ​​input into the second layer attention operation network to obtain a third weighted context matrix; Adding the third weighted context matrix to the second weighted context matrix, and inputting the addition result into the second layer multilayer perceptron network to obtain a fourth weighted context matrix; Inputting the fourth weighted context matrix into the third layer of the multi-layer perceptron network to obtain a fifth weighted context matrix; The fifth weighted context matrix is ​​passed through a softmax function to predict the probability distribution of the information to be compressed corresponding to the current node to be compressed.

3. The compression method according to claim 2, characterized in that: The context information of the current node to be compressed is input into the first layer of the attention operation network of the attention neural network model to obtain a first weighted context matrix of the current node to be compressed, including: Inputting the context information of the current node to be compressed into a first multi-layer perceptron to obtain a first output matrix; Inputting the context information of the current node to be compressed into a second multi-layer perceptron to obtain a second output matrix; Inputting the context information of the current node to be compressed into a third multi-layer perceptron to obtain a third output matrix; multiplying the transposed matrix of the second output matrix by the first output matrix to obtain a matrix inner product; Add the inner product of the matrix to the mask matrix, and input the result of the addition into the softmax function to obtain the attention matrix; The attention matrix is ​​multiplied by the third output matrix to obtain a first weighted context matrix of the current node to be compressed.

4. The compression method according to claim 3, characterized in that: Each element in the attention matrix is ​​an attention value; The attention value of the attention matrix is ​​calculated by the following formula: In formula (1), refers to the attention value between the jth node and the kth node context. The jth node is the jth node in the same layer and is the current node to be compressed. j is the context information of the jth node, f k is the context information of the kth node, and the kth node is the previous node of the jth node; molecular Represents the similarity value between the context of the jth node and the kth node, the denominator It refers to the sum of similarity values ​​between the j-th node and the contexts from the 1st node to the j-th node; MLP2(f j ) refers to inputting the context information of the j-th node into the second multilayer perceptron to obtain a second output matrix corresponding to the j-th node; It refers to inputting the context information of the k-th node into the first multi-layer perceptron to obtain the transposed matrix of the first output matrix corresponding to the k-th node.

5. The compression method according to claim 1, characterized in that: The step of inputting the context information of the current node to be compressed into a pre-trained attention neural network model to predict the probability distribution of the information to be compressed corresponding to the current node to be compressed includes: Expanding the context information of the current node to be compressed to generate expanded context information; The expanded context information is input into a pre-trained attention neural network model to predict the probability distribution of the information to be compressed corresponding to the current node to be compressed.

6. A point cloud decompression method, characterized in that: The method comprises: The preset window is moved for the tree structure, and the undecompressed node in the preset window is determined as the current node to be decompressed each time; Inputting the context information of the current node to be decompressed into a pre-trained attention neural network model to predict the probability distribution of the information to be decompressed corresponding to the current node to be decompressed; The probability distribution of the information to be decompressed and the point cloud code stream corresponding to the current node to be decompressed are input into the arithmetic decoder for entropy decoding to obtain the information to be decompressed corresponding to the current node to be decompressed; The information to be decompressed is constructed into a tree structure, the point cloud is obtained from the tree structure, and then the decompressed point cloud is obtained through inverse quantization; The context information of the current node to be decompressed includes: the multi-layer parent nodes of the same-layer nodes corresponding to the current node to be decompressed, the node occupancy code and position information of the previous same-layer nodes, and the position information of the current node to be decompressed; The multi-layer parent nodes of the same-layer nodes corresponding to the current node to be decompressed, the node occupancy codes and position information of the previous nodes in the same layer, and the position information of the current node to be decompressed are used as the context information corresponding to the node to be decompressed, including: Using the node occupancy code and position information of the previous peer node of the current node to be decompressed and the multi-layer parent node corresponding to the previous peer node as the first context information corresponding to the current node to be decompressed; The position information of the current node to be decompressed, the node occupancy code of the previous decompressed node of the same layer of the current node to be decompressed, and the node occupancy code and position information of the multi-layer parent node corresponding to the current node to be decompressed are used as the second context information corresponding to the current node to be decompressed, Using the first context information and the second context information as context information corresponding to the current node to be decompressed; The to-be-decompressed information corresponding to the current to-be-decompressed node includes: a node occupancy code of the current to-be-decompressed node; The context information of the current node to be decompressed also includes: node occupancy codes and position information of multi-layer parent nodes of the same-layer nodes corresponding to the current node to be decompressed, node occupancy codes and position information of the same-layer nodes, and node attributes and / or node attribute residuals corresponding to the previous same-layer nodes; The to-be-decompressed information corresponding to the current to-be-decompressed node further includes: node attributes and / or node attribute residuals of the current to-be-decompressed node.

7. An electronic device, characterized in that: include: A processor, a memory and a bus, wherein the memory stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor and the memory communicate through the bus, and when the machine-readable instructions are run by the processor, the steps of the point cloud compression method as described in any one of claims 1 to 5 are executed, and / or the steps of the point cloud decompression method as described in claim 6 are executed.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the computer program executes the steps of the point cloud compression method according to any one of claims 1 to 5, and / or the steps of the point cloud decompression method according to claim 6.

Images