Point cloud data compression method and device, electronic equipment and storage medium

Abstract

The invention provides a point cloud data compression method and device, electronic equipment and a storage medium, and relates to the technical field of computers. The point cloud data compression method comprises the following steps: acquiring a depth image of a target point cloud; performing quantization processing on a depth value in the depth image to obtain a quantization result of the depth image; and generating a compressed target point cloud based on the quantization result of the depth image. According to the method, lossless compression of the point cloud can be efficiently realized, so that the requirement of a low-delay lossless transmission scene of the point cloud data is met.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a point cloud data compression method, device, electronic device and storage medium. Background Art

[0002] As an important representation of three-dimensional spatial information, point cloud data has been widely used in autonomous driving, remote sensing mapping, virtual reality and other fields. However, point cloud data is usually massive and high-dimensional, which brings great challenges to storage and transmission. Therefore, point cloud compression technology has become a hot topic of research.

[0003] There are many point cloud compression technologies in related technologies. For example, the octree-based point cloud compression method converts point cloud data into an octree structure and uses the hierarchical and spatial segmentation characteristics of the octree to organize and encode point cloud data. However, most of the existing octree-based deep learning models use the Transformer architecture and large-scale context for prediction, resulting in high computational complexity and long reasoning time. Other point cloud compression methods also have a contradiction between compression efficiency and reasoning speed, and it is difficult to meet the needs of low-latency transmission scenarios of point cloud data.

[0004] Therefore, how to efficiently achieve lossless compression of point clouds to meet the needs of low-latency lossless transmission scenarios of point cloud data is a technical problem that needs to be solved urgently. Summary of the invention

[0005] In view of the above-mentioned problems existing in the prior art, the present invention provides a point cloud data compression method, device, electronic device and storage medium to efficiently realize lossless compression of point clouds and meet the needs of low-latency lossless transmission scenarios of point cloud data.

[0006] The present invention provides a point cloud data compression method, comprising the following steps.

[0007] Acquire a depth image of the target point cloud; quantize the depth values in the depth image to obtain a quantization result of the depth image; and generate a compressed target point cloud based on the quantization result of the depth image.

[0008] A method for compressing point cloud data provided by the present invention. The quantization process of the depth values in the depth image to obtain the quantization result of the depth image includes: quantizing the depth values in the depth image using a first quantization step size to obtain a first quantization result of the depth image; using a quantization step size prediction model to predict a second quantization step size based on the first quantization result, where the quantization step size prediction model is a trained machine learning model; quantizing the difference between each element in the depth image and its corresponding element in the first quantization result based on the second quantization step size to obtain a second quantization result; and obtaining the quantization result of the depth image according to the first quantization result and the second quantization result.

[0009] A method for compressing point cloud data provided by the present invention. The generation of the compressed target point cloud based on the quantization result of the depth image includes: extracting a latent vector of the quantization result of the depth image using a neural network; using a depth value probability prediction model to predict a first probability distribution of the depth values in the quantization result of the depth image based on the quantization result of the latent vector, where the depth value probability prediction model is a trained machine learning model for predicting the probability distribution of the depth values in the corresponding quantization result of the depth image according to the quantization result of the latent vector; performing entropy coding on the quantization result of the depth image according to the first probability distribution to obtain the compressed target point cloud; and performing entropy coding on the quantization result of the latent vector to obtain the entropy coding of the latent vector, so as to decode the compressed target point cloud based on a second probability distribution of the depth values in the quantization result of the depth image predicted using the entropy coding of the latent vector during the reconstruction process of the compressed target point cloud.

[0010] A compression method for point cloud data provided by the present invention, wherein the latent vector includes a first latent vector and a second latent vector, the neural network includes a first feature extraction network and a second feature extraction network, and the entropy coding includes a first entropy coding and a second entropy coding; the extracting the latent vector of the quantization result of the depth image by using the neural network includes: extracting the first latent vector from the quantization result of the depth image by using the first feature extraction network; extracting the second latent vector from the first latent vector by using the second feature extraction network; the predicting the first probability distribution of the depth values in the quantization result of the depth image by using the depth value probability prediction model according to the quantization result of the latent vector includes: predicting the first probability distribution by using the depth value probability prediction model according to the quantization result of the first latent vector; the performing entropy coding on the quantization result of the latent vector to obtain the entropy coding of the latent vector includes: predicting the probability distribution of each element value in the first latent vector by using the latent vector element value distribution probability prediction model according to the quantization result of the second latent vector; performing entropy coding on the quantization result of the first latent vector according to the probability distribution of each element value in the first latent vector to obtain the first entropy coding; performing entropy coding on the quantization result of the second latent vector to obtain the second entropy coding.

[0011] A method for compressing point cloud data provided by the present invention, wherein the depth value probability prediction model includes a first depth value probability prediction model, a second depth value probability prediction model, a third depth value probability prediction model, and a fourth depth value probability prediction model; the method further includes: based on the checkerboard context structure, dividing the first quantization result of the depth image into two parts to obtain a first context group and a second context group; based on the checkerboard context structure, dividing the second quantization result of the depth image into two parts to obtain a third context group and a fourth context group; the step of using the depth value probability prediction model to predict the first probability distribution of the depth values in the quantization result of the depth image according to the quantization result of the latent vector includes: using the first depth value probability prediction model to predict the third probability distribution of the depth values in the first context group according to the quantization result of the first latent vector; using the second depth value probability prediction model to predict the fourth probability distribution of the depth values in the second context group according to the quantization result of the first latent vector and the first context group; using the third depth value probability prediction model to predict the fifth probability distribution of the depth values in the third context group according to the quantization result of the first latent vector, the first context group, and the second context group; using the fourth depth value probability prediction model to predict the sixth probability distribution of the depth values in the fourth context group according to the quantization result of the first latent vector, the first context group, the second context group, and the third context group; and obtaining the first probability distribution according to the third probability distribution, the fourth probability distribution, the fifth probability distribution, and the sixth probability distribution.

[0012] The present invention provides a method for reconstructing point cloud data, including the following steps.

[0013] Obtain compressed point cloud; wherein, the compressed point cloud is generated according to the quantization result of the depth image of the target point cloud; decode the compressed point cloud to obtain the quantization result of the depth image of the target point cloud; and reconstruct the target point cloud based on the quantization result of the depth image of the target point cloud.

[0014] A method for reconstructing point cloud data according to the present invention, the quantization result of the depth image is obtained in the following manner: using a first quantization step size, quantizing the pixels in the depth image to obtain a first quantization result of the depth image; using a quantization step size prediction model, predicting a second quantization step size according to the first quantization result; wherein, the quantization step size prediction model is a trained machine learning model; based on the second quantization step size, quantizing the difference between each element in the depth image and its corresponding element in the first quantization result to obtain a second quantization result; and obtaining the quantization result of the depth image according to the first quantization result and the second quantization result.

[0015] The present invention also provides a point cloud data compression device, including the following modules: an acquisition module, configured to acquire a depth image of a target point cloud; a quantization module, configured to perform quantization processing on the depth values in the depth image to obtain a quantization result of the depth image; and a generation module, configured to generate the compressed target point cloud based on the quantization result of the depth image.

[0016] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the computer program, it implements the point cloud data compression method as described in any one of the above.

[0017] The present invention also provides a non-transitory computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, it implements the point cloud data compression method as described in any one of the above.

[0018] The present invention also provides a computer program product, including a computer program. When the computer program is executed by a processor, it implements the point cloud data compression method as described in any one of the above.

[0019] The point cloud data compression method provided by the present invention includes: acquiring a depth image of a target point cloud; performing quantization processing on the depth values in the depth image to obtain a quantization result of the depth image; and generating the compressed target point cloud based on the quantization result of the depth image. By converting the three-dimensional point cloud into a two-dimensional depth image and then compressing the point cloud based on the quantization result of the depth image, the complexity of the point cloud compression algorithm is effectively reduced, and the encoding and decoding speed of the point cloud compression algorithm is accelerated. Thus, lossless compression of the point cloud can be efficiently achieved, meeting the requirements of the lossless transmission scenario of point cloud data with low latency. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] To more clearly illustrate the technical solutions in the present invention or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the drawings in the following description are some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0021] Figure 1 It is a schematic flowchart of the method for compressing point cloud data provided by the present invention.

[0022] Figure 2 It is a schematic flowchart of the method for generating compressed target point cloud based on the quantization result of depth image provided by the present invention.

[0023] Figure 3 It is a schematic flowchart of the method for reconstructing point cloud data provided by the present invention.

[0024] Figure 4 It is a schematic flowchart of the method for training the machine learning model used in the compression and reconstruction process of point cloud data provided by the present invention.

[0025] Figure 5 It is a schematic diagram of the process of encoding and decoding point cloud data provided by the present invention.

[0026] Figure 6 It is a schematic diagram of the process of extracting and encoding latent vectors provided by the present invention.

[0027] Figure 7 It is a schematic diagram of the structure of the point cloud data compression device provided by the present invention.

[0028] Figure 8 It is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed implementation manners

[0029] To make the objectives, technical solutions, and advantages of the present invention clearer, the following will clearly and completely describe the technical solutions in the present invention with reference to the accompanying drawings in the present invention. Obviously, the described embodiments are some but not all of the embodiments of the present invention. Based on the embodiments in the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts belong to the scope of protection of the present invention.

[0030] The following will describe Figures 1-6 the method for compressing point cloud data of the present invention.

[0031] Figure 1 It is a schematic flowchart of the method for compressing point cloud data provided by the present invention. As Figure 1 shown, the method includes the following: Step 101: Obtain the depth image of the target point cloud.

[0032] The target point cloud is the point cloud to be compressed. The target point cloud may include, but is not limited to, radar point cloud, infrared point cloud, and lidar point cloud, etc.

[0033] In some embodiments, the depth image of the target point cloud can be a Range Image. A Range Image is a two-dimensional image generated based on the lidar scanning results. Each pixel in the image stores the distance (i.e., depth value) of a certain point cloud point from the lidar sensor. The abscissa of the pixel gives the azimuth angle of the point in the polar coordinate system, and the ordinate corresponds to the elevation angle of the point in the polar coordinate system.

[0034] In the specific implementation process, the target point cloud can be obtained in various ways according to the application scenario, not limited by the description in this specification. For example, during the driving of an autonomous vehicle, the on-vehicle lidar is used to continuously scan the surrounding environment to obtain the target point cloud data.

[0035] Step 102: Quantize the depth values in the depth image to obtain the quantization result of the depth image.

[0036] In the specific implementation process, the depth values in the depth image can be quantized in various ways to obtain the quantization result of the depth image.

[0037] In some embodiments, use the first quantization step size to quantize the depth values in the depth image to obtain the first quantization result of the depth image; use the quantization step size prediction model to predict the second quantization step size according to the first quantization result; where the quantization step size prediction model is a trained machine learning model; based on the second quantization step size, quantize the difference between each element in the depth image and its corresponding element in the first quantization result to obtain the second quantization result; according to the first quantization result and the second quantization result, obtain the quantization result of the depth image.

[0038] In the specific implementation process, the value of the first quantization step size can be determined according to experience or experimental results. For example, the first quantization step size can be 1, 2, etc., not limited by the description in this specification.

[0039] In the specific implementation process, the quantization step size prediction model can be constructed based on various ways. For example, the quantization step size prediction model can be constructed based on a convolutional neural network. Regarding the training method of the quantization step size prediction model, see Figure 4 the relevant content in, which will not be elaborated here.

[0040] Only as an example, the above quantization process in the embodiments can be implemented by the following formula.

[0041] (1) Among them, is the first quantization result; is the depth value with continuous values in the depth image of the target point cloud; The function is used for floor operation.

[0042] The following steps are realized through formula (1): Using the first quantization step with a value of 1, the depth values in the depth image are quantized to obtain the first quantization result of the depth image .

[0043] (2) (3) Among them,[[]] is the second quantization result, is the quantization step prediction model; is the predicted second quantization step; is the rounding operation, The operation means restricting the operation result within range.

[0044] The following steps are realized through formula (2): Using the quantization step prediction model, according to the first quantization result, the second quantization step is predicted.

[0045] The following steps are realized through formula (3): Based on the second quantization step, the difference between each element in the depth image and its corresponding element in the first quantization result is quantized to obtain the second quantization result.

[0046] Finally, the first quantization result and the second quantization result are used as the quantization result of the depth image.

[0047] In the embodiments provided by the present invention, through the use of a first quantization step size to perform preliminary quantization processing on the depth values in the depth image, a first quantization result of the depth image is obtained, thereby mapping continuous depth values to discrete numerical values; by using a quantization step size prediction model, a second quantization step size is predicted based on the first quantization result. Since the powerful inference and prediction capabilities of the machine learning model are fully utilized, the quantization step size can be dynamically adjusted according to the existing quantization results, thereby obtaining a second quantization step size that can effectively improve the quantization accuracy; based on the predicted second quantization step size, the difference between each element in the depth image and its corresponding element in the first quantization result is quantized to obtain a second quantization result. By quantizing the difference, the detailed information in the depth image can be effectively captured while maintaining a high compression ratio. Furthermore, the quantization result of the depth image that combines the first quantization result and the second quantization result can be used to efficiently complete the lossless compression of the target point cloud.

[0048] Step 103: Generate a compressed target point cloud based on the quantization result of the depth image.

[0049] In the specific implementation process, there are various ways to generate a compressed target point cloud based on the quantization result of the depth image, which is not limited by the description in this specification.

[0050] For an embodiment of generating a compressed target point cloud based on the quantization result of the depth image, refer to Figure 2 the relevant content therein, which will not be elaborated here.

[0051] Figure 2 is a schematic flowchart of a method for generating a compressed target point cloud based on the quantization result of the depth image provided by the present invention. As Figure 2 shown, the method includes the following: Step 201: Use a neural network to extract the latent vector of the quantization result of the depth image.

[0052] In the specific implementation process, a convolutional neural network can be used to extract the latent vector of the quantization result of the depth image.

[0053] In some embodiments, in order to further improve the compression efficiency, as Figure 5 shown, on the basis of extracting the latent vector of the quantization result of the depth image, the hyper prior of the latent vector can be extracted.

[0054] In the above embodiment, the latent vector includes a first latent vector (for example, Figure 5 the latent vector y in Figure 5 and a second latent vector (for example, Figure 6As shown, a first feature extraction network (e.g., using a convolutional neural network) can be utilized to extract a first latent vector from the quantization result of the depth image; a second feature extraction network (e.g., using a convolutional neural network) can be utilized to extract a second latent vector from the first latent vector.

[0055] In a specific implementation process, the first feature extraction network and the second feature extraction network can be constructed in various ways. For example, based on the convolutional neural network, the first feature extraction network and the second feature extraction network can be respectively constructed.

[0056] Regarding the training process of the first feature extraction network and the second feature extraction network, refer to Figure 4 the relevant content in [reference], which will not be elaborated here.

[0057] Step 202: Utilize the depth value probability prediction model to predict a first probability distribution of the depth values in the quantization result of the depth image according to the quantization result of the latent vector.

[0058] The depth value probability prediction model is a trained machine learning model, which is used to predict the probability distribution of the depth values in the quantization result of the corresponding depth image according to the quantization result of the latent vector.

[0059] In a specific implementation process, the depth value probability prediction model can be constructed in various ways. For example, the depth value probability prediction model can be constructed based on the convolutional neural network. The quantization result of the latent vector is used as the input of the depth value probability prediction model, and the depth value probability prediction model outputs a first probability distribution of the depth values in the predicted quantization result of the depth image.

[0060] In the embodiment including the second latent vector described in step 201, the depth value probability prediction model can be utilized to predict the first probability distribution according to the quantization result of the first latent vector.

[0061] In some embodiments, the depth value probability prediction model includes a first depth value probability prediction model, a second depth value probability prediction model, a third depth value probability prediction model, and a fourth depth value probability prediction model. Based on the checkerboard context structure, the first quantization result of the depth image is divided into two parts to obtain a first context group and a second context group; based on the checkerboard context structure, the second quantization result of the depth image is divided into two parts to obtain a third context group and a fourth context group.

[0062] Using the first depth value probability prediction model, based on the quantization result of the first hidden vector, the third probability distribution of the depth values in the first context group is predicted; using the second depth value probability prediction model, based on the quantization result of the first hidden vector and the first context group, the fourth probability distribution of the depth values in the second context group is predicted; using the third depth value probability prediction model, based on the quantization result of the first hidden vector, the first context group and the second context group, the fifth probability distribution of the depth values in the third context group is predicted; using the fourth depth value probability prediction model, based on the quantization result of the first hidden vector, the first context group, the second context group and the third context group, the sixth probability distribution of the depth values in the fourth context group is predicted; based on the third probability distribution, the fourth probability distribution, the fifth probability distribution and the sixth probability distribution, the first probability distribution is obtained.

[0063] In the specific implementation process, the first depth value probability prediction model, the second depth value probability prediction model, the third depth value probability prediction model and the fourth depth value probability prediction model can be constructed based on the convolutional neural network respectively.

[0064] In the specific implementation process, the quantization result of the hidden vector can be input into the first depth value probability prediction model, and the first depth value probability prediction model outputs the third probability distribution of the depth values in the first context group; the quantization result of the first hidden vector and the first context group are input into the second depth value probability prediction model, and the second depth value probability prediction model outputs the fourth probability distribution of the depth values in the second context group; the quantization result of the first hidden vector, the first context group and the second context group are input into the third depth value probability prediction model, and the third depth value probability prediction model outputs the fifth probability distribution of the depth values in the third context group; the quantization result of the first hidden vector, the first context group, the second context group and the third context group are input into the fourth depth value probability prediction model, and the third depth value probability prediction model outputs the sixth probability distribution of the depth values in the fourth context group; the third probability distribution, the fourth probability distribution, the fifth probability distribution and the sixth probability distribution are combined to obtain the first probability distribution.

[0065] Regarding the training process of the first depth value probability prediction model, the second depth value probability prediction model, the third depth value probability prediction model and the fourth depth value probability prediction model, see Figure 4 the relevant content in, which will not be elaborated here.

[0066] Step 203, according to the first probability distribution, perform entropy coding on the quantization result of the depth image to obtain the compressed target point cloud.

[0067] Entropy coding is a lossless compression method that assigns codes of different lengths according to the probability distribution of symbols. Symbols with higher probabilities are assigned shorter codes, while symbols with lower probabilities are assigned longer codes.

[0068] Based on the high-precision first probability distribution predicted by using a neural network, lossless compression of the target point cloud can be achieved with less code stream.

[0069] Step 204: Perform entropy coding on the quantization result of the latent vector to obtain the entropy coding of the latent vector, so that in the process of reconstructing the compressed target point cloud, based on the second probability distribution of the depth values in the quantization result of the depth image predicted by using the entropy coding of the latent vector, decode the compressed target point cloud.

[0070] In the embodiment including the second latent vector described in step 201, the entropy coding includes the first entropy coding and the second entropy coding. The entropy coding of the latent vector is implemented in the following manner.

[0071] Using the latent vector element value distribution probability prediction model, according to the quantization result of the second latent vector, predict the probability distribution of each element value in the first latent vector; according to the probability distribution of each element value in the first latent vector, perform entropy coding on the quantization result of the first latent vector to obtain the first entropy coding; use a probability model (such as the probability model of Huffman coding, etc.) or a coding table to perform entropy coding on the quantization result of the second latent vector to obtain the second entropy coding.

[0072] The latent vector element value distribution probability prediction model is a trained machine learning model, such as a trained convolutional neural network. Regarding the training process of the latent vector element value distribution probability prediction model, see Figure 4 the relevant content in, which will not be elaborated here.

[0073] Regarding the detailed description of decoding the compressed target point cloud based on the second probability distribution of the depth values in the quantization result of the depth image predicted by using the entropy coding of the latent vector in the process of reconstructing the compressed target point cloud, see Figure 3 the relevant content in, which will not be elaborated here.

[0074] In the embodiment provided by the present invention, first, a neural network is used to extract the latent vector of the depth image. Then, using the depth value probability prediction model, according to the quantization result of the latent vector, accurately and efficiently predict the first probability distribution of the depth values in the depth image. Based on this probability distribution, the entropy coding algorithm can accurately compress the depth image with less code stream, and finally obtain the compressed target point cloud. This process utilizes the powerful reasoning ability and data processing efficiency of the depth value probability prediction model to achieve low-latency lossless compression of the point cloud.

[0075] Figure 3It is a schematic flowchart of the method for reconstructing point cloud data provided by the present invention. As Figure 3 shown, the method includes the following: Step 301: Obtain compressed point cloud; wherein, the compressed point cloud is generated according to the quantization result of the depth image of the target point cloud.

[0076] In the specific implementation process, for different application scenarios, the compressed point cloud can be obtained in different ways.

[0077] Only as an example, in the application scenario of autonomous driving, during the driving process of an autonomous vehicle, the on-vehicle lidar will continuously scan objects such as roads, vehicles, and pedestrians, and generate a large amount of point cloud data. Using Figure 1 and Figure 2 the methods shown, after compressing these point cloud data, the compressed point cloud data is transmitted in real time to the central processing unit (CPU) or graphics processing unit (GPU) of the vehicle for decoding and reconstruction. Based on the reconstructed point cloud data, the vehicle can perform various processing and analysis tasks, such as identifying road signs, detecting obstacles, tracking other vehicles and pedestrians, etc., so as to provide accurate navigation and obstacle avoidance capabilities for the autonomous vehicle.

[0078] Regarding the generation process of the compressed point cloud, refer to the relevant content in Figure 1 and Figure 2 , which will not be elaborated here.

[0079] Step 302: Decode the compressed point cloud to obtain the quantization result of the depth image of the target point cloud.

[0080] In the specific implementation process, the compressed point cloud can be decoded based on the generation method of the compressed point cloud to obtain the quantization result of the depth image of the target point cloud.

[0081] In some embodiments, the compressed point cloud is obtained by performing entropy coding on the quantization result of the depth image according to the first probability distribution of the depth values in the quantization result of the depth image; the first probability distribution is predicted by using a depth value probability prediction model according to the quantization result of the latent vector; the latent vector is extracted from the quantization result of the depth image by using a neural network; during the generation of the compressed point cloud, the entropy coding of the quantization result of the latent vector is also generated. In this embodiment, the compressed point cloud is decoded in the following manner to obtain the quantization result of the depth image of the target point cloud.

[0082] According to the entropy coding of the quantization result of the latent vector and the probability model or coding table used when encoding the quantization result of the latent vector, decode the quantization result of the latent vector.

[0083] Using a depth value probability prediction model, according to the quantization result of the latent vector, a second probability distribution of depth values in the quantization result of the depth image is predicted.

[0084] According to the second probability distribution, entropy decoding is performed on the compressed point cloud to obtain the quantization result of the depth image of the target point cloud.

[0085] In some embodiments, the latent vector includes a first latent vector and a second latent vector. Entropy encoding includes first entropy encoding and second entropy encoding.

[0086] Among them, the first latent vector is extracted from the quantization result of the depth image by using a first feature extraction network; the second latent vector is extracted from the first latent vector by using a second feature extraction network.

[0087] Among them, the first entropy encoding is obtained by performing entropy encoding on the quantization result of the first latent vector according to the probability distribution of each element value in the first latent vector, and the probability distribution of each element value in the first latent vector is predicted according to the quantization result of the second latent vector by using a latent vector element value distribution probability prediction model; the second entropy encoding is obtained by performing entropy encoding on the quantization result of the second latent vector by using a probability model or a coding table. The first probability distribution is predicted according to the quantization result of the first latent vector by using a depth value probability prediction model.

[0088] In the above embodiments, the compressed point cloud is decoded in the following manner to obtain the quantization result of the depth image of the target point cloud: Using the probability model or coding table used when generating the second entropy encoding, the second entropy encoding is decoded to obtain the quantization result of the second latent vector.

[0089] Using the latent vector element value distribution probability prediction model, according to the quantization result of the second latent vector, the probability distribution of each element in the first latent vector is predicted.

[0090] Using the probability distribution of each element in the first latent vector, the first entropy encoding is decoded to obtain the quantization result of the first latent vector.

[0091] Using the depth value probability prediction model, according to the quantization result of the first latent vector, a second probability distribution of depth values in the quantization result of the depth image is predicted.

[0092] Using the second probability distribution, entropy decoding is performed on the compressed point cloud to obtain the quantization result of the depth image of the target point cloud.

[0093] In some embodiments, the quantization result of the depth image is obtained in the following manner: Quantize the pixels in the depth image using a first quantization step size to obtain a first quantization result of the depth image; use a quantization step size prediction model to predict a second quantization step size based on the first quantization result, where the quantization step size prediction model is a trained machine learning model; based on the second quantization step size, quantize the difference between each element in the depth image and its corresponding element in the first quantization result to obtain a second quantization result; and obtain the quantization result of the depth image according to the first quantization result and the second quantization result.

[0094] Based on the checkerboard context structure, divide the first quantization result of the depth image into two parts to obtain a first context group and a second context group; based on the checkerboard context structure, divide the second quantization result of the depth image into two parts to obtain a third context group and a fourth context group.

[0095] In the above embodiment, the depth value probability prediction model includes a first depth value probability prediction model, a second depth value probability prediction model, a third depth value probability prediction model, and a fourth depth value probability prediction model. The following method can be used to predict, using the depth value probability prediction model, a second probability distribution of depth values in the quantization result of the depth image based on the quantization result of the first latent vector.

[0096] Input the quantization result of the first latent vector into the first depth value probability prediction model, and the first depth value probability prediction model outputs a seventh probability distribution of depth values in the first context group; input the quantization result of the first latent vector and the predicted first context group into the second depth value probability prediction model, and the second depth value probability prediction model outputs an eighth probability distribution of depth values in the second context group; input the quantization result of the first latent vector, the first context group, and the second context group into the third depth value probability prediction model, and the third depth value probability prediction model outputs a ninth probability distribution of depth values in the third context group; input the quantization result of the first latent vector, the first context group, the second context group, and the third context group into the fourth depth value probability prediction model, and the third depth value probability prediction model outputs a tenth probability distribution of depth values in the fourth context group; combine the seventh probability distribution, the eighth probability distribution, the ninth probability distribution, and the tenth probability distribution to obtain the second probability distribution.

[0097] Step 303, reconstruct the target point cloud based on the quantization result of the depth image of the target point cloud.

[0098] In the specific implementation process, the depth image can be restored according to the quantization result of the depth image based on the generation method of the quantization result of the depth image of the target point cloud.

[0099] In the embodiment described in step 302, the first context group and the second context group can be combined based on the chessboard context structure to obtain a first quantization result; the third context group and the fourth context group can be combined to obtain a second quantization result. Then, after adding the corresponding elements in the first quantization result and the second quantization result, a depth image of the target point cloud is obtained.

[0100] Finally, based on the depth image of the target point cloud, the target point cloud is reconstructed.

[0101] Figure 4 is a schematic flowchart of a method for training a machine learning model used in the compression and reconstruction process of point cloud data provided by the present invention, as Figure 4 shown, the method includes the following steps.

[0102] Step 401, obtain a training data set.

[0103] In a specific implementation process, for different application scenarios (such as an autonomous driving scenario), point cloud data can be collected in different ways as sample data. Then, the lossless compressed point cloud data after compression and reconstruction of these sample data is obtained and used as the label corresponding to the sample data.

[0104] The training data set is composed of the sample data and its corresponding label.

[0105] Step 402, use the training data set to perform end-to-end training on the depth value probability prediction model, the latent vector element value distribution probability prediction model, the quantization step prediction model, the first feature extraction network, and the second feature extraction network.

[0106] In a specific implementation process, a loss function (for example, a cross-entropy loss function) can be established for the depth value probability prediction model (including the first depth value probability prediction model, the second depth value probability prediction model, the third depth value probability prediction model, and the fourth depth value probability prediction model), the latent vector element value distribution probability prediction model, the quantization step prediction model, the first feature extraction network, and the second feature extraction network.

[0107] In each round of training, the sample data is used as the target point cloud, and the method provided by the embodiment of the present invention is applied to compress and reconstruct the sample data, so as to obtain a predicted reconstructed point cloud. The loss function is used to evaluate the difference between the label corresponding to the sample data and the predicted reconstructed point cloud.

[0108] The model parameters are adjusted using a preset optimization algorithm (such as a gradient descent algorithm) until the joint loss function converges or reaches a preset number of training times, and the trained depth value probability prediction model, the latent vector element value distribution probability prediction model, and the quantization step prediction model are obtained.

[0109] The compression device for point cloud data provided by the present invention will be described below. The compression device for point cloud data described below can be correspondingly referred to the compression method for point cloud data described above.

[0110] Figure 7 It is a schematic structural diagram of the compression device for point cloud data provided by the present invention. As Figure 7 shown, the device 700 includes the following modules.

[0111] An acquisition module 710, configured to acquire a depth image of a target point cloud.

[0112] A quantization module 720, configured to perform quantization processing on the depth values in the depth image to obtain a quantization result of the depth image.

[0113] A generation module 730, configured to generate the compressed target point cloud based on the quantization result of the depth image.

[0114] Figure 8 Illustrates a schematic structural diagram of an electronic device. As Figure 8 shown, the electronic device may include: a processor 810, a communication interface 820, a memory 830, and a communication bus 840. Among them, the processor 810, the communication interface 820, and the memory 830 complete communication with each other through the communication bus 840. The processor 810 can call the logical instructions in the memory 830 to execute the compression method for point cloud data. The method includes: acquiring a depth image of a target point cloud; performing quantization processing on the depth values in the depth image to obtain a quantization result of the depth image; generating the compressed target point cloud based on the quantization result of the depth image.

[0115] In addition, when the logical instructions in the above-mentioned memory 830 are implemented in the form of software functional units and sold or used as an independent product, they can be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or a part of this technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of the present invention. The foregoing storage medium includes: various media such as a USB flash drive, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a magnetic disk, or an optical disc that can store program codes.

[0116] On the other hand, the present invention also provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the point cloud data compression method provided by each of the above methods. The method includes: obtaining a depth image of a target point cloud; performing quantization processing on the depth values in the depth image to obtain a quantization result of the depth image; and generating the compressed target point cloud based on the quantization result of the depth image.

[0117] In another aspect, the present invention also provides a non-transitory computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, it implements the point cloud data compression method provided by each of the above methods. The method includes: obtaining a depth image of a target point cloud; performing quantization processing on the depth values in the depth image to obtain a quantization result of the depth image; and generating the compressed target point cloud based on the quantization result of the depth image.

[0118] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed to multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment. Those of ordinary skill in the art can understand and implement it without creative efforts.

[0119] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solution, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. The computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disc, etc., and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0120] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements on some of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for compressing point cloud data, characterized in that, Including: Obtaining a depth image of a target point cloud; Performing quantization processing on the depth values in the depth image to obtain a quantization result of the depth image; Generating the compressed target point cloud based on the quantization result of the depth image.

2. The compression method of point cloud data according to claim 1, characterized in that, The performing quantization processing on the depth values in the depth image to obtain a quantization result of the depth image includes: Using a first quantization step size to perform quantization processing on the depth values in the depth image to obtain a first quantization result of the depth image; Using a quantization step size prediction model to predict a second quantization step size according to the first quantization result; wherein, the quantization step size prediction model is a trained machine learning model; Quantizing the difference between each element in the depth image and its corresponding element in the first quantization result based on the second quantization step size to obtain a second quantization result; Obtaining the quantization result of the depth image according to the first quantization result and the second quantization result.

3. The compression method of point cloud data according to claim 1 or 2, characterized in that, The generating the compressed target point cloud based on the quantization result of the depth image includes: Using a neural network to extract a latent vector of the quantization result of the depth image; Using a depth value probability prediction model to predict a first probability distribution of the depth values in the quantization result of the depth image according to the quantization result of the latent vector; wherein, the depth value probability prediction model is a trained machine learning model for predicting the probability distribution of the depth values in the corresponding quantization result of the depth image according to the quantization result of the latent vector; Performing entropy coding on the quantization result of the depth image according to the first probability distribution to obtain the compressed target point cloud; Performing entropy coding on the quantization result of the latent vector to obtain the entropy coding of the latent vector, so as to decode the compressed target point cloud based on a second probability distribution of the depth values in the quantization result of the depth image predicted by using the entropy coding of the latent vector during the reconstruction process of the compressed target point cloud.

4. The compression method of point cloud data according to claim 3, wherein The latent vector includes a first latent vector and a second latent vector, the neural network includes a first feature extraction network and a second feature extraction network, and the entropy coding includes a first entropy coding and a second entropy coding; the using a neural network to extract a latent vector of the quantization result of the depth image includes: Using the first feature extraction network to extract the first latent vector from the quantization result of the depth image; Using the second feature extraction network to extract the second latent vector from the first latent vector; The using a depth value probability prediction model to predict a first probability distribution of the depth values in the quantization result of the depth image according to the quantization result of the latent vector includes: Using the depth value probability prediction model to predict the first probability distribution according to the quantization result of the first latent vector; The performing entropy coding on the quantization result of the latent vector to obtain the entropy coding of the latent vector includes: Using a latent vector element value distribution probability prediction model to predict the probability distribution of each element value in the first latent vector according to the quantization result of the second latent vector; Entropy-encode the quantization result of the first hidden vector according to the probability distribution of each element value in the first hidden vector to obtain the first entropy encoding; Entropy-encode the quantization result of the second hidden vector to obtain the second entropy encoding.

5. The compression method of point cloud data according to claim 2, characterized in that, The depth value probability prediction model includes a first depth value probability prediction model, a second depth value probability prediction model, a third depth value probability prediction model, and a fourth depth value probability prediction model; the method further includes: Based on the checkerboard context structure, divide the first quantization result of the depth image into two parts to obtain a first context group and a second context group; Based on the checkerboard context structure, divide the second quantization result of the depth image into two parts to obtain a third context group and a fourth context group; The step of using the depth value probability prediction model to predict the first probability distribution of the depth values in the quantization result of the depth image according to the quantization result of the hidden vector includes: Use the first depth value probability prediction model to predict the third probability distribution of the depth values in the first context group according to the quantization result of the first hidden vector; Use the second depth value probability prediction model to predict the fourth probability distribution of the depth values in the second context group according to the quantization result of the first hidden vector and the first context group; Use the third depth value probability prediction model to predict the fifth probability distribution of the depth values in the third context group according to the quantization result of the first hidden vector, the first context group, and the second context group; Use the fourth depth value probability prediction model to predict the sixth probability distribution of the depth values in the fourth context group according to the quantization result of the first hidden vector, the first context group, the second context group, and the third context group; Obtain the first probability distribution according to the third probability distribution, the fourth probability distribution, the fifth probability distribution, and the sixth probability distribution.

6. A method for reconstructing point cloud data, characterized in that, Include: Obtain a compressed point cloud; wherein, the compressed point cloud is generated according to the quantization result of the depth image of the target point cloud; Decode the compressed point cloud to obtain the quantization result of the depth image of the target point cloud; Reconstruct the target point cloud based on the quantization result of the depth image of the target point cloud.

7. The method for reconstructing point cloud data according to claim 6, wherein, The quantization result of the depth image is obtained by the following method: Use a first quantization step to perform quantization processing on the pixels in the depth image to obtain the first quantization result of the depth image; Use a quantization step prediction model to predict a second quantization step according to the first quantization result; wherein, the quantization step prediction model is a trained machine learning model; Based on the second quantization step, perform quantization on the difference between each element in the depth image and its corresponding element in the first quantization result to obtain a second quantization result; Obtain the quantization result of the depth image according to the first quantization result and the second quantization result.

8. A compression device for point cloud data, characterized in that, Include: An acquisition module for acquiring the depth image of the target point cloud; A quantization module, configured to perform quantization processing on the depth values in the depth image to obtain a quantization result of the depth image; A generation module, configured to generate the compressed target point cloud based on the quantization result of the depth image.

9. An electronic device, comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, When the processor executes the computer program, the compression method of the point cloud data according to any one of claims 1 to 7 is implemented.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, the compression method of the point cloud data according to any one of claims 1 to 7 is implemented.

Images