Three-dimensional point cloud prediction geometric coding method based on deep learning

Abstract

The invention relates to a three-dimensional point cloud prediction geometric coding method based on deep learning, and the method comprises the steps: 1, forming a prediction tree through points collected by each laser transmitter, and converting an original laser radar point cloud LPC into a plurality of prediction trees for representation; 2, respectively designing different predictors and entropy encoders for each component of the coordinates of the points, and compressing each component of the coordinates of the points by applying different quantization step lengths; 3, selecting a quantization step size for quantifying each component by adopting a quantization step size selection strategy; 4, adopting an entropy model to model the probability distribution of the residual error of each component so as to encode the residual error entropy; and step 5, decoding is realized through a reverse process from the step 1 to the step 4. Compared with other methods, the method provided by the invention obtains the best rate distortion performance.

Description

Technical Field

[0001] This invention relates to a deep learning-based method for predicting geometric codes for 3D point clouds, belonging to the field of image processing technology. Background Technology

[0002] LiDAR is a sensor that measures distance by emitting laser beams and detecting their reflections. It is currently widely used in autonomous driving, robot navigation, and geographic information systems. LiDAR incorporates multiple laser emitters with different elevation angles, scanning horizontally at a predefined azimuth resolution and calculating the distance between objects and the LiDAR based on their reflections. It then converts the azimuth, elevation, and measured distance of all points into Cartesian coordinates, forming a LiDAR point cloud (LPC). Due to the massive data volume of LPC, efficient compression techniques are urgently needed to reduce the storage and transmission costs of LPC.

[0003] In recent years, the Moving Picture Experts Group (MPEG) has released the Geometry-Based Point Cloud Compression (G-PCC) standard, which includes two LPC geometric information coding methods: octree coding and predictive geometry coding. The predictive geometry coding method first transforms the point cloud into a spherical coordinate system. Then, it connects points acquired by the same laser emitter in ascending order of azimuth angle to construct a prediction tree. Finally, it predicts the coordinates of each point sequentially starting from the root point and entropy-encodes the residuals. On the one hand, LiDAR has a fixed azimuth resolution, allowing the azimuth angle of the current point to be predicted based on the azimuth angle of the previous point. On the other hand, points from the same laser emitter have similar elevation angles. These characteristics make the azimuth and elevation angles of the prediction tree structure highly efficient for coding. However, the predictive geometry coding method only uses a simple linear model based on adjacent points for prediction, ignoring the geometric correlation between long-distance points within the prediction tree.

[0004] In our previous work, we proposed a learning-based predictive coding method (LPCM), which represents LPC as a prediction tree before compression. Specifically, LPCM uses an incremental coding (DC) approach to compress azimuth and radius, and proposes a learning-based prediction module to minimize the error between the actual and theoretical pitch angles. We then analyze the impact of distortion in the radius, pitch, and azimuth components on overall geometric distortion and propose a quantization step size selection strategy. However, LPCM still has some limitations. First, similar to predictive geometric coding, LPCM uses a simple linear model based on neighboring points to predict geometric coordinates. Since the radius of a point is determined only by the distance between the lidar and the object, lacking prior information, the coding efficiency is low. Second, the quantization step size selection strategy in LPCM is based solely on peak signal-to-noise ratio (PSNR) to select the optimal quantization step size, making it difficult to achieve optimal global rate-distortion performance. Third, LPCM uses a general context-adaptive binary arithmetic encoder (CABAC) to compress the residuals, without specific optimization for the characteristics of spherical coordinate residuals. Summary of the Invention

[0005] To address the shortcomings of existing technologies, this invention proposes a learning-based LiDAR point cloud prediction geometry coding method. LiDAR (Light Detection and Ranging) systems typically acquire point cloud data by scanning the surrounding environment at a predefined angular resolution. This acquisition characteristic allows for the regular representation and efficient compression of point clouds. In recent years, point cloud compression methods based on spherical coordinates have demonstrated extremely high compression efficiency. However, deep learning-based spherical coordinate compression methods do not utilize inter-frame correlation to eliminate geometric redundancy, resulting in a high bit rate for the compression radius. Furthermore, while the predictive geometry coding method in the Geometry-Based Point Cloud Compression (G-PCC) standard provides inter-frame patterns, it only predicts the spherical coordinates of the current point using a simple linear model, without utilizing the spatial relationships of points within a prediction tree structure to further improve prediction accuracy.

[0006] To further improve point cloud compression efficiency, this invention proposes a learning-based inter-frame predictive coding method (Inter-LPCM). In the prediction stage, differential coding is used to predict the azimuth angle, taking into account the azimuth resolution of the point cloud. To improve radius compression efficiency, an inter-frame radius prediction (Inter-RP) model is proposed, which uses neighborhood points in the current frame and the registered reference frame to predict the radius of the current point. Simultaneously, a lightweight attention prediction (LAEP) model is proposed, which predicts the pitch angle based on the spatial relationships between points. In the quantization stage, a rate-distortion optimization-based quantization step size selection method is designed to determine the optimal quantization step size in the spherical coordinate system. In the entropy coding stage, different entropy models are designed based on the statistical priors of each component of the spherical coordinate system, thereby achieving more accurate probability estimation and more efficient entropy coding.

[0007] Terminology Explanation: The LSTM network module is a type of time-recurrent neural network, specifically designed to address the long-term dependency problem inherent in general recurrent neural networks.

[0008] The technical solution of this invention is as follows: Deep learning-based geometric coding methods for predicting 3D point clouds include: Step 1: A prediction tree is constructed from the points collected by each laser emitter, thereby transforming the original LiDAR point cloud LPC into multiple prediction trees for representation. Step 2: For each component of the point's coordinates, design different predictors and entropy encoders, and apply different quantization step sizes to compress each component of the point's coordinates; specifically including: For azimuth, the azimuth resolution of the lidar is used to predict the azimuth by representing the increment between the azimuth of the current point and the azimuth of the previous point. For pitch angle, the Latitude and Longitude Prediction (LAEP) model based on attention mechanism and LSTM network predicts the pitch angle of the current point by utilizing the correlation between neighboring points and the correlation between radius and pitch angle; it introduces time information to improve the compression efficiency of radius; specifically including: Whether the current frame should use intra-frame coding (I-frame) is determined by calculating the similarity between the current frame and the previous frame; for I-frames, a DC-based method is used to compress the radius; for predictive coded frames (P-frames), the transformation matrix between the current frame and the reference frame is calculated and encoded, and the reference frame is registered to the current frame. For the radius, the radius of the current point is predicted using the proposed inter-frame radius prediction model (Inter-RP model) by utilizing the neighboring points in the current frame and the registered R frame; Step 3: Use a quantization step size selection strategy to select the quantization step size for each component; Step 4: Use an entropy model to model the probability distribution of the residuals for each component, thereby encoding the residual entropy; Step 5: Decoding is achieved by reversing the process from Step 1 to Step 4.

[0009] According to a preferred embodiment of the present invention, a prediction tree is constructed from the points collected by each laser emitter, thereby transforming the original lidar point cloud (LPC) into multiple prediction trees for representation; including: First, point cloud The process of transforming the coordinates of all points from the Cartesian coordinate system to the spherical coordinate system is expressed as: ; in, Represents the Cartesian coordinates of a point. Represents the spherical coordinates of a point. Indicates pitch angle, Indicates azimuth. Indicates radius, id This indicates the number (LarseID) of the laser that captured the data at this point. Subsequently, having the same The points will be grouped together, and within each group, all points will be arranged in ascending order of azimuth angle, thus constructing a prediction tree. .

[0010] According to a preferred embodiment of the present invention, during encoding, the prediction tree starts from the root node, i.e., the point with the smallest azimuth angle, and predicts the coordinates of the next point one by one, and entropy encoding the predicted residuals; within the same point, the azimuth angle is encoded first, then the radius is encoded, and finally the pitch angle is encoded.

[0011] A further preferred option is azimuth encoding; including: When encoding the azimuth of a point cloud, the azimuth of each point is... Represented as: ; in, It is an integer. This indicates the set unit azimuth angle. Set the azimuth resolution of the lidar , This is the preset quantization step size; Differential coding is Encoding via entropy model; It is an integer, based on the azimuth angle. To obtain is Integer representation, Immediately Integer representation of the azimuth angle; Output bias coefficients using the entropy model mean and standard deviation This allows us to model a biased normal distribution (Skew normal distribution) and obtain the result through the integral of the probability density function. The probability mass function, thus affecting Entropy coding.

[0012] A further preferred option is radius encoding; including: The radius is compressed through four stages: segmentation, registration, prediction, quantization, and entropy coding; specifically including: During the segmentation phase, the current frame is segmented according to the laser number at each point. With the reconstructed reference frame Divided into upper half and the lower half ; During the registration stage, the first step is to evaluate the upper half of the current frame. With the upper half of the reference frame The similarity is used to determine whether the current frame is an I-frame or a P-frame; if it is an I-frame, it is directly... Represented as a prediction tree If it is a P-frame, the upper half of the reconstructed reference frame is calculated. With the upper half of the current frame The transformation matrix between them, thus making Registration ; During the prediction phase, for all points in the I-frame, and The radius of all points is predicted using a differential coding method; for Then the Inter-RP model is used for prediction; In the quantization and entropy coding stage, the predicted residuals are quantized and entropy coding is performed using an entropy model.

[0013] Further preferred, during the registration stage, by calculating the upper half of the current frame With the upper half of the reference frame The PSNR is used to determine whether the current frame is an I-frame or a P-frame. If the PSNR is greater than the set threshold, it is a P-frame; otherwise, it is an I-frame. If it is an I-frame, directly... Represented as a prediction tree If it is a P-frame, first calculate the upper half of the reconstructed reference frame using the ICP algorithm. With the upper half of the current frame The transformation matrix between them will Registration Next, after registration , as well as These are respectively represented as sets of prediction trees. , as well as .

[0014] Further optimization, in the prediction phase, for as well as Predicted value of the radius at the midpoint Obtained directly through incremental encoding DC: ; in, Point The reconstructed value of the radial distance; for The radius is then predicted using the Inter-RP model. The Inter-RP model consists of three LSTM network modules, an attention layer, and a multilayer perceptron (MLP). The inputs to the Inter-RP model are respectively from as well as The already encoded part Selected time neighborhood Space Neighbors ,as well as Corresponding prediction residuals ;for First of all Find three prediction trees within the range of LaserID plus or minus 1 for the current point. Then, find the point in each prediction tree whose azimuth is closest to the current point. Finally, take this point, along with the 24 points preceding it and the 25 points following it in the prediction tree sequence, to form a 50×3 point set. ;for ,from Take the prediction tree with the same Laser ID as the current point, and construct it from the 50 previously encoded points before the current point. ; Then it is The residual of each point entropy encoding; The output of the Inter-RP model is the predicted value of the radius; After completing the prediction, point The prediction residuals and quantization residuals of the radial distance are: ; ; in, The predicted residuals of the radial distance of the point cloud. For the quantization step size of radial distance, To quantify the residual; then the point Reconstructed value of radial distance Represented as: ; Finally, the entropy model is used to analyze the set of quantization residuals. Perform entropy encoding.

[0015] According to a preferred embodiment of the present invention, pitch angle encoding includes: Pitch angle prediction model based on attention mechanism and LSTM network; First, by leveraging the LSTM network's sensitivity to the order of the input sequence, spatial correlations within the prediction tree are modeled. Then, when encoding points within a prediction tree When determining the pitch angle, the already encoded points arrive The reconstruction information is input into the LSTM network, and the reconstruction information for each point includes... , They are Azimuth angle reconstruction value, radius reconstruction value, laser number, yes The reconstructed pitch angle value; Furthermore, the attention mechanism is used to model the relationship between different coordinate axes of the point cloud, thereby weighting the features output by the LSTM layer; Finally, the weighted features are aggregated by a multilayer perceptron (MLP) to output the predicted pitch angle. Then point The prediction residuals and quantization residuals of the pitch angle are: ; ; in, The prediction residual for the pitch angle, For the quantization step size of radial distance, To quantify the residual; then the point The reconstructed pitch angle is expressed as: ; right and the set of quantized residuals of pitch angle Perform entropy encoding.

[0016] A further optimized pitch angle prediction model based on an attention mechanism and an LSTM network calculates predicted values ​​during training. Compared with the true value The loss function is constructed using the MSE loss between the two values.

[0017] According to a preferred embodiment of the present invention, a quantization step size selection strategy is employed to select the quantization step size for each component; including: The problem of selecting encoding parameters is transformed into an optimization problem under constraints. Approximately optimal encoding parameters are obtained through iterative optimization on a small dataset containing only 20 point clouds. Other point clouds are then encoded using these approximately optimal encoding parameters. First, the problem of choosing encoding parameters is transformed into an optimization problem of minimizing the BD-rate under bit rate constraints: ; ; in, For the set of quantization parameters, , , This represents the set of reference quantization step sizes. , The upper and lower bounds of the set bit rate, N The number of point clouds; For the above optimization problem, the approximate optimal quantization step size is solved by the differential evolution algorithm.

[0018] Further preferably, when selecting the optimal quantization step size at 6 bit rates, the approximate optimal quantization step size is solved using a differential evolution algorithm; including: Step 1: Population initialization; The initial population is represented as: ; In the formula, Represents the initial population. The first term of the population k Individual; Based on empirical values, the following settings are made: ; ; After population initialization, the quantization step size at different code rates in each individual will be used for encoding a small dataset consisting of 20 point clouds, thereby obtaining the 6 bit rates and corresponding distortions for encoding this small dataset, and thus calculating the individual's... The fitness function of an individual is obtained by comparing its encoding performance with the BD-Rate; for individuals whose bit rate exceeds a preset range, their fitness function is assigned a value of positive infinity. Subsequently, the population undergoes mutation, crossover, and selection operations, and the next generation of the population is selected based on the fitness function.

[0019] After several rounds of iteration, the optimal set of quantization step sizes is obtained. .

[0020] The beneficial effects of this invention are as follows: Compared with other methods, the method proposed in this invention achieves the best rate-distortion performance. Experimental results on the SemanticKITTI and Ford datasets demonstrate that the proposed method achieves optimal compression performance. Attached Figure Description

[0021] Figure 1 This is an implementation architecture diagram of the deep learning-based 3D point cloud prediction geometry encoding method of the present invention; Figure 2 This is a statistical probability diagram of the incremental coding (DC) of azimuth angle; Figure 3 This is a schematic diagram of the entropy model proposed in this invention; Figure 4 This is a schematic diagram of the radius encoding structure; Figure 5 This is a schematic diagram of the pitch angle prediction model based on LSTM network and attention mechanism of the present invention; Figure 6 This is a schematic diagram comparing the rate-distortion performance of the method of the present invention with other methods. Detailed Implementation

[0022] The present invention will be further defined below with reference to the accompanying drawings and embodiments, but is not limited thereto.

[0023] Example 1 Deep learning-based 3D point cloud prediction geometric coding method, such as Figure 1 As shown, it includes: Step 1: A prediction tree is constructed from the points collected by each laser emitter, thereby transforming the original LiDAR point cloud LPC into multiple prediction trees for representation. Step 2: For each component of the point's coordinates, design different predictors and entropy encoders, and apply different quantization step sizes to compress each component of the point's coordinates; specifically including: For azimuth, the azimuth resolution of the lidar is used to predict the azimuth by representing the increment between the azimuth of the current point and the azimuth of the previous point. For pitch angle, the Latitude and Longitude Prediction (LAEP) model based on attention mechanism and LSTM network predicts the pitch angle of the current point by utilizing the correlation between neighboring points and the correlation between radius and pitch angle. Due to the lack of prior information, the bit rate of compressed radius is significantly higher than that of azimuth and pitch angle. Therefore, time information is introduced to improve the compression efficiency of radius; specifically including: Whether the current frame should use intra-frame coding (I-frame) is determined by calculating the similarity between the current frame and the previous frame; for I-frames, a DC-based method is used to compress the radius; for predictive coded frames (P-frames), the transformation matrix (T matrix) between the current frame and the reference frame is calculated and encoded, and the reference frame (R-frame) is registered to the current frame. For the radius, the radius of the current point is predicted using the proposed inter-frame radius prediction model (Inter-RP model) by utilizing the neighboring points in the current frame and the registered R frame; Step 3: Use a quantization step size selection strategy to select the quantization step size for each component; Step 4: Use an entropy model to model the probability distribution of the residuals for each component, thereby encoding the residual entropy; Step 5: Decoding is achieved by reversing the process from Step 1 to Step 4.

[0024] Example 2 The difference between the deep learning-based 3D point cloud prediction geometric coding method described in Example 1 and the following is: Each point collected by a laser emitter forms a prediction tree, thus transforming the original LiDAR point cloud (LPC) into multiple prediction trees; including: First, point cloud The process of transforming the coordinates of all points from the Cartesian coordinate system to the spherical coordinate system is expressed as: ; in, Represents the Cartesian coordinates of a point. Represents the spherical coordinates of a point. Indicates pitch angle, Indicates azimuth. Indicates radius, id This indicates the number (LarseID) of the laser that captured the data at this point. Subsequently, having the same The points will be grouped together, and within each group, all points will be arranged in ascending order of azimuth angle, thus constructing a prediction tree. .

[0025] During encoding, the prediction tree starts from the root node, which is the point with the smallest azimuth angle, and predicts the coordinates of the next point one by one. The prediction residuals are then quantized and entropy encoded. Within the same point, the azimuth angle is encoded first, then the radius, and finally the elevation angle.

[0026] Azimuth encoding; including: When encoding the azimuth of a point cloud, the azimuth of each point is... Represented as: ; in, It is an integer. This indicates the set unit azimuth angle. Set the azimuth resolution of the lidar , This is the preset quantization step size; the advantage of this representation is that it utilizes the azimuth resolution of the lidar to process the difference between azimuth angles into easily encoded integers.

[0027] Differential coding is Encoding via entropy model; It is an integer, based on the azimuth angle. To obtain is Integer representation of azimuth is a way to represent azimuth angles and can improve coding efficiency. Immediately Integer representation of the azimuth angle;

[0028] Specifically, such as Figure 2 As shown, Figure 2 In the middle, the horizontal axis is... The value is represented by two lines of different colors, which are the probability density function curves of the biased normal distribution and the normal distribution, respectively; the acquisition characteristics of the lidar and the arrangement order of the prediction tree determine the probability density function of the normal distribution. Generally, values ​​greater than 0 and greater than or equal to the angular resolution of the lidar do not conform to a symmetrical distribution. Therefore, as... Figure 3 As shown, the bias coefficient is output through the entropy model. mean and standard deviation This allows us to model a biased normal distribution (Skew normal distribution) and obtain the result through the integral of the probability density function. The probability mass function, thus affecting Entropy coding.

[0029] The entropy model is composed of a neural network model and outputs bias coefficients. mean and standard deviation This yields the probability density function of the biased normal distribution (it's just a function, including the bias coefficients). mean and standard deviation With three variables, we can obtain the probability table for entropy encoding through this probability density function, thus completing the entropy encoding. A schematic diagram of the entropy model is shown below. Figure 2 Given the encoded residual as input, the last layer of the entropy model (neural network) outputs three numbers, which are considered as bias coefficients. mean and standard deviation These three parameters are the variables of the probability density function of the biased normal distribution. The probability mass function is obtained by calculating the integral of the probability density function.

[0030] Radius encoding; including: like Figure 4 As shown, radius compression is achieved through four stages: segmentation, registration, prediction, quantization, and entropy coding; specifically including: During the segmentation phase, the current frame is segmented according to the laser number at each point. With the reconstructed reference frame Divided into upper half and the lower half ; During the registration stage, the first step is to evaluate the upper half of the current frame. With the upper half of the reference frame The similarity is used to determine whether the current frame is an I-frame or a P-frame; if it is an I-frame, it is directly... Represented as a prediction tree If it is a P-frame, the upper half of the reconstructed reference frame is calculated. With the upper half of the current frame The transformation matrix between them, thus making Registration ; During the prediction phase, for all points in the I-frame, and The radius of all points is predicted using a differential coding method; for Then the Inter-RP model proposed in this paper is used for prediction; In the quantization and entropy coding stage, the predicted residuals are quantized and entropy coding is performed using an entropy model.

[0031] During the segmentation phase, such as Figure 4As shown, laser emitters with larger pitch angles can acquire points from farther distances, while those with smaller pitch angles primarily capture ground points closer to the lidar. This is because these ground points have similar radii; furthermore, in the world coordinate system, static objects such as buildings and trees maintain consistent positions in both the reference and current frames. However, due to the lidar's motion, these ground points originate from different locations, hindering accurate registration between the reference and current frames. Therefore, we divide the current frame into an upper and lower half based on the Laser ID and encode each half separately.

[0032] During the registration stage, the upper half of the current frame is calculated. With the upper half of the reference frame The PSNR between the two frames determines whether the current frame is an I-frame or a P-frame. PSNR is a commonly used metric for calculating similarity in the coding field. When the PSNR is greater than a set threshold, it is a P-frame; otherwise, it is an I-frame. If it is an I-frame, directly... Represented as a prediction tree If it is a P-frame, first calculate the upper half of the reconstructed reference frame using the ICP algorithm. With the upper half of the current frame The transformation matrix between the two sides, the ICP algorithm is a very common method for calculating the transformation matrix, and it will... Registration Next, after registration , as well as These are respectively represented as sets of prediction trees. , as well as .

[0033] In the prediction phase, for as well as Predicted value of the radius at the midpoint It can be obtained directly through incremental coding DC: The prediction method of incremental coding DC is very simple, which is to use the value of the previous point as the predicted value of the current point; ; in, Point The reconstructed value of the radial distance; for The radius is then predicted using the Inter-RP model. The Inter-RP model consists of three LSTM network modules, an attention layer, and a multilayer perceptron (MLP). The inputs to the Inter-RP model are respectively from as well as The already encoded part Selected time neighborhood Space Neighbors ,as well as Corresponding prediction residuals ;for First of all Find three prediction trees within the range of LaserID plus or minus 1 for the current point. Then, find the point in each prediction tree whose azimuth is closest to the current point. Finally, take this point, along with the 24 points preceding it and the 25 points following it in the prediction tree sequence, to form a 50×3 point set. ;for ,from Take the prediction tree with the same Laser ID as the current point, and construct it from the 50 previously encoded points before the current point. ; Then it is The residual of each point entropy encoding; The output of the Inter-RP model is the predicted value of the radius; After completing the prediction, point The prediction residuals and quantization residuals of the radial distance are: ; ; in, The predicted residuals of the radial distance of the point cloud. For the quantization step size of radial distance, To quantify the residual; then the point Reconstructed value of radial distance Represented as: ; Finally, an entropy model similar to that described above is used to analyze the set of quantization residuals. Entropy encoding is performed. It's important to note that since the radius residual does not have bias characteristics, the only difference between the entropy model used here, and the entropy model used later to encode the pitch angle residual, and the entropy model described above, is that the entropy model here only outputs the mean. and standard deviation This allows us to model a normal distribution.

[0034] Pitch angle encoding; including: like Figure 5 As shown, this invention proposes a pitch angle prediction model based on LSTM networks and an attention mechanism. The pitch angle prediction model based on the attention mechanism and LSTM network predicts the pitch angle;

[0035] First, the spatial correlation within the prediction tree is modeled by utilizing the LSTM network's sensitivity to the order of the input sequence. During the training phase of the neural network, the LSTM network's sensitivity to the order of the input sequence is used to model the spatial correlation within the prediction tree.

[0036] Then, when encoding points within a prediction tree When determining the pitch angle, the already encoded points arrive The reconstruction information is input into the LSTM network, and the reconstruction information for each point includes... , They are Azimuth angle reconstruction value, radius reconstruction value, laser number, yes The reconstructed pitch angle value; Furthermore, the attention mechanism is used to model the relationship between different coordinate axes of the point cloud, thereby weighting the features output by the LSTM layer; Finally, the weighted features are aggregated by a multilayer perceptron (MLP) to output the predicted pitch angle. Then point The prediction residuals and quantization residuals of the pitch angle are: ; ; in, The prediction residual for the pitch angle, For the quantization step size of radial distance, To quantify the residual; then the point The reconstructed pitch angle is expressed as: ; right and the set of quantized residuals of pitch angle Perform entropy encoding.

[0037] The pitch angle prediction model based on the attention mechanism and LSTM network calculates the predicted values ​​during training. Compared with the true value The loss function is constructed using the MSE loss between the two values.

[0038] A quantization step size selection strategy is used to select the quantization step size for each component; including: In the predictive coding process, there are coexisting , , Several parameters affect the rate-distortion performance of point cloud coding. In the quantization of coordinate residuals in Cartesian coordinates, residuals originating from different coordinate axes can be quantized using the same quantization step size because the residuals of different coordinate axes in Cartesian coordinates have the same impact on the distortion of the reconstructed point cloud. However, since the physical meaning of each coordinate axis is different in spherical coordinates, the impact of distortion in each coordinate axis direction in spherical coordinates on the global distortion after being mapped back to Cartesian coordinates is also different. In Cartesian coordinates, when modeling the relationship between quantization parameters, bit rate, and distortion, only one quantization step size is used as the dependent variable. However, in spherical coordinates, three quantization step sizes are involved, as well as nonlinear transformations between different coordinate systems, making it difficult to select the optimal quantization step size through rate-distortion curve modeling.

[0039] Therefore, this scheme proposes a quantization step size selection method based on differential evolution, which transforms the problem of selecting encoding parameters into an optimization problem under constraints. It obtains the near-optimal encoding parameters through iterative optimization on a small dataset containing only 20 point clouds, and encodes other point clouds using the near-optimal encoding parameters. First, the problem of choosing encoding parameters is transformed into an optimization problem of minimizing the BD-rate under bit rate constraints: ; ; in, For the set of quantization parameters, , , This represents the set of reference quantization step sizes. This set is designed only to calculate the BD-rate and has no impact on the final optimization result. , The upper and lower bounds of the set bit rate, N The number of point clouds;

[0040] For the above optimization problem, the approximate optimal quantization step size is solved by the differential evolution algorithm.

[0041] When selecting the optimal quantization step size at 6 bit rates, the approximate optimal quantization step size is found using a differential evolution algorithm; including: Step 1: Population initialization; The initial population is represented as: ; In the formula, Represents the initial population. The first term of the population k Individual; To improve the solution efficiency of the DE algorithm, based on empirical values, the following settings are made: ; ; After population initialization, the quantization step size at different code rates in each individual will be used for encoding a small dataset consisting of 20 point clouds, thereby obtaining the 6 bit rates and corresponding distortions for encoding this small dataset, and thus calculating the individual's... The BD-Rate between the encoding performance and the fitness function of an individual is obtained; using the BD-Rate as the fitness function, the fitness function is determined based on the 6 bit rates and their corresponding distortions. After determining the encoding performance, input it into the BD-Rate calculation formula. BD-Rate is a commonly used calculation metric in this field. For individuals whose bit rate exceeds the preset range, their fitness function is assigned a value of positive infinity. Subsequently, similar to the widely used DE algorithm, the population undergoes mutation, crossover, and selection operations, and the next generation of the population is selected based on the fitness function.

[0042] After several rounds (50 rounds) of iteration, the optimal set of quantization step sizes is obtained. .

[0043] Figure 6 This is a schematic diagram comparing the rate-distortion performance of the method of the present invention with other methods; Figure 6 In (a) to (f), the horizontal axis represents the bit rate, and the vertical axis represents the PSNR. Figure 6 Tables (a) to (c) show the experimental results on the SemanticKitti dataset. Figure 6 In the figure, (a), (b), and (c) represent D1-PSNR, D2-PSNR, and chamfer distance, respectively; Figure 6 Figures (d) to (f) show the experimental results on the Ford dataset. Figure 6 In the diagram, (d), (e), and (f) represent D1-PSNR, D2-PSNR, and chamfer distance, respectively.

[0044] Table 1 compares the rate-distortion performance of this invention with other methods on the FORD dataset and the SemanticKITTI dataset.

[0045] Table 1 like Figure 6As shown in Table 1, the method proposed in this invention achieves the best rate-distortion performance compared to other methods. On the Ford dataset, compared to the current best-performing LPCM, the D1 BD-Rate is -4.5%, and the D2 BD-Rate is -4.7%. Compared to the point cloud coding standard G-PCC, the D1 BD-Rate is -28.7%, and the D2 BD-Rate is -28.9%. On the SemanticKITTI dataset, compared to LPCM, the D1 BD-Rate is -6.5%, and the D2 BD-Rate is -8.1%. Compared to G-PCC, the BD-Rates based on D1 and D2 are -26.4% and -27.2%, respectively.

Claims

1. A deep learning-based method for predicting geometric encoding of 3D point clouds, characterized in that, include: Step 1: A prediction tree is constructed from the points collected by each laser emitter, thereby transforming the original LiDAR point cloud LPC into multiple prediction trees for representation. Step 2: For each component of the point's coordinates, design different predictors and entropy encoders, and apply different quantization step sizes to compress each component of the point's coordinates; specifically including: For azimuth, the azimuth resolution of the lidar is used to predict the azimuth by representing the increment between the azimuth of the current point and the azimuth of the previous point. For pitch angle, the pitch angle prediction model based on attention mechanism and LSTM network uses the correlation between neighboring points and the correlation between radius and pitch angle to predict the pitch angle of the current point. For the radius, time information is introduced to improve the compression efficiency; specifically including: Whether the current frame should use intra-frame coding (I-frame) is determined by calculating the similarity between the current frame and the previous frame; for I-frames, a DC-based method is used to compress the radius; for predictive coded frames (P-frames), the transformation matrix between the current frame and the reference frame is calculated and encoded, and the reference frame is registered to the current frame. For the radius, the radius of the current point is predicted using the proposed inter-frame radius prediction model, taking into account the neighboring points in the current frame and the registered R frame. Step 3: Use a quantization step size selection strategy to select the quantization step size for each component; Step 4: Use an entropy model to model the probability distribution of the residuals for each component, thereby encoding the residual entropy; Step 5: Decoding is achieved by reversing the process from Step 1 to Step 4.

2. The deep learning-based 3D point cloud prediction geometric coding method according to claim 1, characterized in that, Each point collected by a laser emitter forms a prediction tree, thus transforming the original LiDAR point cloud (LPC) into multiple prediction trees; including: First, point cloud The process of transforming the coordinates of all points from the Cartesian coordinate system to the spherical coordinate system is expressed as: ； in, Represents the Cartesian coordinates of a point. Represents the spherical coordinates of a point. Indicates pitch angle, Indicates azimuth. Indicates radius, id This indicates the number of the laser that was detected at this point; Subsequently, having the same The points will be grouped together, and within each group, all points will be arranged in ascending order of azimuth angle, thus constructing a prediction tree. .

3. The deep learning-based 3D point cloud prediction geometric coding method according to claim 1, characterized in that, During encoding, the prediction tree starts from the root node, which is the point with the smallest azimuth angle, and predicts the coordinates of the next point one by one. The prediction residuals are then quantized and entropy encoded. Within the same point, the azimuth angle is encoded first, then the radius, and finally the elevation angle.

4. The deep learning-based 3D point cloud prediction geometric coding method according to claim 3, characterized in that, Azimuth encoding; including: When encoding the azimuth of a point cloud, the azimuth of each point is... Represented as: ； in, It is an integer. This indicates the set unit azimuth angle. Set the azimuth resolution of the lidar , This is the preset quantization step size; Differential coding is Encoding via entropy model; It is an integer, based on the azimuth angle. To obtain is Integer representation, Immediately Integer representation of the azimuth angle; Output bias coefficients using the entropy model mean and standard deviation This allows us to model a biased normal distribution and obtain the result through the integral of the probability density function. The probability mass function, thus affecting Entropy coding.

5. The deep learning-based 3D point cloud prediction geometric coding method according to claim 3, characterized in that, Radius encoding; including: The radius is compressed through four stages: segmentation, registration, prediction, quantization, and entropy coding; specifically including: During the segmentation phase, the current frame is segmented according to the laser number at each point. With the reconstructed reference frame Divided into upper half and the lower half ; During the registration stage, the first step is to evaluate the upper half of the current frame. With the upper half of the reference frame The similarity is used to determine whether the current frame is an I-frame or a P-frame; if it is an I-frame, it is directly... Represented as a prediction tree If it is a P-frame, the upper half of the reconstructed reference frame is calculated. With the upper half of the current frame The transformation matrix between them, thus making Registration ; During the prediction phase, for all points in the I-frame, and The radius of all points is predicted using a differential coding method; for Then the Inter-RP model is used for prediction; In the quantization and entropy coding stage, the predicted residuals are quantized and entropy coding is performed using an entropy model.

6. The deep learning-based 3D point cloud prediction geometric coding method according to claim 3, characterized in that, During the registration stage, the upper half of the current frame is calculated. With the upper half of the reference frame The PSNR is used to determine whether the current frame is an I-frame or a P-frame. If the PSNR is greater than the set threshold, it is a P-frame; otherwise, it is an I-frame. If it is an I-frame, directly... Represented as a prediction tree If it is a P-frame, first calculate the upper half of the reconstructed reference frame using the ICP algorithm. With the upper half of the current frame The transformation matrix between them will Registration Next, after registration , as well as These are respectively represented as sets of prediction trees. , as well as ; Further optimization, in the prediction phase, for as well as Predicted value of the radius at the midpoint Obtained directly through incremental encoding DC: ； in, Point The reconstructed value of the radial distance; for The radius is then predicted using the Inter-RP model. The Inter-RP model consists of three LSTM network modules, an attention layer, and a multilayer perceptron (MLP). The inputs to the Inter-RP model are respectively from as well as The already encoded part Selected time neighborhood Space Neighbors ,as well as Corresponding prediction residuals ;for First of all Find three prediction trees within the range of LaserID plus or minus 1 for the current point. Then, find the point in each prediction tree whose azimuth is closest to the current point. Finally, take this point, along with the 24 points preceding it and the 25 points following it in the prediction tree sequence, to form a 50×3 point set. ;for ,from Take the prediction tree with the same Laser ID as the current point, and construct it from the 50 previously encoded points before the current point. ; Then it is The residual of each point entropy encoding; The output of the Inter-RP model is the predicted value of the radius; After completing the prediction, point The prediction residuals and quantization residuals of the radial distance are: ； ； in, The predicted residuals of the radial distance of the point cloud. For the quantization step size of radial distance, To quantify the residual; then the point Reconstructed value of radial distance Represented as: ； Finally, the entropy model is used to analyze the set of quantization residuals. Perform entropy encoding.

7. The deep learning-based 3D point cloud prediction geometric coding method according to claim 3, characterized in that, Pitch angle encoding; including: Pitch angle prediction model based on attention mechanism and LSTM network; First, by leveraging the LSTM network's sensitivity to the order of the input sequence, spatial correlations within the prediction tree are modeled. Then, when encoding points within a prediction tree When determining the pitch angle, the already encoded points arrive The reconstruction information is input into the LSTM network, and the reconstruction information for each point includes... , They are Azimuth angle reconstruction value, radius reconstruction value, laser number, yes The reconstructed pitch angle value; Furthermore, the attention mechanism is used to model the relationship between different coordinate axes of the point cloud, thereby weighting the features output by the LSTM layer; Finally, the weighted features are aggregated by a multilayer perceptron (MLP) to output the predicted pitch angle. Then point The prediction residuals and quantization residuals of the pitch angle are: ； ； in, The prediction residual for the pitch angle, For the quantization step size of radial distance, To quantify the residual; then the point The reconstructed pitch angle is expressed as: ； right and the set of quantized residuals of pitch angle Perform entropy encoding.

8. The deep learning-based 3D point cloud prediction geometric coding method according to claim 1, characterized in that, The pitch angle prediction model based on the attention mechanism and LSTM network calculates the predicted values ​​during training. Compared with the true value The loss function is constructed using the MSE loss between the two values.

9. The deep learning-based 3D point cloud prediction geometric coding method according to any one of claims 1-8, characterized in that, A quantization step size selection strategy is used to select the quantization step size for each component; including: The problem of selecting encoding parameters is transformed into an optimization problem under constraints. Approximately optimal encoding parameters are obtained through iterative optimization on a small dataset containing only 20 point clouds. Other point clouds are then encoded using these approximately optimal encoding parameters. First, the problem of choosing encoding parameters is transformed into an optimization problem of minimizing the BD-rate under bit rate constraints: ； ; in, For the set of quantization parameters, , , This represents the set of reference quantization step sizes. , The upper and lower bounds of the set bit rate, N The number of point clouds; For the above optimization problem, the approximate optimal quantization step size is solved by the differential evolution algorithm.

10. The deep learning-based 3D point cloud prediction geometric coding method according to claim 9, characterized in that, When selecting the optimal quantization step size at 6 bit rates, the approximate optimal quantization step size is found using a differential evolution algorithm; including: Step 1: Population initialization; The initial population is represented as: ； In the formula, Represents the initial population. The first term of the population k Individual; Based on empirical values, the following settings are made: ； ； After population initialization, the quantization step size at different code rates in each individual will be used for encoding a small dataset consisting of 20 point clouds, thereby obtaining the 6 bit rates and corresponding distortions for encoding this small dataset, and thus calculating the individual's... The fitness function of an individual is obtained by comparing its encoding performance with the BD-Rate; for individuals whose bit rate exceeds a preset range, their fitness function is assigned a value of positive infinity. Subsequently, the population undergoes mutation, crossover, and selection operations, and the next generation of the population is selected based on the fitness function; After several rounds of iteration, the optimal set of quantization step sizes is obtained. .

Images