Deep learning based three-dimensional point cloud prediction geometry encoding method and system

Abstract

The present application relates to a three-dimensional point cloud prediction geometry coding method and system based on deep learning, comprising: 1) constructing a prediction tree; constructing a prediction tree based on radar parameters; or constructing a prediction tree by using a threshold segmentation method; 2) predicting coding; high code rate mode and low code rate mode are designed to meet the coding needs under different bit rates; 3) selecting coding parameters based on differential evolution: through the quantization step selection method based on differential evolution, the selection problem of coding parameters is converted into an optimization problem under a constraint condition, and the approximate optimal coding parameters are obtained through iterative optimization on a small data set containing only ten point clouds, and finally other point clouds are encoded through the coding parameters. The present application improves the rate distortion performance of Lidar point cloud coding, and compared with the previous method, the reconstructed point cloud after decoding has higher fidelity under the same bit rate.

Description

TECHNICAL FIELD

[0001] The present application relates to a three-dimensional point cloud prediction geometry coding method and system based on deep learning, and belongs to the technical field of image processing. BACKGROUND

[0002] LiDAR can accurately collect 3D scene information and is widely used in automatic driving, robot navigation and geographic information systems. The point cloud data collected by LiDAR generally contains geometric information (such as Cartesian coordinates x, y, z) and attribute information (such as reflectivity). However, due to the very large amount of data of LiDAR point cloud, there is an urgent need for efficient compression technology to reduce the storage and transmission cost of LiDAR point cloud.

[0003] In recent years, the Moving Picture Experts Group (MPEG) has released a geometry-based point cloud compression (G-PCC) standard, and is still studying more efficient point cloud compression technology. G-PCC contains two LiDAR point cloud geometry coding methods: octree coding method and prediction geometry coding method. Among them, the prediction geometry coding method makes full use of the collection principle of LiDAR, and is more efficient than the octree coding method.

[0004] Specifically, LiDAR collects point cloud data through multiple built-in lasers with fixed elevation angles and fixed azimuth angle resolution. This collection principle results in a stronger correlation of point cloud in the spherical coordinate system. In order to take advantage of this feature, the angle mode in the prediction geometry coding method converts the geometric information of the point cloud into the spherical coordinate system, and groups the points according to the number of lasers. Within each group, the points are connected in order of their azimuth angle size, thereby constructing a prediction tree. Then, starting from the root node, the geometric information is sequentially predicted by the coded neighboring points, and the residual error is entropy coded.

[0005] However, the prediction geometry coding method in G-PCC still has the following shortcomings. First, this method models the simple linear relationship between neighboring points in the prediction tree to predict the geometric information of the current point, but ignores the long-distance geometric correlation between different coordinates. Second, at low bit rates, the distortion increases, and the geometric correlation also decreases accordingly, resulting in a performance that is not as good as the octree coding method. Finally, in the spherical coordinate system, the distortion caused by the quantization of the residual error on each axis has different effects on the reconstruction distortion, but this method does not explore the rate-distortion (RD) optimization problem of the quantization parameter (QP) in the spherical coordinate system, so it cannot achieve the best RD performance. SUMMARY

[0006] In view of the shortcomings of the prior art, the present application provides a three-dimensional point cloud prediction geometry coding method based on deep learning;

[0007] As Figure 1As shown, first, according to different known information of the point cloud to be encoded, the application proposes two prediction tree construction methods, and selects the most suitable one for the construction of the prediction tree in the spherical coordinate system. Since the laser radar point cloud has different annotation contents related to the collection device in addition to geometry and attributes, and the arrangement order of the geometric coordinates of the points in the point cloud file is also different, the method has two built-in prediction tree construction methods. For point cloud files containing the elevation angle of each laser scanner of the laser radar and the height of each laser radar scanner coordinate system relative to the radar coordinate system, such as the Ford point cloud dataset, the prediction tree construction method based on the radar parameters is preferred. For point cloud files that do not contain the above-mentioned related parameters, but the coordinates of the points are arranged in the point cloud file according to the height order of the laser scanner and the collection order of the laser scanner, such as the SemanticKITTI dataset, the threshold segmentation method is preferred to construct the prediction tree.

[0008] Then, as the code rate decreases, the correlation of the points will be weakened, and therefore the application proposes a high code rate encoding mode and a low code rate encoding mode. In the high code rate encoding mode, the method trains an elevation angle prediction model based on LSTM by exploring the relationship between the radius and the elevation angle of the Lidar point cloud, and encodes the prediction residual of the elevation angle; the azimuth angle and the radial distance are directly differential encoded. In the low code rate mode, the radial distance is encoded by a differential autoencoder; the azimuth angle and the elevation angle are directly differential encoded.

[0009] Since the same quantization step is used to process the residual of the elevation angle, the azimuth angle and the radius in the spherical coordinate system, the best rate-distortion performance cannot be obtained, the application proposes a quantization step selection method based on differential evolution, thereby further improving the encoding efficiency.

[0010] The application also provides a three-dimensional point cloud prediction geometry encoding system based on deep learning.

[0011] Term explanation:

[0012] 1. High code rate mode, the high code rate mode compresses the original point cloud into a larger bit stream, so that the reconstructed point cloud after decoding has higher fidelity.

[0013] 2. Entropy encoding, that is, encoding without losing any information according to the entropy principle in the encoding process.

[0014] The technical scheme of the application is:

[0015] The three-dimensional point cloud prediction geometry encoding method based on deep learning comprises:

[0016] 1) Constructing a prediction tree; for a point cloud file containing the elevation angle of each laser scanner of the laser radar and the height of each laser radar scanner coordinate system relative to the radar coordinate system, a prediction tree is constructed based on the radar parameters; otherwise, for a point cloud file in which the coordinates of the points are arranged in the point cloud file according to the height order of the laser scanner and the collection order of the laser scanner, a threshold segmentation method is used to construct a prediction tree;

[0017] 2) Prediction coding;

[0018] In the high code rate mode, in each prediction tree, the next point coordinate is predicted in sequence from the root node, and the quantized residual error is coded; when the coordinate of the coded point in the spherical coordinate system is obtained, the radial distance and the azimuth angle are first differentially coded; at the same time, the relationship between the elevation angle and the radius is fitted through the elevation angle prediction model based on the LSTM network, the elevation angle is predicted by using the reconstructed coordinates of the coded point and the elevation angle and the radial distance of the current point, and the quantized prediction residual error is coded;

[0019] In the low code rate mode, the azimuth angle is coded by using the same method as in the high code rate mode, and the elevation angle is represented by differential coding of the reconstructed value; for the radius r, a self-encoder based on an entropy model is used for compression;

[0020] 3) Selecting coding parameters based on differential evolution;

[0021] By using the quantization step selection method based on differential evolution, the selection problem of the coding parameters is converted into an optimization problem under a constraint condition, and the approximate optimal coding parameters are obtained through iterative optimization on a small data set containing only ten point clouds, and finally the other point clouds are coded by using the coding parameters.

[0022] According to the preferred embodiment of the present application, the selection problem of the coding parameters is converted into an optimization problem under a constraint condition, which comprises:

[0023] The selection problem of the coding parameters is converted into a single-objective optimization problem under a bit rate constraint:

[0024]

[0025] Wherein, Q is a set of quantization parameters, Q=(q θ ,q r ,q δ ,Φ unit ), q θ is the quantization step for the elevation angle θ, q r is the quantization step for the radius R, q δ ,Φ unit is a related parameter for representing the azimuth angle, and D g (Q,P i) represents the reconstructed point cloud in the point cloud P i the mean square error measured on the point cloud P θ R represents the bit rate of the encoded elevation angle, r R represents the bit rate of the encoded radius, Φ R represents the bit rate of the encoded azimuth angle, i ) represents the point cloud P i the total bit rate, also tested on small data sets.

[0026] According to the application, the prediction tree is preferably constructed based on radar parameters; including:

[0027] The process of converting the coordinates (x, y, z) of point n in the Cartesian coordinate system to the spherical coordinate system is represented as:

[0028]

[0029] Φ = atan (y, x);

[0030]

[0031] where r represents the radial distance, Φ represents the azimuth angle, i and j represent the number of laser scanners, θ(j) represents the preset elevation angle of laser scanner j, and N represents the number of laser scanners;

[0032] Through the above calculation, the radial distance, azimuth angle and laser scanner number i of point n in the spherical coordinate system are obtained.

[0033] According to the laser scanner number i, the height θ(i) of the laser scanner under the laser radar scanner is obtained, so that the elevation angle θ of point n is calculated.

[0034]

[0035] Thus, the coordinates (Φ, θ, r) of point n in the spherical coordinate system are obtained.

[0036] According to the laser scanner number i of each point, all points are divided into N groups, and within each group, the points are sorted according to the value of φ. Each group of points constitutes a prediction tree.

[0037] According to the application, the threshold segmentation method is preferably used to construct the prediction tree; including:

[0038] First, the coordinates (x, y, z) of point n in the Cartesian coordinate system are converted to the spherical coordinate system:

[0039]

[0040] Φ = atan(y,x);

[0041] θ1 = atan(z,r);

[0042] Thus, the coordinates (Φ, θ1, r) of point n in the spherical coordinate system are obtained;

[0043] Next, the points are grouped, which means: starting from the azimuth angle Φ1 of the first point in the point cloud file, traversing point by point, and calculating the threshold difference between the current point and its adjacent points in the storage order of the point cloud file; when there are n points in the point cloud file... i The azimuth and point n i-1 When the difference in azimuth angles exceeds a preset threshold, the data is grouped at this point, and each group forms a prediction tree.

[0044]

[0045] Among them, G j Let G represent the j-th group. j+1 This represents the (j+1)th group, and t represents the preset threshold. This represents the azimuth angle of the i-th point in the point cloud file. This represents the azimuth angle of the (i-1)th point in the point cloud file.

[0046] After constructing the prediction tree using the above method, the encoding process begins. According to a preferred embodiment of the present invention, differential encoding is performed on the radial distance; including:

[0047] First, when performing differential encoding on the radial distance, the radial distance r1 of the root node in each prediction tree is directly entropy encoded;

[0048] Then, point n i Predicted radial distance Obtained through differential encoding:

[0049]

[0050] in, Representing point n i-1 The reconstructed radial distance of point n i The prediction residuals and quantization residuals of the radial distance are:

[0051]

[0052] Among them, res r,i Let q be the prediction residual of the radial distance of the point cloud. r For the quantization step size of radial distance, To quantize the residual; then point n i Reconstructed value of radial distance Represented as:

[0053]

[0054] Finally, the quantized residuals of r1 and the radial distance are entropy coded.

[0055] According to the present application, the azimuth angle is preferably differentially encoded; including:

[0056] When encoding the azimuth angle of the point cloud, the azimuth angle Φ i of each point is expressed as:

[0057] Φ i = Φ unit × s i + δ i ;

[0058] wherein δ i represents an angle, s i is an integer, Φ unit represents a set unit azimuth angle, and Φ unit is set as x is a preset integer;

[0059] At this time, the encoding of Φ i is converted to the encoding of s i and δ i ;

[0060] For s i , differential encoding is performed before entropy encoding; for δ i , quantization is performed first, and then entropy encoding is performed;

[0061]

[0062] wherein, represents the quantized value of δ i , Φ is the reconstructed value, and q δ is the quantization step; the reconstructed value of Φ i is expressed as:

[0063]

[0064] The data that needs to be entropy coded is: the differential encoding of s i and the set of quantized residuals of δ i

[0065] ​​According to the application, the relationship between the pitch angle and the radius is fitted by the pitch angle prediction model based on the LSTM network, the pitch angle is predicted by using the reconstructed coordinates of the coded points and the pitch angle and the radial distance of the current point, and the quantized prediction residual is coded; including:

[0066] The relationship between the pitch angle and the radius under the prediction tree sequence is fitted by the pitch angle prediction model based on the LSTM network, the pitch angle prediction model based on the LSTM network includes 3 layers of LSTM network and 5 layers of full connection network, and parallel calculation of multiple prediction trees is realized;

[0067] Firstly, when coding the pitch angle of point n i in a certain prediction tree, the reconstructed information of points n i-1 to n i-50 that have been coded is input into the LSTM network, the reconstructed information of each point includes j∈[i-50,i-1], wherein, l j represents the prediction tree number to which point n j belongs; the missing points are padded by adding padding;

[0068] After feature extraction by the LSTM network and feature aggregation by the full connection layer, part of the reconstructed information of the current point is spliced together, wherein, are the reconstructed values of the current point, is the reconstructed value of the pitch angle of point n i-1 .

[0069] Subsequently, the spliced features are aggregated again by two layers of full connection network, and finally the predicted value of the pitch angle of point n i is output . The prediction residual and the quantized residual of the pitch angle of point n i are:

[0070]

[0071] Wherein, res θ,i is the prediction residual of the pitch angle, q θ is the quantization step for the radial distance, is the quantized residual; the reconstructed value of the pitch angle of point n i is represented as:

[0072]

[0073] Finally, θ1 and the set of quantized residuals of the pitch angle are entropy coded.

[0074] According to a preferred embodiment of the present invention, the pitch angle prediction model based on an LSTM network calculates the predicted value during training. Compared with the true value θ i The MSE loss between them is used to construct the loss function l mse :

[0075]

[0076] Where N represents the number of points in the point cloud.

[0077] According to a preferred embodiment of the present invention, in low bit rate mode, the pitch angle is represented by differential coding of the reconstructed values; including:

[0078] Point n i The predicted pitch angle Obtained through differential encoding:

[0079]

[0080] in, Representing point n i-1 The reconstructed value of the radial distance at point n i The prediction residuals and quantization residuals of the radial distance are:

[0081]

[0082] Among them, res θ,i Let q be the prediction residual of the radial distance of the point cloud. θ For the quantization step size of radial distance, To quantize the residual; then point n i The reconstructed radial distance is expressed as:

[0083]

[0084] Finally, the set of quantized residuals for θ1 and radial distance. Perform entropy encoding; the pitch angle is then represented as...

[0085] According to a preferred embodiment of the present invention, the specific implementation process of the quantization step size selection method based on differential evolution includes:

[0086] Step 1: Population Initialization

[0087] First, generate an i-th number of individuals X in the 4-dimensional feasible solution space to form the initial population;

[0088] X(0) = (X1,X2,X3,…,X) i )=((x 1,1 ,x 1,2 ,x1,T ,x 1,4 ),…,(x i,1 ,x i,2 ,x i,3 ,x i,4 ));

[0089] In the formula, X(0) represents the initial population, X i represents the ith individual of the population, x i,j is the jth element of the ith individual of the population; wherein X1 and the elements in Q are one-to-one corresponding, that is, x i,1 corresponds to q θ , that is, x i,2 corresponds to q r , x i,3 corresponds to q δ , x i,4 corresponds to Φ unit , and all are represented by X i ; according to the empirical value, set:

[0090]

[0091] After randomly initializing the population according to the above set value range, first, the fitness value of each individual X is calculated, that is, ∑ i D g (Q, P i ) and the corresponding bit rate ∑ i R(Q, P i ); for the individual whose ∑ i R(Q, P i ) is greater than R T , the fitness value is revalued as positive infinity;

[0092] Step 2: Mutation operation

[0093] First, randomly select two different individuals to subtract, as shown in the following formula:

[0094] B i,j = X i (t)-X j (t),

[0095] X i (t) represents the individual in the population after iteration t times, and i and j represent the serial number of the individual in the population after iteration t times. After weighted processing, the difference vector B i,j is summed with another individual X k (t) in the population after iteration t times, and the offspring mutation individual V i (t+1) is obtained; the specific mutation operation is as follows:

[0096] V i(t+1) = X k (t) + μB i,j ,

[0097] where i is the current target individual index, i, j and k are the indices of individuals in the t-th generation population which are different from the current target index, i.e. i≠j≠k; X k (t) is the base vector of the mutation vector V i (t+1); μ∈[0, 2] is the mutation scale factor parameter of the differential evolution algorithm;

[0098] Step 3: crossover operation

[0099] each element u i (t+1) in the trial individual U i,j (t+1) is calculated as follows:

[0100]

[0101] where v i,j (t+1) is the j-th dimensional component of the mutation individual V i (t+1), x i,j (t) is the j-th dimensional component of the parent population target individual, i = 1, 2, …, NP, j = 1, 2, …, d; r i,j is the random number corresponding to the j-th dimensional component, and satisfies the normal distribution between (0, 1); CR is the crossover probability factor of the algorithm, and the value range is [0, 1], m∈{1, 2, …, d}, which ensures that at least one dimensional component of the trial individual U i (t+1) comes from the mutation individual V i (t+1);

[0102] When the random number r i,j corresponding to the j-th component of the population individual is less than the crossover rate CR or j = m, the j-th dimensional component of the trial individual U i (t+1) is provided by the mutation individual V i (t+1), otherwise it is provided by the parent target individual X i (t);

[0103] Step 4: selection operation

[0104] According to the fitness value and whether the constraint condition of the trial individual U i (t+1) and the parent target individual X i (t) are satisfied, wherein the constraint condition is the measured bit rate on the small data set, and the fitness value is the total distortion measured on the small data set; the individual which satisfies the bit rate constraint and has better fitness value is selected into the next generation population;

[0105]

[0106] Through multiple iterations of the differential algorithm, the approximately optimal quantization parameter Q under the bit rate is obtained;

[0107] Subsequently, the quantization parameter Q is directly used for compression of all point clouds under the bit rate.

[0108] The deep learning-based three-dimensional point cloud prediction geometry coding system comprises:

[0109] The prediction tree construction module is configured to: for a point cloud file containing the elevation angle of each laser scanner of the lidar and the height of each laser scanner coordinate system relative to the radar coordinate system, construct a prediction tree based on the radar parameters; otherwise, for a point cloud file in which the coordinates of points are arranged in the order of the height of the laser scanner and the collection order of the laser scanner in the point cloud file, a threshold segmentation method is used to construct a prediction tree;

[0110] The prediction coding module is configured to: in the high code rate mode, in each prediction tree, starting from the root node in order, the next point coordinate is predicted one by one, and the quantized residual error is coded; when the coordinates of the coded points in the spherical coordinate system are obtained, the radial distance and the azimuth angle are first differentially coded; at the same time, the relationship between the elevation angle and the radius is fitted through the elevation angle prediction model based on the LSTM network, the elevation angle is predicted by using the reconstructed coordinates of the coded points and the elevation angle and the radial distance of the current point, and the quantized prediction residual error is coded; in the low code rate mode, the azimuth angle is coded by using the same method as in the high code rate mode, and the elevation angle is represented by differentially coding the reconstructed value; for the radius r, the self-encoder based on the entropy model is used for compression;

[0111] The coding parameter selection module is configured to: through the quantization step selection method based on differential evolution, the coding parameter selection problem is converted into an optimization problem under a constraint condition, and the approximately optimal coding parameter is obtained through iterative optimization on a small data set containing only ten point clouds, and finally the other point clouds are coded by using the coding parameter.

[0112] The present application has the following advantages:

[0113] 1. The present application improves the rate-distortion performance of Lidar point cloud coding, that is, compared with the previous method, the reconstructed point cloud after decoding has higher fidelity under the same bit rate.

[0114] 2. The present application proposes a quantization parameter calculation method for the residual error of each coordinate axis of the point cloud in the spherical coordinate system, which can further improve the rate-distortion performance of point cloud coding and can also be applied in similar spherical coordinate coding methods. BRIEF DESCRIPTION OF DRAWINGS

[0115] Figure 1is an implementation block diagram of a three-dimensional point cloud prediction geometry coding method based on deep learning of the present application;

[0116] Figure 2 is a diagram showing the relationship between the radius and the pitch angle of points originating from the same laser;

[0117] Figure 3 is a diagram showing the relationship between the radius and the pitch angle of points originating from the same laser;

[0118] Figure 4 is a diagram showing the relationship between the radius and the pitch angle of points originating from the same laser;

[0119] Figure 5 is a diagram showing the relationship between the radius and the pitch angle of points originating from the same laser;

[0120] Figure 6 is a structural diagram of a pitch angle prediction model based on an LSTM network proposed by the present application;

[0121] Figure 7 is a structural diagram of a radius compression model in a low code rate mode;

[0122] Figure 8 is a diagram showing the rate-distortion curve based on D1-PSNR measured on the Semantic dataset;

[0123] Figure 9 is a diagram showing the rate-distortion curve based on D1-PSNR measured on the Semantic dataset;

[0124] Figure 10 is a diagram showing the rate-distortion curve based on D2-PSNR measured on the FORD dataset;

[0125] Figure 11 is a diagram showing the rate-distortion curve based on D2-PSNR measured on the FORD dataset. DETAILED DESCRIPTION

[0126] The present application will be further limited by the following description and examples with reference to the accompanying drawings, but is not limited thereto.

[0127] Example 1

[0128] A three-dimensional point cloud prediction geometry coding method based on deep learning, as shown in Figure 1 , includes:

[0129] 1) Constructing prediction tree; Since the annotation content of the laser radar point cloud is not the same in terms of geometry, attributes, and related information about the collection device, and the arrangement order of the point geometry coordinates in the point cloud file is also different, two prediction tree construction methods are built in this method. For point cloud files containing the elevation angle of each laser scanner of the laser radar and the height of each laser radar scanner coordinate system relative to the radar coordinate system, such as the Ford point cloud dataset, a prediction tree is constructed based on the radar parameters; otherwise, for point cloud files that do not contain the above related parameters, for the coordinates of the points arranged in the point cloud file according to the height order of the laser scanner and the collection order of the laser scanner, such as the SemanticKITTI dataset, a threshold segmentation method is used to construct a prediction tree;

[0130] 2) Prediction encoding;

[0131] In the high code rate mode, after the construction of the prediction tree, in each prediction tree, the next point coordinate is predicted in order from the root node, and the quantized residual is encoded; when the encoded point coordinates in the spherical coordinate system are obtained, the radial distance and azimuth are first differentially encoded; at the same time, the relationship between the elevation angle and the radius is fitted through the LSTM network-based elevation angle prediction model, the elevation angle is predicted using the reconstructed coordinates of the encoded point and the elevation angle and radial distance of the current point, and the quantized prediction residual is encoded; since prediction encoding only depends on the information in the prediction tree, multiple prediction trees can be encoded in parallel.

[0132] In the low code rate mode, since the correlation between points is destroyed, prediction encoding is difficult to accurately predict the coordinates of adjacent points. Therefore, in the low code rate mode, the same method as in the high code rate mode is used to encode the azimuth angle, and the elevation angle is represented by differential encoding of the reconstructed value; for the radius r, an autoencoder based on an entropy model is used for compression;

[0133] 3) Selecting encoding parameters based on differential evolution;

[0134] In the prediction encoding process, there are q θ , q r , q δ , Φ unitFour parameters affect the rate-distortion performance of point cloud coding. In the quantization of coordinate residuals in the Cartesian coordinate system, residuals from different coordinate axes can be quantized using the same quantization step size because the residuals from different coordinate axes in the Cartesian coordinate system have the same impact on the distortion of the reconstructed point cloud. However, in the spherical coordinate system, the pitch angle, azimuth angle, and radial distance need to be converted to the Cartesian coordinate system first, and this conversion is nonlinear, which makes the impact of the above-mentioned spherical coordinate residuals on the distortion of the reconstructed point cloud different, and using the same quantization step size cannot obtain the optimal quantization result. However, the quantization step size has a large value interval, forming a very large number of combination methods, and it is undoubtedly very high in time and calculation cost to obtain the optimal quantization step size by the exhaustive method. Therefore, by using the quantization step size selection method based on differential evolution, the coding parameter selection problem is converted into an optimization problem under the constraint condition, and the approximate optimal coding parameter is obtained through iterative optimization on a small data set containing only ten point clouds, and finally the other point clouds are encoded by using the coding parameter. Compared with the pre-encoding method, this method has less rate-distortion performance loss, but can significantly improve the encoding speed.

[0135] Embodiment 2

[0136] The three-dimensional point cloud prediction geometry coding method based on deep learning is distinguished in that:

[0137] The coding parameter selection problem is converted into a single-objective optimization problem under the bit rate constraint:

[0138]

[0139] Wherein, Q is a set of quantization parameters, Q=(q θ ,q r ,q δ ,Φ unit ), q θ is the quantization step size for the pitch angle θ, q r is the quantization step size for the radius R, q δ ,Φ unit is a related parameter for representing the azimuth angle, D g (Q,P i ) represents the mean square error of the reconstructed point cloud measured on the point cloud P i , R θ represents the bit rate of encoding the pitch angle, R r represents the bit rate of encoding the radius, R Φ represents the bit rate of encoding the azimuth angle, and R(Q,P i ) represents the total bit rate of the point cloud P i , which is also tested on a small data set.

[0140] A prediction tree is constructed based on radar parameters; comprising:

[0141] The method is the same as the prediction tree construction method in the G-PCC point cloud encoding standard, and the process of converting the coordinates (x, y, z) of point n in the Cartesian coordinate system to the spherical coordinate system is represented as:

[0142]

[0143] Φ = atan (y, x) ;

[0144]

[0145] wherein r represents the radial distance, Φ represents the azimuth angle, i and j represent the number of laser scanners, represents the height of laser scanner j under the laser radar scanner, θ(j) represents the preset pitch angle of laser scanner j, and N represents the number of laser scanners;

[0146] Through the above calculation, the radial distance, azimuth angle and point n laser scanner number i of point n in the spherical coordinate system are obtained;

[0147] According to the laser scanner number i, the height θ(i) of the laser scanner under the laser radar scanner is obtained, so that the pitch angle θ of point n is calculated:

[0148]

[0149] Thus, the coordinates (Φ, θ, r) of point n in the spherical coordinate system are obtained;

[0150] According to the laser scanner number i of each point, all points are divided into N groups, and within each group, the points are sorted according to the value of φ. Each group of points constitutes a prediction tree. This method makes full use of the parameter information of the laser radar, and the prediction tree constructed is the most accurate. For a point cloud file containing the pitch angles of each laser scanner of the laser radar and the heights of each laser radar scanner coordinate system relative to the radar coordinate system, the prediction tree construction method based on the radar parameters is preferred.

[0151] A threshold segmentation method is used to construct the prediction tree; including:

[0152] This method is suitable for point cloud files in which the coordinates of points are arranged in the order of laser scanners and the collection order within the laser scanners. First, the coordinates (x, y, z) of point n in the Cartesian coordinate system are converted to the spherical coordinate system:

[0153]

[0154] Φ = atan (y, x) ;

[0155] θ1 = atan (z, r) ;

[0156] Thus, the coordinates (Φ, θ1, r) of point n in the spherical coordinate system are obtained;

[0157] Next, we group the points. Similar to the method described above, we need to group points originating from the same laser scanner into the same group. Since points originating from the same laser scanner are arranged adjacently in the point cloud file, and further arranged according to the order of acquisition (i.e., from smallest to largest azimuth angle), we only need to find the boundary between points originating from two laser scanners to achieve grouping. Because the boundary point n... i With point n i-1 These are the first and last points scanned by two different laser scanners within a single scanning cycle. Therefore, the azimuth angles of these two points often differ significantly. When point n... i The azimuth and point n i-1 When the difference between the azimuth angles of points n exceeds a set threshold, they can be assigned to two different laser scanners. This means: starting from the azimuth angle Φ1 of the first point in the point cloud file, traversing point by point, calculating the threshold difference between the current point and its adjacent points in the point cloud file's storage order; when the difference between the azimuth angles of points n in the point cloud file exceeds a set threshold, they can be assigned to two different laser scanners. i The azimuth and point n i-1 When the difference in azimuth angles exceeds a preset threshold, the data is grouped at this point, and each group forms a prediction tree.

[0158]

[0159] Among them, G j Let G represent the j-th group. j+1 This indicates the (j+1)th group, and t represents the preset threshold, which is typically set to 60. This represents the azimuth angle of the i-th point in the point cloud file. This represents the azimuth angle of the (i-1)th point in the point cloud file. This method utilizes the potential laser scanner grouping information in the point cloud file. Although the final partitioning result is not accurate enough and may result in erroneous partitioning where the number of prediction trees does not equal the actual number of laser scanners, a large number of points originating from the same laser scanner will still be partitioned into the same prediction tree.

[0160] In high bitrate mode, radial distance is differentially encoded; including:

[0161] First, when performing differential encoding on the radial distance, the radial distance r1 of the root node in each prediction tree is directly entropy encoded with high precision.

[0162] Then, point n i Predicted radial distance Obtained through differential encoding:

[0163]

[0164] wherein, denotes the reconstructed value of the radial distance of point n i-1 , the prediction residual and the quantized residual of the radial distance of point n i are:

[0165]

[0166] wherein, res r,i is the prediction residual of the radial distance of the point cloud, q r is the quantization step for the radial distance, is the quantized residual; the reconstructed value of the radial distance of point n i is expressed as:

[0167]

[0168] Finally, the set of r1 and the quantized residual of the radial distance are entropy coded.

[0169] differential coding is performed on the azimuth angle; including:

[0170] When the azimuth angle of the point cloud is coded, the azimuth angle Φ i of each point is expressed as:

[0171] Φ i = Φ unit x s i + δ i ;

[0172] wherein, δ i denotes an angle, s i is an integer, Φ unit denotes a set unit azimuth angle, and Φ unit is set as x is a preset integer; the advantage of this expression is that the difference between s i and s i-1 is often x or an integer multiple of x, and the information entropy is small, and at the same time, the difference between the azimuth angles is processed as an integer which is easy to code, thereby reducing the number of bits consumed by the coding of δ i .

[0173] At this time, the coding of Φ i is converted to the coding of s i and δ i .

[0174] For s​​i , the differential encoding is performed before the entropy encoding; for i , the quantization is performed before the entropy encoding;

[0175]

[0176] wherein, denotes the quantized value of i , is the reconstructed value, q δ is the quantization step size; Φ i is the reconstructed value of is denoted as:

[0177]

[0178] The data that needs to be entropy encoded is: s i the differential encoding of i the quantized residual error of is also the result of differential encoding on the azimuth angle.

[0179] The relationship between the pitch angle and the radius is fitted by the pitch angle prediction model based on the LSTM network, the reconstructed coordinates of the encoded points, and the pitch angle and radial distance of the current point are used to predict the pitch angle, and the quantized prediction residual error is encoded; including:

[0180] As shown in Figure 2 , Figure 3 , Figure 4 and Figure 5 , the abscissa represents the radius, and the ordinate represents the pitch angle. The radius and the pitch angle of the point cloud show a more obvious negative correlation. This relationship is mainly caused by the internal calibration process of the laser radar. However, due to the internal error of the laser radar and the influence of the external environment, it is difficult to effectively model the relationship between the radius and the pitch angle of the point cloud by function fitting.

[0181] Through the pitch angle prediction model based on the LSTM network, the relationship between the pitch angle and the radius under the prediction tree sequence is fitted by the long and short modeling ability of the LSTM network. The pitch angle prediction model based on the LSTM network is based on lightweight design, as shown in Figure 6 , the pitch angle prediction model based on the LSTM network includes 3 layers of LSTM network and 5 layers of fully connected network; the LSTM network and the fully connected network are common neural network module structures in the art and are widely used, and parallel computing of multiple prediction trees is realized;

[0182] First, when encoding the pitch angle of point n i in a certain prediction tree, the encoded points n i-1 to n i-50The reconstruction information of each point is input into the LSTM network, and the reconstruction information of each point includes j∈[i-50,i-1], where l j represents the reconstruction value of the pitch angle of point n j The prediction tree number to which the point belongs; since the first 50 points under the prediction tree do not have sufficient coded points as inputs of the neural network, the missing points are padded; specifically, the first point under the prediction tree can be decoded without relying on adjacent points, and therefore, 49 points are generated according to the coordinates of the first point and placed in front of the first point of the prediction tree, to make up 50 points of input.

[0183] After feature extraction by the LSTM network and feature aggregation by the fully connected layer, part of the reconstruction information of the current point is concatenated together, where are the reconstruction values of the current point, is the reconstruction value of the pitch angle of point n i-1 .

[0184] Subsequently, the concatenated features are aggregated again by two fully connected layers, and finally, the prediction value of the pitch angle of point n i is output. The prediction residual of the pitch angle of point n i is:

[0185]

[0186] where res θ,i is the prediction residual of the pitch angle, q θ is the quantization step for the radial distance, and is the quantization residual; the reconstruction value of the pitch angle of point n i is represented as:

[0187]

[0188] Finally, the set of θ0and the quantization residual of the pitch angle is entropy coded.

[0189] In the training of the pitch angle prediction model based on the LSTM network, the loss function l mse is constructed by calculating the MSE loss between the prediction value and the true value θ i .

[0190]

[0191] where N represents the number of points in the point cloud.

[0192] In the low code rate mode, the pitch angle is represented by differential coding of the reconstruction value; including:

[0193] Point n i The predicted pitch angle Obtained through differential encoding:

[0194]

[0195] in, Representing point n i-1 The reconstructed value of the radial distance at point n i The prediction residuals and quantization residuals of the radial distance are:

[0196]

[0197] Among them, res θ,i Let q be the prediction residual of the radial distance of the point cloud. θ For the quantization step size of radial distance, To quantize the residual; then point n i The reconstructed radial distance is expressed as:

[0198]

[0199] Finally, the set of quantized residuals for θ1 and radial distance. Perform entropy encoding; the pitch angle is then represented as...

[0200] For radius r, compression is achieved through an autoencoder that incorporates an entropy model, such as... Figure 7 As shown, the autoencoder consists of a convolutional neural network (CNN) stacked with a generalized normalized activation function (GDN) and a rectified linear function (ReLU). It includes:

[0201] First, the radius values ​​of each point are arranged in the order of the octree to form a radius matrix R;

[0202] Then, the radius matrix R is input into the autoencoder to extract the latent representation Y;

[0203] Subsequently, the entropy model models the distribution of Y and entropy encodes Y;

[0204] At the decoding end, the decoder obtains the reconstructed matrix from the reconstructed Y. This allows us to reconstruct the radius of the point cloud.

[0205] The specific implementation process of the quantization step size selection method based on differential evolution includes:

[0206] Step 1: Population Initialization

[0207] In this invention, the differential evolution algorithm is mainly used in the process of Q = (q θ,q r ,q δ ,Φ unit ) of the 4-dimensional real parameter space, first, in the 4-dimensional feasible solution space, the number of individuals X is generated, which constitutes the initial population; here, the number of the generated population is 20.

[0208] X(0)=(X1,X2,X3,…,X i )=((x 1,1 ,x 1,2 ,x 1,3 ,x 1,4 ),…,(x i,1 ,x i,2 ,x i,3 ,x i,4 ));

[0209] In the formula, X(0) represents the initial population, X i represents the i-th individual of the population, and x i,j is the j-th element of the i-th individual of the population; wherein X1 and the elements in Q are one-to-one corresponding, that is, x i,1 corresponds to q θ , that is, x i,2 corresponds to q r , x i,3 corresponds to q δ , x i,4 corresponds to Φ unit , and X i is used to represent the individual of the population; according to the experience value, it is set that:

[0210]

[0211] After randomly initializing the population according to the above setting value range, first, the fitness value of each individual X is calculated, that is, ∑ i D g (Q,P i ) and the corresponding bit rate ∑ i R(Q,P i ); for the individual whose ∑ i R(Q,P i ) is greater than R T , its fitness value is revalued as positive infinity;

[0212] Step 2: mutation operation

[0213] In order to maintain the diversity of the population, mutation operation needs to be performed on each individual. The mutation operation of the differential evolution algorithm is to add disturbance to other individuals by using the difference between individuals. First, two different individuals are randomly selected to be subtracted, as shown in the following formula:

[0214] Bi,j = X i (t) - X j (t),

[0215] X i (t) represents the individual in the population after iteration t times, i and j represent the serial number of the individual in the population after iteration t times, and the difference vector B i,j is obtained by summing up the weighted processing of another individual X k (t) in the population after iteration t times, and the offspring mutation individual V i (t+1) is obtained; the specific mutation operation is as follows:

[0216] V i (t+1) = X k (t) + μB i,j ,

[0217] In the formula, i is the serial number of the current target individual, i, j and k are the serial numbers of the individuals in the population after iteration t times which are different from the serial number of the current target in X k (t), that is, i≠j≠k; X i (t) is the basis vector of the mutation vector V i (t+1); μ∈[0,2] is the mutation scale factor parameter of the differential evolution algorithm; when the value of an element in V i (t+1) is greater than the preset value range of X i (t), the maximum value of the value range is taken, and when the value is less than the preset value range of X i (t), the minimum value of the value range is taken. According to the number of population individuals currently used, the population number NP of the differential evolution is at least 4, otherwise the mutation operation cannot be performed.

[0218] Step 3: crossover operation

[0219] After the compilation operation, the population needs to be subjected to a crossover operation. The mutation individual and the target individual exchange part of the components by a certain proportion to form a trial individual U i (t+1). In order to ensure that the target individual X i (t) evolves, it is necessary to ensure that at least one element in the trial individual U i (t+1) comes from the mutation individual V i (t+1). Each element u i (t+1) in the trial individual U i,j (t+1) is calculated as follows:

[0220]

[0221] In the formula, v i,j (t+1) is the j-dimensional component of the mutation individual V i (t+1), and x i,j(t) represents the j-th dimension component of the target individual in the parent population, i = 1, 2, ..., NP, j = 1, 2, ..., d; r i,j is a random number corresponding to the j-dimensional component, and satisfies a normal distribution between (0,1); CR is the crossover probability factor of the algorithm, with a value range of [0,1], which directly affects the mutated individual V. i (t+1) in the experimental individual U i The proportion of (t+1) in m∈{1,2,..,d}, ensuring that the experimental individual U i At least one dimension of (t+1) comes from the mutant individual V. i (t+1);

[0222] When the random number r corresponding to the j-th component of an individual in the population i,. When the crossover rate CR is less than or j = m, the experimental individual U i The j-th dimension component of (t+1) is determined by the mutated individual V. i (t+1) is provided; otherwise, it is provided by the parent target individual X. i (t) provides;

[0223] Step 4: Select Operation

[0224] To ensure the algorithm approaches the global optimum, the fitness value is selected for the next generation of individuals. Specifically, based on the experimental individual U... i (t+1) and the parent target individual X i The fitness value of (t) is determined by whether the constraints are met, where the constraints are the bit rate measured on a small dataset, and the fitness value is the total distortion measured on a small dataset; individuals that meet the bit rate constraints and have better fitness values ​​are selected to enter the next generation of the population.

[0225]

[0226] Through multiple iterations of the differential algorithm, the approximately optimal quantization parameter Q at the bit rate is obtained;

[0227] Subsequently, the quantization parameter Q will be directly used for compression of the entire point cloud at the bit rate.

[0228] Table 1 is a comparison table of the rate-distortion performance of the present invention and other methods on the FORD dataset.

[0229] Table 1

[0230]

[0231] Table 2 compares the rate-distortion performance of this invention with other methods on the SemanticKITTI dataset.

[0232] Table 2

[0233]

[0234] As Figure 8 , Figure 9 , Figure 10 , Figure 11 shown, the abscissa represents the bit rate, and the ordinate represents the peak signal-to-noise ratio. In combination with Tables 1 and 2, the method proposed in the present application obtains the best rate-distortion performance compared with other methods. On the Ford dataset, D1BD-Rate is reduced by 6.8% and D2BD-Rate is reduced by 8.5% compared with the currently optimal SCP-EHEM. Compared with the point cloud encoding standard G-PCC, D1BD-Rate is reduced by 24.9% and D2BD-Rate is reduced by 21.7%. On the SemanticKITTI dataset, D1BD-Rate is reduced by 2.5% and D2BD-Rate is reduced by 1.9% compared with SCP-EHEM. Compared with G-PCC, the BD-Rate based on D1 and D2 is reduced by -23.1% and -23.9%, respectively.

[0235] Embodiment 3

[0236] The deep learning-based three-dimensional point cloud prediction geometry encoding system comprises:

[0237] The prediction tree construction module is configured to: since the annotation content of the laser radar point cloud related to the collection device is not the same in addition to the geometry, attributes, and the arrangement order of the point geometry coordinates in the point cloud file is also not the same, two prediction tree construction methods are built in the method. For point cloud files containing the elevation angle of each laser scanner of the laser radar and the height of each laser radar scanner coordinate system relative to the radar coordinate system, such as the Ford point cloud dataset, a prediction tree is constructed based on the radar parameters; otherwise, for point cloud files that do not contain the above-mentioned related parameters, for the point coordinates arranged in the point cloud file according to the height order of the laser scanner and the collection order of the laser scanner, such as the SemanticKITTI dataset, a threshold segmentation method is used to construct a prediction tree;

[0238] The prediction coding module is configured to: in the high code rate mode, after the construction of the prediction tree is completed, in each prediction tree, starting from the root node, the next point coordinate is predicted one by one in sequence, and the quantized residual error is coded; when the coordinate of the coded point in the spherical coordinate system is obtained, the radial distance and the azimuth angle are first differentially coded; at the same time, the relationship between the pitch angle and the radius is fitted through the pitch angle prediction model based on the LSTM network, the pitch angle is predicted by using the reconstructed coordinate of the coded point and the pitch angle and the radial distance of the current point, and the quantized prediction residual error is coded; since the prediction coding only depends on the information in the prediction tree, multiple prediction trees can be coded in parallel. In the low code rate mode, since the correlation between the points is destroyed, the prediction coding is difficult to accurately predict the coordinates of the adjacent points. Therefore, in the low code rate, the same method as in the high code rate is used to code the azimuth angle, and the pitch angle is expressed by differential coding of the reconstructed value; for the radius r, an autoencoder based on an entropy model is used for compression;

[0239] The coding parameter selection module is configured to: in the prediction coding process, there are four parameters that affect the rate-distortion performance of the point cloud coding, namely q θ , q r , q δ , and Φ unit In the quantization of the coordinate residual error in the Cartesian coordinate system, the residual errors from different coordinate axes can be quantized using the same quantization step, because the residual errors from different coordinate axes in the Cartesian coordinate system have the same effect on the reconstruction point cloud distortion. However, in the spherical coordinate system, the pitch angle, the azimuth angle, and the radial distance need to be converted to the Cartesian coordinate system first, and this conversion is nonlinear, which makes the above-mentioned residual errors of the spherical coordinate system have different effects on the reconstruction point cloud distortion, and using the same quantization step cannot obtain the optimal quantization result. However, the quantization step has a large value interval, forming a very large number of combination methods, and it is undoubtedly very high in time and calculation cost to obtain the optimal quantization step by the exhaustive method. Therefore, by using the quantization step selection method based on differential evolution, the coding parameter selection problem is converted into an optimization problem under a constraint condition, and the approximate optimal coding parameters are obtained through iterative optimization on a small data set containing only ten point clouds, and finally other point clouds are coded by using the coding parameters. Compared with the pre-coding method, this method has less rate-distortion performance loss, but can significantly improve the coding speed.

Claims

1. A deep learning-based method for predicting geometric encoding of 3D point clouds, characterized in that, include: 1) Construct a prediction tree; For point cloud files containing the elevation angles of each laser scanner of the lidar and the height of each lidar scanner's coordinate system relative to the radar coordinate system, a prediction tree is constructed based on the radar parameters; or, for point cloud files where the coordinates of the points are arranged in the order of the laser scanner's height and the order in which the laser scanners were acquired, a threshold segmentation method is used to construct a prediction tree. 2) Predictive coding; In high bitrate mode, within each prediction tree, starting from the root node, the coordinates of the next point are predicted sequentially, and the quantized residuals are encoded. When the coordinates of the encoded point in the spherical coordinate system are obtained, the radial distance and azimuth angle are differentially encoded first. At the same time, the relationship between the pitch angle and the radius is fitted by the pitch angle prediction model based on the LSTM network. The pitch angle is predicted using the reconstructed coordinates of the encoded points and the pitch angle and radial distance of the current point, and the quantized prediction residuals are encoded. In low bitrate mode, the azimuth angle is encoded, while the elevation angle is represented by differential encoding of the reconstructed values; for the radius r, it is compressed using an autoencoder based on an entropy model. 3) Select encoding parameters based on differential evolution; By using a quantization step size selection method based on differential evolution, the problem of selecting encoding parameters is transformed into an optimization problem under constraints. The approximate optimal encoding parameters are obtained through iterative optimization on a small dataset containing only ten point clouds. Finally, other point clouds are encoded using these encoding parameters.

2. The deep learning-based 3D point cloud prediction geometric coding method according to claim 1, characterized in that, The problem of selecting encoding parameters is transformed into an optimization problem under constraints, including: The problem of selecting encoding parameters is transformed into a single-objective optimization problem under bit rate constraints: Where Q is the set of quantization parameters, Q = (q θ ,q r ,q δ ,Φ unit ), q θ q represents the quantization step size for the pitch angle θ. r q represents the quantization step size with respect to radius R. δ ,Φ unit It is a parameter used to represent the azimuth angle, D g (Q,P i ) indicates the reconstruction of the point cloud in point cloud P i The mean square error measured above, R θ The bit rate R represents the encoded pitch angle. r R represents the bit rate of the coding radius. Φ The bit rate representing the encoded azimuth angle, R(Q,P) i ) represents point cloud P i The total bit rate was also obtained by testing on a small dataset.

3. The deep learning-based 3D point cloud prediction geometric coding method according to claim 1, characterized in that, Constructing a prediction tree based on radar parameters; including: The process of transforming the coordinates (x, y, z) of point n in the Cartesian coordinate system to the spherical coordinate system is represented as follows: Φ = atan(y,x); Where r represents the radial distance, Φ represents the azimuth angle, and i and j represent the laser scanner numbers. θ(j) represents the height of laser scanner j under the lidar scanner, θ(j) represents the preset pitch angle of laser scanner j, and N represents the number of laser scanners. Through the above calculations, we obtain the radial distance and azimuth of point n in the spherical coordinate system, as well as the laser scanner number i of point n; Based on the laser scanner number i, the height θ(i) of the laser scanner under the lidar scanner is obtained, and the pitch angle θ of point n is calculated accordingly: Thus, the coordinates (Φ, θ, r) of point n in the spherical coordinate system are obtained; Based on the laser scanner number i for each point, all points are divided into N groups. Within each group, they are sorted according to the value of Φ. Each group of points forms a prediction tree. A prediction tree is constructed using a threshold segmentation method; including: First, transform the coordinates (x, y, z) of point n in the Cartesian coordinate system to the spherical coordinate system: Φ = atan(y,x); θ1 = atan(z,r); Thus, the coordinates (Φ, θ1, r) of point n in the spherical coordinate system are obtained; Next, the points are grouped, which means: starting from the azimuth angle Φ1 of the first point in the point cloud file, traversing point by point, and calculating the threshold difference between the current point and its adjacent points in the storage order of the point cloud file; when there are n points in the point cloud file... i The azimuth and point n i-1 When the difference in azimuth angles exceeds a preset threshold, the data is grouped at this point, and each group forms a prediction tree. Among them, G j Let G represent the j-th group. j+1 This represents the (j+1)th group, and t represents the preset threshold. This represents the azimuth angle of the i-th point in the point cloud file. This represents the azimuth angle of the (i-1)th point in the point cloud file.

4. The deep learning-based 3D point cloud prediction geometric coding method according to claim 1, characterized in that, Differential encoding of radial distance; including: First, when performing differential encoding on the radial distance, the radial distance r1 of the root node in each prediction tree is directly entropy encoded; Then, point n i Predicted radial distance Obtained through differential encoding: in, Representing point n i-1 The reconstructed radial distance of point n i The prediction residuals and quantization residuals of the radial distance are: Among them, res r,i Let q be the prediction residual of the radial distance of the point cloud. r For the quantization step size of radial distance, To quantize the residual; then point n i Reconstructed value of radial distance Represented as: Finally, the set of quantized residuals for r1 and radial distance. Perform entropy encoding.

5. The deep learning-based 3D point cloud prediction geometric coding method according to claim 1, characterized in that, Differential encoding of the azimuth angle; including: When encoding the azimuth angle of a point cloud, the azimuth angle Φ of each point is... i Represented as: F i =Φ unit ×s i +d i ; Where, δ i To represent an angle, s i Φ is an integer. unit Φ represents the set unit azimuth angle. unit Set the azimuth resolution of the lidar x is a preset integer; At this time, for Φ i The encoding was converted into s i and δ i The encoding; For s i Differential encoding is followed by entropy encoding; for δ i Therefore, quantization is performed first, followed by entropy encoding. in, δ i Quantization value, For the reconstructed value, q δ Φ is the quantization step size. i Reconstruction value Represented as: The data that needs to be entropy encoded is: s i Differential coding and δ i The set of quantized residuals 6. The deep learning-based 3D point cloud prediction geometric coding method according to claim 1, characterized in that, A pitch angle prediction model based on an LSTM network is used to fit the relationship between pitch angle and radius. The pitch angle is predicted using the reconstructed coordinates of encoded points, the pitch angle of the current point, and the radial distance. The quantized prediction residuals are then encoded. This includes: The pitch angle prediction model based on LSTM network is used to fit the relationship between pitch angle and radius under the prediction tree sequence. The pitch angle prediction model based on LSTM network consists of 3 layers of LSTM network and 5 layers of fully connected network; and parallel computation of multiple prediction trees is realized at the same time. First, when encoding a point n within a prediction tree i When determining the pitch angle, the already encoded point n i-1 to n i-50 The reconstruction information is input into the LSTM network, and the reconstruction information for each point includes... j∈[i-50,i-1], where, l j Representing point n j The corresponding prediction tree number; missing points are filled by adding padding; After feature extraction by the LSTM network and feature aggregation by the fully connected layer, the partial reconstructed information of the current point is combined with... When pieced together, l i All are reconstructed values ​​of the current point. For point n i-1 Reconstructed pitch angle value; Subsequently, the spliced ​​features are aggregated again through two layers of fully connected networks, finally outputting point n. i Predicted pitch angle Then point n i The prediction residuals and quantization residuals of the pitch angle are: Among them, res θ,i For the prediction residual of the pitch angle, q θ For the quantization step size of radial distance, To quantize the residual; then point n i The reconstructed pitch angle is expressed as: Finally, the set of quantization residuals for θ1 and pitch angle. Perform entropy encoding.

7. The deep learning-based 3D point cloud prediction geometric coding method according to claim 1, characterized in that, The pitch angle prediction model based on LSTM network calculates the predicted values ​​during training. Compared with the true value θ i The MSE loss between them is used to construct the loss function l mse : Where N represents the number of points in the point cloud.

8. The deep learning-based 3D point cloud prediction geometric coding method according to claim 1, characterized in that, In low bitrate mode, the pitch angle is represented by differential coding of the reconstructed values; include: Point n i The predicted pitch angle Obtained through differential encoding: in, Representing point n i-1 The reconstructed radial distance of point n i The prediction residuals and quantization residuals of the radial distance are: Among them, res θ,i Let q be the prediction residual of the radial distance of the point cloud. θ For the quantization step size of radial distance, To quantize the residual; then point n i The reconstructed radial distance is expressed as: Finally, the set of quantized residuals for θ1 and radial distance. Perform entropy encoding; the pitch angle is then represented as...

9. The deep learning-based 3D point cloud prediction geometric coding method according to claim 2, characterized in that, The specific implementation process of the quantization step size selection method based on differential evolution includes: Step 1: Population Initialization First, generate an i-th number of individuals X in the 4-dimensional feasible solution space to form the initial population; X(0)=(X1,X2,X3,…,X i )=((x 1,1 ,x 1,2 ,x 1,3 ,x 1,4 ),…,(x i,1 ,x i,2 ,x i,3 ,x i,4 )); In the formula, X(0) represents the initial population, X i Let x represent the i-th individual in the population. i,j Let X1 be the j-th element of the i-th individual in the population; where X1 corresponds one-to-one with the elements in Q, i.e., x i,1 Corresponding to q θ , that is, x i,2 Corresponding to q r x i,3 Corresponding to q δ x i,4 Corresponding Φ unit Use X uniformly i Represents an individual in the population; based on empirical values, the following is defined: After randomly initializing the population according to the above-defined value range, the fitness value of each individual X is first calculated, i.e., ∑ i D g (Q,P i and the corresponding bit rate ∑ i R(Q,P i For ∑ i R(Q,P i () greater than R T For individuals, their fitness value is reassigned to positive infinity; Step 2: Mutation Operation First, randomly select two different individuals and calculate the difference, as shown in the following formula: B i,j =X i (t)-X j (t), X i (t) represents an individual in the population after t iterations, where i and j represent the individual indices after t iterations. The difference vector B... i,j After weighting, it is compared with individual X in the population after another iteration t. k Summing (t) yields the offspring variant individual V. i (t+1); The specific mutation operation is as follows: V i (t+1)=X k (t)+μB i,j , In the formula, i is the current target individual index, and i,j, and k are the indices of individuals in the t-th generation of the population that are different from the current target index, i ≠ j ≠ k; X k (t) is the mutation vector V i (t+1) basis vectors; μ∈[0,2] is the mutation scale factor parameter of the differential evolution algorithm; Step 3: Cross-operation Test individual U i Each element u in (t+1) i,j The calculation of (t+1) is as follows: In the formula, v i,j (t+1) represents the mutated individual V. i The j-th dimension component of (t+1), x i,j (t) represents the j-th dimension component of the target individual in the parent population, i = 1, 2, ..., NP, j = 1, 2, ..., d; r i,j Let be the random number corresponding to the j-dimensional component, and satisfy a normal distribution between (0,1); CR is the crossover probability factor of the algorithm, with a value range of [0,1], m∈{1,2,..,d}, ensuring that the experimental individual U i At least one dimension of (t+1) comes from the mutant individual V. i (t+1); When the random number r corresponding to the j-th component of an individual in the population i,j When the crossover rate CR is less than or j = m, the experimental individual U i The j-th dimension component of (t+1) is determined by the mutated individual V. i (t+1) is provided; otherwise, it is provided by the parent target individual X. i (t) provides; Step 4: Select Operation According to the experimental individual U i (t+1) and the parent target individual X i The fitness value of (t) is determined by whether the constraints are met, where the constraints are the bit rate measured on a small dataset, and the fitness value is the total distortion measured on a small dataset; individuals that meet the bit rate constraints and have better fitness values ​​are selected to enter the next generation of the population. Through multiple iterations of the differential algorithm, the approximately optimal quantization parameter Q at the bit rate is obtained; Subsequently, the quantization parameter Q will be directly used for compression of the entire point cloud at the bit rate.

10. A deep learning-based 3D point cloud prediction geometry coding system, characterized in that, include: The prediction tree construction module is configured to: construct a prediction tree based on radar parameters for point cloud files containing the elevation angles of each laser scanner of the LiDAR and the height of each LiDAR scanner coordinate system relative to the radar coordinate system; or, construct a prediction tree using a threshold segmentation method for point cloud files in which the coordinates of the points are arranged in the order of the laser scanner height and the acquisition order of the laser scanner. The predictive coding module is configured as follows: In high bitrate mode, within each prediction tree, starting from the root node, the coordinates of the next point are predicted sequentially, and the quantized residuals are encoded. When the coordinates of the coded point in the spherical coordinate system are obtained, the radial distance and azimuth angle are differentially encoded first. Simultaneously, the relationship between the pitch angle and the radius is fitted by an LSTM-based pitch angle prediction model. The pitch angle is predicted using the reconstructed coordinates of the encoded points, the pitch angle of the current point, and the radial distance, and the quantized prediction residuals are encoded. In low bitrate mode, the azimuth angle is encoded, while the pitch angle is represented by differential encoding of the reconstructed values. For the radius r, an entropy-based autoencoder is used for compression. The encoding parameter selection module is configured to: transform the encoding parameter selection problem into a constrained optimization problem by using a quantization step size selection method based on differential evolution, obtain approximately optimal encoding parameters through iterative optimization on a small dataset containing only ten point clouds, and finally encode other point clouds using these encoding parameters.

Images