A LiDAR point cloud inter-frame encoding and decoding method based on pose state

Abstract

The present invention specifically discloses a posture state-based inter-frame encoding and decoding method for laser radar point clouds. During the encoding process, the geometric information of a three-dimensional point cloud is first mapped into a two-dimensional distance map; the point cloud sequence is divided into a reference point cloud and a point cloud to be encoded, and the posture states of the reference point cloud and the point cloud to be encoded are used to design an inter-frame prediction method for the point cloud to be encoded, and finally the prediction residual is quantized and encoded, and the transformation matrix, residual coding information and reference point cloud coding information are synthesized to output a code stream; during the decoding process, the transformation matrix, residual coding information and reference point cloud coding information are sequentially decrypted from the code stream to obtain the original distance between the reference point cloud and the point cloud to be encoded on the two-dimensional distance map, and finally the distance map is converted into a point cloud to restore the three-dimensional point cloud; the method provided by the present invention has high encoding performance when encoding and decoding a laser radar point cloud sequence.

Description

Technical Field

[0001] The present invention belongs to the field of image coding technology, and in particular relates to a posture state-based inter-frame encoding and decoding method for a laser radar point cloud. Background Art

[0002] A point cloud is a massive collection of points that express the spatial distribution and surface specifics of a target within a single spatial reference frame. Point clouds typically consist of two parts: geometric information and attribute information. Geometric information refers to positional information in three-dimensional space, while attribute information includes reflectivity, color, and other information. Because point clouds can accurately represent scene or object information, they are widely used in fields such as intelligent driving, virtual and augmented reality, and map modeling. However, point clouds are massive, sparse, and disordered, posing significant challenges to their storage and transmission. Therefore, the exploration and research of efficient point cloud encoding methods is crucial.

[0003] In a continuous point cloud sequence, the time interval between two adjacent frames of point cloud is very short, and the laser radar only moves a small distance. Therefore, the point clouds of adjacent frames have a similar structure in space over a large range, and the objects in the scene only undergo small position changes, and there is a lot of redundancy in the time dimension. The Chinese patent with publication number CN111899152A discloses a point cloud data compression method based on projection and video stitching, which specifically discloses analyzing the geometric attribute characteristics of point cloud data to determine the corresponding projection strategy and the optimal projection angle; projecting the projection area at the same projection angle multiple times to obtain a set of two-dimensional images with spatial correlation from the same angle; and stitching a set of two-dimensional images into a video file. However, this method is suitable for dense point cloud data and has poor versatility in multiple point cloud types. Summary of the Invention

[0004] In order to solve the above technical problems, the present invention provides a lidar point cloud inter-frame coding method based on posture state, which can reduce the time domain redundancy in the point cloud sequence and improve the data compression performance by making use of the posture relationship between adjacent frame point clouds.

[0005] The technical solutions of the present invention are as follows:

[0006] A method for inter-frame coding of a laser radar point cloud based on a posture state comprises the following steps:

[0007] Coding process: Map the geometric information of the 3D point cloud into a 2D distance map; obtain the pose state, calculate the transformation matrix M based on the pose state, calculate the predicted distance map by comparing the I frame with the transformation matrix M, and obtain the residual coding information of the P frame and the I frame coding information based on the predicted distance map. The transformation matrix M and the coding information are then combined to output the bitstream.

[0008] Decoding process: The transformation matrix, residual coding information and reference point cloud coding information are sequentially extracted from the bitstream to obtain the original distance between the reference point cloud and the point cloud to be encoded on the two-dimensional distance map. Finally, the distance map is converted into a point cloud to restore the three-dimensional point cloud.

[0009] Furthermore, the posture relationship is obtained based on the IMU information in the KITTI dataset.

[0010] Furthermore, the reference point cloud is an I-frame point cloud and a P-frame point cloud to be encoded, and the point cloud sequence is encoded in the form of "IPPPIPPP...".

[0011] Furthermore, the specific steps of the encoding process are as follows:

[0012] S1. Divide the point cloud sequence into a reference point cloud and a point cloud to be encoded. Map the geometric information of the reference point cloud and the point cloud to be encoded into a two-dimensional distance map. Obtain the original distance map of the I frame of the reference point cloud and the original distance map of the P frame with the encoded point cloud, respectively. The point cloud sequence is encoded in the form of "IPPPIPPP..."

[0013] S2, obtain the posture state according to the IMU information in the Kitti data set, and calculate the transformation matrix M through the posture state;

[0014] S3, performing a formula operation on the reference point cloud I frame and the transformation matrix M to obtain a predicted distance map of the point cloud P frame to be encoded;

[0015] S4, performing a difference between the predicted distance map obtained in step S3 and the original distance map of the P frame in step S1 to obtain a prediction residual;

[0016] S5. quantize and JPEG-LS lossless encode the prediction residual obtained in step S4 to obtain residual coding information;

[0017] S6. The original distance map of the I frame in step S1 is also encoded using JPEG-LS to obtain I frame encoding information;

[0018] S7. Combine the transformation matrix M in step S2, the residual coding information in step S5, and the I frame coding information in step S6 to output a code stream.

[0019] Furthermore, the specific steps of the decoding process are as follows:

[0020] M1, after receiving the code stream, extracts the I frame coding information in the code stream and obtains the I frame distance map through JPEG-LS decoding;

[0021] M2, after receiving the code stream, extracts the transformation matrix M in the code stream, combines the information of the transformation matrix M with the I frame distance map obtained in step M1 to calculate the predicted distance map;

[0022] M3, after receiving the code stream, extracts the residual coding information in the code stream, and performs inverse quantization and JPEG-LS decoding on it to obtain the prediction residual;

[0023] M4, reconstruct the P frame distance map by combining the prediction residual obtained by M3 and the prediction distance map obtained by M2;

[0024] M5. Convert the I-frame distance map obtained in step M1 and the P-frame distance map obtained in step M4 into point clouds, and finally restore the two-dimensional distance map into a three-dimensional point cloud.

[0025] Furthermore, the formula for calculating the reference point cloud I frame and the transformation matrix M during the encoding and decoding processes is as follows:

[0026] Among them, i is any point in the I frame point cloud, p′ is any point in the predicted point cloud, M is the transformation matrix, R 3×3 is the rotation matrix, T 3×1 is the translation matrix.

[0027] Compared with the prior art, the present invention has the following beneficial effects:

[0028] 1. The present invention creatively maps the geometric information of a three-dimensional point cloud into a two-dimensional distance map, and simultaneously divides a point cloud sequence into a reference point cloud and a point cloud to be encoded. Utilizing the pose relationship between the reference point cloud and the point cloud to be encoded, an inter-frame prediction method for the point cloud to be encoded is designed. The reference point cloud is combined with a transformation matrix to predict the point cloud to be encoded. The point cloud to be encoded only needs to encode the differences between it and the original point cloud. The entire encoding and decoding process achieves data compression for continuous point cloud sequences, reducing the amount of data required for transmission, reducing the temporal redundancy of the point cloud sequence, and improving data compression performance.

[0029] 2. The method provided by the present invention has high coding performance when encoding and decoding lidar point cloud sequences. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 This is a schematic diagram of the encoding process of the present invention;

[0031] Figure 2 Schematic diagram of the decoding process of the present invention. DETAILED DESCRIPTION

[0032] The present invention will be further described below with reference to preferred drawings and embodiments.

[0033] The present invention provides a posture-based inter-frame encoding and decoding method for a laser radar point cloud. First, during the encoding process, the geometric information of a three-dimensional point cloud is mapped to a two-dimensional distance map, and the point cloud sequence is divided into a reference point cloud and a point cloud to be encoded. The posture relationship between the reference point cloud and the point cloud to be encoded is used to design an inter-frame prediction method for the point cloud to be encoded. Finally, the prediction residual is quantized and encoded, and the transformation matrix, residual coding information, and reference point cloud coding information are synthesized to output a code stream. During the decoding process, the transformation matrix, residual coding information, and reference point cloud coding information are sequentially decrypted from the code stream to obtain the original distance between the reference point cloud and the point cloud to be encoded on the two-dimensional distance map. Finally, the distance map is converted into a point cloud to restore the three-dimensional point cloud.

[0034] The encoding process includes the following steps:

[0035] S1. Divide the point cloud sequence into a reference point cloud and a point cloud to be encoded, map the geometric information of the reference point cloud and the point cloud to be encoded into a two-dimensional distance map, and obtain the original distance map of the I frame of the reference point cloud and the original distance map of the P frame with the encoded point cloud, respectively; the I frame point cloud is combined with the pose information to predict the P frame point cloud. The number of frames of the point cloud to be encoded predicted by one frame of the reference point cloud is determined by the parameter n. In this embodiment, the parameter n is set to 3, that is, the point cloud sequence is encoded in the form of "IPPIPPP...";

[0036] S2, obtain the posture state according to the IMU information in the Kitti data set, and calculate the transformation matrix M through the posture state; the calculation of the transformation matrix is ​​based on the translation acceleration in the IMU information and rotation rate Calculated; the transformation matrix includes the rotation matrix R 3×3 and the translation matrix T 3×1 ; The translation matrix calculation is shown in formula 1, where Yes The translation parameters calculated by the first-order Runge-Kutta numerical method;

[0037]

[0038] Similarly, the rotation matrix R 3×3 The calculation is shown in formula 2, Yes The rotation parameters are calculated using the first-order Runge-Kutta numerical method;

[0039]

[0040] Finally, according to the translation matrix T 3×1 and the rotation matrix R 3×3 , we get the transformation matrix M, as shown in formula 3;

[0041]

[0042] S3, performing a formula operation on the reference point cloud I frame and the transformation matrix M to obtain a predicted distance map of the point cloud P frame to be encoded;

[0043] S4, performing a difference between the predicted distance map obtained in step S3 and the original distance map of the P frame in step S1 to obtain a prediction residual;

[0044] S5. Perform quantization and JPEG-LS lossless encoding on the prediction residual obtained in step S4 to obtain residual coding information. The operations of the relevant quantization and JPEG-LS lossless encoding are described in the following paper:

[0045] Quantization processing: TU C, TAKEUCHI E, CARBALLO A, et al. Real-time streaming pointcloud compression for 3D LiDAR sensor using U-NET[J]. IEEE Access, 2019, 7: 113616-113625.

[0046] JPEG-LS: MARKOS E, PAPADONIKOLAKIS, ATHANASIOS P, et al. Efficient high-performance implementation of JPEG-LS encoder[J]. Journal of Real-Time ImageProcessing, 2008, 3(4): 303-310.

[0047] S6. The original distance map of the I frame in step S1 is also encoded using JPEG-LS to obtain I frame encoding information;

[0048] S7. Combine the transformation matrix M in step S2, the residual coding information in step S5, and the I frame coding information in step S6 to output a code stream.

[0049] The specific steps of the decoding process are as follows:

[0050] M1, after receiving the code stream, extracts the I frame coding information in the code stream and obtains the I frame distance map through JPEG-LS decoding;

[0051] M2, after receiving the code stream, extracts the transformation matrix M in the code stream, combines the information of the transformation matrix M with the I frame distance map obtained in step M1 to calculate the predicted distance map;

[0052] M3, after receiving the code stream, extracts the residual coding information in the code stream, and performs inverse quantization and JPEG-LS decoding on it to obtain the prediction residual;

[0053] M4, reconstruct the P frame distance map by combining the prediction residual obtained by M3 and the prediction distance map obtained by M2;

[0054] M5. Convert the I-frame distance map obtained in step M1 and the P-frame distance map obtained in step M4 into point clouds, and finally restore the two-dimensional distance map into a three-dimensional point cloud.

[0055] The formula for calculating the reference point cloud I frame and the transformation matrix M during the encoding and decoding processes is as follows:

[0056] Among them, i is any point in the I frame point cloud, p′ is any point in the predicted point cloud, M is the transformation matrix, R 3×3 is the rotation matrix, T 3×1 is the translation matrix.

[0057] In this embodiment, the above method is used to conduct experimental tests on four scene sequences of the KITTI dataset. The experimental results are shown in Table 1. In the experiment, 100 frames of each scene sequence are selected for encoding and decoding. The data in the table are the experimental results of the average point cloud of one frame, where the residual quantization parameter is selected to be the minimum. The experimental evaluation criteria are the compression rate and the peak signal-to-noise ratio (PSNR) of the distance map. The compression rate determines the compression performance of the data, and the PSNR determines the reconstruction quality of the point cloud, as shown in Formulas 5 and 6.

[0058] Where CR represents the compression ratio, Size coding Indicates the size of the output stream after encoding, Size original Indicates the size of the original data, RMSE is the root mean square error, MAX=13000;

[0059]

[0060]

[0061] Table 1 Experimental test results

[0062]

[0063] It can be seen from the above test results that the method provided by the present invention has high encoding performance when encoding and decoding lidar point cloud sequences.

[0064] Although the present invention has been disclosed above with preferred embodiments, it is not intended to limit the present invention. Any person skilled in the art can make possible changes and modifications to the technical solutions of the present invention by using the methods and technical contents disclosed above without departing from the spirit and scope of the present invention. Therefore, any simple modification, equivalent change and modification made to the above embodiments based on the technical essence of the present invention without departing from the content of the technical solution of the present invention shall fall within the scope of protection of the technical solution of the present invention. The above is only a preferred embodiment of the present invention. All equivalent changes and modifications made according to the scope of the patent application of the present invention shall fall within the scope of coverage of the present invention.

Claims

1. A method for inter-frame encoding and decoding of LiDAR point clouds based on posture state, characterized by: Encoding process: S1. Divide the point cloud sequence into a reference point cloud and a point cloud to be encoded. Map the geometric information of the reference point cloud and the point cloud to be encoded into a two-dimensional distance map. Obtain the original distance map of the I frame of the reference point cloud and the original distance map of the P frame of the point cloud to be encoded. The point cloud sequence is encoded in the form of "IPPPIPPP..."; S2, obtain the posture state according to the IMU information in the Kitti data set, and calculate the transformation matrix M through the posture state; S3, the reference point cloud I frame and the transformation matrix M are used to calculate the predicted distance map of the point cloud P frame to be encoded , the formula is expressed as: ； in, is any point in the I-frame point cloud, is any point in the predicted point cloud, M is the transformation matrix, is the rotation matrix, is the translation matrix; S4, the predicted distance map obtained in step S3 The original distance map of the P frame in step S1 Take the difference to get the prediction residual , , and then get the prediction residual; S5. quantize and JPEG-LS lossless encode the prediction residual obtained in step S4 to obtain residual coding information; S6. The original distance map of the I frame in step S1 is also encoded using JPEG-LS to obtain I frame encoding information; S7, synthesize the transformation matrix M in step S2, the residual coding information in step S5 and the I frame coding information in step S6 into an output stream; Decoding process: The transformation matrix, residual coding information and reference point cloud coding information are sequentially extracted from the bitstream to obtain the original distance between the reference point cloud and the point cloud to be encoded on the two-dimensional distance map. Finally, the distance map is converted into a point cloud to restore the three-dimensional point cloud.

2. The inter-frame encoding and decoding method of a laser radar point cloud based on posture state according to claim 1, characterized in that: The specific steps of the decoding process are as follows: M1, after receiving the code stream, extracts the I frame encoding information in the code stream and obtains the I frame original distance map through JPEG-LS decoding; M2, after receiving the code stream, extracts the transformation matrix M in the code stream, combines the transformation matrix M information with the original distance map of the I frame obtained in step M1 to calculate the predicted distance map; M3, after receiving the code stream, extracts the residual coding information in the code stream, and performs inverse quantization and JPEG-LS decoding on it to obtain the prediction residual; M4, reconstruct the original distance map of the P frame by combining the prediction residual obtained by M3 and the predicted distance map obtained by M2; M5. Convert the original distance map of the I frame obtained in step M1 and the original distance map of the P frame obtained in step M4 into a point cloud, and finally restore the two-dimensional distance map into a three-dimensional point cloud.

Images