Laser radar point cloud data coding method

Abstract

The invention discloses a method for coding a laser radar point cloud in unmanned driving, which comprises the following steps of: firstly, dividing the point cloud into an intra-frame point cloud and an inter-frame point cloud, removing point cloud data space redundancy by adopting a clustering-based intra-frame prediction technology aiming at the intra-frame point cloud, and predicting the inter-frame point cloud based on the point cloud prediction technology aiming at the inter-frame point cloud; and the point cloud data time redundancy is removed. And the residual data is coded, so that the point cloud compression rate is improved.

Description

Technical Field

[0001] The present invention belongs to the field of unmanned driving, and in particular relates to data processing technology in unmanned driving, and more specifically, to a method for encoding laser radar point clouds in unmanned driving. Background Art

[0002] The establishment, loading and updating of unmanned driving maps require the transmission of massive point cloud data to the server for processing, and the transmission of the generated map to the car or robot terminal to achieve positioning, navigation and other functions. At the same time, as construction robots are continuously put into use on construction sites, the loading and updating of navigation maps also require the transmission of a large amount of point cloud data. In addition, robots dedicated to building surveying and mapping need to transmit a large amount of high-precision point cloud data from the collection site to the server for storage and processing. The remote transmission of a large amount of point cloud data is very time-consuming, especially when the network is unstable. If the point cloud data is compressed like video stream data before transmission, it will greatly reduce the data transmission time and reduce the impact of network instability.

[0003] With the widespread application of 16-line, 32-line and 64-line LiDARs in unmanned vehicles, 128-line LiDAR (VLS-128) and solid-state LiDAR are also being gradually promoted. Compared with Velodyne's 64-line LiDAR data, VLS-128 has a 10-fold increase in resolution and a 3-fold increase in detection distance to 300 meters compared to Velodyne's 64-line LiDAR HDL-64. Point cloud data will grow exponentially. This brings greater challenges to the already immature LiDAR point cloud data encoding scheme. At present, the compression scheme for LiDAR point cloud data is still being formulated and improved. The old encoding technology has low compression efficiency, and the emerging compression technology still has a lot of room for improvement.

[0004] The compression of disordered point cloud data mainly adopts the point cloud data compression algorithm based on octree, which is the most common method for compressing point cloud. These algorithms generally have some problems: on the one hand, the data compression ratio is low and the spatial and temporal redundancy of point cloud data cannot be fully removed; on the other hand, at a high compression ratio, the point cloud is distorted. Since the tasks of mobile robot navigation, obstacle avoidance, target detection and tracking have high requirements on point cloud accuracy, these algorithms are not suitable for the field of mobile robots. The literature "Ahn JK, Lee KY, Sim JY, et al. Large-Scale 3D Point Cloud Compression Using Adaptive Radial Distance Prediction in Hybrid Coordinate Domains[J]. IEEE Journal of Selected Topics in Signal Processing, 2015, 9(3): 422-434." proposes a point cloud data compression algorithm based on adaptive prediction for large-scale 3D point cloud data generated by terrestrial laser scanners. The point cloud data is mapped to the depth image. Compared with traditional image or video encoding algorithms, their technology achieves better compression performance, but their algorithm cannot remove the temporal redundancy of point cloud data. The paper "Tu Chenxi, Takeuchi E, Carballo, A, et al. Motion Analysis and Performance Improved Method for 3D LiDAR Sensor Data Compression [J]. IEEE Transactions on Intelligent Transportation Systems, 2021, 22 (1): 243-256." proposes an image-based LiDAR point cloud sequence compression algorithm. They convert LiDAR data losslessly into a depth map and then use image / video encoding technology to remove data redundancy. Each pixel value in the image represents a color value, while each pixel value in the depth map represents the distance from the object to the center of the LiDAR. Using image or video encoding methods cannot effectively remove the information redundancy of the point cloud depth map sequence, and the compression efficiency is low. Summary of the invention

[0005] The purpose of the present invention is to overcome the deficiencies in the prior art and provide a method for encoding laser radar point cloud data. The technical solution of the present invention is to design a laser radar point cloud encoding algorithm framework, divide the point cloud into intra-frame point cloud and inter-frame point cloud, adopt clustering-based intra-frame prediction technology for intra-frame point cloud to remove spatial redundancy of point cloud data, and predict inter-frame point cloud based on point cloud prediction technology for inter-frame point cloud to remove temporal redundancy of point cloud data. Encode the residual data to improve the compression rate of point cloud.

[0006] The objective of the present invention is achieved through the following technical solutions:

[0007] A laser radar point cloud data encoding method, comprising:

[0008] S1. Preprocess the point cloud data obtained by the laser radar to remove noise points and ensure the purity and reliability of the point cloud data;

[0009] S2. dividing the point cloud data into intra-frame point cloud and inter-frame point cloud, encoding the intra-frame point cloud using intra-frame prediction coding technology, and encoding the inter-frame point cloud using inter-frame prediction technology based on the encoded intra-frame point cloud;

[0010] S3. Convert the point cloud data into a depth map. Since the vehicle-mounted LiDAR point cloud data is ordered, the formula for converting the point cloud data into a depth map is:

[0011]

[0012] Where (x, y, z) is the coordinate of the point cloud data in space, θ is the angle between the point cloud and the z-axis, and ф is the angle between the projection of the point cloud in the xoy plane and the x-axis;

[0013] S4. In the intra-frame prediction stage, the point cloud data is segmented into independent regions;

[0014] S5. Based on the segmentation results, the data fitting method is used to fit the ideal plane to the ground points. The formula is:

[0015]

[0016] Where n represents the normal vector of the ideal plane, d represents the vertical distance parameter from the origin to the fitted plane, and p i represents each point on the real LiDAR point cloud data, and s1 represents the fitted plane;

[0017] S6. Use sphere and cylinder to fit other segmented areas, the formula is:

[0018] d(s2,p i )=||cp i ||-r (5)

[0019]

[0020] Formula (5) is the spherical surface fitting equation, c represents the center of the sphere, and r is the radius of the sphere; Formula (6) is the cylindrical surface fitting equation, where q0 represents the coordinates of a point on the cylinder axis, a represents the normal vector of the cylinder axis, and r is the radius of the cylinder, p i represents each point on the real LiDAR point cloud data, s2 and s3 represent the fitted sphere and cylinder respectively;

[0021] S7. Using the fitted surface and LiDAR parameters, calculate the fitted LiDAR point cloud data, and convert the fitted LiDAR point cloud data into intra-frame predicted depth map data, which is calculated as X pred-intra ; The fitted surface includes the fitted plane, spherical surface and cylindrical surface;

[0022] S8. Convert the real laser radar point cloud data into a depth map, counted as X real ;

[0023] S9. Calculate X real and X pred-intra The difference is taken as the intra-frame residual data R intra , and use the lossless coding algorithm to encode the residual data R intra Encode to obtain intra-frame code stream;

[0024] R intra =X real -X pred-intra (7)

[0025] S10. Constructing inter-frame lidar point cloud prediction network based on convolutional LSTM module;

[0026] S11. Decode the intra-frame encoded laser radar point cloud data to obtain the decoded point cloud sequence X at time t=0, t=1, ..., t=T, calculated as, X={X t＝0 , X t＝1 , …, X t＝T};

[0027] S12. The input of the inter-frame lidar point cloud prediction network is X, and the input point cloud sequence is P, where P = {P0, P1, ..., P T , P T+1}, where P0, P1, ..., P T , P T+1 Respectively represent the predicted point cloud data at time t=0, t=1,…, t=T, t=T+1; Represents the initialization information input to the ConvLSTM module at the Lth layer at time t=0; Represents the initialization information input to the ConvLSTM module at the lth layer at time t=0; It is the output of the GHU module at the lth layer at time t; It is the output result of the GHU module at the 0th layer at time t; the inter-frame lidar point cloud prediction network consists of a convolutional LSTM module and a GHU module. The formula of the convolutional LSTM module is:

[0028]

[0029]

[0030]

[0031] Among them, ConvLSTM is a convolutional LSTM module. is the prediction error of layer l at time t, is the intermediate state information output of the ConvLSTM module at time t-1 of layer l, They are the intermediate state information output of the ConvLSTM modules at time t in layers l-1 and l+1 respectively. Output result of ConvLSTM module at layer l at time t-1.

[0032] S13. Calculate the real lidar point cloud X at time t+1 t+1 The lidar point cloud P at time t+1 predicted by the inter-frame lidar point cloud prediction network t+1 The difference is calculated as P inter

[0033] P inter = X t+1 -P t+1 (7)

[0034] S14. Use lossless coding algorithm to encode the inter-frame residual data P inter Encoding is performed to obtain the inter-frame code stream, thereby removing the time redundancy of the point cloud sequence.

[0035] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the laser radar point cloud data encoding method when executing the program.

[0036] The present invention also provides a computer-readable storage medium on which a computer program is stored, and when the computer program is executed by a processor, the steps of the laser radar point cloud data encoding method are implemented.

[0037] Compared with the prior art, the technical solution of the present invention has the following beneficial effects:

[0038] 1. Significantly improve the compression efficiency of point cloud data: By adopting clustering-based intra-frame prediction technology and point cloud prediction technology, the spatial and temporal redundancy of point cloud data is effectively removed, thereby greatly improving the compression rate of point cloud data while ensuring data accuracy and reducing the pressure of data storage and transmission.

[0039] 2. Improve the restoration accuracy of point cloud data: Intra-frame point cloud coding technology uses surface fitting technology to make the fitted LiDAR point cloud data highly close to the real point cloud data, reducing the fitting error. By losslessly encoding the residual data, the restoration accuracy of the point cloud data is further improved, ensuring that the point cloud data quality can still be maintained at a high compression ratio.

[0040] 3. Reduce the delay in point cloud data transmission: Since the amount of point cloud data is greatly reduced after compression, the data transmission time is significantly shortened. Especially in the case of unstable network, the data transmission delay can be significantly reduced, thereby improving the real-time and stability of the unmanned driving system.

[0041] 4. Adapt to the needs of various lidar devices: The encoding method proposed in the present invention can be widely applied to various types of lidar devices, such as 64-line, 128-line and solid-state LiDAR. It has strong adaptability and versatility, and meets the needs of different devices for efficient data processing.

[0042] 5. Support high-precision navigation and target detection: The encoding method of the present invention can still retain key point cloud detail information under high compression ratio conditions. It is suitable for tasks such as mobile robot navigation, obstacle avoidance and target detection that require high point cloud accuracy, thereby improving the overall performance of the unmanned driving system.

[0043] Through the above technical solution, the present invention achieves higher efficiency and accuracy in the compression and transmission of point cloud data, providing strong support for the development of unmanned driving technology. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 .The laser radar point cloud data compression algorithm framework of the present invention.

[0045] Figure 2 .Schematic diagram of converting the laser radar spherical coordinate system into the Cartesian coordinate system.

[0046] Figure 3 .Block diagram of the intra-frame lidar point cloud encoding algorithm.

[0047] Figure 4 .Block diagram of the inter-frame lidar point cloud encoding algorithm. DETAILED DESCRIPTION

[0048] In order to enable those skilled in the art to better understand the technical solution of the present invention, the present invention is described in detail below in conjunction with the accompanying drawings and specific implementation methods. Figure 1 A lossless compression algorithm for laser radar point cloud data according to an embodiment of the present invention is shown. Figure 1 As shown, an embodiment of the present invention provides a lossless compression algorithm for lidar point cloud data, and the method includes the following steps.

[0049] A laser radar point cloud data compression method, the method comprising:

[0050] Step 1. Preprocess the point cloud data acquired by the lidar to remove noise points and ensure the purity and reliability of the point cloud data;

[0051] Step 2. Divide the point cloud data into intra-frame point cloud and inter-frame point cloud, and use intra-frame prediction coding technology to encode the intra-frame point cloud, see Figure 3 ; According to the encoded intra-frame point cloud, the inter-frame point cloud is encoded using the inter-frame prediction technology; see Figure 4 .

[0052] Step 3. Convert the point cloud data into a depth map. Since the vehicle-mounted LiDAR point cloud data is ordered, convert the point cloud data into a depth map; see Figure 2 , the formula is:

[0053]

[0054] Where (x, y, z) is the coordinate of the point cloud data in space, θ is the angle between the point cloud and the z-axis, and ф is the angle between the projection of the point cloud in the xoy plane and the x-axis;

[0055] Step 4. In the intra-frame prediction phase, the point cloud data is segmented into independent regions;

[0056] Step 5. Based on the segmentation results, the data fitting method is used to fit the ideal plane to the ground points; the formula is:

[0057]

[0058] Where n represents the normal vector of the ideal plane, d represents the vertical distance parameter from the origin to the fitted plane, and p i represents each point on the real LiDAR point cloud data, and s1 represents the fitted plane;

[0059] Step 6. Use spheres and cylinders to fit other segmented areas; the formula is:

[0060] d(s2,p i )=||cp i ||-r (5)

[0061]

[0062] Formula (5) is the spherical surface fitting equation, c represents the center of the sphere, and r is the radius of the sphere; Formula (6) is the cylindrical surface fitting equation, where q0 represents the coordinates of a point on the cylinder axis, a represents the normal vector of the cylinder axis, and r is the radius of the cylinder, p i represents each point on the real LiDAR point cloud data, s2 and s3 represent the fitted sphere and cylinder respectively;

[0063] Step 7. Using the fitted surface and LiDAR parameters, calculate the fitted LiDAR point cloud data and convert these fitted LiDAR point cloud data into intra-frame predicted depth map data, counted as X pred-intra ; The fitted surface includes the fitted plane, spherical surface and cylindrical surface;

[0064] Step 8. Convert the real LiDAR point cloud data into a depth map, counted as X real ;

[0065] Step 9. Calculate X real and X pred-intra The difference is taken as the intra-frame residual data R intra , and use the lossless coding algorithm to encode the residual data R intra Encode to obtain intra-frame code stream;

[0066] R intra =X real -X pred-intra (7)

[0067] Step 10. Build an inter-frame lidar point cloud prediction network based on the convolutional LSTM module;

[0068] Step 11. Decode the intra-frame encoded LiDAR point cloud data to obtain the decoded point cloud sequence X at time t=0, t=1, …, t=T, calculated as, X={X t＝0 , X t＝1 , …, X t＝T};

[0069] Step 12. The input of the inter-frame lidar point cloud prediction network is X, and the input point cloud sequence is P, where P = {P0, P1, ..., P T , P T+1}, where P0, P1, ..., P T , P T+1 Respectively represent the predicted point cloud data at time t=0, t=1,…, t=T, t=T+1; Represents the initialization information input to the ConvLSTM module at the Lth layer at time t=0; Represents the initialization information input to the ConvLSTM module at the lth layer at time t=0; It is the output of the GHU module at the lth layer at time t; It is the output result of the GHU module at the 0th layer at time t; the inter-frame lidar point cloud prediction network is composed of convolutional LSTM modules, and the formula of the convolutional LSTM module is:

[0070]

[0071]

[0072] Among them, ConvLSTM is a convolutional LSTM module. is the prediction error of layer l at time t, is the intermediate state information output of the ConvLSTM module at time t-1 of layer l, They are the intermediate state information output of the ConvLSTM modules at time t in layers l-1 and l+1 respectively. Output result of ConvLSTM module at layer l at time t-1.

[0073] Step 13. Calculate the real lidar point cloud X at time t+1 t+1 The lidar point cloud P at time t+1 predicted by the inter-frame lidar point cloud prediction network t+1 The difference is calculated as P inter ;

[0074] P inter = X t+1 -P t+1 (7)

[0075] Step 14: Use a lossless coding algorithm to encode the inter-frame residual data P inter Encoding is performed to obtain the inter-frame code stream, thereby removing the time redundancy of the point cloud sequence.

[0076] Preferably, the embodiments of the present application also provide a specific implementation of an electronic device capable of implementing all steps in the laser radar point cloud data encoding method in the above embodiment, and the electronic device specifically includes the following contents:

[0077] Processor, memory, communications interface and bus;

[0078] Among them, the processor, memory, and communication interface communicate with each other through the bus; the communication interface is used to realize information transmission between related devices such as server-side devices, metering devices, and user-side devices.

[0079] The processor is used to call the computer program in the memory, and when the processor executes the computer program, all the steps in the laser radar point cloud data encoding method in the above embodiment are implemented.

[0080] An embodiment of the present application also provides a computer-readable storage medium capable of implementing all the steps of the laser radar point cloud data encoding method in the above embodiment. A computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, all the steps of the laser radar point cloud data encoding method in the above embodiment are implemented.

[0081] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the hardware + program embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0082] The above is a description of a specific embodiment of the specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0083] Although the present application provides method operation steps such as embodiments or flow charts, more or fewer operation steps may be included based on conventional or non-creative labor. The order of steps listed in the embodiments is only one way of executing the order of many steps and does not represent the only execution order. When the actual device or client product is executed, it can be executed in the order of the method shown in the embodiments or the drawings or in parallel (for example, in a parallel processor or multi-threaded processing environment).

[0084] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0085] These computer program instructions may also be stored in a computer readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0086] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0087] The present invention is not limited to the embodiments described above. The above description of the specific embodiments is intended to describe and illustrate the technical solution of the present invention. The above specific embodiments are merely illustrative and not restrictive. Without departing from the scope of the present invention and the scope of protection of the claims, a person of ordinary skill in the art can also make many forms of specific changes under the guidance of the present invention, which all fall within the scope of protection of the present invention.

Claims

1. A laser radar point cloud data encoding method, characterized in that: include S1. Preprocess the point cloud data obtained by the laser radar to remove noise points and ensure the purity and reliability of the point cloud data; S2. dividing the point cloud data into intra-frame point cloud and inter-frame point cloud, encoding the intra-frame point cloud using intra-frame prediction coding technology, and encoding the inter-frame point cloud using inter-frame prediction technology based on the encoded intra-frame point cloud; S3. Convert the point cloud data into a depth map. Since the vehicle-mounted LiDAR point cloud data is ordered, the formula for converting the point cloud data into a depth map is: Where (x, y, z) is the coordinate of the point cloud data in space, θ is the angle between the point cloud and the z-axis, and ф is the angle between the projection of the point cloud in the xoy plane and the x-axis; S4. In the intra-frame prediction stage, the point cloud data is segmented into independent regions; S5. Based on the segmentation results, the data fitting method is used to fit the ideal plane to the ground points. The formula is: Where n represents the normal vector of the ideal plane, d represents the vertical distance parameter from the origin to the fitted plane, and p i represents each point on the real LiDAR point cloud data, and s1 represents the fitted plane; S6. Use sphere and cylinder to fit other segmented areas, the formula is: d(s2,p i )=||c-p i ||-r (5) Formula (5) is the spherical surface fitting equation, c represents the center of the sphere, and r is the radius of the sphere; Formula (6) is the cylindrical surface fitting equation, where q0 represents the coordinates of a point on the cylinder axis, a represents the normal vector of the cylinder axis, and r is the radius of the cylinder, p i represents each point on the real LiDAR point cloud data, s2 and s3 represent the fitted sphere and cylinder respectively; S7. Using the fitted surface and LiDAR parameters, calculate the fitted LiDAR point cloud data, and convert the fitted LiDAR point cloud data into intra-frame predicted depth map data, which is calculated as X pred-intra ; The fitted surface includes the fitted plane, spherical surface and cylindrical surface; S8. Convert the real laser radar point cloud data into a depth map, counted as X real ; S9. Calculate X real and X pred-intra The difference is taken as the intra-frame residual data R intra , and use the lossless coding algorithm to encode the residual data R intra Encode to obtain intra-frame code stream; R intra = X real - X pred -intra (7) S10. Constructing inter-frame lidar point cloud prediction network based on convolutional LSTM module; S11. Decode the intra-frame encoded laser radar point cloud data to obtain the decoded point cloud sequence X at time t=0, t=1, ..., t=T, calculated as, X={X t＝0 , X t＝1 , …, X t＝T }; S12. The input of the inter-frame lidar point cloud prediction network is X, and the input point cloud sequence is P, where P = {P0, P1, ..., P T , P T+1 }, where P0, P1, ..., P T , P T+1 Respectively represent the predicted point cloud data at time t=0, t=1,…, t=T, t=T+1; Represents the initialization information input to the ConvLSTM module at the Lth layer at time t=0; Represents the initialization information input to the ConvLSTM module at the lth layer at time t=0; It is the output of the GHU module at the lth layer at time t; It is the output result of the GHU module at the 0th layer at time t; the inter-frame lidar point cloud prediction network consists of a convolutional LSTM module and a GHU module. The formula of the convolutional LSTM module is: Among them, ConvLSTM is a convolutional LSTM module. is the prediction error of layer l at time t, is the intermediate state information output of the ConvLSTM module at time t-1 of layer l, They are the intermediate state information output of the ConvLSTM modules at time t in layers l-1 and l+1 respectively. Output result of ConvLSTM module at layer l at time t-1. S13. Calculate the real lidar point cloud X at time t+1 t+1 The lidar point cloud P at time t+1 predicted by the inter-frame lidar point cloud prediction network t+1 The difference is calculated as P inter P inter = X t+1 - P t+1 (7) S14. Use lossless coding algorithm to encode the inter-frame residual data P inter Encoding is performed to obtain the inter-frame code stream, thereby removing the time redundancy of the point cloud sequence.

2. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the steps of the laser radar point cloud data encoding method according to claim 1 are implemented.

3. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the laser radar point cloud data encoding method according to claim 1 are implemented.

Images