A LiDAR point cloud coding method and system based on inter-frame motion prediction

Abstract

The application relates to a LiDAR point cloud coding and decoding method and system based on inter-frame motion prediction, which comprises an encoding process and a decoding process; the encoding process comprises: inputting a laser radar point cloud sequence and preprocessing; dividing the point cloud through a classification network to separate environment point cloud and target point cloud; according to the characteristics of different types of point cloud, corresponding methods are used for motion prediction to estimate inter-frame motion transformation information; the obtained motion transformation information and the reference point cloud required for prediction are selected to write a corresponding encoding method into a bit stream for transmission; the decoding process comprises: receiving and decoding the bit stream of the encoding end to generate decoded reference point cloud and motion transformation information; the decoded motion transformation information is applied to the corresponding point cloud after decoding according to the corresponding relationship to realize inter-frame prediction of the point cloud; and the point cloud after prediction is fused to output complete reconstructed point cloud. The method and system are beneficial to improving inter-frame registration accuracy, eliminating time domain redundancy and realizing space-time coding of large-scale point cloud sequences.

Description

Technical Field

[0001] The present invention belongs to the technical field of point cloud processing, and in particular relates to a LiDAR point cloud encoding and decoding method and system based on inter-frame motion prediction. Background Art

[0002] LiDAR is an essential sensor for autonomous vehicles, drones, robots, and other autonomous vehicles. It plays a vital role in applications such as positioning / path planning, target detection, and high-precision mapping. The Moving Picture Experts Group (MPEG) standard defines LiDAR point clouds as the third type of dynamically acquired point cloud. These data volumes are enormous. The Velodyne LiDAR HDL-64 sensor can generate over 100,000 points per scan, or over 84 billion points per day. Without compression, this massive amount of data poses significant challenges to storage space and network transmission bandwidth, potentially severely restricting its widespread use in various potential applications. Therefore, research on efficient point cloud compression coding techniques is both urgent and necessary.

[0003] In recent years, to address the need for point cloud data compression and improve algorithm performance, many researchers have proposed various compression methods, including those based on three-dimensional space partitioning, two-dimensional image projection, and one-dimensional prediction. Representative examples include the Geometric Point Cloud Compression (G-PCC) algorithm and the Video-based Point Cloud Compression (V-PCC) algorithm. While these methods have achieved some success, due to the sparsity and uneven distribution of LiDAR point clouds, most existing methods focus on reducing spatial redundancy, leaving significant room for improvement in compressing LiDAR point clouds. Large-scale scene point clouds not only contain a significant amount of spatially redundant information but can also be viewed as continuous frames of data that change along the time axis. Frame-by-frame point clouds have high temporal correlation, so utilizing inter-frame prediction can further improve compression performance. However, inter-frame prediction typically comes with higher computational costs and suffers from inaccurate global motion estimation during the inter-frame prediction process. Therefore, it is urgent to deeply analyze the unique data characteristics of LiDAR point clouds and design more adaptive compression schemes to achieve more efficient point cloud compression. Summary of the Invention

[0004] The purpose of the present invention is to provide a LiDAR point cloud encoding and decoding method and system based on inter-frame motion prediction, which is conducive to improving the inter-frame registration accuracy, eliminating temporal redundancy, and realizing spatiotemporal encoding of large-scale point cloud sequences.

[0005] In order to achieve the above object, the technical solution adopted by the present invention is: a LiDAR point cloud encoding and decoding method based on inter-frame motion prediction, including an encoding process and a decoding process;

[0006] The encoding process includes the following steps:

[0007] Step E1: Input the lidar point cloud sequence, including the three-dimensional coordinates, timestamp and intensity information of the point cloud, and perform preprocessing to obtain preprocessed point cloud data;

[0008] Step E2: Perform point cloud segmentation through a classification network, using the point information and category information of the point cloud data to separate the environment point cloud and the target point cloud;

[0009] Step E3: Using corresponding methods to perform motion prediction based on different types of point cloud characteristics to estimate inter-frame motion transformation information;

[0010] Step E4: Select a corresponding encoding method to write the obtained motion transformation information and the reference point cloud required for prediction into a bit stream for transmission;

[0011] The decoding process includes the following steps:

[0012] Step F1: Receive and decode the bit stream from the encoder to generate a decoded reference point cloud and motion transformation information;

[0013] Step F2: applying the decoded motion change information to the corresponding portion of the decoded point cloud according to the corresponding relationship to achieve inter-frame prediction of the point cloud;

[0014] Step F3: Fuse the predicted point clouds and output the complete reconstructed point cloud.

[0015] Furthermore, the step E1 includes the following steps:

[0016] E11: Divide the input lidar point cloud sequence into point cloud reference frames and prediction frames;

[0017] E12: Use radius filter to perform point cloud filtering to remove point cloud outliers and noise.

[0018] Furthermore, in step E2, the point cloud is segmented by a classification network based on the LiDAR point cloud. The classification network combines the geometric coordinate information and category information of the point and obtains the semantic score S by constructing a semantic branch learning. cn And group them to obtain the category segmentation results of the environment point cloud and the target point cloud; among them, the environment point cloud is a relatively fixed point in the scene, and the target point cloud is a point whose state has changed.

[0019] Furthermore, the step E3 includes the following steps:

[0020] E31: Complete the calibration of the LiDAR-IMU sensor and use the motion information provided by the IMU sensor to perform coarse registration of the environment point cloud;

[0021] E32: Using the result obtained in step E31 as the initial value of the transformation, further perform fine alignment through ICP to obtain the motion transformation information T s ;

[0022] E33: Use the DBSCAN clustering algorithm to extract target point cloud clusters and map a cluster label to each cluster. Use the cluster label as the key and the corresponding point cloud data as the value to construct the cluster data table h1, including the I-frame cluster data table h1_I and the P-frame cluster data table h1_P.

[0023] E34: Use the I frame cluster data table h1_I and the P frame cluster data table h1_P for inter-cluster matching and perform ICP registration. The obtained motion transformation information is {T di ,i is the cluster label value}, construct the cluster conversion table h2.

[0024] Furthermore, in the step E33, the DBSCAN algorithm is used to perform density-based clustering analysis on the target point cloud, and a cluster division result C = {C_1, C_2, ..., C_k} is output. A cluster label is mapped to each cluster, and clusters in different states are analyzed separately. Key-value pairs are constructed based on the correspondence between cluster labels and clusters, and a cluster data table h1 is created, including an I-frame cluster data table h1_I and a P-frame cluster data table h1_P. In the step E34, inter-cluster matching is performed using the I-frame cluster data table h1_I and the P-frame cluster data table h1_P to obtain inter-cluster correspondence, and then ICP inter-frame registration is performed to obtain motion transformation information {T di , i is the cluster label value}, and a cluster conversion table h2 is constructed for the I frame point cloud according to the obtained inter-cluster correspondence.

[0025] Furthermore, the step E4 includes the following steps:

[0026] E41: The pose transformation matrix obtained by ICP registration is input into the adaptive entropy encoder as motion transformation information to calculate the frequency of occurrence of matrix characters and determine the coding mode;

[0027] E42: In the case of uneven character frequency distribution, Huffman coding is used. It constructs a binary tree based on the frequency of characters, using long codes for low-frequency characters and short codes for high-frequency characters, thereby reducing computational complexity and improving real-time performance.

[0028] E43: In the data continuous or character frequency distribution uniform mode, arithmetic coding is used to get closer to the information entropy limit and achieve higher compression quality and compression efficiency;

[0029] E44: The outlier points filtered out of the P frame are sent together with the environment point cloud and target point cloud of the I frame as the reference point cloud required for prediction to the G-PCC encoder for encoding.

[0030] Furthermore, the step F1 includes the following steps:

[0031] F11: The motion transformation information T after Huffman decoding is obtained through Huffman decoding in the adaptive entropy decoder s ' and {T di ', i is the cluster label value};

[0032] F12: The motion transformation information T after arithmetic decoding is obtained by arithmetic decoding in the adaptive entropy decoder s ' and {T di ', i is the cluster label value};

[0033] F13: Obtain the decoded environment point cloud, decoded target point cloud and decoded outlier point cloud of the I frame through the G-PCC decoder, and construct the reconstructed I frame cluster data table h1_I'.

[0034] Furthermore, the step F2 includes the following steps:

[0035] F21: Transform the motion information T s 'Act on the decoded I-frame environment point cloud to predict the P-frame environment point cloud;

[0036] F22: Construct the decoded cluster conversion table h2' based on the inter-cluster correspondence relationship of the decoded motion transformation information. Each target point cloud cluster searches the decoded cluster conversion table h2' and converts the corresponding motion transformation information {T di ', i is the cluster label value} acts on the decoded I frame target point cloud in the cluster data table h1_I' to predict the target point cloud of the P frame.

[0037] Furthermore, in step F3, the predicted environment point cloud, target point cloud and decoded outlier point cloud are fused to finally output a complete reconstructed point cloud.

[0038] The present invention also provides a LiDAR point cloud encoding and decoding system based on inter-frame motion prediction, including a memory, a processor, and computer program instructions stored in the memory and capable of being executed by the processor. When the processor executes the computer program instructions, the above-mentioned method can be implemented.

[0039] Compared with the existing technology, the present invention has the following beneficial effects: the present invention provides a LiDAR point cloud encoding and decoding method and system based on inter-frame motion prediction. The method can analyze the impact of point clouds with different states on inter-frame motion estimation, take into account motion diversity, thereby improving the inter-frame registration accuracy, effectively reducing temporal redundancy, and achieving higher compression performance. It is suitable for the compression encoding of large-scale LiDAR continuous point cloud frame sequences. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 is a flowchart of a method implementation of an embodiment of the present invention;

[0041] Figure 2 2. Frame division diagram according to an embodiment of the present invention;

[0042] Figure 3 is a schematic diagram of a radius filter in an embodiment of the present invention;

[0043] Figure 4 Schematic diagram of point cloud classification in an embodiment of the present invention;

[0044] Figure 5 Schematic diagram of the LiDAR-IMU coordinate system in an embodiment of the present invention;

[0045] Figure 6 is a flow chart of point cloud registration in an embodiment of the present invention;

[0046] Figure 7 Schematic diagram of density-based clustering relationships in an embodiment of the present invention;

[0047] Figure 8 Flowchart of inter-cluster transformation in an embodiment of the present invention;

[0048] Figure 9 Adaptive entropy coding flow chart in an embodiment of the present invention;

[0049] Figure 10 This is a flowchart based on octree geometric coding in an embodiment of the present invention. DETAILED DESCRIPTION

[0050] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0051] It should be noted that the following detailed descriptions are exemplary and are intended to provide further explanation of the present application. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which the present application belongs.

[0052] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit the exemplary embodiments according to the present application. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that when the terms "comprise" and / or "include" are used in this specification, they indicate the presence of features, steps, operations, devices, components and / or combinations thereof.

[0053] The present invention provides a LiDAR (Light Detection and Ranging, LiDAR) point cloud encoding and decoding method based on inter-frame motion prediction.

[0054] like Figure 1 As shown, this embodiment provides a LiDAR (Light Detection and Ranging, LiDAR) point cloud encoding and decoding method based on inter-frame motion prediction, which includes two parts: encoding process and decoding process.

[0055] The encoding process includes the following steps:

[0056] Step E1: Input the lidar point cloud sequence, including the three-dimensional coordinates, timestamp and intensity information of the point cloud, and perform necessary preprocessing to obtain preprocessed point cloud data.

[0057] E11: Divide the input lidar point cloud sequence into point cloud reference frames and prediction frames.

[0058] E12: Use a radius filter to filter the point cloud to remove outliers and noise. Specifically, a search radius is defined to determine the neighborhood of each point, and the number of points within its neighborhood is counted. A threshold is set to determine whether a point is an outlier and filter it out.

[0059] Step E2: Perform point cloud segmentation through a classification network, making full use of the point information and category information of the point cloud data to separate the environment point cloud (scene fixed points) and the target point cloud (state change points).

[0060] Specifically, the point cloud is segmented by a classification network based on the LiDAR point cloud. The classification network combines the geometric coordinate information and category information of the point and obtains the semantic score S by constructing a semantic branch learning. cn The points are grouped and the category segmentation results of the environment point cloud and the target point cloud are obtained. The environment point cloud is the relatively fixed points in the scene, and the target point cloud is the points whose state changes.

[0061] Step E3: Use appropriate methods to perform motion prediction based on different types of point cloud characteristics and estimate inter-frame motion transformation information.

[0062] E31: Complete the calibration of the LiDAR-IMU sensor. Use the motion information provided by the IMU (Inertial Measurement Unit) sensor, including acceleration and angular velocity information, to reflect the vehicle's motion state. Align the LiDAR and IMU coordinate systems. After the two coordinate systems are aligned, the LiDAR's motion can be reflected by the IMU sensor, thereby achieving coarse registration with the environmental point cloud.

[0063] E32: Using the result obtained in step E31 as the initial value of the transformation, further fine-alignment is performed through ICP (Iterative Closest Point) to obtain more accurate motion transformation information T s .

[0064] E33: The target point cloud represents the traffic participants in the scene, with a distinct clustered distribution and strong local density. The DBSCAN (Density-Based Spatial Clustering of Applications with Noise) clustering algorithm is used to extract target point cloud clusters and assign a cluster label to each cluster. Cluster data table h1 is constructed, using cluster labels as keys and corresponding point cloud data as values. This table includes the I-frame cluster data table h1_I and the P-frame cluster data table h1_P.

[0065] E34: Use the I frame cluster data table h1_I and the P frame cluster data table h1_P for inter-cluster matching and perform ICP registration. The obtained motion transformation information is {T di ,i is the cluster label value corresponding to the motion transformation information}, and the cluster conversion table h2 is constructed.

[0066] In step E33, the DBSCAN algorithm is used to perform density-based cluster analysis on the target point cloud, outputting a cluster division result C = {C_1, C_2, ..., C_k}, and mapping a cluster label to each cluster. Clusters in different states are analyzed separately, and key-value pairs are constructed based on the correspondence between cluster labels and clusters to create a cluster data table h1, including an I-frame cluster data table h1_I and a P-frame cluster data table h1_P. In step E34, inter-cluster matching is performed using the I-frame cluster data table h1_I and the P-frame cluster data table h1_P to obtain inter-cluster correspondences, and then ICP inter-frame registration is performed to obtain motion transformation information {T di , i is the cluster label value}, and a cluster conversion table h2 is constructed for the I frame point cloud according to the obtained inter-cluster correspondence.

[0067] Step E4: Select a suitable encoding method to write the obtained motion transformation information and the reference point cloud required for prediction into a bit stream for transmission.

[0068] E41: The pose transformation matrix obtained by ICP registration is input into the adaptive entropy encoder as motion transformation information to calculate the frequency of occurrence of matrix characters and determine the coding mode.

[0069] E42: In the case of uneven character frequency distribution, Huffman coding is used. It constructs a binary tree based on the frequency of character occurrence, using longer codes for low-frequency characters and shorter codes for high-frequency characters, thereby effectively reducing computational complexity, achieving faster encoding and decoding speeds, and improving real-time performance.

[0070] E43: In modes where data is continuous or character frequency distribution is uniform, arithmetic coding is used to enable the reconstructed data to be closer to the information entropy limit, achieving higher compression quality and efficiency.

[0071] E44: The outlier points filtered out of the P frame are used together with the environment point cloud and target point cloud of the I frame as the reference point cloud required for prediction, and are transmitted to the G-PCC (Geometry-based Point Cloud Compression) encoder for encoding.

[0072] The decoding process includes the following steps:

[0073] Step F1: Receive and decode the bit stream from the encoding end, and generate decoded reference point cloud and motion transformation information.

[0074] F11: The motion transformation information T after Huffman decoding is obtained through Huffman decoding in the adaptive entropy decoder s ' and {T di ', i is the cluster label value}.

[0075] F12: The motion transformation information T after arithmetic decoding is obtained by arithmetic decoding in the adaptive entropy decoder s ' and {T di ', i is the cluster label value}.

[0076] F13: Obtain the decoded environment point cloud, decoded target point cloud and decoded outlier point cloud of the I frame through the G-PCC decoder, and construct the reconstructed I frame cluster data table h1_I'.

[0077] Step F2: Apply the decoded motion transformation information to the corresponding portion of the decoded point cloud according to the corresponding relationship to achieve inter-frame prediction of the point cloud.

[0078] F21: Transform the motion information T s 'Act on the decoded I-frame environment point cloud to perform pose conversion and predict the P-frame environment point cloud.

[0079] F22: Construct the decoded cluster conversion table h2' based on the inter-cluster correspondence of the decoded motion transformation information. Each target point cloud cluster searches for the corresponding relationship based on the decoded cluster conversion table h2'. di ', i is the cluster label value} acts on the decoded I-frame target point cloud in the cluster data table h1_I' to perform the corresponding pose transformation and predict the target point cloud of the P-frame.

[0080] Step F3: Fuse the predicted point clouds and output the complete reconstructed point cloud.

[0081] Specifically, by fusing the predicted environment point cloud, target point cloud and decoded outlier point cloud, a complete reconstructed point cloud is finally output.

[0082] In this example, the point cloud sequence frame division diagram of step E11 is as follows: Figure 2 As shown in the figure, the point cloud sequence is divided into reference frames (I frames) and prediction frames (P frames). I frames do not depend on other frames and can be encoded independently. P frames need to rely on the previous I frames to use inter-frame coding.

[0083] Since the laser radar sensor is affected by its own noise, uneven reflection of the object surface, and occlusion, a large number of abnormal values ​​may be generated during the distance measurement process. Therefore, in this embodiment, step E12 uses a radius filter to filter these abnormal points to improve the efficiency of subsequent registration and reduce the registration error. The principle is as follows Figure 3 As shown in the figure, to determine whether points 1, 2, and 3 are abnormal noise points, we need to set two thresholds: radius d and the number of points in the neighborhood num. Based on experience, we set d = 0.38 and th_num = 2. At this time, points 1 and 3 are determined to be noise points and are filtered out, while point 2 is retained.

[0084] In this embodiment, step E2 uses a classification network based on LiDAR point cloud to implement point cloud segmentation, and its implementation flow chart is as follows: Figure 4 As shown in Figure 2. The input of the network is the denoised LiDAR point cloud, which is pixelated into an ordered voxel grid and fed into a U-Net-style backbone network to obtain point features, including coordinate features and category features. Semantic branches are constructed based on point features, and semantic scores {S cn cn is the number of semantic categories in the point cloud. Semantic scores are grouped directly. The category of each point is determined by comparing the semantic score with a score threshold S_th. This method allows each point to be associated with multiple categories, reducing the risk of prediction errors. Finally, through the refinement stage, the object category with the highest category confidence score is obtained. The environment point cloud and target point cloud are derived through category label mapping.

[0085] In this embodiment, step E31 uses the acceleration and angular velocity information provided by the IMU sensor to reflect the vehicle's motion state, thereby obtaining the relative motion state of the point cloud. The environment point cloud represents relatively fixed points in the scene, while the target point cloud represents points whose states have changed. The formula is as follows:

[0086]

[0087] Among them, p e (t) represents the position of a point in the environment point cloud at time t, p o (t) represents the position of a point in the target point cloud at time t, V IMU (t) represents the velocity vector of the IMU sensor at time t, v e (t) and v o (t) represents the velocity vector of the midpoint of the environment point cloud and the target point cloud relative to the IMU.

[0088] However, due to the installation error angle and position error between the lidar and IMU, Figure 5 Different coordinate systems are shown, the world coordinate system is (OX1Y1Z1), the IMU coordinate system is (OX2Y2Z2), and the LiDAR coordinate system is (OX3Y3Z3). Therefore, it is necessary to obtain the rigid body transformation between the IMU coordinate system and the LiDAR coordinate system through LiDAR-IMU joint calibration. Used to describe the relative pose between the two to achieve environmental point cloud IMU registration and provide more accurate initial values ​​for subsequent fine registration. Expressed as:

[0089]

[0090] Among them, t imu is a 3×1 translation vector, R imu It represents a 3×3 rotation matrix.

[0091] Furthermore, in this embodiment, step E32 uses the IMU coarse registration result as the initial value and performs the following steps: Figure 6 The ICP fine registration process shown in the figure obtains the motion transformation information T of the environment point cloud. s First, the source point cloud s i With the target point cloud p i The corresponding points are matched to construct the corresponding point set. Then, for the corresponding points, the following objective function is constructed by the least squares method:

[0092]

[0093] Through continuous iterative optimization, the optimal rotation matrix R* and translation vector t* are calculated when the sum of squared errors reaches a minimum, and then the motion change information T is obtained. s , used to predict the target point cloud.

[0094] In this embodiment, step E33 uses the DBSCAN algorithm based on density clustering to determine whether to classify points into the same category by judging whether there is a density relationship between points (density direct, density reachable, density connected), thereby achieving clustering of the target point cloud. Figure 7 As shown, the neighborhood radius r and the minimum number of points in the neighborhood min_points are two important parameters. Points with a radius of r and a number greater than min_points are considered core points, such as p1 to p4. Points (boundary points) located in the neighborhood of each core point are directly accessible to the core point density. The core points connected by dotted arrows in the figure are all density-reachable points. In the neighborhood of these density-reachable points, all points are density-connected to each other, and these points can be clustered into one category. Outliers and non-core points outside the neighborhood are non-density-connected and are not classified into the same category. The final output cluster partition result C = {C_1, C_2, ..., C_k}, and the cluster index value is used as the cluster label;

[0095] In this embodiment, step E34 analyzes clusters of different states separately, constructs key-value pairs based on the correspondence between cluster labels and clusters, and constructs cluster data tables h1_I and h1_P for I frames and P frames respectively. Figure 8 As shown, by finding the point cloud clusters whose difference between the number of point cloud points (counts_P) and the three-dimensional coordinate range (coord_P(x), coord_P(y), coord_P(z)) in h1_P and the number of point cloud points (counts_I) and the three-dimensional coordinate range (coord_I(x), coord_I(y), coord_I(z)) in h1_I is less than the threshold, the cluster matching relationship is confirmed and ICP inter-frame registration is performed, and the obtained motion transformation information {T di , i is the cluster label value} is stored in the cluster conversion table h2. The threshold is set as follows:

[0096] Th_coord(k)=(maxcoord(k)-mincoord(k)) / n

[0097] Th_counts=(counts_I+counts_P) / n

[0098] Where Th_coord(k) and Th_counts represent the thresholds of the coordinate range and number of points, respectively. maxcoord(k) and mincoord(k) represent the maximum and minimum values ​​of the coordinate range. k represents the three coordinate axes, and n is empirically set to 20. counts_I and counts_P represent the number of points in the I-frame and P-frame point clouds, respectively.

[0099] In this embodiment, in order to achieve a balance between compression efficiency and computational complexity, the motion transformation information is compressed by adaptive entropy coding. The adaptive entropy coding flow chart is as follows: Figure 9 As shown, first calculate the frequency of occurrence of characters in the input data, and dynamically select the encoding method based on its frequency distribution characteristics. In the mode where the frequency of characters is unevenly distributed, select Huffman encoding through step E41. It is an encoding method that constructs a binary tree based on the frequency of occurrence of characters. Characters with high frequencies use shorter codes, which can effectively reduce the computational complexity and improve real-time performance. In the mode where the data is continuous or the frequency of occurrence of characters is very close, select arithmetic coding through step E42, which can be closer to the information entropy, improve the compression rate, and ensure the reconstruction quality. After adaptive encoding, calculate the bit rate and perform decoding quality verification. If the compression requirements are not met, adjust the encoding parameters and re-encode, and finally output a satisfactory result.

[0100] In this embodiment, step E43 compresses and encodes the I frame environment point cloud and target point cloud, as well as the P frame outlier point cloud using the mature international standard G-PCC. The geometric encoding process is as follows: Figure 10 As shown, first convert the original point cloud data coordinates into a d The octet is quantized and deduplicated within a 3D bounding box of a certain size. An occupancy code is predicted using an octree-based spatial structure, and finally arithmetic coding is performed to complete the geometric encoding. The octree structure recursively partitions the 3D geometry to find the maximum size of the scene. A cube is created at this size as the root node, and the voxels with points are subdivided into eight subcubes until a leaf node with a volume of 1 is obtained at the maximum recursive depth. The generated occupancy code is then encoded and transmitted.

[0101] In this embodiment, step F1 receives and decodes the bit stream of the encoding end to generate the decoded point cloud and pose transformation matrix. Specifically, step F11 obtains the decoded motion transformation information T through the Huffman decoding process of adaptive entropy coding. s ' and {T di ', i is the cluster label value}, stored in the decoded cluster conversion table h2'. Similarly, step F12 obtains the decoded motion transformation information T through the arithmetic decoding process of adaptive entropy coding. s ' and {Tdi ', i is the cluster label value}, stored in the decoded cluster conversion table h2'. Step F13 obtains the decoded environment point cloud and the decoded target point cloud of the I frame through the G-PCC decoder, and constructs the cluster data table h1_I' after the I frame is decoded, and the outlier point cloud after the P frame is decoded.

[0102] In this embodiment, the decoded motion transformation information is applied to the decoded corresponding category point cloud by searching the corresponding relationship in step F2 to perform posture conversion and realize inter-frame prediction. Specifically, step F21 converts T s The motion transformation information acts on the I frame environment point cloud to predict the P frame environment point cloud. In step F22, each target point cloud cluster converts the corresponding motion transformation information {T di ', i is the cluster label value} acts on the target point cloud cluster in h1_I' to predict the target point cloud of the P frame.

[0103] In this embodiment, the predicted environment point cloud and target point cloud are merged in step F3, and the decoded outlier point cloud is fused to finally output a complete reconstructed point cloud.

[0104] This embodiment also provides a LiDAR point cloud encoding and decoding system based on inter-frame motion prediction, including a memory, a processor, and computer program instructions stored in the memory and capable of being executed by the processor. When the processor executes the computer program instructions, the above method can be implemented.

[0105] The present invention provides a LiDAR point cloud encoding and decoding method and system based on inter-frame motion prediction. At the encoding end, the reference frame (I frame) is compressed by spatial redundancy, and the prediction frame (P frame) is compressed by eliminating temporal redundancy. First, the denoised LiDAR data is divided into an environmental point cloud and a target point cloud; then, the IMU coarse registration result is used as the initial value of the ICP registration algorithm to perform fine registration on the environmental point cloud. After the target point cloud is clustered by the DBSAN algorithm, ICP registration is performed according to the correspondence between the cluster labels to achieve inter-frame prediction; finally, the posture transformation matrix obtained by the registration is sent to the adaptive entropy encoder as motion transformation information, and the denoised I frame point cloud and the P frame outlier points are encoded together using G-PCC. At the decoding end, the motion transformation information and point cloud data are first decoded, and then the P frame is predicted and reconstructed based on the correspondence between the various point clouds of the I frame and the motion transformation information. Finally, the various parts are merged to obtain a complete reconstructed point cloud. The present invention adopts multiple technologies to improve the accuracy of inter-frame registration and effectively reduce temporal redundancy, which is of great significance to the research on large-scale point cloud compression.

[0106] Those skilled in the art will appreciate that the embodiments of the present application can be provided as methods, systems, or computer program products. Therefore, the present application can adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application can adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0107] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the steps in the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0108] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0109] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0110] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention in any other manner. Any person skilled in the art may utilize the above-disclosed technical content to modify or modify the present invention into equivalent embodiments. However, any simple modifications, equivalent variations, and modifications to the above embodiments that do not depart from the technical content of the present invention and are based on the technical essence of the present invention remain within the scope of protection of the present invention.

Claims

1. A LiDAR point cloud encoding and decoding method based on inter-frame motion prediction, characterized in that: Including encoding process and decoding process; The encoding process includes the following steps: Step E1: Input the lidar point cloud sequence, including the three-dimensional coordinates, timestamp and intensity information of the point cloud, and perform preprocessing to obtain preprocessed point cloud data; Step E2: Perform point cloud segmentation through a classification network, using the point information and category information of the point cloud data to separate the environment point cloud and the target point cloud; Step E3: Using corresponding methods to perform motion prediction based on different types of point cloud characteristics to estimate inter-frame motion transformation information; Step E4: Select a corresponding encoding method to write the obtained motion transformation information and the reference point cloud required for prediction into a bit stream for transmission; The decoding process includes the following steps: Step F1: Receive and decode the bit stream from the encoder to generate a decoded reference point cloud and motion transformation information; Step F2: applying the decoded motion transformation information to the corresponding decoded point cloud according to the corresponding relationship to achieve inter-frame prediction of the point cloud; Step F3: Fuse the predicted point clouds and output the complete reconstructed point cloud; The step E3 comprises the following steps: E31: Complete the calibration of the LiDAR-IMU sensor and use the motion information provided by the IMU sensor to perform coarse registration of the environment point cloud; E32: Using the result obtained in step E31 as the initial value of the transformation, further perform fine alignment through ICP to obtain the motion transformation information T s ; E33: Use the DBSCAN clustering algorithm to extract target point cloud clusters and map a cluster label to each cluster. Use the cluster label as the key and the corresponding point cloud data as the value to construct the cluster data table h1, including the I-frame cluster data table h1_I and the P-frame cluster data table h1_P. E34: Use the I frame cluster data table h1_I and the P frame cluster data table h1_P for inter-cluster matching and perform ICP registration. The obtained motion transformation information is {T di ,i is the cluster label value}, construct the cluster conversion table h2.

2. The LiDAR point cloud encoding and decoding method based on inter-frame motion prediction according to claim 1, characterized in that: The step E1 comprises the following steps: E11: Divide the input lidar point cloud sequence into point cloud reference frames and prediction frames; E12: Use radius filter to perform point cloud filtering to remove point cloud outliers and noise.

3. The LiDAR point cloud encoding and decoding method based on inter-frame motion prediction according to claim 1, characterized in that: In step E2, the point cloud is segmented by a classification network based on the LiDAR point cloud. The classification network combines the geometric coordinate information and category information of the point and obtains the semantic score S by constructing a semantic branch learning. cn And group them to obtain the category segmentation results of the environment point cloud and the target point cloud; among them, the environment point cloud is a relatively fixed point in the scene, and the target point cloud is a point whose state has changed.

4. The LiDAR point cloud encoding and decoding method based on inter-frame motion prediction according to claim 1, characterized in that: In step E33, the DBSCAN algorithm is used to perform density-based clustering analysis on the target point cloud, and a cluster division result C = {C_1, C_2, ..., C_k} is output. A cluster label is mapped to each cluster, and clusters in different states are analyzed separately. Key-value pairs are constructed based on the correspondence between cluster labels and clusters, and a cluster data table h1 is created, including an I-frame cluster data table h1_I and a P-frame cluster data table h1_P. In step E34, inter-cluster matching is performed using the I-frame cluster data table h1_I and the P-frame cluster data table h1_P to obtain inter-cluster correspondence, and then ICP inter-frame registration is performed to obtain motion transformation information {T di , i is the cluster label value}, and a cluster conversion table h2 is constructed for the I-frame point cloud according to the obtained inter-cluster correspondence.

5. The LiDAR point cloud encoding and decoding method based on inter-frame motion prediction according to claim 1, characterized in that: The step E4 comprises the following steps: E41: The pose transformation matrix obtained by ICP registration is input into the adaptive entropy encoder as motion transformation information to calculate the frequency of occurrence of matrix characters and determine the coding mode; E42: In the case of uneven character frequency distribution, Huffman coding is used. It constructs a binary tree based on the frequency of characters, using long codes for low-frequency characters and short codes for high-frequency characters, thereby reducing computational complexity and improving real-time performance. E43: In the data continuous or character frequency distribution uniform mode, arithmetic coding is used to get closer to the information entropy limit and achieve higher compression quality and compression efficiency; E44: The outlier points filtered out of the P frame are sent together with the environment point cloud and target point cloud of the I frame as the reference point cloud required for prediction to the G-PCC encoder for encoding.

6. The LiDAR point cloud encoding and decoding method based on inter-frame motion prediction according to claim 1, characterized in that: The step F1 comprises the following steps: F11: The motion transformation information T after Huffman decoding is obtained through Huffman decoding in the adaptive entropy decoder s ' and {T di ', i is the cluster label value}; F12: The motion transformation information T after arithmetic decoding is obtained by arithmetic decoding in the adaptive entropy decoder s ' and {T di ', i is the cluster label value}; F13: Obtain the decoded environment point cloud, decoded target point cloud and decoded outlier point cloud of the I frame through the G-PCC decoder, and construct the reconstructed I frame cluster data table h1_I'.

7. The LiDAR point cloud encoding and decoding method based on inter-frame motion prediction according to claim 1, characterized in that: The step F2 comprises the following steps: F21: Transform the motion information T s 'Act on the decoded I-frame environment point cloud to predict the P-frame environment point cloud; F22: Construct the decoded cluster conversion table h2' based on the inter-cluster correspondence relationship of the decoded motion transformation information. Each target point cloud cluster searches the decoded cluster conversion table h2' and converts the corresponding motion transformation information {T di ', i is the cluster label value} acts on the decoded I frame target point cloud in the cluster data table h1_I' to predict the target point cloud of the P frame.

8. The LiDAR point cloud encoding and decoding method based on inter-frame motion prediction according to claim 1, characterized in that: In step F3, the predicted environment point cloud, target point cloud and decoded outlier point cloud are fused to finally output a complete reconstructed point cloud.

9. A LiDAR point cloud encoding and decoding system based on inter-frame motion prediction, characterized in that: The method comprises a memory, a processor, and computer program instructions stored in the memory and capable of being executed by the processor. When the processor executes the computer program instructions, the method according to any one of claims 1 to 8 can be implemented.

Images