LiDAR point cloud coding and decoding method and system based on inter-frame motion prediction

Abstract

The invention relates to a LiDAR point cloud coding and decoding method and system based on inter-frame motion prediction. The method comprises a coding process and a decoding process. The coding process comprises the following steps: inputting and preprocessing a laser radar point cloud sequence; dividing point clouds through a classification network, and separating environment point clouds and target point clouds; performing motion prediction by using corresponding methods according to different types of point cloud characteristics, and estimating inter-frame motion transformation information; the obtained motion transformation information and the reference point cloud required by prediction are written into a bit stream for transmission by selecting a corresponding coding method; the decoding process comprises the following steps: receiving and decoding a bit stream of a coding end, and generating decoded reference point cloud and motion transformation information; the decoded motion transformation information acts on the decoded corresponding point cloud according to the corresponding relation, and point cloud inter-frame prediction is achieved; and fusing the predicted point clouds of all parts, and outputting a complete reconstructed point cloud. According to the method and the system, the inter-frame registration precision can be improved, the time domain redundancy can be eliminated, and the space-time coding of the large-scale point cloud sequence can be realized.

Description

Technical Field

[0001] The present invention belongs to the technical field of point cloud processing, and in particular relates to a LiDAR point cloud encoding and decoding method and system based on inter-frame motion prediction. Background Art

[0002] LiDAR is an essential sensor for autonomous vehicles, drones, robots and other automatic machines. It plays a vital role in positioning / path planning, target detection and high-precision mapping. The Moving Picture Experts Group (MPEG) defines LiDAR point cloud as the third type of dynamically acquired point cloud in the standard. The data volume is huge. The Velodyne LiDAR HDL-64 sensor can generate more than 100,000 points per scan and more than 84 billion points per day. Without compression, such a large amount of data will pose a huge challenge to storage space and network transmission bandwidth, which may seriously restrict its widespread use in various potential applications. Therefore, it is particularly urgent and necessary to study efficient point cloud compression coding technology.

[0003] In recent years, in order to meet the needs of point cloud data compression and improve algorithm performance, many researchers have proposed various compression methods, including point cloud compression algorithms based on three-dimensional space division, two-dimensional image projection and one-dimensional prediction. For example, the representative ones are the geometric point cloud compression (G-PCC) algorithm and the video-based point cloud compression (V-PCC) algorithm. Among these methods, although certain results have been achieved, due to the sparsity and uneven distribution of LiDAR point clouds, most existing methods focus on reducing spatial redundancy, and there is still a lot of room for improvement when compressing LiDAR point clouds. Large-scale scene point clouds not only contain a lot of spatial redundant information, but can also be regarded as continuous frame data that changes along the time axis. Frame-by-frame point clouds have high temporal correlation, so inter-frame prediction can further improve compression performance. However, inter-frame prediction is usually accompanied by higher computational costs, and there is a problem of inaccurate global motion estimation in the inter-frame prediction process. Therefore, it is urgent to deeply analyze the unique data characteristics of LiDAR point clouds and design more suitable compression schemes to achieve more efficient point cloud compression effects. Summary of the invention

[0004] The purpose of the present invention is to provide a LiDAR point cloud encoding and decoding method and system based on inter-frame motion prediction, which is conducive to improving the inter-frame registration accuracy, eliminating time domain redundancy, and realizing spatiotemporal encoding of large-scale point cloud sequences.

[0005] In order to achieve the above object, the technical solution adopted by the present invention is: a LiDAR point cloud encoding and decoding method based on inter-frame motion prediction, including an encoding process and a decoding process;

[0006] The encoding process comprises the following steps:

[0007] Step E1: inputting a laser radar point cloud sequence, including the three-dimensional coordinates, timestamp and intensity information of the point cloud, and performing preprocessing to obtain preprocessed point cloud data;

[0008] Step E2: perform point cloud segmentation through a classification network, and use the point information and category information of the point cloud data to separate the environment point cloud and the target point cloud;

[0009] Step E3: using corresponding methods to perform motion prediction according to different types of point cloud characteristics and estimate inter-frame motion transformation information;

[0010] Step E4: Select a corresponding encoding method to write the obtained motion transformation information and the reference point cloud required for prediction into a bit stream for transmission;

[0011] The decoding process comprises the following steps:

[0012] Step F1: receiving and decoding the bit stream from the encoding end, generating a decoded reference point cloud and motion transformation information;

[0013] Step F2: applying the decoded motion change information to the decoded corresponding part of the point cloud according to the corresponding relationship to achieve point cloud inter-frame prediction;

[0014] Step F3: Fuse the predicted point clouds and output the complete reconstructed point cloud.

[0015] Furthermore, the step E1 comprises the following steps:

[0016] E11: Divide the input lidar point cloud sequence into point cloud reference frames and prediction frames;

[0017] E12: Use radius filter to perform point cloud filtering to remove point cloud outliers and noise.

[0018] Furthermore, in step E2, the point cloud is segmented by a classification network based on the LiDAR point cloud. The classification network combines the geometric coordinate information and category information of the point and obtains the semantic score S by constructing a semantic branch learning. cn And group them to obtain the category segmentation results of the environment point cloud and the target point cloud; among them, the environment point cloud is a relatively fixed point in the scene, and the target point cloud is a point whose state has changed.

[0019] Furthermore, the step E3 comprises the following steps:

[0020] E31: Complete the calibration of the LiDAR-IMU sensor and use the motion information provided by the IMU sensor to perform coarse registration of the environment point cloud;

[0021] E32: Using the result obtained in step E31 as the initial value of the transformation, further perform fine alignment through ICP to obtain motion transformation information T s ;

[0022] E33: Use the DBSCAN clustering algorithm to extract the target point cloud clusters and map a cluster label to each cluster; use the cluster label as the key and the corresponding point cloud data as the value to form a cluster data table h1, including the I frame cluster data table h1_I and the P frame cluster data table h1_P;

[0023] E34: Use the I frame cluster data table h1_I and the P frame cluster data table h1_P for inter-cluster matching and perform ICP registration. The obtained motion transformation information is {T di , i is the cluster label value}, and build the cluster conversion table h2.

[0024] Furthermore, in the step E33, the DBSCAN algorithm is used to perform density-based clustering analysis on the target point cloud, and a cluster division result C={C_1, C_2,..., C_k} is output, and a cluster label is mapped to each cluster, and clusters in different states are analyzed separately, and key-value pairs are constructed based on the correspondence between cluster labels and clusters to create a cluster data table h1, including an I-frame cluster data table h1_I and a P-frame cluster data table h1_P; in the step E34, the I-frame cluster data table h1_I and the P-frame cluster data table h1_P are used to perform inter-cluster matching to obtain the inter-cluster correspondence, and then ICP inter-frame registration is performed to obtain motion transformation information {T di , i is the cluster label value}, and a cluster conversion table h2 is constructed for the I frame point cloud according to the obtained inter-cluster correspondence.

[0025] Furthermore, the step E4 comprises the following steps:

[0026] E41: Input the pose transformation matrix obtained by ICP registration into the adaptive entropy encoder as motion transformation information to calculate the frequency of occurrence of matrix characters and determine the encoding mode;

[0027] E42: In the case of uneven character frequency distribution, Huffman coding is used. It builds a binary tree based on the frequency of characters, uses long coding for low-frequency characters and short coding for high-frequency characters, thereby reducing computational complexity and improving real-time performance.

[0028] E43: In the mode of continuous data or uniform character frequency distribution, arithmetic coding is used to get closer to the information entropy limit and achieve higher compression quality and compression efficiency;

[0029] E44: The outlier points filtered out of the P frame are sent together with the environment point cloud and target point cloud of the I frame as the reference point cloud required for prediction to the G-PCC encoder for encoding.

[0030] Furthermore, the step F1 comprises the following steps:

[0031] F11: The motion transformation information T obtained by Huffman decoding in the adaptive entropy decoder s ' and {T di ', i is the cluster label value};

[0032] F12: The motion transformation information T after arithmetic decoding is obtained by arithmetic decoding in the adaptive entropy decoder s ' and {T di ', i is the cluster label value};

[0033] F13: Obtain the decoded environment point cloud, decoded target point cloud and decoded outlier point cloud of the I frame through the G-PCC decoder, and construct the reconstructed I frame cluster data table h1_I'.

[0034] Furthermore, the step F2 comprises the following steps:

[0035] F21: Transform the motion information T s 'Act on the decoded I frame environment point cloud to predict the P frame environment point cloud;

[0036] F22: Construct the decoded cluster conversion table h2' according to the inter-cluster correspondence relationship of the decoded motion transformation information. Each target point cloud cluster searches the decoded cluster conversion table h2' and converts the corresponding motion transformation information {T di ', i is the cluster label value} acts on the decoded I frame target point cloud in the cluster data table h1_I' to predict the P frame target point cloud.

[0037] Furthermore, in step F3, the predicted environment point cloud, target point cloud and decoded outlier point cloud are fused to finally output a complete reconstructed point cloud.

[0038] The present invention also provides a LiDAR point cloud encoding and decoding system based on inter-frame motion prediction, including a memory, a processor, and computer program instructions stored in the memory and capable of being executed by the processor. When the processor executes the computer program instructions, the above method can be implemented.

[0039] Compared with the prior art, the present invention has the following beneficial effects: the present invention provides a LiDAR point cloud encoding and decoding method and system based on inter-frame motion prediction, which can analyze the influence of point clouds with different states on inter-frame motion estimation, takes into account motion diversity, thereby improving the inter-frame registration accuracy, effectively reducing temporal redundancy, and achieving higher compression performance, which is suitable for compression encoding of large-scale LiDAR continuous point cloud frame sequences. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 is a flow chart of a method implementation of an embodiment of the present invention;

[0041] Figure 2 is a schematic diagram of frame division in an embodiment of the present invention;

[0042] Figure 3 is a schematic diagram of a radius filter in an embodiment of the present invention;

[0043] Figure 4 is a schematic diagram of point cloud classification in an embodiment of the present invention;

[0044] Figure 5 is a schematic diagram of the LiDAR-IMU coordinate system in an embodiment of the present invention;

[0045] Figure 6 is a point cloud registration flow chart in an embodiment of the present invention;

[0046] Figure 7 Schematic diagram of density-based clustering relationship in an embodiment of the present invention;

[0047] Figure 8 Inter-cluster transformation flow chart in an embodiment of the present invention;

[0048] Fig. 9 Adaptive entropy coding flow chart in an embodiment of the present invention;

[0049] Fig.10 It is a flow chart based on octree geometric coding in an embodiment of the present invention. DETAILED DESCRIPTION

[0050] The present invention will be further described below in conjunction with the accompanying drawings and embodiments.

[0051] It should be noted that the following detailed descriptions are exemplary and are intended to provide further explanation of the present application. Unless otherwise specified, all technical and scientific terms used herein have the same meanings as those commonly understood by those skilled in the art to which the present application belongs.

[0052] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit the exemplary embodiments according to the present application. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that when the terms "comprise" and / or "include" are used in this specification, it indicates the presence of features, steps, operations, devices, components and / or combinations thereof.

[0053] The present invention provides a LiDAR (Light Detection and Ranging, LiDAR) point cloud encoding and decoding method based on inter-frame motion prediction.

[0054] like Figure 1 As shown, this embodiment provides a LiDAR (Light Detection and Ranging, LiDAR) point cloud encoding and decoding method based on inter-frame motion prediction, including two parts: encoding process and decoding process.

[0055] The encoding process comprises the following steps:

[0056] Step E1: Input the laser radar point cloud sequence, including the three-dimensional coordinates, timestamp and intensity information of the point cloud, and perform necessary preprocessing to obtain preprocessed point cloud data.

[0057] E11: Divide the input lidar point cloud sequence into point cloud reference frames and prediction frames.

[0058] E12: Use radius filter to filter the point cloud and remove outliers and noise. Specifically, define a search radius to determine the neighborhood of each point, count all points in its neighborhood, set a point count threshold to determine whether it is an outlier point, and filter it out.

[0059] Step E2: Perform point cloud segmentation through a classification network, making full use of the point information and category information of the point cloud data to separate the environment point cloud (scene fixed points) and the target point cloud (state change points).

[0060] Specifically, the point cloud is segmented by a classification network based on the LiDAR point cloud. The classification network combines the geometric coordinate information and category information of the point and obtains the semantic score S by constructing a semantic branch learning. cn And group them to get the category segmentation results of the environment point cloud and the target point cloud. Among them, the environment point cloud is a relatively fixed point in the scene, and the target point cloud is a point whose state has changed.

[0061] Step E3: Use appropriate methods to perform motion prediction according to different types of point cloud characteristics and estimate inter-frame motion transformation information.

[0062] E31: Complete the calibration of the LiDAR-IMU sensor. Use the motion information provided by the IMU (Inertial Measurement Unit) sensor, including acceleration and angular velocity information, to reflect the vehicle's motion state. Align the LiDAR and IMU coordinate systems. After the two coordinate systems are aligned, the motion of the LiDAR can be reflected by the IMU sensor, thereby achieving coarse registration of the environmental point cloud.

[0063] E32: Using the result obtained in step E31 as the initial value of the transformation, further perform fine alignment through ICP (Iterative Closest Point) to obtain more accurate motion transformation information T s .

[0064] E33: The target point cloud represents the traffic participants in the scene, with obvious cluster distribution and strong local density. The DBSCAN (Density-Based Spatial Clustering of Applications with Noise) clustering algorithm is used to extract the target point cloud clusters, and a cluster label is mapped to each cluster; the cluster label is used as the key and the corresponding point cloud data is used as the value to form the cluster data table h1, including the I frame cluster data table h1_I and the P frame cluster data table h1_P.

[0065] E34: Use the I frame cluster data table h1_I and the P frame cluster data table h1_P for inter-cluster matching and perform ICP registration. The obtained motion transformation information is {T di , i is the cluster label value corresponding to the motion transformation information}, and the cluster conversion table h2 is constructed.

[0066] In step E33, the DBSCAN algorithm is used to perform density-based clustering analysis on the target point cloud, and the cluster division result C = {C_1, C_2, ..., C_k} is output, and a cluster label is mapped to each cluster, and clusters in different states are analyzed separately, and key-value pairs are constructed based on the correspondence between cluster labels and clusters to create a cluster data table h1, including an I-frame cluster data table h1_I and a P-frame cluster data table h1_P. In step E34, the I-frame cluster data table h1_I and the P-frame cluster data table h1_P are used to perform inter-cluster matching to obtain the inter-cluster correspondence, and then ICP inter-frame registration is performed to obtain motion transformation information {T di , i is the cluster label value}, and a cluster conversion table h2 is constructed for the I frame point cloud according to the obtained inter-cluster correspondence.

[0067] Step E4: Select a suitable encoding method to write the obtained motion transformation information and the reference point cloud required for prediction into a bit stream for transmission.

[0068] E41: The pose transformation matrix obtained by ICP registration is input into the adaptive entropy encoder as motion transformation information to calculate the frequency of occurrence of matrix characters and determine the coding mode.

[0069] E42: In the mode of uneven character frequency distribution, Huffman coding is used; it builds a binary tree based on the frequency of character occurrence, uses longer codes for low-frequency characters and shorter codes for high-frequency characters, thereby effectively reducing the computational complexity, achieving faster encoding and decoding speeds, and improving real-time performance.

[0070] E43: In the mode of continuous data or uniform character frequency distribution, arithmetic coding is used to make the reconstructed data closer to the information entropy limit, achieving higher compression quality and compression efficiency.

[0071] E44: The outlier points filtered out of the P frame are used together with the environment point cloud and target point cloud of the I frame as the reference point cloud required for prediction, and are transmitted to the G-PCC (Geometry-based Point Cloud Compression) encoder for encoding.

[0072] The decoding process comprises the following steps:

[0073] Step F1: Receive and decode the bit stream from the encoding end, and generate a decoded reference point cloud and motion transformation information.

[0074] F11: The motion transformation information T obtained by Huffman decoding in the adaptive entropy decoder s ' and {T di ', i is the cluster label value}.

[0075] F12: The motion transformation information T after arithmetic decoding is obtained by arithmetic decoding in the adaptive entropy decoder s ' and {T di ', i is the cluster label value}.

[0076] F13: Obtain the decoded environment point cloud, decoded target point cloud and decoded outlier point cloud of the I frame through the G-PCC decoder, and construct the reconstructed I frame cluster data table h1_I'.

[0077] Step F2: Apply the decoded motion transformation information to the decoded corresponding part of the point cloud according to the corresponding relationship to achieve point cloud inter-frame prediction.

[0078] F21: Transform the motion information T s 'Act on the decoded I-frame environment point cloud to perform pose conversion and predict the P-frame environment point cloud.

[0079] F22: Construct the decoded cluster conversion table h2' according to the inter-cluster correspondence relationship of the decoded motion transformation information. Each target point cloud cluster searches for the correspondence relationship according to the decoded cluster conversion table h2'. di ', i is the cluster label value} acts on the decoded I frame target point cloud in the cluster data table h1_I' to perform the corresponding posture transformation, and predict the target point cloud of the P frame.

[0080] Step F3: Fuse the predicted point clouds and output the complete reconstructed point cloud.

[0081] Specifically, by fusing the predicted environment point cloud, target point cloud and decoded outlier point cloud, a complete reconstructed point cloud is finally output.

[0082] In this example, the schematic diagram of point cloud sequence frame division in step E11 is as follows: Figure 2 As shown, the point cloud sequence is divided into reference frames (I frames) and prediction frames (P frames). I frames do not depend on other frames and can be encoded independently, while P frames need to rely on the previous I frames to use inter-frame coding.

[0083] Since the laser radar sensor is affected by its own noise, uneven reflection of the object surface, and occlusion, a large number of abnormal values ​​may be generated during the distance measurement process. Therefore, in this embodiment, step E12 uses a radius filter to filter these abnormal points to improve the efficiency of subsequent registration and reduce the registration error. The principle is as follows Figure 3 As shown in the figure, to examine whether point 1, point 2, and point 3 are abnormal noise points, two thresholds, radius d and number of points num in the neighborhood, need to be set. According to experience, d = 0.38 and th_num = 2 are set. At this time, point 1 and point 3 are determined to be noise points, and they are filtered out, and point 2 is retained.

[0084] In this embodiment, step E2 uses a classification network based on LiDAR point cloud to implement point cloud segmentation, and its implementation flow chart is as follows: Figure 4 As shown in Figure 1. The input of the network is the denoised LiDAR point cloud. The disordered point cloud is voxelized into an ordered voxel grid and fed into a U-Net style backbone network to obtain point features, including coordinate features and category features. The semantic branch is constructed based on the point features, and the semantic score {S cn cn is the number of semantic categories of the point cloud}, and the semantic scores are grouped directly. The category of each point is determined by comparing the semantic score with the score threshold S_th. This method allows each point to be associated with multiple categories, thereby reducing the risk of prediction errors. Finally, through the refinement stage, the object category with the highest category confidence score is obtained, and the environment point cloud and target point cloud are derived through category label mapping.

[0085] In this embodiment, step E31 uses the acceleration and angular velocity information provided by the IMU sensor to reflect the vehicle motion state, and then obtains the point cloud relative motion state. Among them, the environment point cloud represents a relatively fixed point in the scene, and the target point cloud represents a point whose state has changed. The formula is as follows:

[0086]

[0087] Among them, p e (t) represents the position of a point in the environment point cloud at time t, p o (t) represents the position of a point in the target point cloud at time t, V IMU (t) represents the velocity vector of the IMU sensor at time t, v e (t) and v o (t) represents the velocity vector of the midpoint of the environment point cloud and the target point cloud relative to the IMU.

[0088] However, due to the installation error angle and position error between the LiDAR and IMU, Figure 5 Different coordinate systems are shown, the world coordinate system is (OX1Y1Z1), the IMU coordinate system is (OX2Y2Z2), and the LiDAR coordinate system is (OX3Y3Z3). Therefore, it is necessary to obtain the rigid body transformation between the IMU coordinate system and the LiDAR coordinate system through the joint calibration of LiDAR-IMU. It is used to describe the relative position between the two to achieve environmental point cloud IMU registration and provide more accurate initial values ​​for subsequent fine registration. It is expressed as:

[0089]

[0090] Among them, t imu is a 3×1 translation vector, R imu It represents a 3×3 rotation matrix.

[0091] Further, in this embodiment, step E32 uses the IMU coarse registration result as the initial value and performs the following steps: Figure 6 The ICP fine registration process shown in the figure obtains the environmental point cloud motion transformation information T s First, the source point cloud s i With the target point cloud p i The corresponding points are matched to construct the corresponding point set. Then, for the corresponding points, the following objective function is constructed by the least squares method:

[0092]

[0093] Through continuous iterative optimization, the optimal rotation matrix R* and translation vector t* are calculated when the sum of squared errors reaches a minimum, and then the motion change information T is obtained. s , used to predict the target point cloud.

[0094] In this embodiment, step E33 uses the DBSCAN algorithm based on density clustering to determine whether to classify points into the same category by judging whether there is a density relationship between points (density direct, density reachable, density connected), thereby achieving clustering of the target point cloud. Figure 7 As shown in the figure, the neighborhood radius r and the minimum number of points in the neighborhood min_points are two important parameters. The points in the circle with a radius of r that contain more than min_points are considered core points, such as p1~p4. The points (boundary points) in the neighborhood of each core point are directly accessible to the core point density. The core points connected by the dotted arrows in the figure are density-reachable points. In the neighborhood of these density-reachable points, all points are density-connected to each other, and these points can be clustered into one category. The outlier points and non-core points outside the neighborhood are not density-connected and are not classified into the same category. The final output cluster division result C={C_1,C_2,...,C_k}, and the cluster index value is used as the cluster label;

[0095] In this embodiment, step E34 analyzes clusters of different states separately, constructs key-value pairs based on the correspondence between cluster labels and clusters, and constructs cluster data tables h1_I and h1_P for I frames and P frames respectively. Figure 8 As shown in FIG. 1 , by finding the point cloud clusters whose difference between the number of point cloud points (counts_P) and the three-dimensional coordinate range (coord_P(x), coord_P(y), coord_P(z)) in h1_P and the number of point cloud points (counts_I) and the three-dimensional coordinate range (coord_I(x), coord_I(y), coord_I(z)) in h1_I is less than the threshold, the cluster matching relationship is confirmed and ICP inter-frame registration is performed, and the obtained motion transformation information {T di , i is the cluster label value} is stored in the cluster conversion table h2. The threshold is set as follows:

[0096] Th_coord(k)=(maxcoord(k)-mincoord(k)) / n

[0097] Th_counts=(counts_I+counts_P) / n

[0098] Among them, Th_coord(k) and Th_counts represent the thresholds of the coordinate range and the number of points, maxcoord(k) and mincoord(k) represent the maximum and minimum values ​​of the coordinate range, k represents the three coordinate axes, and n is set to 20 based on experience. counts_I and counts_P represent the number of points in the I frame and P frame point cloud, respectively.

[0099] In this embodiment, in order to achieve a balance between compression efficiency and computational complexity, the motion transformation information is compressed by adaptive entropy coding. The adaptive entropy coding flow chart is as follows: Fig. 9 As shown, first calculate the frequency of occurrence of characters in the input data, and dynamically select the encoding method according to its frequency distribution characteristics. In the mode of uneven distribution of character frequencies, select Huffman encoding through step E41, which is a coding method that constructs a binary tree based on the frequency of character occurrence. Characters with high frequencies use shorter codes, which can effectively reduce the computational complexity and improve real-time performance. In the mode where the data is continuous or the frequency of occurrence of characters is very close, select arithmetic coding through step E42, which can be closer to the information entropy, improve the compression rate, and ensure the reconstruction quality. After adaptive encoding, calculate the bit rate and perform decoding quality verification. If the compression requirements are not met, adjust the encoding parameters and re-encode, and finally output a satisfactory result.

[0100] In this embodiment, step E43 compresses and encodes the I frame environment point cloud and target point cloud, as well as the P frame outlier point cloud using the mature international standard G-PCC. The geometric encoding process is as follows: Fig.10 As shown, first convert the original point cloud data coordinates to a point with a side length of 2 d The occupancy code is predicted using the octree spatial structure, and finally the geometric coding is completed after arithmetic coding. Among them, the octree structure recursively divides the geometry of the three-dimensional space to find the maximum size of the scene, and uses this size to establish the first cube as the root node, and subdivides the voxels with points into 8 sub-cubes until the maximum recursive depth obtains a leaf node with a volume of 1, and finally encodes and transmits the generated occupancy code.

[0101] In this embodiment, step F1 receives and decodes the bit stream of the encoding end to generate a decoded point cloud and a pose transformation matrix. Specifically, step F11 obtains the decoded motion transformation information T by the Huffman decoding process of the adaptive entropy coding. s ' and {T di ', i is the cluster label value}, stored in the decoded cluster conversion table h2'. Similarly, step F12 obtains the decoded motion transformation information T through the arithmetic decoding process of adaptive entropy coding. s ' and {Tdi ', i is the cluster label value}, stored in the decoded cluster conversion table h2'. Step F13 obtains the decoded environment point cloud and the decoded target point cloud of the I frame through the G-PCC decoder, and constructs the cluster data table h1_I' after the I frame is decoded, and the outlier point cloud after the P frame is decoded.

[0102] In this embodiment, the decoded motion transformation information is applied to the decoded corresponding category point cloud by searching the corresponding relationship in step F2 to perform posture conversion and realize inter-frame prediction. s The motion transformation information acts on the I frame environment point cloud to predict the P frame environment point cloud. In step F22, each target point cloud cluster converts the corresponding motion transformation information {T di ', i is the cluster label value} acts on the target point cloud cluster in h1_I' to predict the target point cloud of the P frame.

[0103] In this embodiment, the predicted environment point cloud and target point cloud are merged in step F3, and the decoded outlier point cloud is fused to finally output a complete reconstructed point cloud.

[0104] This embodiment also provides a LiDAR point cloud encoding and decoding system based on inter-frame motion prediction, including a memory, a processor, and computer program instructions stored in the memory and capable of being executed by the processor. When the processor executes the computer program instructions, the above method can be implemented.

[0105] The present invention provides a LiDAR point cloud encoding and decoding method and system based on inter-frame motion prediction. At the encoding end, the reference frame (I frame) is compressed by spatial redundancy, and the prediction frame (P frame) is compressed by eliminating temporal redundancy. First, the denoised LiDAR data is divided into an environmental point cloud and a target point cloud; then, the IMU coarse registration result is used as the initial value of the ICP registration algorithm to perform fine registration on the environmental point cloud. After the target point cloud is clustered by the DBSAN algorithm, ICP registration is performed according to the correspondence between cluster labels to achieve inter-frame prediction; finally, the registered pose transformation matrix is ​​sent to the adaptive entropy encoder as motion transformation information, and the denoised I frame point cloud and the P frame outlier points are encoded together using G-PCC. At the decoding end, the motion transformation information and point cloud data are first decoded, and then the P frame is predicted and reconstructed according to the correspondence between various point clouds and motion transformation information of the I frame, and finally the parts are merged to obtain a complete reconstructed point cloud. The present invention adopts a variety of technologies to improve the accuracy of inter-frame registration and effectively reduce temporal redundancy, which is of great significance to the research on large-scale point cloud compression.

[0106] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0107] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0108] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0109] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0110] The above is only a preferred embodiment of the present invention, and does not limit the present invention in other forms. Any technician familiar with the profession may use the above disclosed technical content to change or modify it into an equivalent embodiment with equivalent changes. However, any simple modification, equivalent change and modification made to the above embodiment according to the technical essence of the present invention without departing from the technical solution of the present invention still belongs to the protection scope of the technical solution of the present invention.

Claims

1. A LiDAR point cloud encoding and decoding method based on inter-frame motion prediction, characterized in that: Including encoding process and decoding process; The encoding process comprises the following steps: Step E1: inputting a laser radar point cloud sequence, including the three-dimensional coordinates, timestamp and intensity information of the point cloud, and performing preprocessing to obtain preprocessed point cloud data; Step E2: perform point cloud segmentation through a classification network, and use the point information and category information of the point cloud data to separate the environment point cloud and the target point cloud; Step E3: using corresponding methods to perform motion prediction according to different types of point cloud characteristics and estimate inter-frame motion transformation information; Step E4: Select a corresponding encoding method to write the obtained motion transformation information and the reference point cloud required for prediction into a bit stream for transmission; The decoding process comprises the following steps: Step F1: receiving and decoding the bit stream from the encoding end, generating a decoded reference point cloud and motion transformation information; Step F2: applying the decoded motion transformation information to the decoded corresponding part of the point cloud according to the corresponding relationship to achieve point cloud inter-frame prediction; Step F3: Fuse the predicted point clouds and output the complete reconstructed point cloud.

2. The LiDAR point cloud encoding and decoding method based on inter-frame motion prediction according to claim 1, characterized in that: The step E1 comprises the following steps: E11: Divide the input lidar point cloud sequence into point cloud reference frames and prediction frames; E12: Use radius filter to perform point cloud filtering to remove point cloud outliers and noise.

3. The LiDAR point cloud encoding and decoding method based on inter-frame motion prediction according to claim 1, characterized in that: In step E2, point cloud segmentation is performed using a classification network based on LiDAR point cloud. The classification network combines the geometric coordinate information and category information of the point and obtains the semantic score S by constructing a semantic branch learning. cn And group them to obtain the category segmentation results of the environment point cloud and the target point cloud; among them, the environment point cloud is a relatively fixed point in the scene, and the target point cloud is a point whose state has changed.

4. The LiDAR point cloud encoding and decoding method based on inter-frame motion prediction according to claim 1, characterized in that: The step E3 comprises the following steps: E31: Complete the calibration of the LiDAR-IMU sensor and use the motion information provided by the IMU sensor to perform coarse registration of the environment point cloud; E32: Using the result obtained in step E31 as the initial value of the transformation, further perform fine alignment through ICP to obtain motion transformation information T s ; E33: Use the DBSCAN clustering algorithm to extract the target point cloud clusters and map a cluster label to each cluster; use the cluster label as the key and the corresponding point cloud data as the value to form a cluster data table h1, including the I frame cluster data table h1_I and the P frame cluster data table h1_P; E34: Use the I frame cluster data table h1_I and the P frame cluster data table h1_P for inter-cluster matching and perform ICP registration. The obtained motion transformation information is {T di , i is the cluster label value}, and build the cluster conversion table h2.

5. The LiDAR point cloud encoding and decoding method based on inter-frame motion prediction according to claim 4, characterized in that: In the step E33, the DBSCAN algorithm is used to perform density-based clustering analysis on the target point cloud, and a cluster division result C={C_1,C_2,...,C_k} is output, and a cluster label is mapped to each cluster, and clusters in different states are analyzed separately, and key-value pairs are constructed based on the correspondence between cluster labels and clusters to create a cluster data table h1, including an I-frame cluster data table h1_I and a P-frame cluster data table h1_P; in the step E34, the I-frame cluster data table h1_I and the P-frame cluster data table h1_P are used to perform inter-cluster matching to obtain the inter-cluster correspondence, and then ICP inter-frame registration is performed to obtain motion transformation information {T di , i is the cluster label value}, and a cluster conversion table h2 is constructed for the I frame point cloud according to the obtained inter-cluster correspondence.

6. The LiDAR point cloud encoding and decoding method based on inter-frame motion prediction according to claim 1, characterized in that: The step E4 comprises the following steps: E41: Input the pose transformation matrix obtained by ICP registration into the adaptive entropy encoder as motion transformation information to calculate the frequency of occurrence of matrix characters and determine the encoding mode; E42: In the case of uneven character frequency distribution, Huffman coding is used. It builds a binary tree based on the frequency of characters, uses long coding for low-frequency characters and short coding for high-frequency characters, thereby reducing computational complexity and improving real-time performance. E43: In the mode of continuous data or uniform character frequency distribution, arithmetic coding is used to get closer to the information entropy limit and achieve higher compression quality and compression efficiency; E44: The outlier points filtered out of the P frame are sent together with the environment point cloud and target point cloud of the I frame as the reference point cloud required for prediction to the G-PCC encoder for encoding.

7. The LiDAR point cloud encoding and decoding method based on inter-frame motion prediction according to claim 1, characterized in that: The step F1 comprises the following steps: F11: The motion transformation information T obtained by Huffman decoding in the adaptive entropy decoder s ' and {T di ', i is the cluster label value}; F12: The motion transformation information T after arithmetic decoding is obtained by arithmetic decoding in the adaptive entropy decoder s ' and {T di ', i is the cluster label value}; F13: Obtain the decoded environment point cloud, decoded target point cloud and decoded outlier point cloud of the I frame through the G-PCC decoder, and construct the reconstructed I frame cluster data table h1_I'.

8. The LiDAR point cloud encoding and decoding method based on inter-frame motion prediction according to claim 1, characterized in that: The step F2 comprises the following steps: F21: Transform the motion information T s 'Act on the decoded I frame environment point cloud to predict the P frame environment point cloud; F22: Construct the decoded cluster conversion table h2' according to the inter-cluster correspondence relationship of the decoded motion transformation information. Each target point cloud cluster searches the decoded cluster conversion table h2' and converts the corresponding motion transformation information {T di ', i is the cluster label value} acts on the decoded I frame target point cloud in the cluster data table h1_I' to predict the P frame target point cloud.

9. The LiDAR point cloud encoding and decoding method based on inter-frame motion prediction according to claim 1, characterized in that: In step F3, the predicted environment point cloud, the target point cloud and the decoded outlier point cloud are fused to finally output a complete reconstructed point cloud.

10. A LiDAR point cloud encoding and decoding system based on inter-frame motion prediction, characterized in that: The method comprises a memory, a processor, and computer program instructions stored in the memory and executable by the processor. When the processor executes the computer program instructions, the method according to any one of claims 1 to 9 can be implemented.

Images