A LiDAR point cloud encoding method and decoding method based on enhanced mapping correlation

Abstract

The present invention relates to a LiDAR point cloud encoding method and decoding method based on enhanced mapping correlation, the encoding method comprising: obtaining a point cloud to be encoded, taking part of the point cloud frames as I frames and the remaining point cloud frames as P frames; mapping the geometric information of the I frame and the P frame into a 2D distance map, dividing into blocks with a resolution of N1×N2, converting each block of the I frame and the P frame into a three-dimensional point cloud, using an ICP algorithm to register the three-dimensional point cloud of each block of the I frame with the three-dimensional point cloud of each block of the P frame, and obtaining motion information H; converting the three-dimensional point cloud of each block of the P' frame into a 2D distance map; mapping the attribute information of the I frame and the P frame into a 2D attribute map; encoding the 2D distance map and the 2D attribute map into a video stream, encoding the video stream into a binary code stream; quantizing the motion information H into a 2D distance map, and compressing it into a binary code stream. The present invention reduces spatial and temporal redundancy, and the exploration and research of efficient point cloud compression schemes have important strategic significance for the development of related applications such as autonomous driving and mobile robots.

Description

Technical Field

[0001] The invention relates to a LiDAR point cloud encoding method and a decoding method based on enhanced mapping correlation, belonging to the technical field of point cloud processing. Background Art

[0002] The development of three-dimensional sensor data acquisition has created new opportunities and challenges in industries such as virtual reality technology, smart cities, intelligent robots, and autonomous driving. Autonomous driving systems have a wide range of applications in reducing road traffic pressure and improving the natural environment for transportation. Using lidar to identify the surrounding environment and status of the vehicle is one of the foundations of autonomous driving.

[0003] Since 3D point cloud acquisition has many advantages such as high efficiency, high precision, high resolution, and digital acquisition, many autonomous driving systems including Google and Uber are actually developing and applying 3D point cloud data collected by on-board laser radar. The point cloud model represents the visual scene through a series of 3D points, describing the topological information of the surface of the object and the related attributes of each point. However, the non-uniform sampling of point clouds and the disordered distribution of spatial structure make point cloud compression and processing quite challenging. Taking the 64-line radar sensor produced by Velodyne as an example, the number of points collected in one rotation exceeds 100,000. Under normal settings, the data collected in half an hour of work is as high as 50G. Therefore, the amount of 3D data generated by the point cloud acquisition equipment poses a higher challenge to data storage and transmission. In recent years, 3D point cloud data compression technology has gradually been studied and explored in depth. The inventors found that the 3D point cloud has the problem of wide distribution range and uneven density. Summary of the invention

[0004] In order to overcome the above problems, the present invention provides a LiDAR point cloud encoding method and a decoding method based on enhanced mapping correlation. The encoding method reduces spatial and temporal redundancy. The exploration and research of efficient point cloud compression schemes has important strategic significance for the development of related applications such as autonomous driving and mobile robots.

[0005] The detachable electromechanical mounting frame has a simple structure, is easy to disassemble, has replaceable components, and is suitable for installing electromechanical equipment of different specifications.

[0006] The technical solution of the present invention is as follows:

[0007] First aspect

[0008] A LiDAR point cloud encoding method based on enhanced mapping correlation includes the following steps:

[0009] Get the point cloud to be encoded, take part of the point cloud frames as I frames, and take the rest of the point cloud frames as P frames;

[0010] The geometric information of the I frame and the P frame is mapped into a 2D distance map, and divided into blocks with a resolution of N1×N2, where N1 is the number of laser radar lines, and each block of the I frame and the P frame is converted into a three-dimensional point cloud, and the three-dimensional point cloud of each block of the P frame is aligned with the three-dimensional point cloud of each block of the I frame using the ICP algorithm to obtain the motion information H of the aligned three-dimensional point cloud of each block of the P' frame and the three-dimensional point cloud of each block of the P frame to the three-dimensional point cloud of each block of the I frame; the motion information H includes a rotation matrix and a translation matrix;

[0011] Converting the three-dimensional point clouds of each block of the P' frame into a 2D distance map;

[0012] Mapping the attribute information of the I frame and the P frame into a 2D attribute map;

[0013] Encoding the 2D distance map of the I frame, the 2D distance map of the P' frame, and the 2D attribute map into video streams respectively, and encoding the video streams into binary code streams through HEVC;

[0014] The motion information H is quantized into a 2D distance map and compressed into a binary code stream through entropy coding.

[0015] Furthermore, N2 is the same as N1.

[0016] Furthermore, the geometric information is mapped into a 2D distance map, specifically:

[0017] Calculate the yaw angle ω of point i based on the 3D point cloud data i , pitch angle θ i and the distance r between point i and the center i , the formula is as follows:

[0018]

[0019] Among them, x i is the left value of the Cartesian coordinate x-axis of point i, y i is the left value of the Cartesian coordinates of point i on the y axis; z i The left side value of the Cartesian coordinate z axis of point i;

[0020] The point (r i ,θ i ,ω i ) is mapped to pixel P(px i ,py i ), the formula is as follows:

[0021]

[0022] P(px i ,py i )=round(ri );

[0023] Wherein, x_quan and y_quan are the yaw angle ω i and the pitch angle θ i The quantization coefficient of is quantized to an integer;

[0024] According to the pixel P (px i ,py i ) to obtain a 2D distance map.

[0025] Furthermore, the geometric information is mapped into a 2D distance map of 16-bit depth values.

[0026] Furthermore, the ICP algorithm is an improved ICP algorithm, and the improved ICP algorithm is specifically:

[0027] According to the improved ICP algorithm based on the translation model, the motion information between point cloud frames is estimated. The formula is as follows:

[0028]

[0029] Among them, f(R, t) is the error function value of the point cloud when it is not registered, fmin(R, t) is the error function of the motion information after registration, n is the total number of point clouds in the source point set, and p i is the ICP source point set, t is the translation matrix, q i is the ICP target point set, and R is the rotation matrix.

[0030] Furthermore, the ICP improved algorithm also includes:

[0031] After the ICP improved algorithm is iterated N times, let the rotation matrix be equal to the identity matrix, that is, R = R0:

[0032]

[0033] Solve for the translation matrix and use the ICP algorithm on the newly obtained translation matrix and the identity matrix.

[0034] Further, the I frame 2D distance map, the P' frame 2D distance map and the 2D attribute map are respectively encoded into video streams, and the video streams are encoded into binary code streams through HEVC, specifically:

[0035] The I frame 2D distance map, the P' frame 2D distance map and the 2D attribute map are encoded into a video stream, and compressed using a video encoding method to obtain a binary code stream.

[0036] Furthermore, the motion information H is quantized into a 2D distance map as follows:

[0037] The motion information H is quantized using uniform quantization, and the formula is as follows:

[0038]

[0039] Among them, P m (i, j) is the quantized pixel, H(i, j) is the value of the rigid transformation matrix H in the i row and j column, H min (i,j) is the minimum value in the rigid transformation matrix H, max (i,j) is the maximum value in the rigid transformation matrix H.

[0040] Furthermore, the compression into a binary code stream by entropy coding is specifically:

[0041] Perform 4-level 5 / 3 wavelet transform on the 2D distance map and calculate the optimal quantization step size for each subband;

[0042] The JPEG_LS algorithm is used to remove the correlation redundancy between pixels from the data of the lowest frequency sub-band, and the data is encoded using zero-run coding to obtain the zero-run coding result;

[0043] Context modeling is performed based on the zero-run coding result, and the series of exponential Golomb coding is updated, and the binary code stream is obtained using the exponential Golomb coding of the corresponding series.

[0044] Second aspect

[0045] A decoding method for LiDAR point cloud coding based on enhanced mapping correlation, decoding and restoring the point cloud encoded by the LiDAR point cloud coding method based on enhanced mapping correlation described in the first aspect to the original point cloud, comprising the following steps:

[0046] Motion information decoding: entropy decoding is performed on the binary code stream obtained by entropy coding to obtain a 2D distance map of the P' frame, and the registration information in the 2D distance map of the P' frame is converted into motion information H;

[0047] Geometric information decoding: Perform HEVC decoding on the binary code stream obtained by HEVC encoding to obtain the 2D distance map of the I frame and the 2D distance map of the P' frame containing geometric information, convert the 2D distance maps of the I frame and the P' frame into 3D point clouds of the I frame and the P' frame, and use the decoded motion information H to restore the 3D point cloud of the P' frame to obtain the geometric information of the P frame;

[0048] Attribute information decoding: performing HEVC decoding on the binary code stream obtained by HEVC encoding to obtain a 2D attribute map containing attribute information, and converting the 2D attribute map of the I frame and the P frame into attribute information of the I frame and the P' frame;

[0049] The 3D point clouds of I-frames and P-frames are restored based on the 2D distance map, 2D attribute map, geometric information and attribute information.

[0050] The present invention has the following beneficial effects:

[0051] The method of mapping point cloud data to distance map and attribute map is adopted; in order to improve the temporal correlation of the distance map of the point cloud, the method of point cloud segmentation and registration is proposed; in order to improve the accuracy of registration, a translation model is proposed to improve the ICP registration method; the registered point cloud is converted into a distance map again, and finally the distance map and attribute map are used to form a video sequence, which is encoded using HEVC, and the motion information required for registration is mapped to a motion map and encoded using entropy coding. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] Figure 1 The figure is a flow chart of the encoding method of the present invention.

[0053] Figure 2 This is a diagram of the improved ICP algorithm of the present invention.

[0054] Figure 3 The algorithm flow framework of the adaptive exponential Golomb coding of the present invention.

[0055] Figure 4 The figure is a flowchart of the decoding method of the present invention. DETAILED DESCRIPTION

[0056] The present invention will be described in detail below with reference to the accompanying drawings and specific embodiments.

[0057] First aspect

[0058] Embodiment 1

[0059] refer to Figure 1 , a LiDAR point cloud encoding method based on enhanced mapping correlation, comprising the following steps:

[0060] Get the point cloud to be encoded, take part of the point cloud frames as I frames, and take the rest of the point cloud frames as P frames;

[0061] The geometric information of the I frame and the P frame is mapped into a 2D distance map, and divided into blocks with a resolution of N1×N2, where N1 is the number of laser radar lines, and each block of the I frame and the P frame is converted into a three-dimensional point cloud, and the three-dimensional point cloud of each block of the P frame is aligned with the three-dimensional point cloud of each block of the I frame using the ICP algorithm to obtain the motion information H of the aligned three-dimensional point cloud of each block of the P' frame and the three-dimensional point cloud of each block of the P frame to the three-dimensional point cloud of each block of the I frame; the motion information includes a rotation matrix and a translation matrix;

[0062] Converting the three-dimensional point clouds of each block of the P' frame into a 2D distance map;

[0063] Mapping the attribute information of the I frame and the P frame into a 2D attribute map;

[0064] Encoding the 2D distance map of the I frame, the 2D distance map of the P' frame, and the 2D attribute map into video streams respectively, and encoding the video streams into binary code streams through HEVC;

[0065] The motion information H is quantized into a 2D distance map and compressed into a binary code stream through entropy coding.

[0066] The present invention designates a portion of point cloud frames as reference frames, also known as I frames (intra-frame coded frames), which are key frames; and uses them to predict other non-key point cloud frames, and the predicted non-key frames are called P frames (inter-frame prediction frames).

[0067] In one embodiment of the present invention, in order to make the number of points in the point cloud registration at a reasonable and stable value, N2 is considered to be the same as N1 from the perspective of experimental experience.

[0068] Embodiment 2

[0069] A LiDAR point cloud encoding method based on enhanced mapping correlation, based on the first embodiment, the geometric information is mapped into a 2D distance map, specifically:

[0070] Calculate the yaw angle ω of point i based on the 3D point cloud data i , pitch angle θ i and the distance r between point i and the center i , the formula is as follows:

[0071]

[0072] Among them, x i is the left value of the Cartesian coordinate x-axis of point i, y i is the left value of the Cartesian coordinates of point i on the y axis; z i The left side value of the Cartesian coordinate z axis of point i;

[0073] The point (r i ,θ i ,ω i ) is mapped to pixel P(px i ,py i ), the formula is as follows:

[0074]

[0075] P(px i ,py i )=round(r i );

[0076] Wherein, x_quan and y_quan are the yaw angle ω i and the pitch angle θ i The quantization coefficient of is quantized to an integer;

[0077] According to the pixel P (px i ,py i ) to obtain a 2D distance map.

[0078] The pitch angle is determined by the inherent resolution of the vehicle-mounted lidar. For example, for 16-line, 32-line, and 64-line lidars, the yaw angle is generally 0 to a little more than 360 degrees. At present, the effective sampling distance of the lidar generally does not exceed 130 meters. Since the original data is obtained by measuring the radial distance from the center of the lidar to the object, the pitch angle is used as the row index and the yaw angle is used as the column index. The distance value corresponding to a certain pitch angle and yaw angle is the pixel value of the grayscale image, which converts the originally compressed, disordered and scattered point cloud into a compression of a distance map (grayscale image) with a stronger correlation.

[0079] In one embodiment of the present invention, the geometric information is mapped into a 2D distance map of 16-bit depth values.

[0080] Embodiment 3

[0081] A LiDAR point cloud coding method based on enhanced mapping correlation, based on the first embodiment, the ICP algorithm is an ICP improved algorithm, and the ICP improved algorithm is specifically:

[0082] According to the improved ICP algorithm based on the translation model, the motion information between point cloud frames is estimated. The formula is as follows:

[0083]

[0084] Among them, f(R, t) is the error function value of the point cloud when it is not registered, fmin(R, t) is the error function of the motion information after registration, n is the total number of point clouds in the source point set, and p i is the ICP source point set, t is the translation matrix, q i is the ICP target point set, and R is the rotation matrix.

[0085] The ICP algorithm is a rigid transformation algorithm widely used in point cloud registration. The present invention estimates the motion information between point cloud frames by using an improved ICP algorithm based on a translation model.

[0086] Dividing the point cloud into blocks and aligning the corresponding modules can further improve the accuracy of the alignment and reduce the operation speed. Because the lidar point cloud is sparse and the collector obtains a frame of data after one rotation vertically of the vehicle, using octree blocking will result in a large number of macroblocks with no data, and encoding and indexing the macroblocks with point cloud geometry data in each frame will waste a lot of encoding data and time.

[0087] refer to Figure 2 In one embodiment of the present invention, the ICP improved algorithm further includes:

[0088] After the ICP improved algorithm is iterated N times, let the rotation matrix be equal to the identity matrix, that is, R = R0:

[0089]

[0090] Solve for the translation matrix and use the ICP algorithm on the newly obtained translation matrix and the identity matrix.

[0091] According to the data characteristics of the collected vehicle-mounted laser radar point cloud: the collection time interval between frames is short and the road surface is flat. At this time, the running information between frames is mainly a large translation vector and a small rotation vector. Therefore, this method can not only ensure close to the real value, but also improve the coding efficiency.

[0092] Embodiment 4

[0093] A LiDAR point cloud encoding method based on enhanced mapping correlation, based on the first embodiment, encodes the I frame 2D distance map, the P' frame 2D distance map and the 2D attribute map into video streams respectively, and encodes the video stream into a binary code stream through HEVC, specifically:

[0094] The I frame 2D distance map, the P' frame 2D distance map and the 2D attribute map are encoded into a video stream, and compressed using a video encoding method to obtain a binary code stream.

[0095] Embodiment 5

[0096] A LiDAR point cloud coding method based on enhanced mapping correlation, based on the first embodiment, quantizes the motion information H into a 2D distance map as follows:

[0097] The motion information H is quantized using uniform quantization, and the formula is as follows:

[0098]

[0099] Among them, P m (i, j) is the quantized pixel, H(i, j) is the value of the rigid transformation matrix H in the i row and j column, H min(i,j) is the minimum value in the rigid transformation matrix H, max (i,j) is the maximum value in the rigid transformation matrix H.

[0100] Embodiment 6

[0101] Referring to FIG. 3 , a LiDAR point cloud coding method based on enhanced mapping correlation is shown in the first embodiment. The method is compressed into a binary code stream by entropy coding, specifically:

[0102] Perform 4-level 5 / 3 wavelet transform on the 2D distance map and calculate the optimal quantization step size for each subband;

[0103] The JPEG_LS algorithm is used to remove the correlation redundancy between pixels from the data of the lowest frequency sub-band, and the data is encoded using zero-run coding to obtain the zero-run coding result;

[0104] Context modeling is performed based on the zero-run coding result, and the series of exponential Golomb coding is updated, and the binary code stream is obtained using the exponential Golomb coding of the corresponding series.

[0105] Second aspect

[0106] Embodiment 7

[0107] refer to Figure 4 A decoding method for LiDAR point cloud coding based on enhanced mapping correlation is provided, wherein the point cloud encoded by the LiDAR point cloud coding method based on enhanced mapping correlation according to the first aspect is decoded and restored to the original point cloud, comprising the following steps:

[0108] Motion information decoding: entropy decoding is performed on the binary code stream obtained by entropy coding to obtain a 2D distance map of the P' frame, and the registration information in the 2D distance map of the P' frame is converted into motion information H;

[0109] Geometric information decoding: Perform HEVC decoding on the binary code stream obtained by HEVC encoding to obtain the 2D distance map of the I frame and the 2D distance map of the P' frame containing geometric information, convert the 2D distance maps of the I frame and the P' frame into 3D point clouds of the I frame and the P' frame, and use the decoded motion information H to restore the 3D point cloud of the P' frame to obtain the geometric information of the P frame;

[0110] Attribute information decoding: performing HEVC decoding on the binary code stream obtained by HEVC encoding to obtain a 2D attribute map containing attribute information, and converting the 2D attribute map of the I frame and the P frame into attribute information of the I frame and the P' frame;

[0111] The 3D point clouds of I-frames and P-frames are restored based on the 2D distance map, 2D attribute map, geometric information and attribute information.

[0112] The above descriptions are merely embodiments of the present invention and are not intended to limit the patent scope of the present invention. Any equivalent structure made using the contents of the present invention's specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A LiDAR point cloud coding method based on enhanced mapping correlation, characterized in that: The following steps are involved: Get the point cloud to be encoded, take part of the point cloud frames as I frames, and take the rest of the point cloud frames as P frames; The geometric information of the I frame and the P frame is mapped into a 2D distance map, and divided into blocks with a resolution of N1×N2, where N1 is the number of laser radar lines, and each block of the I frame and the P frame is converted into a three-dimensional point cloud, and the three-dimensional point cloud of each block of the P frame is aligned with the three-dimensional point cloud of each block of the I frame using the ICP algorithm to obtain the motion information H of the aligned three-dimensional point cloud of each block of the P' frame and the three-dimensional point cloud of each block of the P frame to the three-dimensional point cloud of each block of the I frame; the motion information H includes a rotation matrix and a translation matrix; Converting the three-dimensional point clouds of each block of the P' frame into a 2D distance map; Mapping the attribute information of the I frame and the P frame into a 2D attribute map; Encoding the 2D distance map of the I frame, the 2D distance map of the P' frame, and the 2D attribute map into video streams respectively, and encoding the video streams into binary code streams through HEVC; quantizing the motion information H into a 2D distance map, and compressing it into a binary code stream through entropy coding; The ICP algorithm is an improved ICP algorithm, and the improved ICP algorithm is specifically: According to the improved ICP algorithm based on the translation model, the motion information between point cloud frames is estimated. The formula is as follows: Among them, f(R, t) is the error function value of the point cloud when it is not registered, fmin(R, t) is the error function of the motion information after registration, n is the total number of point clouds in the source point set, and p i is the ICP source point set, t is the translation matrix, q i is the ICP target point set, R is the rotation matrix; The ICP improved algorithm also includes: After the ICP improved algorithm is iterated N times, let the rotation matrix be equal to the identity matrix, that is, R = R0: Solve the translation matrix and use the ICP algorithm on the newly obtained translation matrix and the identity matrix; The motion information H is quantized into a 2D distance map as follows: The motion information H is quantized using uniform quantization, and the formula is as follows: Among them, P m (i, j) is the quantized pixel, H(i, j) is the value of the rigid transformation matrix H in the i row and j column, H min (i,j) is the minimum value in the rigid transformation matrix H, max (i,j) is the maximum value in the rigid transformation matrix H.

2. The LiDAR point cloud coding method based on enhanced mapping correlation according to claim 1, characterized in that: N2 is the same as N1.

3. The LiDAR point cloud coding method based on enhanced mapping correlation according to claim 1, characterized in that: The geometric information is mapped into a 2D distance map, specifically: Calculate the yaw angle ω of point i based on the 3D point cloud data i , pitch angle θ i and the distance r between point i and the center i , the formula is as follows: Among them, x i is the left value of the Cartesian coordinate x-axis of point i, y i is the left value of the Cartesian coordinates of point i on the y axis; z i The left side value of the Cartesian coordinate z axis of point i; The point (r i ,θ i ,ω i ) is mapped to pixel P(px i ,py i ), the formula is as follows: P(px i ,py i )=round(r i ); Among them, x _ quan and y _ quan are the yaw angles ω i and the pitch angle θ i The quantization coefficient of is quantized to an integer; According to the pixel P (px i ,py i ) to obtain a 2D distance map.

4. The LiDAR point cloud coding method based on enhanced mapping correlation according to claim 3, characterized in that: The geometric information is mapped into a 2D distance map of 16-bit depth values.

5. The LiDAR point cloud encoding method based on enhanced mapping correlation according to claim 1, characterized in that: The I frame 2D distance map, the P' frame 2D distance map and the 2D attribute map are respectively encoded into video streams, and the video streams are encoded into binary code streams through HEVC, specifically: The I frame 2D distance map, the P' frame 2D distance map and the 2D attribute map are encoded into a video stream, and compressed using a video encoding method to obtain a binary code stream.

6. The LiDAR point cloud coding method based on enhanced mapping correlation according to claim 4, characterized in that: The compression into a binary code stream by entropy coding is specifically: Perform 4-level 5 / 3 wavelet transform on the 2D distance map and calculate the optimal quantization step size for each subband; The JPEG_LS algorithm is used to remove the correlation redundancy between pixels from the data of the lowest frequency sub-band, and the data is encoded using zero-run coding to obtain the zero-run coding result; Context modeling is performed based on the zero-run coding result, and the series of exponential Golomb coding is updated, and the binary code stream is obtained using the exponential Golomb coding of the corresponding series.

7. A decoding method for LiDAR point cloud coding based on enhanced mapping correlation, decoding and restoring the point cloud encoded by the LiDAR point cloud coding method based on enhanced mapping correlation according to any one of claims 1 to 6 to the original point cloud, characterized in that: The following steps are involved: Motion information decoding: entropy decoding is performed on the binary code stream obtained by entropy coding to obtain a 2D distance map of the P' frame, and the registration information in the 2D distance map of the P' frame is converted into motion information H; Geometric information decoding: Perform HEVC decoding on the binary code stream obtained by HEVC encoding to obtain the 2D distance map of the I frame and the 2D distance map of the P' frame containing geometric information, convert the 2D distance maps of the I frame and the P' frame into 3D point clouds of the I frame and the P' frame, and use the decoded motion information H to restore the 3D point cloud of the P' frame to obtain the geometric information of the P frame; Attribute information decoding: performing HEVC decoding on the binary code stream obtained by HEVC encoding to obtain a 2D attribute map containing attribute information, and converting the 2D attribute map of the I frame and the P frame into attribute information of the I frame and the P' frame; The 3D point clouds of I-frames and P-frames are restored based on the 2D distance map, 2D attribute map, geometric information and attribute information.

Images