Point cloud polar coordinate encoding method and device

Abstract

The present invention provides a point cloud polar coordinate encoding method and device, comprising the following steps: dividing the circular scanning area scanned by a laser radar at equal angles at an angle Δθ to obtain a plurality of identical polar coordinate areas; dividing each polar coordinate area along the radial direction at equal lengths at a length Δr to obtain a plurality of polar coordinate grids, and generating a plurality of polar coordinate columns corresponding to each polar coordinate grid in a three-dimensional space; generating coordinate column voxels; extracting structural features from all point cloud data in each polar coordinate column voxel; obtaining a two-dimensional point cloud pseudo image; performing boundary compensation on the two-dimensional point cloud pseudo image; and using a convolutional neural network to extract features from the two-dimensional point cloud pseudo image after step F and outputting a final feature map. While realizing the ordered encoding of point cloud data, the present invention can maximize the retention of the inherent characteristics of point cloud data and improve the accuracy of subsequent point cloud target detection.

Description

Technical Field

[0001] The invention relates to a point cloud polar coordinate encoding method and device. Background Art

[0002] LiDAR is widely used in the field of self-driving cars. The point cloud data collected by LiDAR is different from traditional image data. It has a naturally irregular data form and cannot directly transfer the traditional image target detection algorithm to the point cloud. Therefore, by encoding the point cloud data, the disordered point cloud data is ordered, and then the architecture process of using the traditional target detection algorithm to process it can take into account both engineering implementation and the final effect. It is one of the main research directions in the current point cloud target detection field. In order to achieve a higher frame rate, the encoding of point cloud data mostly uses the voxel method, but the current voxel method is mostly based on the Cartesian rectangular coordinate system, which is different from the way LiDAR collects data by rotation. Therefore, more intrinsic characteristics of point cloud data will be lost during the encoding process. Summary of the invention

[0003] The purpose of the present invention is to address the deficiencies in the prior art and propose a point cloud polar coordinate encoding method and device, which can maximize the retention of the inherent characteristics of the point cloud data while realizing the ordered encoding of the point cloud data and improve the accuracy of subsequent point cloud target detection.

[0004] The present invention is achieved through the following technical solutions:

[0005] A point cloud polar coordinate encoding method is used to encode point cloud data obtained by laser radar scanning, comprising the following steps:

[0006] A. Divide the circular scanning area scanned by the laser radar into equal angles at angles Δθ to obtain multiple identical polar coordinate areas;

[0007] B. Divide each polar coordinate area into equal lengths along the radial direction with a length Δr to obtain multiple polar coordinate grids. The radius interval of the (m,n)th polar coordinate grid is [n*Δr, (n+1)*Δr], and the arc interval is [m*Δθ, (m+1)*Δθ], and generate multiple polar coordinate columns corresponding to each polar coordinate grid in three-dimensional space;

[0008] C. All point cloud data in the scanning area are converted into polar coordinates (r, θ), and the polar coordinate column where the point cloud data is located is determined according to the polar coordinate grid radius and arc interval where the polar coordinate (r, θ) falls, so as to obtain the polar coordinate column voxel;

[0009] D. Extract structural features (r,θ,z,I,r) from all point cloud data in each polar coordinate column voxel. c ,θ c ,z c,r p ,θ p ), and ensure that there are L point cloud data in each polar coordinate column voxel, so as to obtain a tensor with a shape of (M, N, L, 9); where (r, θ, z) is the polar coordinate and height of the point cloud data, I is the intensity of the point cloud data, (r c ,θ c ,z c ) is the offset from the point cloud data to the cluster center, (r p ,θ p ) is the offset from the point cloud data to the center of the bottom surface of the polar coordinate column, and M×N is the total number of polar coordinate column voxels;

[0010] E. Perform a 1×1 convolution operation on the K polar coordinate cylinder voxels containing the point cloud data to obtain a tensor with a shape of (K, L, C), and perform maximum pooling on the second dimension of the tensor to obtain a feature tensor with a shape of (K, C), and then map the K features of the feature tensor back to the original position to obtain a two-dimensional point cloud pseudo image with a shape of (M, N, C); where C refers to C different 1×1 convolution operations, and the weighted summation coefficients in the C convolution operations are all different;

[0011] F. Extract the (M-3, M-2, M-1)th row and (0, 1, 2)th row of the two-dimensional point cloud pseudo image, and copy the (M-3, M-2, M-1)th row to the front of the 0th row for padding, and copy the (0, 1, 2)th row to the back of the (M-1)th row for padding, to obtain the two-dimensional point cloud pseudo image after boundary compensation;

[0012] G. Use a convolutional neural network to extract features from the two-dimensional point cloud pseudo image obtained in step F and output the final feature map.

[0013] Furthermore, the radius r of the circular scanning area is 1 The area within is set as a blank area, then the radius interval of the (m,n)th polar coordinate grid is [n*Δr+r 1 ,(n+1)*Δr+r 1 ], the radian interval is [m*Δθ,(m+1)*Δθ].

[0014] Furthermore, the Δθ=1.125°.

[0015] Furthermore, in step C, the point cloud data in the scanning area are converted into polar coordinates (r, θ) by the following formula: Among them, (x, y) is the coordinate of the point cloud data in the rectangular coordinate system.

[0016] Furthermore, L=64.

[0017] Furthermore, in step D, in order to ensure that there are L point cloud data in each polar coordinate column voxel, when the number of point cloud data in the polar coordinate column voxel exceeds L, the point cloud data is randomly downsampled to L, and when the number of point cloud data in the polar coordinate column voxel is less than L, it is supplemented with data point 0.

[0018] Furthermore, the 1 =2m.

[0019] The present invention is also achieved through the following technical solutions:

[0020] A point cloud polar coordinate encoding device, comprising:

[0021] Ordering module: used to divide the circular scanning area scanned by the laser radar into equal angles at an angle of Δθ to obtain multiple identical polar coordinate areas; divide each polar coordinate area into equal lengths along the radial direction at a length of Δr to obtain multiple polar coordinate grids, the radius interval of the (m,n)th polar coordinate grid is [n*Δr, (n+1)*Δr], the arc interval is [m*Δθ, (m+1)*Δθ], and generate multiple polar coordinate columns corresponding to each polar coordinate grid in three-dimensional space;

[0022] Voxel generation module: used to convert all point cloud data in the scanning area into polar coordinates (r, θ), and determine the polar coordinate column where the point cloud data is located according to the polar coordinate grid radius and arc interval where the polar coordinates (r, θ) fall, so as to obtain polar coordinate column voxels;

[0023] Feature extraction module: used to extract structural features (r,θ,z,I,r) from all point cloud data in each polar coordinate column voxel c ,θ c ,z c ,r p ,θ p ), and ensure that there are L point cloud data in each polar coordinate column voxel, so as to obtain a tensor with a shape of (M, N, L, 9); where (r, θ, z) is the polar coordinate and height of the point cloud data, I is the intensity of the point cloud data, (r c ,θ c ,z c ) is the offset from the point cloud data to the cluster center, (r p ,θ p ) is the offset from the point cloud data to the center of the bottom surface of the polar coordinate column, and M×N is the total number of polar coordinate column voxels;

[0024] 2D point cloud pseudo image generation module: used to perform 1×1 convolution operation on K polar coordinate cylinder voxels containing point cloud data to obtain a tensor with a shape of (K, L, C), and perform maximum pooling on the second dimension of the tensor to obtain a feature tensor with a shape of (K, C), and then map the K features of the feature tensor back to the original position to obtain a 2D point cloud pseudo image with a shape of (M, N, C); where C refers to C different 1×1 convolution operations, and the weighted sum coefficients in the C convolution operations are all different;

[0025] 2D point cloud pseudo image compensation module: used to extract the (M-3, M-2, M-1)th row and (0, 1, 2)th row of the 2D point cloud pseudo image, and copy the (M-3, M-2, M-1)th row to the front of the 0th row for padding, and copy the (0, 1, 2)th row to the back of the (M-1)th row for padding, to obtain the 2D point cloud pseudo image after boundary compensation;

[0026] Final feature map acquisition module: used to use convolutional neural network to extract features of the compensated two-dimensional point cloud pseudo image and output the final feature map.

[0027] Furthermore, the radius r of the circular scanning area is 1 The area within is set as a blank area, then the arc interval of the (m,n)th polar coordinate grid is [m*Δθ,(m+1)*Δθ], and the radius interval is [n*Δr+r 1 ,(n+1)*Δr+r 1 ].

[0028] Furthermore, the Δθ=1.125°.

[0029] The present invention has the following beneficial effects:

[0030] 1. The present invention uses polar coordinate coding to process disordered point clouds in an orderly manner, and can convert point cloud data with inconsistent data lengths into structured data of a uniform size, which is convenient for subsequent algorithm model processing; secondly, polar coordinate coding can maximize the fit with the data acquisition method of the rotational scanning of the laser radar, thereby retaining the inherent characteristics of the point cloud data; finally, by copying the (M-3, M-2, M-1) rows of the two-dimensional point cloud pseudo image to the front of the 0th row for padding, and copying the (0, 1, 2) row to the back of the (M-1) row for padding to achieve boundary compensation of the two-dimensional point cloud pseudo image, the two-dimensional point cloud pseudo image is continuous in the radian dimension, which can reduce the error caused by the edge filling operation during the convolution operation. Therefore, the present invention can effectively improve the accuracy of subsequent point cloud target detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] The present invention is further described in detail below with reference to the accompanying drawings.

[0032] Figure 1 The figure is a flow chart of the encoding method of the present invention.

[0033] Figure 2 Schematic diagram of polar coordinate grid division in the encoding method of the present invention.

[0034] Figure 3 Schematic diagram of a polar coordinate grid of the encoding method of the present invention (within a polar coordinate area). DETAILED DESCRIPTION

[0035] like Figure 1 As shown, the point cloud polar coordinate encoding method is used to encode the point cloud data obtained by the laser radar scanning of the vehicle, and includes the following steps:

[0036] A. Divide the circular scanning area scanned by the vehicle's laser radar into equal angles at angles Δθ to obtain multiple identical polar coordinate areas, and divide the radius of the circular scanning area r 1 The area within is set as a blank area; in this embodiment, Δθ=1.125°;

[0037] B. Divide each polar coordinate area into equal lengths along the radial direction with length Δr to obtain multiple polar coordinate grids. The radius interval of the (m,n)th polar coordinate grid is [n*Δr+r 1 ,(n+1)*Δr+r 1 ], the arc interval is [m*Δθ, (m+1)*Δθ], and multiple polar coordinate columns corresponding to each polar coordinate grid are generated in three-dimensional space, such as Figure 2 and Figure 3 As shown; in this embodiment, r 1 =2m;

[0038] C. All point cloud data in the scanning area are converted into polar coordinates (r, θ), and the polar coordinate column where the point cloud data is located is determined according to the polar coordinate grid radius and arc interval where the polar coordinate (r, θ) falls, so as to obtain the polar coordinate column voxel; wherein, the polar coordinate conversion formula is:

[0039] Among them, (x, y) is the coordinate of the point cloud data in the rectangular coordinate system;

[0040] D. Extract structural features (r,θ,z,I,r) from all point cloud data in each polar coordinate column voxel. c ,θ c ,z c ,r p ,θ p ), and ensure that there are L point cloud data in each polar coordinate column voxel, so as to obtain a tensor with a shape of (M, N, L, 9); where (r, θ, z) is the polar coordinate and height of the point cloud data, I is the intensity of the point cloud data, (rc ,θ c ,z c ) is the offset from the point cloud data to the cluster center (the cluster center offset is the center of all the point cloud data in the polar coordinate cylinder voxel), (r p ,θ p ) is the offset from the point cloud data to the center of the bottom surface of the polar coordinate column, and M×N is the total number of polar coordinate column voxels;

[0041] In this embodiment, L=64;

[0042] In order to ensure that the number of point cloud data in each polar coordinate column voxel is L, when the number of point cloud data in the polar coordinate column voxel exceeds L, the point cloud data is randomly downsampled to L. When the number of point cloud data in the polar coordinate column voxel is less than L, it is supplemented with data points with structural features of 0;

[0043] E. Because not all polar coordinate cylinder voxels contain point cloud data, a 1×1 convolution operation is performed on the K polar coordinate cylinder voxels containing point cloud data to obtain a tensor with a shape of (K, L, C), and the second dimension of the tensor is max-pooled to obtain a feature tensor with a shape of (K, C). The K features of the feature tensor are then mapped back to the original positions to obtain a two-dimensional point cloud pseudo image with a shape of (M, N, C); where C refers to C different 1×1 convolution operations, and the weighted sum coefficients in the C convolution operations are all different. This operation is done to further improve the accuracy;

[0044] F. After obtaining the two-dimensional point cloud pseudo image with the shape of (M, N, C), since the first dimension corresponds to the arc change of the polar coordinates, there is no boundary in this dimension, that is, the first row and the last row are connected in space. Therefore, when performing subsequent convolution operations in this dimension, the pixels outside the edge cannot be filled with 0 as in traditional operations. Instead, the (M-3, M-2, M-1)th row and (0, 1, 2)th row (that is, the last three rows and the first three rows) of the two-dimensional point cloud pseudo image are extracted, and the (M-3, M-2, M-1)th row is copied to the front of the 0th row for filling, and the (0, 1, 2)th row is copied to the back of the (M-1)th row for filling, so as to obtain the two-dimensional point cloud pseudo image (M+6, N, C) after boundary compensation;

[0045] G. Use the existing convolutional neural network to extract features from the two-dimensional point cloud pseudo image obtained in step F and output the final feature map.

[0046] Point cloud polar coordinate encoding device, comprising:

[0047] Ordering module: used to divide the circular scanning area scanned by the laser radar into equal angles at an angle of Δθ to obtain multiple identical polar coordinate areas; divide each polar coordinate area into equal lengths along the radial direction at a length of Δr to obtain multiple polar coordinate grids, the radius interval of the (m,n)th polar coordinate grid is [n*Δr, (n+1)*Δr], the arc interval is [m*Δθ, (m+1)*Δθ], and generate multiple polar coordinate columns corresponding to each polar coordinate grid in three-dimensional space;

[0048] Voxel generation module: used to convert all point cloud data in the scanning area into polar coordinates (r, θ), and determine the polar coordinate column where the point cloud data is located according to the polar coordinate grid radius and arc interval where the polar coordinates (r, θ) fall, so as to obtain polar coordinate column voxels;

[0049] Feature extraction module: used to extract structural features (r,θ,z,I,r) from all point cloud data in each polar coordinate column voxel c ,θ c ,z c ,r p ,θ p ), and ensure that there are L point cloud data in each polar coordinate column voxel, so as to obtain a tensor with a shape of (M, N, L, 9); where (r, θ, z) is the polar coordinate and height of the point cloud data, I is the intensity of the point cloud data, (r c ,θ c ,z c ) is the offset from the point cloud data to the cluster center, (r p ,θ p ) is the offset from the point cloud data to the center of the bottom surface of the polar coordinate column, and M×N is the number of voxels in the polar coordinate column;

[0050] 2D point cloud pseudo image generation module: used to perform 1×1 convolution operation on K polar coordinate cylinder voxels containing point cloud data to obtain a tensor with a shape of (K, L, C), and perform maximum pooling on the second dimension of the tensor to obtain a feature tensor with a shape of (K, C), and then map the K features of the feature tensor back to the original position to obtain a 2D point cloud pseudo image with a shape of (M, N, C); where C refers to C different 1×1 convolution operations, and the weighted sum coefficients in the C convolution operations are all different;

[0051] 2D point cloud pseudo image compensation module: used to extract the (M-3, M-2, M-1)th row and (0, 1, 2)th row of the 2D point cloud pseudo image, and copy the (M-3, M-2, M-1)th row to the front of the 0th row for padding, and copy the (0, 1, 2)th row to the back of the (M-1)th row for padding, to obtain the 2D point cloud pseudo image after boundary compensation;

[0052] Final feature map acquisition module: used to use convolutional neural network to extract features of the compensated two-dimensional point cloud pseudo image and output the final feature map.

[0053] The above description is only a preferred embodiment of the present invention, and therefore cannot be used to limit the scope of implementation of the present invention. That is, equivalent changes and modifications made according to the scope of the patent application of the present invention and the contents of the specification should still fall within the scope covered by the patent of the present invention.

Claims

1. A point cloud polar coordinate encoding method for encoding point cloud data obtained by laser radar scanning, characterized in that: The steps include: A. Divide the circular scanning area scanned by the laser radar into equal angles at angles Δθ to obtain multiple identical polar coordinate areas; B. Divide each polar coordinate area into equal lengths along the radial direction with a length Δr to obtain multiple polar coordinate grids. The radius interval of the (m,n)th polar coordinate grid is [n*Δr, (n+1)*Δr], and the arc interval is [m*Δθ, (m+1)*Δθ], and generate multiple polar coordinate columns corresponding to each polar coordinate grid in three-dimensional space; C. All point cloud data in the scanning area are converted into polar coordinates (r, θ), and the polar coordinate column where the point cloud data is located is determined according to the polar coordinate grid radius and arc interval where the polar coordinate (r, θ) falls, so as to obtain the polar coordinate column voxel; D. Extract structural features (r,θ,z,I,r) from all point cloud data in each polar coordinate column voxel. c ,θ c ,z c ,r p ,θ p ), and ensure that there are L point cloud data in each polar coordinate column voxel, so as to obtain a tensor with a shape of (M, N, L, 9); where (r, θ, z) is the polar coordinate and height of the point cloud data, I is the intensity of the point cloud data, (r c ,θ c ,z c ) is the offset from the point cloud data to the cluster center, (r p ,θ p ) is the offset from the point cloud data to the center of the bottom surface of the polar coordinate column, and M×N is the total number of polar coordinate column voxels; E. Perform a 1×1 convolution operation on the K polar coordinate cylinder voxels containing the point cloud data to obtain a tensor with a shape of (K, L, C), and perform maximum pooling on the second dimension of the tensor to obtain a feature tensor with a shape of (K, C), and then map the K features of the feature tensor back to the original position to obtain a two-dimensional point cloud pseudo image with a shape of (M, N, C); where C refers to C different 1×1 convolution operations, and the weighted summation coefficients in the C convolution operations are all different; F. Extract the (M-3, M-2, M-1)th row and (0, 1, 2)th row of the two-dimensional point cloud pseudo image, and copy the (M-3, M-2, M-1)th row to the front of the 0th row for padding, and copy the (0, 1, 2)th row to the back of the (M-1)th row for padding, to obtain the two-dimensional point cloud pseudo image after boundary compensation; G. Use a convolutional neural network to extract features from the two-dimensional point cloud pseudo image obtained in step F and output the final feature map.

2. A point cloud polar coordinate encoding method according to claim 1, characterized in that: The area within the radius r1 of the circular scanning area is set as a blank area, then the radius interval of the (m, n)th polar coordinate grid is [n*Δr+r1, (n+1)*Δr+r1], and the radian interval is [m*Δθ, (m+1)*Δθ].

3. A point cloud polar coordinate encoding method according to claim 1, characterized in that: The Δθ=1.125°.

4. A point cloud polar coordinate encoding method according to claim 1, 2 or 3, characterized in that: In step C, the point cloud data in the scanning area are converted into polar coordinates (r, θ) by the following formula: Among them, (x, y) is the coordinate of the point cloud data in the rectangular coordinate system.

5. A point cloud polar coordinate encoding method according to claim 1, 2 or 3, characterized in that: The L=64.

6. A point cloud polar coordinate encoding method according to claim 1, 2 or 3, characterized in that: In step D, in order to ensure that there are L point cloud data in each polar coordinate column voxel, when the number of point cloud data in the polar coordinate column voxel exceeds L, the point cloud data is randomly downsampled to L, and when the number of point cloud data in the polar coordinate column voxel is less than L, it is supplemented with data point 0.

7. A point cloud polar coordinate encoding method according to claim 2, characterized in that: Said r1=2m.

8. A point cloud polar coordinate encoding device, characterized in that: include: Ordering module: used to divide the circular scanning area scanned by the lidar into equal angles at angles Δθ to obtain multiple identical polar coordinate areas; Each polar coordinate area is divided into equal lengths along the radial direction with a length Δr to obtain a plurality of polar coordinate grids, the radius interval of the (m,n)th polar coordinate grid is [n*Δr, (n+1)*Δr], and the arc interval is [m*Δθ, (m+1)*Δθ], and a plurality of polar coordinate columns corresponding to each polar coordinate grid are generated in three-dimensional space; Voxel generation module: used to convert all point cloud data in the scanning area into polar coordinates (r, θ), and determine the polar coordinate column where the point cloud data is located according to the polar coordinate grid radius and arc interval where the polar coordinates (r, θ) fall, so as to obtain polar coordinate column voxels; Feature extraction module: used to extract structural features (r,θ,z,I,r) from all point cloud data in each polar coordinate column voxel c ,θ c ,z c ,r p ,θ p ), and ensure that there are L point cloud data in each polar coordinate column voxel, so as to obtain a tensor with a shape of (M, N, L, 9); where (r, θ, z) is the polar coordinate and height of the point cloud data, I is the intensity of the point cloud data, (r c ,θ c ,z c ) is the offset from the point cloud data to the cluster center, (r p ,θ p ) is the offset from the point cloud data to the center of the bottom surface of the polar coordinate column, and M×N is the number of voxels in the polar coordinate column; 2D point cloud pseudo image generation module: used to perform 1×1 convolution operation on K polar coordinate cylinder voxels containing point cloud data to obtain a tensor with a shape of (K, L, C), and perform maximum pooling on the second dimension of the tensor to obtain a feature tensor with a shape of (K, C), and then map the K features of the feature tensor back to the original position to obtain a 2D point cloud pseudo image with a shape of (M, N, C); where C refers to C different 1×1 convolution operations, and the weighted sum coefficients in the C convolution operations are all different; 2D point cloud pseudo image compensation module: used to extract the (M-3, M-2, M-1)th row and (0, 1, 2)th row of the 2D point cloud pseudo image, and copy the (M-3, M-2, M-1)th row to the front of the 0th row for padding, and copy the (0, 1, 2)th row to the back of the (M-1)th row for padding, to obtain the 2D point cloud pseudo image after boundary compensation; Final feature map acquisition module: used to use convolutional neural network to extract features of the compensated two-dimensional point cloud pseudo image and output the final feature map.

9. A point cloud polar coordinate encoding device according to claim 8, characterized in that: The area within the radius r1 of the circular scanning area is set as a blank area, then the arc interval of the (m, n)th polar coordinate grid is [m*Δθ, (m+1)*Δθ], and the radius interval is [n*Δr+r1, (n+1)*Δr+r1].

10. A point cloud polar coordinate encoding device according to claim 8 or 9, characterized in that: The Δθ=1.125°.

Images