A fast calibration method for multiple laser radar point cloud deduplication of an autonomous driving system

Abstract

The application provides a kind of automatic driving system multi-laser radar point cloud deduplication fast calibration method, comprising the following steps: S1, data analysis for parent radar and child radar, generate point cloud data, the parent radar refers to the laser radar of final release fusion point cloud data, child radar refers to the laser radar needing to transform point cloud data;S2, roughly calculate the pose of parent radar and child radar;S3, calculate the registration relationship between parent radar and child radar.The application has the beneficial effects: a kind of automatic driving system multi-laser radar point cloud deduplication fast calibration method, through point cloud data analysis and point cloud data feature extraction registration, and the fine registration of ICP algorithm, finally different point cloud data is carried out space-time synchronization and adjacent threshold method is repeated point removal, realizes the fine calibration of the external parameter of multiple laser radars, expresses different laser radar data under the same perspective, greatly improves the density of point cloud data and the field of view of radar.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of radars, and particularly relates to a fast calibration method for multiple laser radar point cloud deduplication of an automatic driving system. BACKGROUND

[0002] In the field of unmanned driving technology, environment perception as an interactive module of an unmanned vehicle in a surrounding environment plays a very important role. In the current technical field, commonly used sensor types are roughly laser radars and monocular cameras. Since the laser radar has the characteristic of not being affected by light and can work all day, its ranging mode can provide accurate obstacle position information for the unmanned vehicle, and provide reliable basis for subsequent decision and control of the unmanned vehicle.

[0003] At present, due to the high price of high-beam radars and the low point cloud density of low-beam radars, in view of this situation, in order to reduce the equipment price of the sensor of the unmanned vehicle, combining multiple low-beam radars into one high-beam radar is a reliable solution, which can not only provide high-density point cloud, but also improve the sensing field of view of the radar.

[0004] In the current calibration method, most methods are implemented based on coordinate system conversion. The point cloud is processed according to the conversion relationship between the coordinate systems through the coordinate system relationship between different sensors. The disadvantage of this operation is that the point cloud data is only simply superimposed, and there is a certain position deviation and a large amount of repeated point cloud. SUMMARY

[0005] Therefore, the application aims to provide a fast calibration method for multiple laser radar point cloud deduplication of an automatic driving system, which performs external parameter calibration between different laser radars based on a point cloud feature matching method.

[0006] To achieve the above purpose, the technical scheme of the application is as follows:

[0007] A fast calibration method for multiple laser radar point cloud deduplication of an automatic driving system, comprising the following steps:

[0008] S1, data analysis of the parent radar and the child radar to generate point cloud data, wherein the parent radar refers to a laser radar that finally publishes fusion point cloud data, and the child radar refers to a laser radar that needs to convert point cloud data. Since each radar is independent of each other, data analysis needs to be performed on each radar. Before data analysis, the received data frame of the radar includes some data frame headers, data contents and data check bits, etc. After data analysis, the point cloud data type is published.

[0009] S2, roughly calculating the pose of the parent radar and the child radar.

[0010] S3, calculate the registration relationship between the parent radar and the child radar;

[0011] S4, perform space-time synchronization on the data frames of the parent radar and the child radar, which are the data frames generated after the parent and child radars are parsed in step S1, and are point cloud data frames formed after the original data is parsed;

[0012] S5, remove redundant data under the same view angle.

[0013] Further, in step S1, the data generated by the parent radar and the child radar is parsed by the driver and published in the form of a topic.

[0014] Further, in step S2, the contents to be calculated include the spatial position difference and the attitude difference of the child radar relative to the parent radar.

[0015] Further, when calculating the spatial position relationship of each child radar relative to the parent radar, the spatial coordinates X, Y, and Z values of each child radar and the parent radar need to be calculated respectively.

[0016] Further, in step S3, the registration relationship between different sensors is calculated using a point cloud feature matching algorithm, including a dynamic grid filtering method based on point cloud clustering, a point cloud feature extraction method, a point cloud feature registration method, and a method for calculating the extrinsic relationship between different radar data frames.

[0017] Further, the dynamic grid filtering method based on point cloud clustering includes a method of performing point cloud filtering operation using a dynamic grid method based on K-Means, including the following steps:

[0018] A1, first, the point cloud data of the parent radar and the child radar is represented by the point cloud data structure of the KD tree, generating an organized KD tree;

[0019] A2, the organized KD tree point cloud in step A1 is spatially clustered and divided using the K-Means algorithm, and the clustered and divided point cloud clusters are recorded and stored in a point cloud container;

[0020] A3, divide the point cloud grid of each point cloud cluster, and the division rule is that when the point cloud in the point cloud cluster contains a number of points that exceeds a set threshold, the point cloud representative of the point cloud region is represented by the point cloud of the fixed threshold voxel block center, and if the number of point cloud clusters in a certain point cloud cluster is small, the point cloud cluster is treated as a whole point cloud block and no downsampling operation is performed.

[0021] Further, point cloud spatial feature extraction is performed on the point cloud clusters after the point cloud grid division in step A3, which is used to extract point cloud corner points and plane points, including:

[0022] The Harris feature point detection method is adopted, first, the point cloud corner point threshold and the point cloud plane point threshold are set, the change rate of the slope of the point cloud data in the XYZ three-axis direction is detected, if the change rate of the slope of the point cloud in the XYZ three-axis direction is greater than the set threshold, it belongs to the corner point, and if it is less than the set threshold, it is determined as a plane point, finally, the extracted point cloud features are stored, that is, if it is greater than the set threshold, the feature point is classified as a corner feature point, and if it is less than the threshold, the feature point is classified as a plane feature point.

[0023] Further, the method for calculating the external parameter relationship between different radar data frames comprises:

[0024] In the point cloud feature registration process, the point cloud data extracted by each laser radar is subjected to step (2) calculation, the point cloud data features under the same viewing angle are obtained, and then the features are described, the point cloud features are described in the form of vectors, and then the point cloud feature descriptors stored by each laser radar are subjected to similarity detection, and the cosine distance measurement method is adopted, and the calculation method is shown in (1);

[0025]

[0026] Wherein, cosθ represents the cosine distance between two points, and the value range is [-1, 1];

[0027] By calculating the feature descriptor of each radar (including the parent radar and the child radar), the cosine distance of the feature descriptor is obtained, which represents the similarity of the two feature descriptors, and the point cloud data with a cosine distance greater than the set threshold is used to calculate the rough position by using the PCL point cloud library, the rough point cloud pose change matrix is obtained, and the point cloud data of the child radar is transformed into the coordinate system of the parent radar, and the rough registration of the point cloud data frame is completed. After the point cloud feature rough registration, the current pose transformation information is used as the initial value of the ICP algorithm iteration, and the ICP algorithm is used for multiple iterations, which can further improve the registration accuracy. When the ICP registration converges and is less than the set convergence threshold, the pose transformation matrix of the multiple child radars to the parent radar is output, that is, the calibrated external parameter matrix.

[0028] Further, the method for removing redundant data under the same viewing angle is:

[0029] The repeated point cloud is removed by using the radius neighbor threshold method, first, the point clouds of different radar data frames after synchronization are received, the received point clouds are organized in the form of the data structure of KD tree, next, the distance calculation is carried out on the point cloud data according to the set detection radius threshold and the threshold of adjacent points, if the distance between two points is less than the set detection threshold, the two points are considered to be coincident points, the index values of the repeated points are sorted and recorded, finally, the index values of the point clouds are deleted according to the Extract class in PCL, and the redundant data of different sensors under the same view angle is removed by the method.

[0030] Compared with the prior art, the automatic driving system multi-laser radar point cloud deduplication rapid calibration method has the following beneficial effects:

[0031] The automatic driving system multi-laser radar point cloud deduplication rapid calibration method can realize the fine calibration of the external parameters of multiple laser radars, express different laser radar data under the same view angle, and greatly improve the density of point cloud data and the field of view of the radar. DETAILED DESCRIPTION

[0032] It should be noted that the embodiments in the present application and the features in the embodiments can be combined with each other without conflict.

[0033] The present application will be described in detail below with reference to the embodiments.

[0034] In the current calibration method, most methods are realized based on coordinate system transformation, the point cloud is processed according to the transformation relationship between the coordinate systems through the coordinate system relationship between different sensors, and the disadvantages of the operation are that the point cloud data is only simply superimposed, and there is a certain position deviation and a large amount of repeated point cloud.

[0035] The present application is based on the method of point cloud feature matching to calibrate the external parameters between different laser radars, and the specific process is as follows:

[0036] Multi-laser radar data analysis;

[0037] Roughly calculate the pose of the parent radar and the child radar;

[0038] The registration relationship between different sensors is calculated by using the point cloud feature matching algorithm;

[0039] The data frames are time and space synchronized;

[0040] Remove the redundant data under the same view angle.

[0041] The specific implementation method is as follows:

[0042] Multi-lidar data analysis

[0043] Since the data communication of the lidar is in the form of UDP communication, data analysis and publishing need to be performed according to the relevant technical protocol of the lidar. The research platform of the present scheme is Ubuntu 18.04 and the ROS (Robot Operating System) robot operating system. The radar data after analysis is finally published in the form of a topic.

[0044] Roughly calculate the pose of the parent radar and the child radar

[0045] In the external parameter calibration of multiple radars, the ultimate goal is to calibrate the point clouds under different radar perspectives to the same laser radar perspective, so it is necessary to determine the parent radar and the child radar. The parent radar refers to the laser radar that finally publishes the fused point cloud data, and the child radar refers to the laser radar that needs to be converted. In the process of calculating the pose of the parent radar and the child radar, the contents that need to be calculated include the spatial position difference and the attitude difference of the child radar relative to the parent radar. Since step 2 is a rough pose calculation process, only the spatial position relationship of each child radar relative to the parent radar, i.e. the values of the spatial coordinates X, Y and Z, needs to be calculated. The initial value can be used for attitude estimation. Through the rough calculation of the relationship between the child radar and the parent radar, the accuracy and speed of the subsequent registration process can be greatly accelerated.

[0046] Calculate the registration relationship between different sensors using a point cloud feature matching algorithm

[0047] In the process of calculating the pose between different sensors using a point cloud feature matching algorithm, the following steps are mainly included:

[0048] They are respectively a dynamic grid filtering method based on point cloud clustering, point cloud feature extraction, point cloud feature registration, and calculation of the external parameter relationship between different radar data frames.

[0049] (1) Dynamic grid filtering method based on point cloud clustering

[0050] The current mainstream point cloud filtering methods mainly include Gaussian filtering and voxel filtering. In the field of unmanned vehicle point cloud processing, most of them use a fixed threshold point cloud voxel filtering method. However, when the point cloud density is low, the filtering method with a fixed threshold will destroy the structure of the point cloud. Moreover, different voxel thresholds need to be adapted to different application scenarios, and it is not universal.

[0051] The scheme adopts a dynamic grid method based on K-Means for point cloud filtering operation. First, the point cloud data is represented by the point cloud data structure of the KD tree. The purpose of this is to divide the distance of different points in the subsequent steps and to speed up the search speed of the point cloud. Next, the organized KD tree point cloud is clustered and divided in space using the K-Means algorithm. The clustered point cloud clusters are recorded and stored in a point cloud container. The purpose of this is to perform point cloud spatial grid division and subsequent point cloud feature comparison. Finally, the point cloud grid of each cluster is divided. The division rule is that when the point cloud in the cluster contains a point cloud number exceeding the set threshold, the point cloud of the fixed threshold voxel block center represents the point cloud in the point cloud area. If the point cloud cluster number of a point cloud cluster is less than the set threshold, the point cloud cluster is treated as a whole point cloud block and no downsampling operation is performed. The dynamic grid filtering method of point cloud clustering can not only ensure that the spatial structure of the point cloud is not damaged and more point cloud features are retained, but also can greatly reduce the data amount of the spatial point cloud and speed up the overall calculation speed.

[0052] (2) Spatial point cloud feature extraction

[0053] The point cloud data grid obtained by step (1) needs to be extracted for spatial features. The point cloud feature points extracted here are mainly point cloud corner points and plane points.

[0054] The detection method adopted is Harris feature point detection, which mainly focuses on the rate of change of the slope of the point cloud data in the three-axis direction. If the slope changes significantly in the three-axis direction, it is an angle point. If it is less than the set threshold, it is determined to be a plane point. Finally, the extracted feature points are stored for subsequent feature similarity measurement and point cloud data frame matching.

[0055] Point cloud feature matching and obtaining calibration parameters

[0056] In the point cloud feature registration process, the point cloud data extracted by each laser radar is calculated by step (2) to obtain the point cloud data features under the same viewing angle. The similarity of the point cloud features stored by each laser radar is detected. The Euclidean distance metric is used here, and the calculation method is shown in formula 1.

[0057]

[0058] Wherein, cos represents the two points between the cosine distance, the value range is [-1, 1]. By calculating the feature points under each radar, the distance greater than the set threshold is calculated, the calculation method is to use PCL point cloud library, the point cloud pose change matrix is converted, that is, the point cloud data of the sub-radar is converted to the coordinate system of the parent radar, which completes the rough registration of the point cloud data frame. In order to improve the accuracy of registration and improve the accuracy of multi-radar external parameter calibration, the main role of the rough registration in the previous step is to provide a better initial position for the subsequent accurate registration ICP algorithm. Because the disadvantage of ICP algorithm is that a better initial iteration position is needed, if there is no accurate rough registration, the iteration efficiency and convergence speed of the subsequent ICP algorithm will be slow. After the rough registration of the point cloud features, the ICP algorithm is used for multiple iterations, which can further improve the accuracy of registration. When the ICP registration converges and is less than the set threshold, the pose transformation matrix of the multiple sub-radar to the parent radar is output, that is, the external parameter matrix of the calibration.

[0059] 4, Space-time synchronization of data frame

[0060] The data between different sensors has different coordinate systems and time stamps, and the multi-radar sensor data calibration process includes time synchronization and space synchronization. The time synchronization is realized by using the timestamp synchronization mechanism of the ROS operating system, and the method adopted in this scheme is the adjacent frame data synchronization method. The space synchronization method is to use the rotation and translation matrix obtained in step (3) to convert the point cloud coordinate data.

[0061] 5, Remove redundant data under the same view angle

[0062] Because there are the same detection view angles between different laser radars, it is necessary to remove the repeated point cloud data under the same view angle, so as to ensure the one-to-one relationship between the spatial position and the point cloud data. This scheme adopts the radius neighbor threshold method to remove the repeated point cloud. First, the point clouds of the synchronized different radar data frames are received, and the received point clouds are organized in the form of KD tree data structure to speed up the search speed of the point clouds. Next, the point cloud data is calculated according to the set detection radius threshold and the threshold of adjacent points, if the distance between two points is less than the set detection threshold, it is considered as a coincident point, the index value of the repeated point is sorted and recorded, finally the Extract class in PCL is used to delete the operation according to the index value of the point cloud, and the redundant data of different sensors under the same view angle is removed by this method.

[0063] The scheme realizes fine calibration of the external parameters of multiple laser radars by point cloud data analysis, point cloud data feature extraction and registration, fine registration of the ICP algorithm, and finally time and space synchronization of different point cloud data and repeated point removal by the neighborhood threshold method, so that different laser radar data is expressed under the same perspective, greatly improving the density of the point cloud data and the field of view of the radar.

[0064] Those skilled in the art can appreciate that the units and method steps of the examples described in combination with the embodiments disclosed herein can be realized in electronic hardware, computer software or a combination of both. In order to clearly illustrate the interchangeability of hardware and software, the components and steps of the examples have been described in the above description in general terms. Whether the functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. A person skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0065] In several embodiments provided in the present application, it should be understood that the disclosed method and system can be implemented in other ways. For example, the division of the above-mentioned units is only a logical functional division, and actual implementation can have another division manner, for example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. The above-mentioned units can be or can not be physically separated, and the components displayed as units can be or can not be physical units, i.e., they can be located in one place or distributed on multiple network units. According to actual needs, part or all of the units can be selected to achieve the purpose of the embodiment of the present application.

[0066] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application, and not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: it can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement for part or all of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the present application, and they should be covered in the scope of the claims and description of the present application.

[0067] The above-mentioned is only the preferred embodiment of the present application, and does not limit the present application, any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the protection scope of the present application.

Claims

1. A rapid calibration method for deduplication of multiple lidar point clouds in an autonomous driving system, characterized in that, Includes the following steps: S1. Data parsing of the parent radar and child radar to generate point cloud data. The parent radar refers to the LiDAR that ultimately releases the fused point cloud data, and the child radar refers to the LiDAR that needs to convert the point cloud data. S2. Roughly calculate the pose of the parent radar and the child radar; S3. Calculate the registration relationship between the parent radar and the child radar; S4. Perform spatiotemporal synchronization on the data frames of the parent radar and the child radar; S5. Remove redundant data from the same viewpoint; In step S3, the registration relationship between different sensors is calculated using a point cloud feature matching algorithm, including a dynamic grid filtering method based on point cloud clustering, a point cloud feature extraction method, a point cloud feature registration method, and a method for calculating the external parameter relationship between different radar data frames. The dynamic mesh filtering method based on point cloud clustering includes: a method for point cloud filtering using a K-Means-based dynamic mesh method, comprising the following steps: A1. First, the point cloud data of the parent radar and child radar are represented through the point cloud data structure of KD tree to generate a well-organized KD tree. A2. Use the K-Means algorithm to perform spatial clustering on the KD tree point cloud organized in step A1, record the clustered point cloud clusters, and store them in a point cloud container. A3. Divide the point cloud mesh of each point cloud cluster. The division rule is that when the number of points in a point cloud cluster exceeds the set threshold, the point cloud of the centroid of the voxel block with a fixed threshold is used to represent the point cloud in that point cloud region. If the number of points in a point cloud cluster is small, this point cloud cluster is treated as the entire point cloud block and no downsampling operation is performed.

2. The rapid calibration method for deduplication of multiple lidar point clouds in an autonomous driving system according to claim 1, characterized in that, In step S1, the data generated by the parent radar and the child radar are parsed by the driver program and then published as messages in the form of topics.

3. The rapid calibration method for deduplication of multiple lidar point clouds in an autonomous driving system according to claim 1, characterized in that, In step S2, the calculations need to include the spatial position difference and attitude difference between the sub-radar and the parent radar.

4. The rapid calibration method for deduplication of multiple lidar point clouds in an autonomous driving system according to claim 3, characterized in that, When calculating the spatial positional relationship of each sub-radar relative to the parent radar, it is necessary to calculate the spatial coordinates X, Y, Z values ​​of each sub-radar and the parent radar separately.

5. The rapid calibration method for deduplication of multiple lidar point clouds in an autonomous driving system according to claim 1, characterized in that, Point cloud spatial feature extraction is performed on the point cloud clusters after point cloud meshing in step A3, to extract point cloud corner points and planar points, including: The Harris feature point detection method is adopted. First, the thresholds for corner points and planar points in the point cloud are set. The rate of change of the slope of the point cloud data in the XYZ axes is detected. If the rate of change of the slope of the point cloud in the XYZ axes is greater than the set threshold, it is a corner point. If it is less than the set threshold, it is a planar point. Finally, the extracted point cloud features are stored.

6. The rapid calibration method for deduplication of multiple lidar point clouds in an autonomous driving system according to claim 5, characterized in that, Methods for calculating the extrinsic parameter relationships between different radar data frames include: In the point cloud feature registration process, the point cloud data extracted by each lidar is calculated in step (2) to obtain the point cloud data features under the same viewpoint. Then, these features are described in terms of features. The point cloud features are described in terms of vectors. Then, the similarity detection is performed on the point cloud feature descriptors stored by each lidar. The cosine distance metric is used, and the calculation method is shown in (1). Where cosθ represents the cosine distance between two points, and its value ranges from [-1, 1]. By calculating the coherent distance of the feature descriptors under each radar, point cloud data with a coherent distance greater than the set threshold are used to perform coarse position calculation using the PCL point cloud library. The coarsely obtained point cloud pose change matrix is ​​then used to transform the point cloud data, and the point cloud data of the sub-radars is transformed into the coordinate system of the parent radar through matrix transformation, thus completing the coarse registration of the point cloud data frames. After the coarse registration of point cloud features, the current pose change information is used as the initial value for the ICP algorithm iteration. The ICP algorithm is then used for multiple iterations to further improve the registration accuracy. When the ICP registration converges and is less than the set convergence threshold, the pose change matrices of multiple sub-radars to the parent radar are output, which are the calibrated extrinsic parameter matrices.

7. The rapid calibration method for deduplication of multiple lidar point clouds in an autonomous driving system according to claim 1, characterized in that, In step S5, the method for removing redundant data from the same viewpoint is as follows: The radius-adjacent threshold method is used to remove duplicate point clouds. First, the point clouds of different synchronized radar data frames are received. The received point clouds are organized in the form of a KD-tree data structure. Next, the distance between the point cloud data is calculated according to the set detection radius threshold and the threshold of adjacent points. If the distance between two points is less than the set detection threshold, they are considered to be overlapping points. The index values ​​of the duplicate points are sorted and recorded. Finally, the Extract class in PCL is used to perform a deletion operation based on the index value of the point cloud. This method removes redundant data from different sensors under the same viewpoint.

Images