A method and system for calibrating external parameters of laser radar

Abstract

The present invention discloses a laser radar external parameter calibration method and system, which relates to the laser radar technical field. It includes: performing initial measurement on the laser radar to obtain the initial posture; obtaining the point cloud data of two laser radars Lidar1 and Lidar2, fusing the point cloud data with the initial posture and initially aligning it to obtain the reference posture; extracting features from the point cloud data of the two laser radars, performing feature matching on the extracted features based on a convolutional neural network, and obtaining the predicted posture through global aggregation of the results after feature matching; respectively calculating the deviation L t , Deviation L q ; Through the deviation L t and the deviation L q A loss function Loss is constructed, a convolutional neural network is trained with the loss function Loss, the trained convolutional neural network is re-applied to feature matching of global features to obtain a final predicted posture, and the final predicted posture is calibrated as external parameter information of the laser radar. The present invention can avoid the influence of posture calibration data on the external parameter calibration process and improve the accuracy of the laser radar external parameter calibration result.

Description

Technical Field

[0001] The present invention relates to the field of laser radar technology, and in particular to a laser radar external parameter calibration method and system. Background Art

[0002] Multi-LiDAR extrinsic calibration is an important issue in the field of sensor fusion, especially in the fields of autonomous driving, robot navigation, and intelligent monitoring. Its core task is to accurately determine the relative position and posture relationship between multiple LiDARs so that the data of different LiDARs can be accurately aligned to the same coordinate system in subsequent data fusion, scene reconstruction, and target tracking tasks.

[0003] In the prior art, for example, the Chinese invention publication number is CN110596683B, and the patent name is a multi-group laser radar external parameter calibration system and method, which includes a point cloud segmentation module, a ground detection point cloud matching module, a map construction module, an information fusion module, and a path optimization module. By using the context information of the laser radar over a period of time, calculating the point motion relationship between the front and back frames of the laser radar, and then combining the posture information provided by the IMU to eliminate the accumulated radar motion distortion, the scene map in the area is finally constructed to complete the calibration of the multi-group laser radar external parameters.

[0004] The defects of the above-mentioned existing technology are: in the case of rapidly changing scenes or large-scale movements of the laser radar, the calibration process of its external parameters is easily affected by the radar posture calibration data, resulting in inaccurate results of the laser radar external parameter calibration. Summary of the invention

[0005] Based on this, it is necessary to provide a laser radar external parameter calibration method and system to address the above technical problems.

[0006] The embodiment of the present invention provides a method and system for calibrating external parameters of a laser radar, including:

[0007] Perform initial measurement on Lidar1 and Lidar2 to obtain the initial pose (q0, t0);

[0008] Obtain the point cloud data of Lidar1 and Lidar2, and perform initial registration between the point cloud data and the initial pose (q0, t0) to obtain the reference pose (q re ,t re );

[0009] The features of each point in the point cloud data of Lidar1 and Lidar2 are extracted to obtain local features, and the local features are aggregated to obtain the global features of Lidar1 and Lidar2. The global features of Lidar1 and Lidar2 are matched using a convolutional neural network, and the results of feature matching are globally aggregated to obtain the predicted pose (q pre ,t pre );

[0010] Determine the translation vector t of the predicted pose pre The initial translation vector t with respect to the reference pose re The deviation L t , and determine the predicted quaternion q of the predicted pose pre Quaternion q with reference pose re The deviation L q ;

[0011] By deviation L t and the deviation L q Construct a loss function Loss, use the loss function Loss to train the convolutional neural network, apply the trained convolutional neural network to the feature matching of the global features of Lidar1 and Lidar2 again, get the final predicted pose, and calibrate the final predicted pose as the external parameter of the Lidar.

[0012] Additionally, the initial measurements include:

[0013] Initialize the system state measured by IMU and predict the state at the next moment;

[0014] The state estimation is updated through IMU measurement values, and the accurate initial pose is obtained through repeated iterations of Kalman filtering.

[0015] Additionally, the initial registration includes:

[0016] The ICP algorithm is used to align the two lidar point clouds. First, the reference pose is initialized;

[0017] Iteratively optimize the spatial correspondence between point clouds to minimize the distance error between them until convergence, and obtain the reference pose (q re ,t re ).

[0018] In addition, the calculated deviation L t The formula is:

[0019]

[0020] The calculated deviation L q The formula is:

[0021]

[0022] Among them, L t represents the translation vector t pre With the initial translation vector t re Deviation of L q Represents the predicted quaternion q pre and the predicted quaternion q re The deviation of n ,b n ,c n ,d n} is the quaternion q pre The amount in.

[0023] In addition, the passing deviation L t and the deviation L q The formula for constructing the loss function Loss is:

[0024]

[0025] Among them, λ1 and λ2 are hyperparameters, L t represents the translation vector t pre With the initial translation vector t re Deviation of L q Represents the predicted quaternion q pre and the predicted quaternion q re deviation.

[0026] In addition, a laser radar external parameter calibration system includes:

[0027] The initial measurement module is used to perform initial measurement on Lidar1 and Lidar2 to obtain the initial pose (q0, t0);

[0028] The point cloud acquisition module is used to obtain the point cloud data of Lidar1 and Lidar2, and to perform initial registration between the point cloud data and the initial pose (q0, t0) to obtain the reference pose (q re ,t re );

[0029] The processing module is used to extract the features of each point in the point cloud data of Lidar1 and Lidar2 to obtain local features, aggregate the local features to obtain the global features of Lidar1 and Lidar2; use the convolutional neural network to perform feature matching on the global features of Lidar1 and Lidar2, and globally aggregate the results after feature matching to obtain the predicted pose (q pre ,t pre );

[0030] Deviation calculation module, used to determine the translation vector t of the predicted pose pre The initial translation vector t with respect to the reference pose re The deviation L t , and determine the predicted quaternion q of the predicted pose pre Quaternion q with reference pose re The deviation L q ;

[0031] External parameter calibration module, used to pass the deviation L t and the deviation L q Construct a loss function Loss, use the loss function Loss to train the convolutional neural network, apply the trained convolutional neural network to the feature matching of the global features of Lidar1 and Lidar2 again, get the final predicted pose, and calibrate the final predicted pose as the external parameter of the Lidar.

[0032] Compared with the prior art, the above-mentioned laser radar extrinsic parameter calibration method and system provided by the embodiment of the present invention have the following beneficial effects:

[0033] Through the reference pose of the laser radar (q re ,t re ) and predicted pose (q pre ,t pre ) Determine the translation vector t of the predicted pose pre The initial translation vector t with respect to the reference pose re The deviation L t And the predicted quaternion q of the predicted pose pre Quaternion q with reference pose re The deviation L q Among them, due to rapidly changing scenes or large-scale movements, the LiDAR will generate posture calibration data, which will affect the accuracy of the context information of the LiDAR over a period of time, resulting in deviations in the calculation results of the point motion relationship between the previous and next frames of the LiDAR. Therefore, the external parameter calibration results of the LiDAR are not accurate enough.

[0034] The present invention adopts the deviation L t and the deviation L q A loss function Loss is constructed, and the convolutional neural network is trained with the loss function Loss. The trained convolutional neural network is re-applied to the feature matching of the global features of the lidar Lidar1 and Lidar2 to obtain the final predicted pose of the lidar extrinsic parameter information. This can avoid the influence of the pose calibration data on the external parameter calibration process, and then calibrate the final predicted pose as the external parameter of the lidar, thereby improving the accuracy of the lidar extrinsic parameter calibration results. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 A schematic diagram of a flow chart of a laser radar extrinsic parameter calibration method provided in an embodiment;

[0036] Figure 2 A schematic diagram of the composition of a laser radar extrinsic parameter calibration system provided in an embodiment;

[0037] Figure 3 A network framework diagram of a laser radar extrinsic parameter calibration method provided in an embodiment. DETAILED DESCRIPTION

[0038] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0039] 1. IMU-assisted multi-lidar external parameter calibration method based on deep learning

[0040] The acquired IMU data is time synchronized with the point cloud data. The flowchart of IMU-assisted multi-lidar external parameter calibration based on deep learning is as follows: Figure 1 As shown, it is divided into the following main steps:

[0041] (1) Perform initial measurement on Lidar1 and Lidar2 to obtain the initial pose (q0, t0).

[0042] The high-precision IMU first performs initial measurements, initializes the system state measured by the IMU, predicts the state at the next moment, updates the state estimate through the IMU measurement value, and repeatedly iterates and optimizes the Kalman filter to improve the accuracy of the pose estimation and obtain an accurate initial pose (q0, t0).

[0043] (2) Obtain the point cloud data of Lidar1 and Lidar2, and perform initial registration between the point cloud data and the initial pose (q0, t0) to obtain the reference pose (q re ,t re ).

[0044] After the two laser radars acquire point cloud data, the registration area is limited to a smaller range by combining the initial pose information in step 1 and the output information of the IMU. The ICP algorithm is used to register the two laser radar point clouds. First, the reference pose is initialized, and then the spatial correspondence between the point clouds is iteratively optimized to minimize the distance error between them until convergence. The reference pose (q re ,t re ).

[0045] (3) Extract the features of each point in the point cloud data of Lidar1 and Lidar2 to obtain local features, aggregate the local features to obtain the global features of Lidar1 and Lidar2; use a convolutional neural network to perform feature matching on the global features of Lidar1 and Lidar2, and globally aggregate the results of feature matching to obtain the predicted pose (q pre ,t pre ).

[0046] (4) Determine the translation vector t of the predicted pose pre The initial translation vector t with respect to the reference pose re The deviation L t , and determine the predicted quaternion q of the predicted pose pre Quaternion q with reference pose re The deviation L q ;

[0047] (5) Through the deviation L t and the deviation L q Construct a loss function Loss, use the loss function Loss to train the convolutional neural network, apply the trained convolutional neural network to the feature matching of the global features of Lidar1 and Lidar2 again, get the final predicted pose, and calibrate the final predicted pose as the external parameter of the Lidar.

[0048] 2. Composition of IMU-assisted multi-lidar external parameter calibration system based on deep learning

[0049] 2.1 System Content

[0050] The initial measurement module is used to perform initial measurement on Lidar1 and Lidar2 to obtain the initial pose (q0, t0);

[0051] The point cloud acquisition module is used to obtain the point cloud data of Lidar1 and Lidar2, and to perform initial registration between the point cloud data and the initial pose (q0, t0) to obtain the reference pose (q re ,t re );

[0052] The processing module is used to extract the features of each point in the point cloud data of Lidar1 and Lidar2 to obtain local features, aggregate the local features to obtain the global features of Lidar1 and Lidar2; use the convolutional neural network to perform feature matching on the global features of Lidar1 and Lidar2, and globally aggregate the results after feature matching to obtain the predicted pose (q pre ,t pre );

[0053] Deviation calculation module, used to determine the translation vector t of the predicted pose pre The initial translation vector t with respect to the reference pose re The deviation L t , and determine the predicted quaternion q of the predicted pose pre Quaternion q with reference pose re The deviation L q ;

[0054] External parameter calibration module, used to pass the deviation L t and the deviation L q Construct a loss function Loss, use the loss function Loss to train the convolutional neural network, apply the trained convolutional neural network to the feature matching of the global features of Lidar1 and Lidar2 again, get the final predicted pose, and calibrate the final predicted pose as the external parameter of the Lidar.

[0055] 2.2 System Specific Composition

[0056] The composition of the IMU-assisted multi-lidar external parameter calibration system based on deep learning is as follows: Figure 2 As shown, the components and their main functions are as follows:

[0057] Carrier: Equipped with IMU, lidar, navigation computer, etc.

[0058] IMU (Inertial Measurement Unit): The accelerometer measures the acceleration on three axes, and the gyroscope measures the angular velocity around the three axes. The IMU's own position and posture are calculated through the data from these two sensors.

[0059] LiDAR: Obtain point cloud data of the environment, where each point has information such as three-dimensional coordinates and distance.

[0060] Navigation computer: A loss function is used to measure the deviation between the reference pose and the estimated pose, and then a deep learning neural network is used for continuous training to estimate and correct the system error in real time to improve the accuracy and stability of the external parameter calibration.

[0061] Airborne connection equipment: interfaces, serial ports, adapter boards, etc. used to connect the navigation computer with the IMU and lidar.

[0062] 3. IMU-assisted multi-lidar external parameter calibration solution algorithm based on deep learning

[0063] The design steps of the IMU-assisted multi-lidar extrinsic calibration algorithm based on deep learning are as follows:

[0064] 3.1 The data measured by the high-precision IMU is filtered by Kalman filtering to obtain the accurate initial pose (q0, t0).

[0065] 3.2 When performing ICP point cloud registration of two LiDAR data, the initial pose of IMU and the output of IMU are combined to reduce the registration area and improve the registration speed. The registration result is used as the reference pose (q re ,t re ). Among them, q re is the reference quaternion with rotation information, t re is a 1×3 reference translation vector.

[0066] 3.3 The point cloud data of Lidar1 and Lidar2 are subjected to feature extraction, feature matching, and global aggregation to obtain the predicted value (q pre ,t pre ). Among them, q pre is the predicted quaternion with rotation information, t pre The 1×3 is the predicted translation vector.

[0067] 3.4 Construct a loss function to measure the deviation L between the predicted value and the reference value t and L q , and then use the deep learning neural network to train it, namely:

[0068] L t =||t pre -t re || (4)

[0069]

[0070] Among them, L t represents the translation vector t pre With the initial translation vector t re Deviation of L q Represents the predicted quaternion q pre and the predicted quaternion q re The deviation of n ,b n ,c n ,d n} is the quaternion q pre Each component in .

[0071] Construct the loss function:

[0072]

[0073] Among them, λ1 and λ2 are hyperparameters and can be set to 0.6.

[0074] 3.5 Use the loss function to continuously train the network to minimize the prediction error, thereby accurately predicting the rotation and translation information and improving the stability and accuracy of the system external parameter calibration. Figure 3 As shown, the network framework of the IMU-assisted multi-lidar extrinsic parameter calibration method based on deep learning.

[0075] The above-mentioned embodiments only express several implementation methods of the present invention, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for ordinary technicians in this field, several variations and improvements can be made without departing from the concept of the present invention, and these all belong to the protection scope of the present invention. Therefore, the protection scope of the patent of the present invention shall be subject to the attached claims.

Claims

1. A method for calibrating laser radar external parameters, characterized in that: include: Perform initial measurement on Lidar1 and Lidar2 to obtain the initial pose (q0, t0); Obtain the point cloud data of Lidar1 and Lidar2, and perform initial registration between the point cloud data and the initial pose (q0, t0) to obtain the reference pose (q re ,t re ); Wherein, the initial registration includes: The ICP algorithm is used to align the two lidar point clouds. First, the reference pose is initialized; Iteratively optimize the spatial correspondence between point clouds to minimize the distance error between them until convergence, and obtain the reference pose (q re ,t re ); The features of each point in the point cloud data of Lidar1 and Lidar2 are extracted to obtain local features, and the local features are aggregated to obtain the global features of Lidar1 and Lidar2. The global features of Lidar1 and Lidar2 are matched using a convolutional neural network, and the results of feature matching are globally aggregated to obtain the predicted pose (q pre ,t pre ); Determine the translation vector t of the predicted pose pre The initial translation vector t with respect to the reference pose re The deviation L t , and determine the predicted quaternion q of the predicted pose pre Quaternion q with reference pose re The deviation L q ; Calculate the deviation L t The formula is: L t =||t pre -t re || Calculate the deviation L q The formula is: L q =D a (q re *(q pre ) -1 ), Among them, L t represents the translation vector t pre With the initial translation vector t re Deviation of L q Represents the predicted quaternion q pre Quaternion q with reference pose re The deviation of n ,b n ,c n ,d n } is the quaternion q pre The amount in By deviation L t and the deviation L q Construct the loss function Loss, which is obtained by using the deviation L t and the deviation L q The formula for constructing the loss function Loss is: Loss=λ1L t +λ2L q Among them, λ1 and λ2 are hyperparameters; The convolutional neural network is trained with the loss function Loss, and the trained convolutional neural network is re-applied to the feature matching of the global features of Lidar Lidar1 and Lidar2 to obtain the final predicted pose, which is calibrated as the external parameter of the Lidar.

2. A laser radar extrinsic parameter calibration method as claimed in claim 1, characterized in that: The initial measurements include: Initialize the system state measured by IMU and predict the state at the next moment; The state estimation is updated through the IMU measurement value, and the accurate posture transformation information is obtained through repeated iterations of Kalman filtering.

3. A laser radar extrinsic parameter calibration system based on the laser radar extrinsic parameter calibration method according to any one of claims 1 to 2, characterized in that: include: The initial measurement module is used to perform initial measurement on Lidar1 and Lidar2 to obtain the initial pose (q0, t0); The point cloud acquisition module is used to obtain the point cloud data of Lidar1 and Lidar2, and to perform initial registration between the point cloud data and the initial pose (q0, t0) to obtain the reference pose (q re ,t re ); The processing module is used to extract the features of each point in the point cloud data of Lidar1 and Lidar2 to obtain local features, aggregate the local features to obtain the global features of Lidar1 and Lidar2; use the convolutional neural network to perform feature matching on the global features of Lidar1 and Lidar2, and globally aggregate the results after feature matching to obtain the predicted pose (q pre ,t pre ); Deviation calculation module, used to determine the translation vector t of the predicted pose pre The initial translation vector t with respect to the reference pose re The deviation L t , and determine the predicted quaternion q of the predicted pose pre Quaternion q with reference pose re The deviation L q ; External parameter calibration module, used to pass the deviation L t and the deviation L q Construct a loss function Loss, use the loss function Loss to train the convolutional neural network, apply the trained convolutional neural network to the feature matching of the global features of Lidar1 and Lidar2 again, get the final predicted pose, and calibrate the final predicted pose as the external parameter of the Lidar.

Images