A rapid calibration method for multiple lidar extrinsic parameters
By combining multi-scale calibration and normal distribution transformation algorithm, the accuracy and efficiency problems of external parameter calibration of multi-LiDAR system under static conditions are solved, realizing efficient and accurate external parameter calibration, reducing initial error and hardware cost, and applicable to various LiDAR installation methods.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- BEIJING MECHANICAL EQUIP INST
- Filing Date
- 2024-06-03
- Publication Date
- 2026-06-30
AI Technical Summary
When calibrating external parameters of a multi-laser radar system under static conditions, the small overlap of the field of view limits the calibration accuracy and efficiency. Existing methods cannot effectively solve the problems of initial error and environmental dependence.
Initial extrinsic parameters are obtained through multi-scale calibration, and iterative fine-tuning is performed by combining yaw angle filtering and normal distribution transformation algorithms to reduce initial errors and improve calibration accuracy. SOR and VGD methods are used to process point cloud data to reduce data volume and improve computational efficiency.
It achieves rapid and accurate calibration of multiple lidar extrinsic parameters under static conditions, reduces hardware costs, improves calibration efficiency and accuracy, and has wide applicability and environmental independence.
Smart Images
Figure CN121069357B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of lidar sensor calibration technology, and in particular to a rapid calibration method for multiple lidar extrinsic parameters. Background Technology
[0002] The rapid development of autonomous driving technology has created a demand for high-precision environmental perception, and LiDAR (Light Detection and Ranging) is widely used in autonomous driving systems as a high-precision, high-resolution environmental perception sensor. However, the installation and use of multiple LiDARs introduces the problem of extrinsic parameter calibration, as they must be accurately positioned and calibrated to ensure that the system can accurately understand the environment and make correct decisions. Modern autonomous driving systems are typically equipped with multiple LiDARs to achieve all-around environmental perception capabilities. Multiple LiDARs are located at different positions on the vehicle, with different scanning angles and resolutions. By combining the point cloud data from multiple LiDARs, the vehicle can obtain a more comprehensive and accurate environmental map, thereby better understanding the surrounding roads and obstacles.
[0003] Multi-LiDAR systems are increasingly used in autonomous driving, robot navigation, and other fields. Ensuring accurate coordinate transformation parameters between multiple LiDARs is crucial for system performance. Extrinsic parameter calibration refers to the process of determining the relative position and attitude relationships between multiple sensors. For LiDAR, extrinsic parameters include translation and rotation. Correct extrinsic parameter calibration ensures that point cloud data acquired by different LiDARs are aligned in the same coordinate system, thereby enabling effective data fusion and processing. Inaccurate extrinsic parameter calibration will lead to data fusion errors, affecting the performance and safety of the autonomous driving system.
[0004] Extrinsic parameter calibration methods typically include target-based methods and optimization-based methods. Target-based methods utilize known calibration board information or feature points for matching, then calculate the relative position and attitude between LiDAR sensors. This method requires accurate target information and high-quality matching algorithms, but in practical applications, it is often limited by the calibration environment conditions, making accurate calibration impossible. Another common method is optimization-based methods, which solve for extrinsic parameters by minimizing the differences between point cloud data acquired by different LiDARs. This method is usually based on optimization algorithms, such as least squares or nonlinear optimization algorithms, and can obtain relatively accurate extrinsic parameter estimates within a small error range. Optimization-based methods have a higher initial error in extrinsic parameter calibration. Some deep learning-based methods have also been applied to LiDAR extrinsic parameter calibration, training neural networks to learn the relative relationships between sensors. These methods require large amounts of data and computational resources, which cannot meet the requirements in many scenarios.
[0005] LiDAR extrinsic parameter calibration faces some challenges. For example, the limited calibration environment conditions make it impossible to use calibration methods based solely on target feature information. At the same time, optimization-based methods have high requirements for the initial error of extrinsic parameter calibration. Summary of the Invention
[0006] Based on the above analysis, the embodiments of the present invention aim to provide a rapid calibration method for multiple lidar extrinsic parameters, in order to solve the technical problems of limited calibration environment and large initial error in existing methods for multiple lidar extrinsic parameters.
[0007] This invention provides a method for rapid calibration of multiple lidar extrinsic parameters, comprising the following steps:
[0008] Step S1: Based on the reference lidar and the lidar to be calibrated, acquire the reference point cloud A, the point cloud to be calibrated B, and the initial overlapping field of view. Based on the reference point cloud A and the point cloud to be calibrated B, perform multi-scale calibration within the initial overlapping field of view to obtain the initial extrinsic parameter p0. Use the yaw angle to extract the point clouds of the reference lidar and the lidar to be calibrated to determine the precise overlapping field of view of the reference lidar and the lidar to be calibrated.
[0009] Step S2: Preprocess the reference point cloud A and the point cloud B to be calibrated within the precise overlapping field of view to obtain the preprocessed reference point cloud C and the point cloud D to be calibrated.
[0010] Step S3: Divide the reference point cloud C into voxel space, iteratively fine-tune the initial extrinsic parameter p0, and use the iteratively fine-tuned extrinsic parameter p0. k Perform coordinate transformation on the point cloud D to be labeled until the extrinsic parameter p is satisfied. k The corresponding normal distribution transformation function converges, yielding the optimal extrinsic parameter p. 优 Where k = 1, 2, 3...;
[0011] Step S4: Use the optimal extrinsic parameter p 优 Perform coordinate transformation on the point cloud B to be calibrated. If the reference point cloud A and the point cloud B to be calibrated completely overlap within the precisely overlapping field of view, then the optimal extrinsic parameter p is... 优 To ensure accurate external parameters, calibration ends; otherwise, return to step S3 for recalibration.
[0012] Furthermore, based on the reference point cloud A and the point cloud to be calibrated B, multi-scale calibration is performed within the initial overlapping field of view to obtain the initial extrinsic parameter p0, including:
[0013] By visually comparing the reference point cloud A and the target point cloud B within the initial overlapping field of view, coordinate transformations are performed on the target point cloud B using translation and rotation scales until the reference point cloud A and the target point cloud B are aligned; wherein...
[0014] The coordinates of each point in the point cloud B to be labeled are transformed as follows:
[0015] x′ i =T(Δp) i ,x i )
[0016] Where, x i Let x′ be the coordinates of the i-th point in the point cloud B to be calibrated. i Let Δp be the coordinates of the i-th point in the point cloud B to be labeled after coordinate transformation. i T represents the change of the extrinsic parameters of the lidar coordinate system to be calibrated after each application of translation and rotation transformations; T is the comprehensive transformation matrix.
[0017]
[0018] Where R is the rotation transformation scale and t is the translation transformation scale;
[0019] Accumulate all Δp i The initial extrinsic parameter p0 of the laser radar to be calibrated relative to the reference laser radar is obtained as follows:
[0020]
[0021] Where n is the number of points in the point cloud B to be labeled.
[0022] Furthermore, the point clouds of the reference lidar and the lidar to be calibrated are extracted using the yaw angle to determine the precise overlapping field of view of the reference lidar and the lidar to be calibrated, including the following steps:
[0023] Step 1: Set the maximum and minimum yaw angles. min yaw max and stride parameter α;
[0024] Step 2: Based on the yaw angle range between the maximum and minimum values of the set yaw angle, remove point cloud data that exceeds the yaw angle range from the reference point cloud A and the point cloud to be calibrated B, and only retain the point cloud data within the yaw angle range.
[0025] Step 3: Gradually increase yaw using the stride parameter α. min Reduce yaw max The adjusted yaw angle range is obtained. min1 yaw max1 Based on the adjusted yaw angle range, the point cloud data removal process in step two is repeated until there is no excess point cloud data outside the yaw angle range, thus obtaining the accurate overlapping field of view {yaw|yaw∈[yaw]}. min1 yaw max1 ]}.
[0026] Further, step S2 includes:
[0027] For the point cloud B to be labeled within the precise overlapping field of view, noise removal is performed using the SOR method, and downsampling is performed using the VGD method to obtain the point cloud E to be labeled after noise removal and downsampling.
[0028] Based on the initial extrinsic parameter p0, the coordinate transformation of the point cloud E to be labeled is performed as follows:
[0029]
[0030] Where, x i Let be the original coordinates of the i-th point in the point cloud E to be calibrated. The coordinates of the point cloud E to be labeled after coordinate transformation;
[0031] The reference point cloud A and the point cloud E to be calibrated within the precisely overlapping field of view are filtered to obtain the preprocessed reference point cloud C and the point cloud D to be calibrated, as shown below:
[0032]
[0033] Where, x Base,i x Calib,i These are the reference point cloud C and the point cloud D to be labeled, respectively, within the filtered, precisely overlapping field of view.
[0034] Further, step S3 includes:
[0035] Preset voxel resolution β and iteration threshold λ;
[0036] Based on the preset voxel resolution β, the reference point cloud C is divided into voxel space, and the mean and covariance matrix of the multidimensional normal distribution parameters of the reference point cloud C in each voxel are calculated.
[0037] Based on the mean and covariance matrix of the multidimensional normal distribution parameters of the reference point cloud C in each voxel, calculate the probability density of each point in the point cloud D to be labeled in the corresponding voxel, and the normal distribution transformation function under the initial extrinsic parameter p0.
[0038] The initial extrinsic parameter p0 is finely adjusted iteratively using Newton's optimization algorithm, and the extrinsic parameter p is calculated sequentially. k-1 The Jacobian vector g of the normal distribution transformation function k-1 H, the Hessian matrix k-1 and external parameter fine-tuning amount Δp k ;
[0039] Based on the initial external parameter p0 and the external parameter fine-tuning amount Δp k The extrinsic parameter p after iterative fine-tuning is obtained.k Where k≥1; calculate the extrinsic parameter p. k The normal distribution transformation function is applied until the normal distribution transformation function converges to the iteration threshold λ, thus obtaining the optimal extrinsic parameter p.
[0040] Furthermore, using the optimal extrinsic parameter p 优 The coordinate transformation of the point cloud B to be calibrated is performed. If the reference point cloud A and the point cloud B to be calibrated do not completely overlap within the precise overlapping field of view, the voxel resolution parameter β and the iteration threshold λ are modified, and step S3 is executed to recalibrate until the normal distribution transformation function converges to the iteration threshold λ to obtain the precise extrinsic parameters.
[0041] Furthermore, the calculation of the mean and covariance matrix of the multidimensional normal distribution parameters of the reference point cloud C within each voxel is as follows:
[0042]
[0043] in, Let n be the mean distribution of the reference point cloud C within voxel j. j Let C be the number of points in the reference point cloud C within voxel j.
[0044] Furthermore, based on the mean and covariance matrix of the multidimensional normal distribution parameters of the reference point cloud C within each voxel, the probability density of each point in the target point cloud D within the corresponding voxel is calculated, as shown below:
[0045]
[0046] Where, x Calib,i Let i be the i-th point cloud to be punctuated.
[0047] Furthermore, the normal distribution transformation function of each point in the point cloud D to be labeled under the initial extrinsic parameter p0 is calculated as follows:
[0048]
[0049] Where s is the normal distribution transformation score function, and i is the i-th voxel.
[0050] Furthermore, the initial extrinsic parameter p0 is iteratively fine-tuned using the Newton optimization algorithm, and the extrinsic parameter p is calculated sequentially. k-1 The Jacobian vector g of the normal distribution transformation function k-1 H, the Hessian matrix k-1 and external parameter fine-tuning amount Δp k ,include:
[0051] Calculate the extrinsic parameter p k-1 The Jacobian vector g of the normal distribution transformation functionk-1 Its vector element gi is calculated as follows:
[0052]
[0053] Where, p i external parameter p k-1 The i-th element of the vector;
[0054] The calculation is based on the extrinsic parameter p. k-1 The Hessian matrix H of the normal distribution transformation function k-1 Its matrix elements are as follows:
[0055]
[0056] Where, p j external parameter p k-1 The j-th element of the vector;
[0057] Based on the Jacobian vector g k-1 and the Hessian matrix H k-1 Calculate the external parameter fine-tuning amount Δp k As shown below:
[0058] Δp k =-H k-1 -1 g k-1 .
[0059] Compared with the prior art, the present invention can achieve at least one of the following beneficial effects:
[0060] 1. This method can significantly improve the environmental independence of multi-LiDAR calibration and enhance the efficiency of extrinsic parameter calibration. Mapping-based multi-LiDAR extrinsic parameter calibration methods require the carrier of the multi-LiDAR to undergo complex kinematic movements to ensure sufficient point cloud data for mapping. However, the mapping process typically consumes significant time and resources and places high demands on the system's hardware configuration. In contrast, the static calibration method proposed in this invention does not require carrier movement; calibration is performed only in a static state, thus improving environmental independence and significantly reducing the time required for extrinsic parameter calibration, thereby increasing calibration efficiency.
[0061] 2. Reduced initial error: Initial extrinsic parameters are obtained through multi-scale calibration, and then iteratively fine-tuned using a normal distribution transformation, effectively reducing initial error. Compared to traditional methods, this reduces initial extrinsic parameter error and improves the accuracy of extrinsic parameter calibration.
[0062] 3. This method can reduce the cost of external parameter calibration and improve its accuracy. Since the static calibration method does not require complex carrier motion, it reduces the need for motion control equipment and accessories, thereby lowering the hardware cost of external parameter calibration. Furthermore, it reduces potential mechanical vibrations and loosening during motion, helping to reduce errors in the calibration process. Fine-tuning the external parameters using the Newton optimization algorithm, combined with the calculation of the Hessian matrix, makes the calibration results more accurate and reliable, thus improving the precision and reliability of the calibration.
[0063] 4. This extrinsic parameter calibration method is universal and applicable. Because this calibration method does not rely on a large overlapping field of view, it can be applied to more lidar installation methods, making it more widely applicable and valuable for promotion. It can be used in automation systems across various fields.
[0064] In summary, the rapid calibration method for multi-lidar extrinsic parameters with small overlapping field of view under static conditions proposed in this invention has significant benefits, including independence from the calibration environment, improved calibration efficiency, reduced calibration costs, improved system accuracy and stability, and advantages in versatility and applicability. This will provide important technical support and assurance for the application and promotion of multi-lidar systems.
[0065] In this invention, the above-described technical solutions can be combined with each other to achieve more preferred combinations. Other features and advantages of this invention will be set forth in the following description, and some advantages may become apparent from the description or be learned by practicing the invention. The objects and other advantages of this invention can be realized and obtained from what is particularly pointed out in the description and drawings. Attached Figure Description
[0066] The accompanying drawings are for illustrative purposes only and are not intended to limit the invention. Throughout the drawings, the same reference numerals denote the same parts.
[0067] Figure 1 This is a flowchart of a method for rapid calibration of multiple lidar extrinsic parameters under static conditions, as described in an embodiment of the present invention.
[0068] Figure 2 This is a schematic diagram illustrating the calibration and verification effect of multiple lidar systems. Detailed Implementation
[0069] Preferred embodiments of the present invention will now be described in detail with reference to the accompanying drawings, which form part of this application and are used together with the embodiments of the present invention to illustrate the principles of the present invention, but are not intended to limit the scope of the present invention.
[0070] To address the challenges in existing technologies, this invention proposes a method combining multi-scale and automatic calibration. This method extracts the initial overlapping field of view of both the laser radar and the reference laser radar, and calibrates their initial extrinsic parameters using multi-scale calibration. This allows for the acquisition of a precise overlapping field of view. Laser point cloud data is preprocessed to remove invalid point cloud data. Then, the calibration extrinsic parameters are accurately calculated using a normal distribution transformation algorithm based on Normal Distributions Transform (NDT, used for point cloud registration and extrinsic parameter calibration). These steps combine multi-scale calibration and automated algorithms for rapid and accurate extrinsic parameter calibration.
[0071] The technical problem this invention aims to solve is how to quickly and accurately determine the extrinsic parameters of multiple lidar systems when the carrier carrying multiple lidars is stationary, given the small overlap of their fields of view. While static calibration typically reduces complexity and cost, traditional methods can be limited in accuracy and efficiency when the overlap is small. Therefore, this invention proposes an innovative method that enables efficient and accurate rapid calibration of multiple lidar extrinsic parameters under static conditions. This method overcomes the problem of small overlap in the lidar fields of view, ensuring accurate estimation of the relative positions and attitude relationships between multiple lidars even under static conditions.
[0072] A specific embodiment of the present invention discloses a method for rapid calibration of multiple lidar extrinsic parameters, such as... Figure 1 As shown, it includes the following steps:
[0073] Step S1: Based on the reference lidar and the lidar to be calibrated, acquire the reference point cloud A, the point cloud to be calibrated B, and the initial overlapping field of view. Based on the reference point cloud A and the point cloud to be calibrated B, perform multi-scale calibration within the initial overlapping field of view to obtain the initial extrinsic parameter p0. Use the yaw angle to extract the point clouds of the reference lidar and the lidar to be calibrated to determine the precise overlapping field of view of the reference lidar and the lidar to be calibrated.
[0074] Step S2: Preprocess the reference point cloud A and the point cloud B to be calibrated within the precise overlapping field of view to obtain the preprocessed reference point cloud C and the point cloud D to be calibrated.
[0075] Step S3: Divide the reference point cloud C into voxel space, iteratively fine-tune the initial extrinsic parameter p0, and use the iteratively fine-tuned extrinsic parameter p0. k Perform coordinate transformation on the point cloud D to be labeled until the extrinsic parameter p is satisfied. k The corresponding normal distribution transformation function converges, yielding the optimal extrinsic parameter p. 优 Where k = 1, 2, 3...;
[0076] Step S4: Use the optimal extrinsic parameter p 优 Perform coordinate transformation on the point cloud B to be calibrated. If the reference point cloud A and the point cloud B to be calibrated completely overlap within the precisely overlapping field of view, then the optimal extrinsic parameter p is... 优 To ensure accurate external parameters, calibration ends; otherwise, return to step S3 for recalibration.
[0077] This method, under static conditions, utilizes point cloud data obtained from a reference lidar and a lidar to be calibrated. The carrier carrying multiple lidars does not need to move and is independent of the environment in which the carrier is located. The use of multi-scale calibration to obtain initial extrinsic parameters reduces initial errors. By fine-tuning the extrinsic parameters, the calibration results of multiple lidars can be made more accurate and reliable, saving time and improving efficiency.
[0078] Step S1 includes steps S11-S13.
[0079] Step S11: Obtain the reference point cloud A, the point cloud B to be calibrated, and the initial overlapping field of view based on the reference lidar and the lidar to be calibrated.
[0080] For example, taking the calibration of two lidars as an example, one is used as the reference lidar and the other is the lidar to be calibrated.
[0081] For the selection of the reference lidar, preferably, a lidar with a stable installation position that is not easily affected by vehicle vibration, a wide field of view, consistent with the vehicle's direction of travel, and a high degree of alignment with the vehicle's coordinate system should be selected as the reference lidar; if a lidar has been calibrated before and there is reliable historical calibration data, that lidar can also be selected as the reference lidar.
[0082] The extrinsic parameters to be solved are the parameters of the reference coordinate system of the lidar to be calibrated relative to the reference lidar.
[0083] The external parameter calibration of multiple lidars is consistent with this principle; the external parameter calibration of the lidars to be calibrated can be performed one by one or simultaneously using the method in this invention.
[0084] The original point cloud acquired by the reference lidar is denoted as reference point cloud A;
[0085] The original point cloud acquired by the lidar to be calibrated is denoted as point cloud B.
[0086] The overlapping field of view of the reference lidar and the lidar to be calibrated is used as a rough initial overlapping field of view. The subsequent step S12 obtains the initial extrinsic parameters based on the initial overlapping field of view, and the precise overlapping field of view is determined by the yaw angle in step S13.
[0087] Step S12: Based on the reference point cloud A and the point cloud to be calibrated B, perform multi-scale calibration within the initial overlapping field of view to obtain the initial extrinsic parameter p0.
[0088] Based on the reference point cloud A and the point cloud to be calibrated B, multi-scale calibration is performed within the overlapping field of view to obtain the initial extrinsic parameter p0, including:
[0089] By visually comparing the reference point cloud A and the target point cloud B within the initial overlapping field of view, coordinate transformations are performed on the target point cloud B using translation and rotation scales until the reference point cloud A and the target point cloud B are aligned; wherein...
[0090] The coordinates of each point in the point cloud B to be labeled are transformed as shown in formula (1):
[0091] x′ i =T(Δp) i ,x i ) Formula (1)
[0092] Where, x i Let x′ be the coordinates of the i-th point in the point cloud B to be calibrated. i Let Δp be the coordinates of the i-th point in the point cloud B to be labeled after coordinate transformation. i T represents the change of the extrinsic parameters of the lidar coordinate system to be calibrated after each application of translation and rotation transformations; T is the comprehensive transformation matrix.
[0093]
[0094] Where R is the rotation transformation scale and t is the translation transformation scale;
[0095] Accumulate all Δp i The initial extrinsic parameter p0 of the laser radar to be calibrated relative to the reference laser radar is obtained as shown in formula (3):
[0096]
[0097] Where n is the number of points in the point cloud B to be labeled.
[0098] In the multi-scale calibration process, the calibration point cloud B of the lidar to be calibrated is compared with the reference point cloud A of the reference lidar through visualization.
[0099] Different translation and rotation transformation scales are applied to perform coordinate transformation on the point cloud B to be marked. Coordinate transformation is a basic operation. For example, the transformation scales are shown in Table 1.
[0100] Table 1: Multiscale calibration transformation values
[0101] Translation scale 0.01m 0.1m 1m 5m Rotational transformation scale 0.1° 1° 10° 50°
[0102] For the coordinates of each point in the point cloud B to be punctured, perform a coordinate transformation x′. i =T(Δp) i ,x i ), where (Δp i ,x i Each multiplication with T represents one scale adjustment.
[0103] Continuously apply translation and rotation transformations to the coordinate transformation of the point cloud B to be calibrated, and continue this process until the two point clouds (reference point cloud A and point cloud B to be calibrated) are aligned within the initial overlapping field of view, accumulating all Δp values. i The initial extrinsic parameters p0 of the laser radar to be calibrated relative to the reference laser radar are obtained.
[0104] Thus, the initial extrinsic parameters p0 of the lidar to be calibrated relative to the reference lidar are obtained.
[0105] By rapidly obtaining the initial extrinsic parameter p0 through multi-scale calibration, we can lay the foundation for further preprocessing operations such as noise removal and data volume reduction of point cloud data, as well as for obtaining accurate extrinsic parameters by applying the normal distribution transformation algorithm. This will shorten the calibration time, improve the calibration efficiency, ensure the calibration accuracy, and guarantee the reliability of the calibration results.
[0106] Step S13: Extracting the point clouds of the reference lidar and the lidar to be calibrated using the yaw angle to determine the precise overlapping field of view of the reference lidar and the lidar to be calibrated, including the following steps:
[0107] Step 1: Set the maximum and minimum yaw angles. min yaw max and stride parameter α;
[0108] Step 2: Based on the yaw angle range between the maximum and minimum values of the set yaw angle, remove point cloud data that exceeds the yaw angle range from the reference point cloud A and the point cloud to be calibrated B, and only retain the point cloud data within the yaw angle range.
[0109] Step 3: Gradually increase yaw using the stride parameter α. min Reduce yaw max The adjusted yaw angle range is obtained. min1 yaw max1 Based on the adjusted yaw angle range, the point cloud data removal process in step two is repeated until there is no excess point cloud data outside the yaw angle range, thus obtaining the accurate overlapping field of view {yaw|yaw∈[yaw]}. min1 yaw max1 ]}.
[0110] By continuously adjusting the yaw angle, the range of the initial overlapping field of view is gradually reduced, thus obtaining a scarce overlapping field of view.
[0111] To reduce the impact of invalid point clouds outside the overlapping field of view on the calibration process, this invention uses yaw angle to denoise two types of point clouds (reference point cloud A and point cloud B to be calibrated), that is, by setting the maximum and minimum yaw angle. min and yaw max The point cloud data is filtered to extract the point cloud data within the precise overlapping field of view.
[0112] Initialize and set the maximum and minimum yaw angles. min yaw max And the stride parameter α, which is used to adjust the maximum and minimum values of the yaw angle; the maximum and minimum values of the yaw angle define the approximate angular range of the expected initial overlapping field of view.
[0113] Adjust the maximum and minimum yaw angles: increase yaw by α steps. min Reduce yaw max The adjusted yaw angle range is obtained. min1 yaw max1 To gradually reduce the yaw angle range, remove point cloud data that exceeds the yaw angle range until there is no excess point cloud data outside the overlapping field of view.
[0114] After each adjustment of the yaw angle range, observe the point cloud visualization results and check whether there is still point cloud data outside the initial overlapping field of view, so as to further adjust the yaw angle until there is no extra point cloud data outside the initial overlapping field of view.
[0115] At this point, we obtain the exact overlapping field of view, i.e., {yaw|yaw∈[yaw]} min1 yaw max1 The precise overlapping field of view serves as the basis for the subsequent S3 normal distribution transformation method, and is used for further precise extrinsic parameter calibration.
[0116] This step determines the yaw. min and yaw max The specific values are obtained to obtain the accurate overlapping field of view. Invalid point cloud data outside the accurate overlapping field of view is effectively removed, providing more accurate input point cloud data for the normal distribution transformation method in the subsequent step S3, thereby improving the accuracy and efficiency of extrinsic parameter calibration.
[0117] The drawback of the normal distribution transformation method in step S3 is its poor robustness to initial errors. Multi-scale calibration calculation of initial extrinsic parameters can reduce initial errors, reduce the calculation error of normal distribution transformation, and increase the accuracy of extrinsic parameter calibration. Filtering the precise overlapping field of view can reduce the amount of calibration point cloud data, eliminate point cloud data of non-precise overlapping fields of view, further speed up the calculation, and reduce the time required for extrinsic parameter calibration.
[0118] Step S2, specifically.
[0119] For the point cloud B to be labeled within the precise overlapping field of view, noise removal is performed using the SOR method, and downsampling is performed using the VGD method to obtain the point cloud E to be labeled after noise removal and downsampling.
[0120] Based on the initial extrinsic parameter p0, the coordinate transformation of the point cloud E to be labeled is performed, as shown in formula (4):
[0121]
[0122] Where, x i Let be the original coordinates of the i-th point in the point cloud E to be calibrated. The coordinates of the point cloud E to be labeled after coordinate transformation;
[0123] The reference point cloud A and the point cloud E to be calibrated within the precise overlapping field of view are filtered to obtain the preprocessed reference point cloud C and the point cloud D to be calibrated, as shown in formula (5):
[0124]
[0125] Where, x Base,i x Calib,i These are the reference point cloud C and the point cloud D to be labeled, respectively, within the filtered, precisely overlapping field of view.
[0126] First, noise removal and downsampling are performed on the point cloud B of the LiDAR to be calibrated to improve the quality of the point cloud data and reduce the order of magnitude of the point cloud, thereby improving the speed of subsequent external parameter calibration.
[0127] In the noise removal process, the Statistical Outlier Removal (SOR) method is selected. This method calculates the distance difference between each point in the point cloud B and its neighboring points to remove outliers (including outliers and noise points), effectively eliminating local noise. The SOR method is used to remove outliers or noise points from point cloud data. This method is based on statistical analysis, identifying and removing outliers by calculating the distance distribution from each point to its neighbors.
[0128] In the downsampling stage, the VGD (Voxel Grid Downsampling) method is selected to divide the point cloud B to be labeled into cubic voxels. Only one point is retained in each voxel as a representative, which quickly reduces the point cloud density, reduces the order of magnitude of the point cloud, and improves the efficiency of subsequent calculations.
[0129] After noise removal and downsampling of the point cloud B to be punctuated, the point cloud E to be punctuated is obtained.
[0130] The reference point cloud A and the coordinate-transformed point cloud E within the precise overlapping field of view are filtered to obtain the preprocessed reference point cloud C and the point cloud D to be calibrated. This further reduces the amount of point cloud data for extrinsic parameter calibration within the precise overlapping field of view, eliminates unnecessary point cloud data, and further improves the efficiency of extrinsic parameter calibration calculation.
[0131] Step S3 includes:
[0132] Preset voxel resolution β and iteration threshold λ;
[0133] Based on the preset voxel resolution β, the reference point cloud C is divided into voxel space, and the mean and covariance matrix of the multidimensional normal distribution parameters of the reference point cloud C in each voxel are calculated.
[0134] Based on the mean and covariance matrix of the multidimensional normal distribution parameters of the reference point cloud C in each voxel, calculate the probability density of each point in the point cloud D to be labeled in the corresponding voxel, and the normal distribution transformation function under the initial extrinsic parameter p0.
[0135] The initial extrinsic parameter p0 is finely adjusted iteratively using Newton's optimization algorithm, and the extrinsic parameter p is calculated sequentially. k-1 The Jacobian vector g of the normal distribution transformation function k-1 H, the Hessian matrix k-1 and external parameter fine-tuning amount Δp k ;
[0136] Based on the initial external parameter p0 and the external parameter fine-tuning amount Δp k The extrinsic parameter p after iterative fine-tuning is obtained. k Where k≥1; calculate the extrinsic parameter p. k The normal distribution transformation function is applied until it converges to the iteration threshold λ, thus obtaining the optimal extrinsic parameter p. 优 .
[0137] Based on the size of the preset voxel resolution β, the voxel space of the reference point cloud C is divided into a set of cubes.
[0138] The calculation of the mean and covariance matrix of the multidimensional normal distribution parameters of the reference point cloud C within each voxel is shown in Equation (6):
[0139]
[0140] in, Let n be the mean distribution of the reference point cloud C within voxel j. j Let C be the number of points in the reference point cloud C within voxel j.
[0141] T is the transpose symbol.
[0142] Based on the mean and covariance matrix of the multidimensional normal distribution parameters of the reference point cloud C within each voxel, the probability density of each point in the target point cloud D within the corresponding voxel is calculated, as shown in formula (7):
[0143]
[0144] Where, x Calib,i Let i be the i-th point cloud to be punctuated.
[0145] Based on the mean and covariance matrix of the multidimensional normal distribution parameters of the reference point cloud C within each voxel, the probability density of each point in the target point cloud D within the corresponding voxel is calculated, as shown in formula (8):
[0146]
[0147] Where, x Calib,i Let i be the i-th point cloud to be punctuated.
[0148] Calculate the normal distribution transformation function of each point in the point cloud D to be labeled under the initial extrinsic parameter p0, as shown in formula (9):
[0149]
[0150] Where s is the normal distribution transformation score function, and i is the i-th voxel.
[0151] Calculate the normal distribution transformation score under the current extrinsic parameter p0, which is to sum the probability densities calculated for all voxels.
[0152] The initial extrinsic parameter p0 is iteratively fine-tuned using Newton's optimization algorithm, and the extrinsic parameter p is calculated sequentially. k-1 The Jacobian vector g of the normal distribution transformation function k-1 H, the Hessian matrix k-1 and external parameter fine-tuning amount Δp k ,include:
[0153] Calculate the extrinsic parameter p k-1 The Jacobian vector g of the normal distribution transformation function k-1 Its vector element g i The calculation is shown in formula (10):
[0154]
[0155] Where, p i external parameter p k-1 The i-th element of the vector;
[0156] The calculation is based on the extrinsic parameter p. k-1 The Hessian matrix H of the normal distribution transformation function k-1 Its matrix elements are shown in formula (11):
[0157]
[0158] Where, p j external parameter p k-1 The j-th element of the vector;
[0159] Based on the Jacobian vector g k-1 and the Hessian matrix H k-1 Calculate the external parameter fine-tuning amount Δp k As shown in formula (12):
[0160] Δp k =-H k-1 -1 g k-1 Formula (12)
[0161] Based on the initial external parameter p0 and the external parameter fine-tuning amount Δp k The extrinsic parameter p after iterative fine-tuning is obtained. k The calculation is shown in formula (13):
[0162] p k =p k-1 +Δp k Formula (13)
[0163] Where k = 1, 2, 3... represents the number of iterations.
[0164] Based on the initial extrinsic parameter p0, the Jacobian vector g0 and the Hessian matrix H0 are solved. Based on the Jacobian vector g0 and the Hessian matrix H0, Δp1 is solved. Based on Δp1, p... k =p k-1 +Δp k Solve for the extrinsic parameter p1; calculate the normal distribution transformation function of p1, and determine whether the normal distribution transformation function converges to within the iteration threshold λ. If it converges, p1 is the optimal extrinsic parameter p. 优 Otherwise, continue solving for the extrinsic parameter p2.
[0165] Based on the extrinsic parameter p1, the Jacobian vector g1 and the Hessian matrix H1 are solved. Based on the Jacobian vector g1 and the Hessian matrix H1, Δp2 is solved. Based on Δp2, p... k =p k-1 +Δp k Solve for the extrinsic parameter p2; calculate the normal distribution transformation function of p2, and determine whether the normal distribution transformation function converges to within the iteration threshold λ. If it converges, p2 is the optimal extrinsic parameter p. 优 Otherwise, continue solving for the extrinsic parameter p3.
[0166] The same principle applies to subsequent cases.
[0167] Iteratively fine-tuned extrinsic parameter p k The normal distribution transformation function is shown in Equation (14):
[0168]
[0169] This invention uses the Newton-Raphson optimization algorithm to optimize the normal distribution transformation function, specifically by calculating the Newton search direction to update the extrinsic parameters. The Newton search direction is calculated by taking the Jacobian vector g of the normal distribution transformation function *s* with respect to the extrinsic parameters. The Newton-Raphson optimization algorithm uses the first-order partial derivative of the normal distribution transformation function with respect to the variables (extrinsic parameters) to estimate the gradient with respect to the variables, i.e., the Newton search direction. The Jacobian vector is the first-order partial derivative used to optimize the probability density.
[0170] Calculate the Hessian matrix H of the normal distribution transformation function s. The Hessian matrix is the second-order partial derivative, and H is a two-dimensional matrix with matrix elements H. ij It is obtained by taking the second-order partial derivatives of the i-th and j-th elements of the external parameters through the normal distribution transformation function s.
[0171] Based on the result of calculating the Hessian matrix, the extrinsic parameters are updated iteratively until the normal distribution transformation function s converges to within the iteration threshold λ.
[0172] Step S3 involves iteratively fine-tuning the initial extrinsic parameter p0 to obtain the optimal extrinsic parameter p for lidar calibration. 优 .
[0173] Step S4: Use the optimal extrinsic parameter p 优 Perform coordinate transformation on the point cloud B to be calibrated. If the reference point cloud A and the point cloud B to be calibrated completely overlap within the precisely overlapping field of view, then the optimal extrinsic parameter p is... 优 To ensure accurate external parameters, calibration ends; otherwise, return to step S3 for recalibration.
[0174] Using the optimal extrinsic parameter p 优The coordinate transformation of the point cloud B to be calibrated is performed. If the reference point cloud A and the point cloud B to be calibrated do not completely overlap within the precise overlapping field of view, the voxel resolution parameter β and the iteration threshold λ are modified, and step S3 is executed to recalibrate until the normal distribution transformation function converges to the iteration threshold λ to obtain the precise extrinsic parameters.
[0175] Decrease the values of the voxel resolution parameter β and the iteration threshold λ, and execute step S3 to perform recalibration.
[0176] The verification of the rapid calibration method for multiple lidar extrinsic parameters in this invention is as follows:
[0177] To verify the effectiveness of the rapid calibration method proposed in this invention for calibrating the external parameters of lidar, the following experiments were conducted.
[0178] The experiment calculated the extrinsic parameters of the radar to be calibrated relative to the reference radar. The point cloud data of the radar to be calibrated was then transformed using the corresponding precise extrinsic parameters. Finally, all point cloud data (including the reference point cloud and the point cloud to be calibrated) were visualized and analyzed together (i.e., observed from different angles and positions in three-dimensional space to determine whether the reference point cloud and the point cloud to be calibrated overlap within the precisely overlapping field of view). The extrinsic parameter calibration process for the radar to be calibrated took less than 5 minutes, and the calibration results were as follows: Figure 2 As shown, by observing the visualization results, it can be found that the point cloud in the precisely overlapping field of view exhibits a good alignment effect, which verifies the effectiveness and reliability of the fast extrinsic parameter calibration method proposed in this invention.
[0179] The proposed rapid calibration method for multiple lidar extrinsic parameters combines multi-scale calibration and automated calculation to address the challenges of limited calibration environment, large initial errors, complexity, and time consumption in the calibration process for multiple lidar systems.
[0180] This method first uses multi-scale calibration to obtain initial extrinsic parameters, and then preprocesses and accurately calculates the laser point cloud data by extracting the initial overlapping field of view and using a normal distribution transformation algorithm, ultimately achieving rapid extrinsic parameter calibration for multiple lidars.
[0181] This invention achieves efficient and accurate extrinsic parameter calibration under static conditions without requiring complex motion of the lidar carrier. By combining multi-scale calibration with automated calculations, calibration time is effectively shortened, calibration efficiency is improved, and the accuracy and stability of the extrinsic parameter calibration results are guaranteed. Experimental results show that point cloud data (including the reference point cloud and the point cloud to be calibrated) within a precisely overlapping field of view exhibits good alignment, verifying the reliability and practicality of this rapid extrinsic parameter calibration method. In summary, this rapid extrinsic parameter calibration method for multi-lidar systems provides an efficient and reliable solution for extrinsic parameter calibration. It provides important technical support and guidance for applications such as autonomous driving and environmental perception, as well as the future development of lidar technology.
[0182] In summary, the rapid calibration method for multiple lidar extrinsic parameters according to embodiments of the present invention has the following beneficial effects:
[0183] 1. This method can significantly improve the environmental independence of multi-LiDAR calibration and enhance the efficiency of extrinsic parameter calibration. Mapping-based multi-LiDAR extrinsic parameter calibration methods require the carrier of the multi-LiDAR to undergo complex kinematic movements to ensure sufficient point cloud data for mapping. However, the mapping process typically consumes significant time and resources and places high demands on the system's hardware configuration. In contrast, the static calibration method proposed in this invention does not require carrier movement; calibration is performed only in a static state, thus improving environmental independence and significantly reducing the time required for extrinsic parameter calibration, thereby increasing calibration efficiency.
[0184] 2. Reduced initial error: Initial extrinsic parameters are obtained through multi-scale calibration, and then iteratively fine-tuned using normal distribution transformation, effectively reducing initial error. Compared to traditional methods, this reduces initial extrinsic parameter error and improves the accuracy of extrinsic parameter calibration.
[0185] 3. This method can reduce the cost of external parameter calibration and improve its accuracy. Since the static calibration method does not require complex carrier motion, it reduces the need for motion control equipment and accessories, thereby lowering the hardware cost of external parameter calibration. Furthermore, it reduces potential mechanical vibrations and loosening during motion, helping to reduce errors in the calibration process. Fine-tuning the external parameters using the Newton optimization algorithm, combined with the calculation of the Hessian matrix, makes the calibration results more accurate and reliable, thus improving the precision and reliability of the calibration.
[0186] 4. This extrinsic parameter calibration method is universal and applicable. Because this calibration method does not rely on a large overlapping field of view, it can be applied to more lidar installation methods, making it more widely applicable and valuable for promotion. It can be used in automation systems across various fields.
[0187] In summary, the rapid calibration method for multi-lidar extrinsic parameters with small overlapping field of view under static conditions proposed in this invention has significant benefits, including independence from the calibration environment, improved calibration efficiency, reduced calibration costs, improved system accuracy and stability, and advantages in versatility and applicability. This will provide important technical support and assurance for the application and promotion of multi-lidar systems.
[0188] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the scope of protection of the present invention.
Claims
1. A method for rapid calibration of multiple lidar extrinsic parameters, characterized in that, Includes the following steps: Step S1: Based on the reference lidar and the lidar to be calibrated, acquire the reference point cloud A, the point cloud to be calibrated B, and the initial overlapping field of view. Based on the reference point cloud A and the point cloud to be calibrated B, perform multi-scale calibration within the initial overlapping field of view to obtain the initial extrinsic parameters. ; The point clouds of the reference lidar and the lidar to be calibrated are extracted using the yaw angle to determine the precise overlapping field of view of the reference lidar and the lidar to be calibrated; Step S2: Preprocess the reference point cloud A and the point cloud B to be calibrated within the precise overlapping field of view to obtain the preprocessed reference point cloud C and the point cloud D to be calibrated. Step S3: Divide the reference point cloud C into voxel space and iteratively fine-tune the initial extrinsic parameters. Using the extrinsic parameters after iterative fine-tuning Perform coordinate transformation on the point cloud D to be labeled until the extrinsic parameters are equal. The corresponding normal distribution transformation function converges, yielding the optimal extrinsic parameters. ;in, ; Step S3 includes: Preset voxel resolution and iteration threshold ; Based on the preset voxel resolution The reference point cloud C is divided into a voxel space, and the mean and covariance matrix of the multidimensional normal distribution parameters of the reference point cloud C within each voxel are calculated. Based on the mean and covariance matrix of the multidimensional normal distribution parameters of the reference point cloud C within each voxel, calculate the probability density of each point in the point cloud D to be labeled within the corresponding voxel, and the probability density of the point in the initial extrinsic parameters. The normal distribution transformation function under the given conditions; The initial extrinsic parameters were iteratively fine-tuned using Newton's optimization algorithm. Calculate the external parameters sequentially The Jacobian vector of the normal distribution transformation function Hesse matrix and external parameter fine-tuning amount ; Based on the initial external parameters External parameter fine-tuning amount Obtain the extrinsic parameters after iterative fine-tuning ,in, ; Calculate the extrinsic parameters The normal distribution transformation function is applied until the normal distribution transformation function converges to the iteration threshold. Internally, the optimal external parameters are obtained. ; Step S4: Use the optimal extrinsic parameters Perform coordinate transformation on the point cloud B to be calibrated. If the reference point cloud A and the point cloud B to be calibrated completely overlap within the precisely overlapping field of view, then the optimal extrinsic parameters are... To ensure accurate external parameters, calibration ends; otherwise, return to step S3 for recalibration.
2. The method according to claim 1, characterized in that, Based on the reference point cloud A and the point cloud to be calibrated B, multi-scale calibration is performed within the initial overlapping field of view to obtain the initial extrinsic parameters. ,include: By visually comparing the reference point cloud A and the target point cloud B within the initial overlapping field of view, coordinate transformations are performed on the target point cloud B using translation and rotation scales until the reference point cloud A and the target point cloud B are aligned; wherein... The coordinates of each point in the point cloud B to be labeled are transformed as follows: in, Let be the coordinates of the i-th point in the point cloud B to be calibrated. These are the coordinates of the i-th point in the point cloud B to be labeled after coordinate transformation. T represents the change of the extrinsic parameters of the lidar coordinate system to be calibrated after each application of translation and rotation transformations; T is the comprehensive transformation matrix. Where R is the rotation transformation scale and t is the translation transformation scale; All The initial extrinsic parameters of the lidar to be calibrated relative to the reference lidar are obtained. ,as follows: Where n is the number of points in the point cloud B to be labeled.
3. The method according to claim 2, characterized in that, The precise overlapping field of view of the reference lidar and the lidar to be calibrated is determined by extracting the point clouds of the reference lidar and the lidar to be calibrated using the yaw angle, including the following steps: Step 1: Set the maximum and minimum yaw angles , and stride parameters ; Step 2: Based on the yaw angle range between the maximum and minimum values of the set yaw angle, remove point cloud data that exceeds the yaw angle range from the reference point cloud A and the point cloud to be calibrated B, and only retain the point cloud data within the yaw angle range. Step 3: Gradually apply the stride parameters Increase , reduce Obtain the adjusted yaw angle range The point cloud data removal process in step two is repeated based on the adjusted yaw angle range until there is no excess point cloud data outside the yaw angle range, thus obtaining an accurate overlapping field of view. .
4. The method according to claim 3, characterized in that, Step S2 includes: For the point cloud B to be labeled within the precise overlapping field of view, noise removal is performed using the SOR method, and downsampling is performed using the VGD method to obtain the point cloud E to be labeled after noise removal and downsampling. Based on the initial external parameters The coordinate transformation of the point cloud E to be labeled is performed as follows: in, Let be the original coordinates of the i-th point in the point cloud E to be calibrated. The coordinates of the point cloud E to be labeled after coordinate transformation; The reference point cloud A and the point cloud E to be calibrated within the precisely overlapping field of view are filtered to obtain the preprocessed reference point cloud C and the point cloud D to be calibrated, as shown below: in, , These are the reference point cloud C and the point cloud D to be labeled, respectively, within the filtered, precisely overlapping field of view.
5. The method according to claim 1, characterized in that, Using the optimal extrinsic parameters A coordinate transformation is performed on the point cloud B to be calibrated. If the reference point cloud A and the point cloud B to be calibrated do not completely overlap within the precisely overlapping field of view, the voxel resolution parameter is then modified. and iteration threshold Then, proceed to step S3 to recalibrate until the normal distribution transformation function converges to the iteration threshold. Internally, accurate external parameters are obtained.
6. The method according to claim 1, characterized in that, The calculation of the mean and covariance matrix of the multidimensional normal distribution parameters of the reference point cloud C within each voxel is as follows: in, voxels The distribution mean of the internal reference point cloud C, voxels The number of points in the inner reference point cloud C.
7. The method according to claim 6, characterized in that, Based on the mean and covariance matrix of the multidimensional normal distribution parameters of the reference point cloud C within each voxel, the probability density of each point in the target point cloud D within the corresponding voxel is calculated as follows: in, Let i be the i-th point cloud to be punctuated.
8. The method according to claim 7, characterized in that, Calculate the initial extrinsic parameters for each point in the point cloud D to be labeled. The normal distribution transformation function is shown below: in, The score function is a normal distribution transformation. For the first Individual factors.
9. The method according to any one of claims 1-8, characterized in that, The initial extrinsic parameters are iteratively fine-tuned using Newton's optimization algorithm. Calculate the external parameters sequentially The Jacobian vector of the normal distribution transformation function Hesse matrix and external parameter fine-tuning amount ,include: Calculate extrinsic parameters The Jacobian vector of the normal distribution transformation function Its vector elements The calculation is as follows: in, external reference The i-th element of the vector; Calculation based on the extrinsic parameters The Hessian matrix of the normal distribution transformation function Its matrix elements are as follows: in, external reference The j-th element of the vector; Based on the Jacobian vector and the Hesse matrix Calculate the external parameter fine-tuning amount As shown below: 。