Calibration method and device
By using an irregularly arranged array of corner reflectors and a data filtering clustering algorithm on large vehicles, high-precision extrinsic parameter calibration of lidar and millimeter-wave radar was achieved, solving the problem of insufficient angle measurement accuracy in traditional methods and improving the environmental perception and safety of autonomous driving systems.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- CHINA COAL CONSTR GRP CO LTD
- Filing Date
- 2026-01-22
- Publication Date
- 2026-07-21
Smart Images
Figure CN121955901B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of autonomous driving technology, and in particular to a calibration method and apparatus. Background Technology
[0002] In the field of autonomous driving technology, installing multiple sensors to collect redundant information is one of the key means to ensure the safety of autonomous driving. Data fusion of multiple sensors (such as LiDAR, millimeter-wave radar, and cameras) can significantly improve the accuracy of environmental perception, providing more comprehensive and reliable environmental information compared to a single sensor. However, to achieve efficient and accurate data fusion, the calibration of extrinsic parameters between the various sensors is particularly important.
[0003] Extrinsic calibration refers to the process of determining the spatial relative positions and attitude relationships between different sensors. In autonomous vehicles, LiDAR and millimeter-wave radar are two important sensors. LiDAR is a sensor that obtains the position information of a target object by emitting a laser beam and measuring the time it takes for the reflected light to pass. It features high precision and high resolution and is used in autonomous vehicles to build accurate environmental models. Millimeter-wave radar is a sensor that uses electromagnetic waves in the millimeter-wave frequency band for detection. It can accurately measure the speed and distance information of target objects and is used in autonomous vehicles for obstacle detection and avoidance.
[0004] In autonomous driving systems, data fusion of LiDAR and millimeter-wave radar is crucial for accurate environmental modeling and path planning, and this fusion relies on accurate extrinsic parameter calibration. The accuracy of extrinsic parameter calibration directly affects the effectiveness of sensor data fusion, and consequently, the safety and reliability of the autonomous driving system. A correct extrinsic parameter calibration result ensures that obstacle information detected by the two sensors can be accurately mapped through simple coordinate transformations, thereby improving the accuracy and efficiency of data fusion.
[0005] In existing technologies, one method involves using a calibration board to calibrate the extrinsic parameters of multiple sensors under different poses. This method is highly dependent on equipment and complex to operate, requiring the use of a calibration board and complex measurements and calculations in multiple coordinate systems, increasing the equipment dependence and operational complexity of the calibration process. Especially in large vehicles such as unmanned mining trucks or in complex environments, the placement and measurement of the calibration board can be even more difficult. Calibration accuracy is affected by various factors such as the placement and pose of the calibration board and ambient lighting conditions. In practical applications, these factors may lead to unstable calibration results, thus affecting the accuracy of subsequent sensor data fusion. Although joint calibration of multiple sensors has been achieved, its applicability may be limited by the size and shape of the calibration board and the measurement environment.
[0006] Especially in scenarios requiring frequent calibration or calibration under varying environments, this method may lack flexibility. For multiple millimeter-wave radars installed on large vehicles, insufficient accuracy may arise when angle measurements are involved. Due to the long body of large vehicles, the relative angles between radars vary significantly, making traditional measurement methods insufficient for high-precision calibration requirements. Summary of the Invention
[0007] To address the aforementioned technical problems, this invention provides a calibration method and apparatus, which is particularly suitable for the installation and calibration of multiple lidar and millimeter-wave radars on large vehicles. It can effectively solve the problem of insufficient accuracy in angle measurement in traditional methods, and provide strong support for the industrial application of autonomous driving technology in complex environments.
[0008] The technical solution provided by this invention is as follows:
[0009] A calibration method, the method comprising:
[0010] S1: Arrange a corner reflector array consisting of multiple corner reflectors arranged in an irregular geometric shape;
[0011] S2: Simultaneously detect the corner reflector array using both lidar and millimeter-wave radar mounted on the same vehicle to obtain lidar data and millimeter-wave radar data;
[0012] S3: Based on the lidar data and millimeter-wave radar data, extract the position points of each corner reflector in the lidar coordinate system and the position points in the millimeter-wave radar coordinate system;
[0013] S4: Match and calibrate the position points of each corner reflector in the lidar coordinate system and the position points in the millimeter-wave radar coordinate system to obtain the transformation relationship between the lidar coordinate system and the millimeter-wave radar coordinate system.
[0014] Furthermore, S1 includes:
[0015] Multiple corner reflectors are placed in an irregular geometric pattern on a flat surface in an open outdoor area;
[0016] Among them, the length of the line connecting any two corner reflector vertices is different and greater than a set length threshold.
[0017] Furthermore, S2 includes:
[0018] S21: Park the vehicle equipped with lidar and millimeter-wave radar in front of the corner reflector array to ensure that both lidar and millimeter-wave radar can effectively detect the corner reflector array;
[0019] S22: Simultaneously activate the pre-calibrated lidar and millimeter-wave radar, collect the echo data of the corner reflector array, and record a unified timestamp to obtain lidar data and millimeter-wave radar data.
[0020] S23: Use visualization tools to initially check whether the corner reflector array has missed or false detections. If there are missed or false detections, adjust the vehicle position or the corner reflector array and collect data again.
[0021] Furthermore, S3 includes:
[0022] S31: Perform directional filtering on lidar and millimeter-wave radar data to remove environmental noise, irrelevant background points, and invalid echoes;
[0023] S32: Cluster the directional filtered lidar data and millimeter-wave radar data respectively to obtain the position points of each corner reflector in the lidar coordinate system and the position points in the millimeter-wave radar coordinate system.
[0024] Furthermore, S31 includes:
[0025] S311: Based on the lidar data, fit the ground plane using a plane fitting algorithm, calculate the vertical distance from each point of the lidar data to the ground plane, and remove points whose distance is less than a set distance threshold;
[0026] S312: Based on the bounding box of the corner reflector array in the lidar coordinate system, only retain the points in the lidar data that fall within the bounding box;
[0027] S313: Remove points in the lidar data whose reflection intensity is below the set lower limit;
[0028] S314: Combine the center of the corner reflector array with the installation position of the millimeter-wave radar to calculate the radial distance range of the corner reflector array within the field of view of the millimeter-wave radar, and remove points in the millimeter-wave radar data that exceed the radial distance range;
[0029] S315: Based on the lateral angle of the corner reflector array, retain the points of the millimeter-wave radar data that fall within the lateral angle range;
[0030] S316: Remove points in millimeter-wave radar data where the signal-to-noise ratio is lower than the set signal-to-noise ratio threshold.
[0031] Furthermore, S32 includes:
[0032] S321: Perform Euclidean clustering on the directional filtered lidar data and millimeter-wave radar data respectively to obtain clusters of lidar data and millimeter-wave radar data respectively;
[0033] The distance threshold for clustering the lidar data is 1.5 times the maximum side length of the corner reflector, and the distance threshold for clustering the millimeter-wave radar data is 1.2 times the ground projection diameter of the corner reflector.
[0034] S322: Obtain the centroid of each cluster in the lidar data and millimeter-wave radar data respectively, and use it as the position point of the corner reflector in the lidar coordinate system and the position point in the millimeter-wave radar coordinate system.
[0035] Furthermore, S4 includes:
[0036] S41: Obtain the set of line segments formed by each pair of position points of each corner reflector in the lidar coordinate system to obtain the lidar line segment set; obtain the set of line segments formed by each pair of position points of each corner reflector in the millimeter-wave radar coordinate system to obtain the millimeter-wave radar line segment set.
[0037] S42: Sort the line segments in the lidar line segment set and the millimeter-wave radar line segment set from longest to shortest, respectively, to obtain the sorted lidar line segment set and the sorted millimeter-wave radar line segment set;
[0038] S43: Traverse each line segment in the sorted lidar line segment set in order, calculate the absolute value of the difference between the length of the line segment and the length of each line segment in the sorted millimeter-wave radar line segment set, and select two line segments whose absolute difference is less than a set difference threshold as candidate line segment pairs.
[0039] S44: For each candidate line segment pair, calculate two candidate transformation relationships based on the first and last endpoints of the two line segments in the candidate line segment pair;
[0040] In this process, the two starting endpoints of the two line segments are matched, and the two ending points of the two line segments are matched. A candidate transformation relationship is calculated based on the two sets of corresponding points.
[0041] The starting point of one line segment is matched with the ending point of another line segment, and the ending point of one line segment is matched with the starting point of another line segment. Another candidate transformation relationship is calculated based on the two sets of corresponding points.
[0042] S45: For each candidate transformation relationship, perform coordinate transformation on each position point of the corner reflector in the millimeter-wave radar coordinate system according to the candidate transformation relationship to obtain each transformed point set;
[0043] S46: For each point in the transformed point set, calculate the minimum distance from that point to each position point in the lidar coordinate system. If the minimum distance is less than the set minimum distance threshold, then increment the number of matching points for the corresponding candidate transformation relationship by 1.
[0044] S47: Obtain the candidate transformation relationship with the largest number of matching points as the transformation relationship. If there are multiple candidate transformation relationships with the largest number of matching points, select the one with the smallest absolute difference as the transformation relationship.
[0045] A calibration device, the device comprising:
[0046] A corner reflector arrangement module is used to arrange a corner reflector array consisting of multiple corner reflectors arranged in an irregular geometric shape.
[0047] The detection module is used to simultaneously detect the corner reflector array using both lidar and millimeter-wave radar mounted on the same vehicle, and obtain lidar data and millimeter-wave radar data.
[0048] The processing module is used to extract the position points of each corner reflector in the lidar coordinate system and the position points in the millimeter-wave radar coordinate system based on the lidar data and millimeter-wave radar data.
[0049] The calibration module is used to match and calibrate the position points of each corner reflector in the lidar coordinate system and the position points in the millimeter-wave radar coordinate system, so as to obtain the transformation relationship between the lidar coordinate system and the millimeter-wave radar coordinate system.
[0050] Furthermore, in the corner reflector arrangement module, multiple corner reflectors are placed on a flat surface in an open outdoor area in an irregular geometric arrangement.
[0051] Among them, the length of the line connecting any two corner reflector vertices is different and greater than a set length threshold.
[0052] Furthermore, the detection module includes:
[0053] The preparation unit is used to park the vehicle equipped with lidar and millimeter-wave radar in front of the corner reflector array to ensure that both lidar and millimeter-wave radar can effectively detect the corner reflector array.
[0054] The acquisition unit is used to simultaneously activate the pre-calibrated lidar and millimeter-wave radar, acquire the echo data of the corner reflector array, and record a unified timestamp to obtain lidar data and millimeter-wave radar data.
[0055] The inspection unit is used to perform a preliminary inspection of the corner reflector array using visualization tools to check whether there are any missed or false detections. If there are any missed or false detections, the vehicle position or the corner reflector array is adjusted and the data is collected again.
[0056] Furthermore, the processing module includes:
[0057] The filtering unit is used to perform directional filtering on lidar data and millimeter-wave radar data to remove environmental noise, irrelevant background points, and invalid echoes.
[0058] The clustering unit is used to cluster the directional filtered lidar data and millimeter-wave radar data respectively, to obtain the position points of each corner reflector in the lidar coordinate system and the position points in the millimeter-wave radar coordinate system.
[0059] Furthermore, the filtering unit includes:
[0060] The ground point elimination subunit is used to fit the ground plane based on the lidar data using a plane fitting algorithm, calculate the vertical distance from each point in the lidar data to the ground plane, and eliminate points whose distance is less than a set distance threshold.
[0061] The spatial range filtering subunit is used to retain only the points in the lidar data that fall within the bounding box of the corner reflector array in the lidar coordinate system.
[0062] The reflection intensity filtering subunit is used to remove points in the lidar data whose reflection intensity is lower than a set lower limit;
[0063] The radial range filtering sub-unit is used to calculate the radial distance range of the corner reflector array within the field of view of the millimeter-wave radar by combining the center of the corner reflector array with the installation position of the millimeter-wave radar, and to remove points in the millimeter-wave radar data that exceed the radial distance range.
[0064] An angle range filtering subunit is used to retain points of the millimeter-wave radar data that fall within the lateral angle range, based on the lateral angle of the corner reflector array.
[0065] The signal-to-noise ratio (SNR) filtering subunit is used to remove points in millimeter-wave radar data whose SNR is lower than a set SNR threshold.
[0066] Furthermore, the clustering unit includes:
[0067] The adaptive clustering subunit is used to perform Euclidean clustering on the directional filtered lidar data and millimeter-wave radar data respectively, to obtain clusters of lidar data and millimeter-wave radar data respectively;
[0068] The distance threshold for clustering the lidar data is 1.5 times the maximum side length of the corner reflector, and the distance threshold for clustering the millimeter-wave radar data is 1.2 times the ground projection diameter of the corner reflector.
[0069] The coordinate point calculation subunit is used to obtain the centroid of each cluster of LiDAR data and millimeter-wave radar data respectively, as the position point of the corner reflector in the LiDAR coordinate system and the position point in the millimeter-wave radar coordinate system.
[0070] Furthermore, the calibration module includes:
[0071] The line segment acquisition unit is used to acquire the set of line segments formed by each pair of position points of each corner reflector in the lidar coordinate system, and to acquire the set of line segments formed by each pair of position points of each corner reflector in the millimeter-wave radar coordinate system, and to acquire the set of line segments formed by each pair of position points of each corner reflector in the millimeter-wave radar coordinate system.
[0072] The sorting unit is used to sort the line segments in the lidar line segment set and the millimeter-wave radar line segment set from longest to shortest, respectively, to obtain the sorted lidar line segment set and the sorted millimeter-wave radar line segment set.
[0073] The line segment matching unit is used to traverse each line segment in the sorted lidar line segment set in order, calculate the absolute value of the difference between the length of the line segment and the length of each line segment in the sorted millimeter-wave radar line segment set, and select two line segments whose absolute difference is less than a set difference threshold as candidate line segment pairs.
[0074] The candidate transformation relationship determination unit is used to calculate two candidate transformation relationships for each candidate line segment pair based on the first endpoint and the last endpoint of the two line segments in the candidate line segment pair.
[0075] In this process, the two starting endpoints of the two line segments are matched, and the two ending points of the two line segments are matched. A candidate transformation relationship is calculated based on the two sets of corresponding points.
[0076] The starting point of one line segment is matched with the ending point of another line segment, and the ending point of one line segment is matched with the starting point of another line segment. Another candidate transformation relationship is calculated based on the two sets of corresponding points.
[0077] The transformation unit is used to perform coordinate transformation on each position point of the corner reflector in the millimeter-wave radar coordinate system according to each candidate transformation relationship, so as to obtain each transformed point set.
[0078] The point matching unit is used to calculate the minimum distance from each point in the transformed point set to each position point in the lidar coordinate system. If the minimum distance is less than the set minimum distance threshold, the number of matching points for the corresponding candidate transformation relationship is incremented by 1.
[0079] The transformation relationship determination unit is used to obtain the candidate transformation relationship with the largest number of matching points as the transformation relationship. If there are multiple candidate transformation relationships with the largest number of matching points, the one with the smallest absolute difference is selected as the transformation relationship.
[0080] The present invention has the following beneficial effects:
[0081] This invention proposes a radar extrinsic parameter calibration method using a corner reflector as the measurement target. The aim is to achieve high-precision extrinsic parameter calibration between lidar and millimeter-wave radar through a precise calibration process, thereby improving the environmental perception capability and safety of autonomous driving systems in complex environments. This method is particularly suitable for the installation and calibration of multiple lidar and millimeter-wave radars on large vehicles, effectively solving the problem of insufficient accuracy in angle measurement using traditional methods, and providing strong support for the industrial application of autonomous driving technology in complex environments. Attached Figure Description
[0082] Figure 1 This is a flowchart of the calibration method of the present invention;
[0083] Figure 2 A schematic diagram of lidar data;
[0084] Figure 3 A schematic diagram of millimeter-wave radar data;
[0085] Figure 4 A schematic diagram for extrinsic parameter calibration of lidar and millimeter-wave radar;
[0086] Figure 5 This is a schematic diagram of the calibration device of the present invention. Detailed Implementation
[0087] To make the technical problems, technical solutions and advantages of the present invention clearer, a detailed description will be given below in conjunction with the accompanying drawings and specific embodiments.
[0088] Example 1:
[0089] This invention provides a calibration method for extrinsic parameters of LiDAR and millimeter-wave radar, applicable to complex environments (such as smart mines) and large vehicles (such as unmanned mining trucks), to meet the sensor data fusion requirements of autonomous vehicles in various complex scenarios. Figure 1 As shown, the method includes:
[0090] S1: Arrange a corner reflector array consisting of multiple corner reflectors arranged in an irregular geometric shape.
[0091] When setting up, an open outdoor site should be selected, away from buildings, trees, and other obstructions. The ground should be flat and away from sources of electromagnetic interference (such as metal equipment and communication base stations) to ensure that the LiDAR and millimeter-wave radar signals are not significantly affected. Multiple corner reflectors should be placed in an irregular geometric shape: the lengths of the lines connecting the vertices should be different (to avoid equidistant repetition) and greater than a set length threshold. The overall coverage area must meet the sensor detection requirements, and the positions should be marked and fixed (e.g., with paint or ground stakes) to ensure stability.
[0092] S2: Simultaneously detect the corner reflector array using both lidar and millimeter-wave radar mounted on the same vehicle to obtain lidar data and millimeter-wave radar data.
[0093] One specific implementation method includes:
[0094] S21: Park the vehicle equipped with lidar and millimeter-wave radar in front of the corner reflector array at a distance that ensures both lidar and millimeter-wave radar can effectively detect the corner reflector array (usually 20-50 meters).
[0095] S22: Simultaneously activate the pre-calibrated lidar and millimeter-wave radar, collect the echo data of the corner reflector array, and record a unified timestamp to obtain lidar data and millimeter-wave radar data.
[0096] Sensors need to be calibrated in advance to ensure time synchronization (e.g., through hardware triggering or GPS synchronization) to avoid data time deviation. Simultaneously activate both LiDAR and millimeter-wave radar to collect target echo data (LiDAR point cloud + millimeter-wave radar signal) and record a unified timestamp to ensure data spatiotemporal consistency.
[0097] S23: Use visualization tools to initially check whether the corner reflector array has missed or false detections. If there are missed or false detections, adjust the vehicle position or the corner reflector array and collect data again.
[0098] During data acquisition, visualization tools can be used to initially check for missed or false detections. If necessary, the vehicle position or sensor angle can be adjusted before re-acquiring data. Environmental parameters (such as weather and temperature) and equipment status (such as sensor frame rate and vehicle attitude) are recorded simultaneously to provide a basic reference for subsequent data processing. Acquisition results containing rich geometric features (irregular corner reflector layout) and multimodal echo data provide effective input for sensor joint calibration or algorithm training. LiDAR data and millimeter-wave radar data are respectively shown below. Figure 2 , Figure 3 As shown, the area inside the white dashed circle represents the array of corner reflectors detected by the sensor.
[0099] This invention involves placing multiple corner reflectors in an open space, with each pair of corner reflectors having a different side length when connected to each other. By parking a vehicle in front of the corner reflectors, both types of sensors can detect them, simultaneously collecting data from lidar and millimeter-wave radar. This method is simple and easy to implement.
[0100] S3: Based on lidar data and millimeter-wave radar data, extract the position points of each corner reflector in the lidar coordinate system and the position points in the millimeter-wave radar coordinate system.
[0101] This step is used for data processing. The data output by the millimeter-wave radar after processing is the two-dimensional position points of the processed corner reflector. The original point cloud data of the lidar is filtered and segmented and clustered to obtain the three-dimensional position points of the corner reflector. Each position point in the lidar coordinate system and the millimeter-wave radar coordinate system represents a corner reflector.
[0102] One implementation method includes:
[0103] S31: Performs directional filtering on lidar and millimeter-wave radar data to remove environmental noise, irrelevant background points, and invalid echoes.
[0104] To remove environmental noise, irrelevant background points, and invalid echoes from the raw data, it is necessary to perform directional filtering on the raw data of lidar and millimeter-wave radar by combining the prior layout information of the corner reflector (the previously recorded center coordinates and array coverage).
[0105] The raw LiDAR data is a 3D point cloud (including x, y, z coordinates and reflection intensity). It needs to be filtered using multi-dimensional thresholding to retain relevant points from the corner reflectors. The steps are as follows:
[0106] S311: Based on LiDAR data, the ground plane is fitted using a plane fitting algorithm, the vertical distance from each point in the LiDAR data to the ground plane is calculated, and points with a distance less than a set distance threshold are removed.
[0107] This step is used for ground point removal: the ground plane is fitted based on the plane fitting algorithm, the vertical distance from each point to the plane is calculated, and points with a distance less than a threshold (such as 0.2 meters) are removed (assuming the ground is flat and there is a significant distance between the bottom of the corner reflector and the ground after it is raised).
[0108] S312: Based on the bounding box of the corner reflector array in the lidar coordinate system, only retain the points in the lidar data that fall within the bounding box.
[0109] This step is used to filter the spatial range: based on the bounding box of the corner reflector array in the lidar coordinate system (calculated from the center coordinates and maximum side length of the corner reflector mentioned above), only the points falling within the bounding box are retained, and background points outside the array (such as reflection points from distant trees and buildings) are excluded.
[0110] S313: Remove points in the lidar data whose reflection intensity is below the set lower limit;
[0111] This step is used for reflection intensity filtering: the corner reflector is made of a high reflectivity material, a lower limit for reflection intensity is set, and weak reflection noise points (such as road surface reflections, low reflectivity debris) are eliminated.
[0112] The raw data from millimeter-wave radar is a two-dimensional point track (containing x, y coordinates, radial velocity, and signal-to-noise ratio, but no height information). Clutter and invalid point tracks are removed through filtering. The steps are as follows:
[0113] S314: Combining the center of the corner reflector array with the installation position of the millimeter-wave radar, calculate the radial distance range of the corner reflector array within the field of view of the millimeter-wave radar, remove points in the millimeter-wave radar data that exceed the radial distance range, and perform radial range filtering.
[0114] S315: Based on the lateral angle of the corner reflector array (determined by the angle between the leftmost / rightmost corner reflector and the radar line), the angle range is filtered to retain the millimeter-wave radar data points that fall within the lateral angle range, while eliminating clutter from irrelevant areas on the left and right sides.
[0115] S316: Removes points in millimeter-wave radar data with a signal-to-noise ratio (SNR) lower than a set SNR threshold, thus achieving SNR filtering and eliminating low SNR echoes.
[0116] S32: Cluster the directional filtered lidar data and millimeter-wave radar data respectively to obtain the position points of each corner reflector in the lidar coordinate system and the position points in the millimeter-wave radar coordinate system.
[0117] This step is used for adaptive clustering and target association of multimodal data: Even after filtering, the LiDAR point cloud / millimeter-wave radar point trace still contains multiple scattering points from the same corner reflector. Target-level localization needs to be achieved through clustering algorithms, adapting to the dimensional differences between LiDAR and millimeter-wave radar. Specific implementation methods include:
[0118] S321: Perform Euclidean clustering on the directional filtered lidar data and millimeter-wave radar data respectively to obtain clusters of lidar data and millimeter-wave radar data respectively.
[0119] LiDAR 3D Clustering: Euclidean distance clustering is performed on the filtered LiDAR point cloud. Core parameters are set based on the physical dimensions of the corner reflectors and the sensor resolution: Distance threshold – 1.5 times the maximum side length of the corner reflector (e.g., if the maximum side length is 0.5 meters, the threshold is 0.75 meters), ensuring that scattering points from the same corner reflector are clustered together, preventing accidental merging of different corner reflectors due to their placement spacing (the vertex spacing of the corner reflectors must be different and ≥0.3 meters during placement). Minimum point count threshold – each cluster must contain at least 5 points (to avoid clustering of isolated noise points). After clustering, the centroid coordinates of each cluster are extracted as the 3D position of the corner reflector in the LiDAR coordinate system.
[0120] Two-dimensional clustering for millimeter-wave radar: Millimeter-wave radar lacks height information and has sparse point clusters (a single corner reflector typically corresponds to 1-3 point clusters). Euclidean distance clustering is employed: the distance threshold is 1.2 times the area projected onto the ground by the corner reflector (e.g., if the bottom projection diameter of the corner reflector is 0.3 meters, the threshold is 0.36 meters), adapting to its two-dimensional scattering characteristics. Output processing: The centroid coordinates of the cluster are directly extracted as the two-dimensional position of the corner reflector in the millimeter-wave radar coordinate system (ignoring the z-axis, as millimeter-wave radar cannot effectively detect height).
[0121] S322: Obtain the centroid of each cluster in the lidar data and millimeter-wave radar data respectively, and use it as the position point of the corner reflector in the lidar coordinate system and the position point in the millimeter-wave radar coordinate system.
[0122] Through the above clustering, the positioning results of the corner reflector in the two sensor coordinate systems can be obtained respectively, providing a basis for subsequent coordinate transformation and joint calibration.
[0123] LiDAR coordinate system position: Records the three-dimensional coordinates of the corner reflector in the LiDAR coordinate system, directly given by the three-dimensional cluster centroid, reflecting the absolute spatial position of the target relative to the LiDAR installation position. Millimeter-wave radar coordinate system position: Records the two-dimensional coordinates of the corner reflector in the millimeter-wave radar coordinate system, given by the two-dimensional cluster centroid, reflecting the planar position of the target relative to the millimeter-wave radar installation position.
[0124] Obtain the position of the corner reflector in the lidar coordinate system. and the position point in the millimeter-wave radar coordinate system This lays the data foundation for subsequent coordinate transformation matrix solving and sensor joint calibration. (e.g.) Figure 2 , Figure 3 As shown, the data points within the white dashed box, after processing, are ultimately extracted as the positioning points of each corner reflector in the corresponding sensor coordinate system.
[0125] S4: Match and calibrate the position points of each corner reflector in the lidar coordinate system and the position points in the millimeter-wave radar coordinate system to obtain the transformation relationship between the lidar coordinate system and the millimeter-wave radar coordinate system.
[0126] This step is used to perform extrinsic parameter calibration. After obtaining the position points of the corner reflector in the lidar coordinate system and the millimeter-wave radar coordinate system, the point maps in the lidar coordinate system and the millimeter-wave radar coordinate system are matched to obtain the rotation and translation relationship between the two coordinate systems. By combining the two coordinate systems, the extrinsic parameters of the millimeter-wave radar relative to the lidar can be obtained.
[0127] During matching, the corresponding transformation relationship is first obtained by pairing line segments. Then, after all points are transformed, corresponding points are paired and compared to obtain the final result, resulting in highly accurate extrinsic parameter coefficients suitable for complex environments and large vehicles. The specific matching method is as follows:
[0128] S41: Obtain the set of line segments formed by pairs of position points of each corner reflector in the lidar coordinate system to obtain the lidar line segment set. Obtain the set of line segments formed by pairs of position points of each corner reflector in the millimeter-wave radar coordinate system to obtain the millimeter-wave radar line segment set.
[0129] right Find the set of line segments between every pair of points in the matrix. (Save two endpoints) , and line segment length ).right Find the set of line segments between each pair of points in the line. (Save endpoints) , and length ).
[0130] S42: Sort the line segments in the lidar line segment set and the millimeter-wave radar line segment set from longest to shortest, respectively, to obtain the sorted lidar line segment set and the sorted millimeter-wave radar line segment set.
[0131] S43: Traverse each line segment in the sorted lidar line segment set in order, calculate the absolute value of the difference between the length of the line segment and the length of each line segment in the sorted millimeter-wave radar line segment set, and select two line segments whose absolute difference is less than the set difference threshold as candidate line segment pairs.
[0132] from After starting the traversal and sorting Each line segment in ,and Each line segment in Compare, calculate and absolute value of the difference ,if Less than the threshold If the pair is 0, it is marked as a candidate line segment pair.
[0133] S44: For each candidate line segment pair, calculate two candidate transformation relationships based on the first and last endpoints of the two line segments in the candidate line segment pair.
[0134] For each candidate line segment pair, there are two cases:
[0135] The first type: The two starting points of the two line segments correspond to each other, and the two ending points of the two line segments correspond to each other. That is... correspond , correspond A set of transformation relationships is obtained. (First, calculate the slope of the two line segments) and , - The difference is the angle of rotation. ,Will and Rotate around the origin We obtain two rotated points, and these two points are then respectively the corresponding... and calculate and The difference in direction yields the translation amount. ,Depend on and Composition of candidate transformation relations .
[0136] The second type: the starting point of one line segment corresponds to the ending point of another line segment, and vice versa. correspond , correspond To obtain another set of candidate transformation relations .
[0137] S45: For each candidate transformation relationship, perform coordinate transformation on each position point of the corner reflector in the millimeter-wave radar coordinate system according to the candidate transformation relationship to obtain each transformed point set.
[0138] Taking the first method as an example, using right Each of them Perform the transformation to obtain the corresponding (Represents the transformed point).
[0139] S46: For each point in the transformed point set, calculate the minimum distance from that point to each position point in the lidar coordinate system. If the minimum distance is less than the set minimum distance threshold, increment the number of matching points for the corresponding candidate transformation relationship by 1.
[0140] Specifically, taking the first type as an example, for exist Traversal of the middle and Find the minimum distance. ,if Less than the threshold Transformation Relationship Corresponding number of matching points ( Indicates will Add one, if the transformed A point has found a corresponding matching point, which is the counter of the matching points. Add one).
[0141] The transformation relationship in the second case is obtained using the same method. Corresponding number of matching points .
[0142] S47: Obtain the candidate transformation relationship with the largest number of matching points as the transformation relationship. If there are multiple candidate transformation relationships with the largest number of matching points, select the one with the smallest absolute difference as the transformation relationship.
[0143] Compare the number of matching points corresponding to all transformation relations. The transformation relation with the largest number of matching points is the final solution. If there are multiple values corresponding to the largest number of matching points, then the one corresponding to the smallest number is the final solution. This is the optimal solution. Figure 4 The diagram shows the calibration results of the external parameters of the lidar and millimeter-wave radar obtained by the external parameter calibration method of the present invention using three corner reflectors. The white dots in the diagram represent the position points of the corner reflectors in the lidar coordinate system, and the square dots represent the position points of the corner reflectors after external parameter calibration, projected from the millimeter-wave radar coordinate system to the lidar coordinate system.
[0144] This invention proposes a radar extrinsic parameter calibration method using a corner reflector as the measurement target. The aim is to achieve high-precision extrinsic parameter calibration between lidar and millimeter-wave radar through a precise calibration process, thereby improving the environmental perception capability and safety of autonomous driving systems in complex environments. This method is particularly suitable for the installation and calibration of multiple lidar and millimeter-wave radars on large vehicles, effectively solving the problem of insufficient accuracy in angle measurement using traditional methods, and providing strong support for the industrial application of autonomous driving technology in complex environments.
[0145] Example 2:
[0146] This invention provides a calibration device, such as... Figure 5 As shown, the device includes:
[0147] Corner reflector arrangement module 1 is used to arrange a corner reflector array consisting of multiple corner reflectors arranged in an irregular geometric shape.
[0148] The detection module 2 is used to simultaneously detect the corner reflector array using both lidar and millimeter-wave radar mounted on the same vehicle, and obtain lidar data and millimeter-wave radar data.
[0149] Processing module 3 is used to extract the position points of each corner reflector in the lidar coordinate system and the position points in the millimeter-wave radar coordinate system based on lidar data and millimeter-wave radar data.
[0150] Calibration module 4 is used to match and calibrate the position points of each corner reflector in the lidar coordinate system and the position points in the millimeter-wave radar coordinate system, so as to obtain the transformation relationship between the lidar coordinate system and the millimeter-wave radar coordinate system.
[0151] In the corner reflector arrangement module of the present invention, multiple corner reflectors are placed on a flat ground in an open outdoor area in an irregular geometric arrangement.
[0152] Among them, the length of the line connecting any two corner reflector vertices is different and greater than a set length threshold.
[0153] As an example, the detection module includes:
[0154] The preparation unit is used to park vehicles equipped with lidar and millimeter-wave radar in front of the corner reflector array, ensuring that both lidar and millimeter-wave radar can effectively detect the corner reflector array.
[0155] The acquisition unit is used to simultaneously activate the pre-calibrated lidar and millimeter-wave radar, acquire the echo data of the corner reflector array, and record a unified timestamp to obtain lidar data and millimeter-wave radar data.
[0156] The inspection unit is used to perform a preliminary inspection of the corner reflector array using visualization tools to check whether there are any missed or false detections. If there are any missed or false detections, the vehicle position or the corner reflector array is adjusted and the data is collected again.
[0157] The aforementioned processing modules include:
[0158] The filtering unit is used to perform directional filtering on lidar data and millimeter-wave radar data to remove environmental noise, irrelevant background points, and invalid echoes.
[0159] The clustering unit is used to cluster the directional filtered lidar data and millimeter-wave radar data respectively, to obtain the position points of each corner reflector in the lidar coordinate system and the position points in the millimeter-wave radar coordinate system.
[0160] Specifically, the filtering unit includes:
[0161] The ground point elimination sub-unit is used to fit the ground plane based on LiDAR data using a plane fitting algorithm, calculate the vertical distance from each point in the LiDAR data to the ground plane, and eliminate points whose distance is less than a set distance threshold.
[0162] The spatial range filtering sub-unit is used to retain only the points in the lidar data that fall within the bounding box of the corner reflector array in the lidar coordinate system.
[0163] The reflection intensity filtering subunit is used to remove points in the lidar data whose reflection intensity is lower than a set lower limit.
[0164] The radial range filtering sub-unit is used to calculate the radial distance range of the corner reflector array within the field of view of the millimeter-wave radar by combining the center of the corner reflector array with the installation position of the millimeter-wave radar, and to remove points in the millimeter-wave radar data that exceed the radial distance range.
[0165] Angle range filtering sub-unit is used to retain millimeter-wave radar data points that fall within the lateral angle range based on the lateral angle of the corner reflector array.
[0166] The signal-to-noise ratio (SNR) filtering subunit is used to remove points in millimeter-wave radar data whose SNR is lower than a set SNR threshold.
[0167] Clustering units include:
[0168] The adaptive clustering subunit is used to perform Euclidean clustering on the directional filtered lidar data and millimeter-wave radar data respectively, to obtain clusters of lidar data and millimeter-wave radar data respectively.
[0169] Among them, the distance threshold for lidar data clustering is 1.5 times the maximum side length of the corner reflector, and the distance threshold for millimeter-wave radar data clustering is 1.2 times the ground projection diameter of the corner reflector.
[0170] The coordinate point calculation subunit is used to obtain the centroid of each cluster of LiDAR data and millimeter-wave radar data respectively, as the position point of the corner reflector in the LiDAR coordinate system and the position point in the millimeter-wave radar coordinate system.
[0171] As an improvement to an embodiment of the present invention, the calibration module includes:
[0172] The line segment acquisition unit is used to acquire the set of line segments formed by each pair of position points of each corner reflector in the lidar coordinate system, and to acquire the set of line segments formed by each pair of position points of each corner reflector in the millimeter-wave radar coordinate system, and to acquire the set of line segments formed by each pair of position points of each corner reflector in the millimeter-wave radar coordinate system.
[0173] The sorting unit is used to sort the line segments in the lidar line segment set and the millimeter-wave radar line segment set from longest to shortest, respectively, to obtain the sorted lidar line segment set and the sorted millimeter-wave radar line segment set.
[0174] The line segment matching unit is used to traverse each line segment in the sorted lidar line segment set in order, calculate the absolute value of the difference between the length of the line segment and each line segment in the sorted millimeter-wave radar line segment set, and select two line segments whose absolute difference is less than a set difference threshold as candidate line segment pairs.
[0175] The candidate transformation relationship determination unit is used to calculate two candidate transformation relationships for each candidate line segment pair based on the first endpoint and the last endpoint of the two line segments in the candidate line segment pair.
[0176] In this process, the two starting endpoints of the two line segments are matched, and the two ending points of the two line segments are matched. A candidate transformation relationship is calculated based on the two sets of corresponding points.
[0177] Match the starting point of one line segment with the ending point of another line segment, and match the ending point of one line segment with the starting point of another line segment. Calculate another candidate transformation relationship based on the two sets of points.
[0178] The transformation unit is used to perform coordinate transformation on each position point of the corner reflector in the millimeter-wave radar coordinate system according to each candidate transformation relationship, so as to obtain each transformed point set.
[0179] The point matching unit is used to calculate the minimum distance from each point in the transformed point set to each position point in the lidar coordinate system. If the minimum distance is less than the set minimum distance threshold, the number of matching points for the corresponding candidate transformation relationship is incremented by 1.
[0180] The transformation relationship determination unit is used to obtain the candidate transformation relationship with the largest number of matching points as the transformation relationship. If there are multiple candidate transformation relationships with the largest number of matching points, the one with the smallest absolute difference is selected as the transformation relationship.
[0181] The apparatus provided in this embodiment of the invention has the same implementation principle and technical effects as the aforementioned method embodiment. For the sake of brevity, any parts not mentioned in the apparatus embodiment can be referred to the corresponding content in the aforementioned method embodiment 1. Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the apparatus and units described above can all be referred to the corresponding processes in the above method embodiments, and will not be repeated here.
[0182] Finally, it should be noted that the above-described embodiments are merely specific implementations of the present invention, used to illustrate the technical solutions of the present invention, and not to limit them. The scope of protection of the present invention is not limited thereto. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that any person skilled in the art can still modify or easily conceive of changes to the technical solutions described in the foregoing embodiments within the scope of the technology disclosed in the present invention, or make equivalent substitutions for some of the technical features; and these modifications, changes, or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be covered within the scope of protection of the present invention.
Claims
1. A calibration method, characterized in that, The method includes: S1: Arrange a corner reflector array consisting of multiple corner reflectors arranged in an irregular geometric shape; S2: Simultaneously detect the corner reflector array using both lidar and millimeter-wave radar mounted on the same vehicle to obtain lidar data and millimeter-wave radar data; S3: Based on the lidar data and millimeter-wave radar data, extract the position points of each corner reflector in the lidar coordinate system and the position points in the millimeter-wave radar coordinate system; S4: Match and calibrate the position points of each corner reflector in the lidar coordinate system and the position points in the millimeter-wave radar coordinate system to obtain the transformation relationship between the lidar coordinate system and the millimeter-wave radar coordinate system. S1 includes: Multiple corner reflectors are placed in an irregular geometric pattern on a flat surface in an open outdoor area; Among them, the length of the line connecting any two corner reflector vertices is different and greater than a set length threshold; S4 includes: S41: Obtain the set of line segments formed by each pair of position points of each corner reflector in the lidar coordinate system to obtain the lidar line segment set; obtain the set of line segments formed by each pair of position points of each corner reflector in the millimeter-wave radar coordinate system to obtain the millimeter-wave radar line segment set. S42: Sort the line segments in the lidar line segment set and the millimeter-wave radar line segment set from longest to shortest, respectively, to obtain the sorted lidar line segment set and the sorted millimeter-wave radar line segment set; S43: Traverse each line segment in the sorted lidar line segment set in order, calculate the absolute value of the difference between the length of the line segment and the length of each line segment in the sorted millimeter-wave radar line segment set, and select two line segments whose absolute difference is less than a set difference threshold as candidate line segment pairs. S44: For each candidate line segment pair, calculate two candidate transformation relationships based on the first and last endpoints of the two line segments in the candidate line segment pair; In this process, the two starting endpoints of the two line segments are matched, and the two ending points of the two line segments are matched. A candidate transformation relationship is calculated based on the two sets of corresponding points. The starting point of one line segment is matched with the ending point of another line segment, and the ending point of one line segment is matched with the starting point of another line segment. Another candidate transformation relationship is calculated based on the two sets of corresponding points. S45: For each candidate transformation relationship, perform coordinate transformation on each position point of the corner reflector in the millimeter-wave radar coordinate system according to the candidate transformation relationship to obtain each transformed point set; S46: For each point in the transformed point set, calculate the minimum distance from that point to each position point in the lidar coordinate system. If the minimum distance is less than the set minimum distance threshold, then increment the number of matching points for the corresponding candidate transformation relationship by 1. S47: Obtain the candidate transformation relationship with the largest number of matching points as the transformation relationship. If there are multiple candidate transformation relationships with the largest number of matching points, select the one with the smallest absolute difference as the transformation relationship.
2. The calibration method according to claim 1, characterized in that, S2 includes: S21: Park the vehicle equipped with lidar and millimeter-wave radar in front of the corner reflector array to ensure that both lidar and millimeter-wave radar can effectively detect the corner reflector array; S22: Simultaneously activate the pre-calibrated lidar and millimeter-wave radar, collect the echo data of the corner reflector array, and record a unified timestamp to obtain lidar data and millimeter-wave radar data. S23: Use visualization tools to initially check whether the corner reflector array has missed or false detections. If there are missed or false detections, adjust the vehicle position or the corner reflector array and collect data again.
3. The calibration method according to claim 1, characterized in that, S3 includes: S31: Perform directional filtering on lidar and millimeter-wave radar data to remove environmental noise, irrelevant background points, and invalid echoes; S32: Cluster the directional filtered lidar data and millimeter-wave radar data respectively to obtain the position points of each corner reflector in the lidar coordinate system and the position points in the millimeter-wave radar coordinate system.
4. The calibration method according to claim 3, characterized in that, S31 includes: S311: Based on the lidar data, fit the ground plane using a plane fitting algorithm, calculate the vertical distance from each point of the lidar data to the ground plane, and remove points whose distance is less than a set distance threshold; S312: Based on the bounding box of the corner reflector array in the lidar coordinate system, only retain the points in the lidar data that fall within the bounding box; S313: Remove points in the lidar data whose reflection intensity is below the set lower limit; S314: Combine the center of the corner reflector array with the installation position of the millimeter-wave radar to calculate the radial distance range of the corner reflector array within the field of view of the millimeter-wave radar, and remove points in the millimeter-wave radar data that exceed the radial distance range; S315: Based on the lateral angle of the corner reflector array, retain the points of the millimeter-wave radar data that fall within the lateral angle range; S316: Remove points in millimeter-wave radar data where the signal-to-noise ratio is lower than the set signal-to-noise ratio threshold.
5. The calibration method according to claim 3, characterized in that, S32 includes: S321: Perform Euclidean clustering on the directional filtered lidar data and millimeter-wave radar data respectively to obtain clusters of lidar data and millimeter-wave radar data respectively; The distance threshold for clustering the lidar data is 1.5 times the maximum side length of the corner reflector, and the distance threshold for clustering the millimeter-wave radar data is 1.2 times the ground projection diameter of the corner reflector. S322: Obtain the centroid of each cluster in the lidar data and millimeter-wave radar data respectively, and use it as the position point of the corner reflector in the lidar coordinate system and the position point in the millimeter-wave radar coordinate system.
6. A calibration device, characterized in that, The device includes: A corner reflector arrangement module is used to arrange a corner reflector array consisting of multiple corner reflectors arranged in an irregular geometric shape. The detection module is used to simultaneously detect the corner reflector array using both lidar and millimeter-wave radar mounted on the same vehicle, and obtain lidar data and millimeter-wave radar data. The processing module is used to extract the position points of each corner reflector in the lidar coordinate system and the position points in the millimeter-wave radar coordinate system based on the lidar data and millimeter-wave radar data. The calibration module is used to match and calibrate the position points of each corner reflector in the lidar coordinate system and the position points in the millimeter-wave radar coordinate system, so as to obtain the transformation relationship between the lidar coordinate system and the millimeter-wave radar coordinate system. In the corner reflector arrangement module, multiple corner reflectors are placed on a flat surface in an open outdoor area in an irregular geometric arrangement. Among them, the length of the line connecting any two corner reflector vertices is different and greater than a set length threshold; The calibration module includes: The line segment acquisition unit is used to acquire the set of line segments formed by each pair of position points of each corner reflector in the lidar coordinate system, and to acquire the set of line segments formed by each pair of position points of each corner reflector in the millimeter-wave radar coordinate system, and to acquire the set of line segments formed by each pair of position points of each corner reflector in the millimeter-wave radar coordinate system. The sorting unit is used to sort the line segments in the lidar line segment set and the millimeter-wave radar line segment set from longest to shortest, respectively, to obtain the sorted lidar line segment set and the sorted millimeter-wave radar line segment set. The line segment matching unit is used to traverse each line segment in the sorted lidar line segment set in order, calculate the absolute value of the difference between the length of the line segment and the length of each line segment in the sorted millimeter-wave radar line segment set, and select two line segments whose absolute difference is less than a set difference threshold as candidate line segment pairs. The candidate transformation relationship determination unit is used to calculate two candidate transformation relationships for each candidate line segment pair based on the first endpoint and the last endpoint of the two line segments in the candidate line segment pair. In this process, the two starting endpoints of the two line segments are matched, and the two ending points of the two line segments are matched. A candidate transformation relationship is calculated based on the two sets of corresponding points. The starting point of one line segment is matched with the ending point of another line segment, and the ending point of one line segment is matched with the starting point of another line segment. Another candidate transformation relationship is calculated based on the two sets of corresponding points. The transformation unit is used to perform coordinate transformation on each position point of the corner reflector in the millimeter-wave radar coordinate system according to each candidate transformation relationship, so as to obtain each transformed point set. The point matching unit is used to calculate the minimum distance from each point in the transformed point set to each position point in the lidar coordinate system. If the minimum distance is less than the set minimum distance threshold, the number of matching points for the corresponding candidate transformation relationship is incremented by 1. The transformation relationship determination unit is used to obtain the candidate transformation relationship with the largest number of matching points as the transformation relationship. If there are multiple candidate transformation relationships with the largest number of matching points, the one with the smallest absolute difference is selected as the transformation relationship.