A method and system for calibrating extrinsic parameters of multi-laser radar

Abstract

This application relates to the field of sensor calibration technology, and discloses a method and system for extrinsic parameter calibration of multiple lidars. The method includes: simultaneously illuminating a calibration board with reflective stickers using a first lidar and a second lidar; determining the spatial pose of the calibration board based on point cloud data detected by the first lidar and the second lidar, respectively; defining a three-dimensional point cloud range based on the spatial pose; extracting the coordinates of the center points of the point cloud data within the three-dimensional point cloud range; and calculating the rigid body transformation matrix between the first lidar and the second lidar based on the relative positional relationship between the coordinates of the first center point and the second center point, thereby achieving extrinsic parameter calibration of both. This application is applicable to the calibration between mechanically rotating multi-line lidars and hybrid solid-state multi-line lidars, as well as between multi-line lidars of the same type.

Description

Technical Field

[0001] This application relates to the field of sensor calibration technology, and more specifically, to a method and system for calibrating the extrinsic parameters of a multi-laser radar. Background Technology

[0002] With technological advancements, multi-laser radars are widely used in various intelligent systems due to their ability to provide high-precision and high-resolution environmental perception data. In practical applications, the sensing range of a single radar is limited, often requiring the deployment of multiple radars to expand coverage and improve sensing accuracy. The extrinsic parameter calibration between different radars, i.e., the determination of their relative pose, is a prerequisite and key to achieving data fusion. Traditional extrinsic parameter calibration methods mainly rely on manual operation, such as manually placing specific calibration objects or manually selecting corresponding point cloud features. This method is not only time-consuming and labor-intensive, but its calibration accuracy is also greatly affected by human factors, making it difficult to meet the requirements of efficiency and accuracy in practical applications. Especially in calibration scenarios between mechanically rotating multi-line radars and hybrid solid-state multi-line radars, traditional calibration methods face even greater challenges due to the significant differences in the working principles and point cloud characteristics of the two types of radars. Summary of the Invention

[0003] To address the aforementioned technical problems, this application discloses a method and system for extrinsic parameter calibration of multiple lidar systems. This method automatically identifies the position and attitude of the calibration plate and calculates the rigid body transformation matrix between two lidar systems by combining multiple sets of calibration data, achieving efficient and accurate extrinsic parameter calibration. It is applicable to calibration between mechanically rotating multi-line lidars and hybrid solid-state multi-line lidars, as well as between lidars of the same type. Specifically, the technical solution of this application is as follows:

[0004] In a first aspect, this application discloses a method for calibrating the extrinsic parameters of a multi-lidar system, comprising the following steps:

[0005] The calibration board with reflective stickers is illuminated simultaneously by a first lidar and a second lidar, and the spatial pose of the calibration board is determined based on the point cloud data detected by the first lidar and the second lidar, respectively.

[0006] Based on the spatial pose, the range of the three-dimensional point cloud is defined; the coordinates of the first center point and the second center point of the point cloud data corresponding to the first lidar and the second lidar within the range of the three-dimensional point cloud are extracted.

[0007] Based on the relative positional relationship between the coordinates of the first center point and the coordinates of the second center point, the rigid body transformation matrix between the first lidar and the second lidar is calculated, thereby realizing the external parameter calibration of the two.

[0008] In some embodiments, the method of simultaneously illuminating a calibration board with a reflective sticker using a first lidar and a second lidar, and determining the spatial pose of the calibration board based on point cloud data detected by the first lidar and the second lidar respectively, includes:

[0009] The first point cloud data is obtained by using the first lidar detection, and the first prediction result of the spatial pose is calculated and obtained based on the first point cloud data.

[0010] The second point cloud data is obtained by using the second lidar, and the second prediction result of the spatial pose is calculated based on the second point cloud data.

[0011] In some embodiments, the step of defining a 3D point cloud range based on the spatial pose, and extracting the first and second center point coordinates of the point cloud data corresponding to the first and second lidars within the 3D point cloud range, specifically includes:

[0012] Based on the first prediction result of the spatial pose, the range of the first stereo point cloud is defined; the coordinates of the first center point of the first point cloud data within the range of the first stereo point cloud are extracted.

[0013] Based on the second prediction result of the spatial pose, the range of the second stereo point cloud is defined; the coordinates of the second center point of the second point cloud data within the range of the second stereo point cloud are extracted.

[0014] In some embodiments, determining the spatial pose of the calibration board based on point cloud data detected by the first lidar and the second lidar respectively includes:

[0015] The first point cloud data and the second point cloud data are subjected to time-series frame fusion processing respectively. By registering and fusing multiple frames of point cloud data from adjacent time series, fused point cloud data is generated.

[0016] Target point cloud data with point cloud intensity higher than a preset intensity threshold are selected from the fused point cloud data, and the largest point cloud cluster in the target point cloud data is extracted using a clustering method;

[0017] Calculate the three-dimensional centroid of the largest point cloud cluster;

[0018] Perform plane fitting on the largest point cloud cluster to obtain the fitting plane, and calculate the yaw angle of the fitting plane;

[0019] The spatial pose of the calibration plate is determined based on the three-dimensional centroid and the yaw angle.

[0020] In some embodiments, the multi-lidar extrinsic parameter calibration method further includes:

[0021] The first point cloud data is obtained by using the first lidar detection, and the first spatial pose of the calibration plate is calculated and obtained based on the first point cloud data.

[0022] The second point cloud data is obtained by using the second lidar, and the second spatial pose of the calibration board is calculated based on the second point cloud data.

[0023] In some implementations, defining the range of the stereo point cloud based on the spatial pose specifically includes:

[0024] Based on the spatial pose of the calibration board, the two-dimensional bounding box of the calibration board is extracted; in a direction perpendicular to the fitting plane of the calibration board, the two-dimensional bounding box is extended by the same preset distance in front of and behind the fitting plane to obtain a three-dimensional bounding box.

[0025] The spatial range defined by the three-dimensional border is taken as the range of the stereo point cloud.

[0026] In some embodiments, the step of extracting the first center point coordinates and the second center point coordinates of the point cloud data corresponding to the first and second lidars within the 3D point cloud range specifically includes:

[0027] Within the range of the three-dimensional point cloud, the point cloud data is grouped according to the laser line number, the boundary points at both ends of each laser line are extracted, and the boundary points are divided into multiple sets of boundary points according to the distance between the points.

[0028] The boundary point set is fitted to a straight line using a fitting algorithm. Based on the property that straight lines intersect each other in space, the nearest point between the straight lines is calculated, thereby determining the four corner points of the boundary point set. The coordinates of the center point are then calculated based on the corner points.

[0029] In other embodiments, the step of extracting the first and second center point coordinates of the point cloud data corresponding to the first and second lidars within the stereo point cloud range specifically includes:

[0030] Within the range of the stereo point cloud, the fused point cloud data is subjected to voxel filtering to reduce the point cloud density;

[0031] The average value of all point cloud points in the filtered point cloud data is calculated to obtain the coordinates of the center point.

[0032] Based on the above embodiments, the first lidar and the second lidar are respectively a mechanical rotating lidar and / or a hybrid solid-state lidar.

[0033] In some implementations, the rigid body transformation matrix between the first and second lidars is calculated based on the relative positional relationship between the coordinates of the first and second center points, thereby achieving extrinsic parameter calibration of both; specifically, this includes:

[0034] Establish a rigid body transformation matrix between the coordinates of the first center point and the coordinates of the second center point; solve the rigid body transformation matrix using the singular value decomposition method; obtain the rotation matrix and translation vector between the first coordinate system of the first lidar and the second coordinate system of the second lidar.

[0035] Secondly, this application also discloses a multi-laser radar extrinsic parameter calibration system, which is used to execute a computer program to implement the steps of a multi-laser radar extrinsic parameter calibration method described in any of the above embodiments.

[0036] Compared with the prior art, this application has at least one of the following beneficial effects:

[0037] 1. This application improves the point cloud density and recognition accuracy of the calibration board by automatically processing the point clouds between multiple lidar systems based on a calibration board. This achieves efficient and accurate extrinsic parameter calibration, applicable to various lidar type combinations, including calibration between mechanically rotating multi-line lidars and hybrid solid-state multi-line lidars, as well as calibration between lidars of the same type. It significantly reduces manual intervention, improves calibration efficiency and the stability of calibration results, and provides a solid foundation for accurate perception in multi-liquid lidar fusion systems.

[0038] 2. In determining the spatial pose of the calibration board, this application first performs temporal frame fusion processing on the first point cloud data and the second point cloud data respectively to increase the point cloud density. Then, target point cloud data with point cloud intensity greater than the target point cloud intensity are selected, and clustering methods are used to extract the largest point cloud cluster. This effectively improves the density of the calibration board's point cloud and the accuracy of feature recognition.

[0039] 3. The center point extraction process in this application employs different calculation methods based on the characteristics of different types of lidar. For mechanically rotating lidar: boundary points are extracted by grouping according to the lidar line number, and a random sampling consensus algorithm is used to fit four boundary lines. The midpoint of the nearest point between the lines is calculated to determine the four corner points, and finally, the center point is obtained. For hybrid solid-state lidar: after voxel filtering of the multi-frame fused point cloud, the mean of all points is calculated as the center point, improving the efficiency of subsequent calculations. The technical solution of this application breaks through the limitations of traditional calibration methods on lidar type matching, realizes automated calibration of cross-type lidar combinations, significantly reduces the need for manual intervention, and has significant improvements in calibration efficiency, applicability, and result stability, providing a precise and reliable spatial reference for multi-library fusion sensing systems. Attached Figure Description

[0040] The preferred embodiments will now be described in a clear and easy-to-understand manner, in conjunction with the accompanying drawings, to further explain the above-mentioned characteristics, technical features, advantages, and implementation methods of this application.

[0041] Figure 1 This is a flowchart illustrating the steps of an embodiment of a multi-lidar extrinsic parameter calibration method according to this application;

[0042] Figure 2 This is a flowchart illustrating the steps for determining the spatial pose of the calibration board based on point cloud data in an embodiment of this application.

[0043] Figure 3 This is a schematic diagram of the center point of the computationally rotating multi-line radar in the embodiments of this application;

[0044] Figure 4 This is a schematic diagram illustrating the calculation of the center point of the hybrid solid-state multi-line radar in an embodiment of this application. Detailed Implementation

[0045] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application can also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this application with unnecessary detail.

[0046] It should be understood that, when used in this specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or sets.

[0047] To keep the drawings concise, each figure only schematically shows the parts relevant to the invention, and these do not represent the actual structure of the product. Furthermore, to facilitate understanding, in some figures, only one of components with the same structure or function is schematically depicted, or only one is labeled. In this document, "one" not only means "only one," but can also mean "more than one."

[0048] It should also be further understood that the term “and / or” as used in this application specification and the appended claims means any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0049] Furthermore, in the description of this application, the terms "first," "second," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0050] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the specific implementation methods of this application will be described below with reference to the accompanying drawings. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings and other implementation methods can be obtained based on these drawings without creative effort.

[0051] Extrinsic parameter calibration technology for multi-lidar systems is one of the core issues in autonomous driving, robotic perception, and multi-sensor fusion, especially the calibration of mechanically rotating radars and hybrid solid-state radars. Mechanically rotating multi-line radars achieve circular scanning of the laser beam through mechanically rotating components, and their point cloud data features uniform distribution and omnidirectional coverage. Hybrid solid-state multi-line radars, on the other hand, use microelectromechanical systems (MEMS) or optical phased arrays to deflect the beam, and their point cloud distribution typically exhibits non-uniformity and a limited field of view. This difference in point cloud characteristics makes direct matching of the data from the two types of radars difficult.

[0052] Traditional point cloud registration-based calibration methods often yield poor results. These methods primarily draw upon calibration techniques from computer vision and photogrammetry, relying on manually placed calibration targets to establish correspondences between different sensors. While these methods can provide high calibration accuracy in static environments, they heavily depend on manual intervention and are difficult to adapt to dynamic or complex environments. Furthermore, the scanning modes of hybrid solid-state radars may dynamically adjust according to the application scenario, further increasing the complexity of calibration.

[0053] While calibration of multi-line lidar systems of the same type exhibits relatively consistent point cloud characteristics, practical applications still suffer from issues such as differences in installation location and obstruction effects. This makes it difficult to simultaneously meet accuracy and automation requirements, necessitating new calibration schemes. Overall, methods for calibrating the extrinsic parameters of multiple lidar systems are of significant importance in fields such as autonomous driving and intelligent robotics.

[0054] In view of the above, this application proposes an automatic extrinsic parameter calibration method for multiple lidar systems based on a reflective calibration plate. This method can automatically identify the position and attitude of the calibration plate, and calculate the rigid body transformation matrix between two lidar systems by combining multiple sets of calibration data, thus achieving efficient and accurate extrinsic parameter calibration. It is applicable to calibration between mechanically rotating multi-line lidar systems and hybrid solid-state multi-line lidar systems, as well as between lidar systems of the same type.

[0055] Reference manual attached Figure 1 As shown, an embodiment of a multi-lidar extrinsic parameter calibration method of this application specifically includes the following steps:

[0056] S1, simultaneously illuminate the calibration board with reflective stickers using a first lidar and a second lidar, and determine the spatial pose of the calibration board based on the point cloud data detected by the first lidar and the second lidar, respectively.

[0057] In this embodiment, the calibration plate is a square rigid plate with a reflective sticker on the front. The side length of the calibration plate is no less than 30cm to ensure that the reflection intensity is significantly higher than the environmental background, facilitating automatic detection. The calibration plate is fixed within the common detection range of the first and second lidars, and is placed at a preset angle. Optionally, the plate surface is placed at an angle of approximately 45 degrees. This tilted placement effectively avoids the problem of inaccurate normal vector estimation caused by the calibration plate being parallel to the radar scanning plane, thereby improving the accuracy of pose estimation.

[0058] Point cloud data of the calibration board were collected from two lidars respectively. The planar features of the point cloud of the calibration board were extracted by analyzing the point cloud. Based on the pose data of the calibration board obtained by each of the two lidars, a precise spatial alignment basis was provided for subsequent multi-library data fusion.

[0059] S2, Based on the spatial pose, define the range of the 3D point cloud. S3, Extract the coordinates of the first and second center points of the point cloud data corresponding to the first and second lidars within the 3D point cloud range.

[0060] Specifically, based on the point cloud data of the calibration board collected by the first and second lidars respectively, the coordinates of the center point of the calibration board in their respective radar coordinate systems are calculated, namely the first center point coordinates and the second center point coordinates.

[0061] This application applies to calibration between mechanically rotating multi-line radars and hybrid solid-state multi-line radars, as well as between multi-line radars of the same type. Different center point extraction methods are employed depending on the type of lidar. Optionally, for mechanically rotating lidars: boundary points are extracted by grouping them according to lidar line numbers. Four boundary lines are fitted using the Random Sample Consensus (RANSAC) algorithm. The four corner points of the calibration board are determined by calculating the midpoint of the nearest point between the lines, and the center point is finally obtained. For hybrid solid-state radars: after voxel filtering of the multi-frame fused point cloud, the mean of all points is calculated as the center point.

[0062] S4. Based on the relative positional relationship between the coordinates of the first center point and the coordinates of the second center point, the rigid body transformation matrix between the first lidar and the second lidar is calculated to achieve the external parameter calibration of the two.

[0063] Specifically, the coordinates of the first center point are calculated from the first point cloud data acquired by the first lidar; the coordinates of the second center point are calculated from the second point cloud data acquired by the second lidar. By analyzing the relative positional relationship between these two center points in three-dimensional space, a rigid body transformation model including rotation matrices and translation vectors is constructed. The coordinates of the center points in the first lidar coordinate system are transformed to the second lidar coordinate system, and the error is minimized by comparing them with the actual observed coordinates of the second center point, ultimately obtaining the optimal rigid body transformation matrix. The rigid body transformation matrix describes the extrinsic parameter calibration results of the first lidar relative to the second lidar, including rotation parameters with three degrees of freedom and translation parameters with three degrees of freedom.

[0064] In one embodiment of this invention, during the calibration process, the calibration plate or radar vehicle is moved multiple times to collect at least three sets of non-collinear points. Optionally, more than six sets of detection data are collected for calculation to ensure calibration accuracy.

[0065] By combining multiple sets of observation data from the calibration board in different attitudes, the influence of random errors in a single measurement can be effectively reduced, improving the stability and accuracy of the calibration results. Compared to a single calibration, multiple calibrations, by fully utilizing the spatial geometric constraints of the calibration board in different attitudes, can more comprehensively optimize all six external parameter degrees of freedom, significantly improving calibration accuracy.

[0066] Based on the above embodiments, this application discloses another embodiment of a multi-LiDAR extrinsic parameter calibration method, wherein step S1 involves determining the spatial pose of the calibration board based on point cloud data detected by the first LiDAR and the second LiDAR, respectively. (See attached specification) Figure 2 As shown. Specifically, it includes the following steps.

[0067] S11, perform time-series frame fusion processing on the first point cloud data and the second point cloud data respectively, and generate fused point cloud data by registering and fusing multiple frames of point cloud data in adjacent time series.

[0068] Specifically, a time-window-based frame fusion algorithm is used to perform point cloud registration and data fusion on N temporally continuous frames of point cloud data, outputting a single-frame fused point cloud dataset. Optionally, 10-20 frames of point cloud data with similar frame numbers are fused into one frame to increase the point cloud density.

[0069] S12, filter target point cloud data with point cloud intensity higher than a preset intensity threshold from the fused point cloud data, and use a clustering method to extract the largest point cloud cluster in the target point cloud data.

[0070] Specifically, point cloud intensity is one of the important attributes of point cloud data, representing the intensity of the reflection of a laser pulse after encountering a target object. Normal point cloud intensity ranges from 0 to 255. Due to the reflective stickers on the calibration board, the reflected light intensity of the point cloud illuminating the calibration board is higher than that of the surrounding environment, generally exceeding 150. When automatically locating the calibration board, the fused point cloud is filtered to select those with an intensity greater than 150. Then, Euclidean clustering is performed on the filtered point cloud, and the largest cluster is extracted from each cluster. The center point and yaw angle of the extracted largest cluster are calculated.

[0071] S13, Calculate the three-dimensional centroid of the largest point cloud cluster.

[0072] Specifically, the three-dimensional centroid coordinates of the largest cluster are calculated using the following formula:

[0073] ;

[0074] Where c represents the three-dimensional centroid coordinates; c x c represents the x-coordinate of the three-dimensional centroid. y c represents the y-coordinate of the three-dimensional centroid. z p represents the z-coordinate of the three-dimensional centroid. i The target point in the largest point cloud cluster; This represents the largest point cloud cluster.

[0075] S14, perform plane fitting on the largest point cloud cluster to obtain the fitting plane, and calculate the yaw angle of the fitting plane.

[0076] Specifically, the process of fitting a plane to the point cloud and calculating it is as follows:

[0077] Input the target point cloud cluster: ;

[0078] Where N is the maximum number of points in the point cloud; x i The x-coordinate of the target point cloud point; y-coordinate i The y-coordinate of the target point cloud point; the z-coordinate. i Let z be the z-coordinate of the target point cloud point;

[0079] A temporary plane is calculated using randomly sampled point clouds, with 3 points randomly selected in each iteration. Temporary plane: ; ;

[0080] in, For temporary planes; p a p is the first point cloud point; b For the second point cloud point; p c For the third point cloud point; n tm It is a planar vector.

[0081] Calculate the distance from all point cloud points in the largest point cloud cluster to the temporary plane, and determine the interior points.

[0082] distance: ;

[0083] Interior point determination criteria: ;in, Distance threshold, optional. =0.03m.

[0084] conduct After several iterations, the temporary plane with the most interior points is selected as the final plane equation:

[0085] ;

[0086] in, , , These are the components of the normal vector in the x, y, and z directions, respectively; It is a constant.

[0087] Perform normalization on the normal vector: ;

[0088] in, This is the normalized normal vector; , , These are the projections of the normal vector onto the x, y, and z directions, respectively; projecting the normal vector onto the XY plane: ;

[0089] Normalized projection vector: ;

[0090] Calculate the yaw angle, which is the angle between the normalized projection vector and the X-axis:

[0091] ;

[0092] in, The projection vector; This is the normalized projection vector.

[0093] S15, determine the spatial pose of the calibration plate based on the three-dimensional centroid and the yaw angle.

[0094] Specifically, the three-dimensional centroid coordinates represent the exact position of the calibration plate in space. The yaw angle reflects the rotation angle of the calibration plate about the vertical axis. By combining the three-dimensional centroid and yaw angle, the position and orientation of the calibration plate in three-dimensional space can be fully described.

[0095] In other embodiments of this example, step S1 specifically includes: using the first lidar to detect first point cloud data, and calculating a first prediction result of the spatial pose based on the first point cloud data. Using the second lidar to detect second point cloud data, and calculating a second prediction result of the spatial pose based on the second point cloud data. The outputs of the first prediction result and the second prediction result are based on the first point cloud data and the second point cloud data, respectively, and the specific processing procedures are the same.

[0096] In the above embodiments, the point cloud data refers to the first point cloud data, and the spatial pose refers to the first prediction result of the spatial pose. A detailed process for determining the spatial pose of the calibration board based on the point cloud data is described. Alternatively, in the above embodiments, the point cloud data refers to the second point cloud data, and the spatial pose refers to the second prediction result of the spatial pose. A detailed process for determining the spatial pose of the calibration board based on the point cloud data is described. The specific implementation process will not be elaborated further.

[0097] Based on the above embodiments, this application discloses another embodiment of a multi-lidar extrinsic parameter calibration method, wherein step S2 specifically includes the following sub-steps:

[0098] S21, based on the spatial pose of the calibration board, extract the two-dimensional bounding box of the calibration board.

[0099] S22, in a direction perpendicular to the fitting plane of the calibration plate, the two-dimensional border is extended forward and backward by the same preset distance to obtain a three-dimensional border.

[0100] S23, the spatial range defined by the three-dimensional border is taken as the range of the stereo point cloud.

[0101] Specifically, based on the spatial pose of the calibration board, the first step is to extract a two-dimensional bounding box from the image or point cloud data to ensure that the bounding box accurately reflects the actual boundary of the calibration board. After obtaining the two-dimensional bounding box, the direction perpendicular to the plane is determined by combining the normal vector of the fitted plane of the calibration board. Along this normal direction, the two-dimensional bounding box is extended forward and backward by a preset distance to form a three-dimensional bounding box structure. This is equivalent to stretching the two-dimensional contour of the calibration board in space to generate a cube containing the forward and backward extension range. The spatial range defined by this three-dimensional bounding box is the required three-dimensional point cloud range. All point cloud data within this range will be retained for subsequent processing, while points outside the range will be filtered out.

[0102] By limiting the effective area of ​​point cloud data, interference from irrelevant data can be avoided, thereby improving the efficiency and accuracy of subsequent registration, segmentation, or recognition algorithms.

[0103] Optionally, in other embodiments, the two-dimensional border of the calibration plate is widened outward by a certain distance based on the size of the calibration plate's own boundary. Furthermore, the two-dimensional border is extended by the same predetermined distance forward and backward along a direction perpendicular to the fitting plane of the calibration plate, resulting in a three-dimensional border. (See attached specification.) Figure 3 , 4 As shown in the figure. The 3D shape highlighted in the figure is the 3D border.

[0104] In other embodiments, the defined three-dimensional point cloud range can be spherical, cylindrical, or other three-dimensional shapes. Obviously, those skilled in the art can make various modifications and variations to this application without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims and their equivalents, this application also intends to include these modifications and variations.

[0105] In some embodiments of this example, step S2 specifically includes: defining a first stereo point cloud range based on a first prediction result of the spatial pose; defining a second stereo point cloud range based on a second prediction result of the spatial pose. The definition of the first and second stereo point cloud ranges is based on the first and second prediction results of the spatial pose, respectively. The specific processing procedures are the same.

[0106] The above embodiments use "spatial pose" to refer to the first predicted result of the spatial pose and "stereo point cloud range" to refer to the first stereo point cloud range, and describe the detailed process of delineating the stereo point cloud range. Alternatively, the above embodiments use "spatial pose" to refer to the second predicted result of the spatial pose and "stereo point cloud range" to refer to the second stereo point cloud range, and describe the detailed process of delineating the stereo point cloud range. The specific implementation process will not be elaborated further.

[0107] Based on the above embodiments, this application discloses another embodiment of a multi-LiDAR extrinsic parameter calibration method, wherein step S3: extract the first center point coordinates and the second center point coordinates of the point cloud data corresponding to the first LiDAR and the second LiDAR within the three-dimensional point cloud range, and use different calculation methods according to the different types and characteristics of the LiDAR.

[0108] Optionally, the first lidar and the second lidar are respectively a mechanical rotating lidar and / or a hybrid solid-state lidar.

[0109] In some implementations, the first lidar is a mechanically rotating lidar. The second lidar is a hybrid solid-state lidar.

[0110] Alternatively, in some other embodiments, both the first and second lidars are mechanically rotating lidars.

[0111] Alternatively, in some other embodiments, both the first lidar and the second lidar are hybrid solid-state radars.

[0112] In this embodiment, the method for calculating the center point of a mechanically rotating radar specifically includes the following sub-steps:

[0113] S311, within the range of the three-dimensional point cloud, the point cloud data is grouped according to the laser line number, the boundary points at both ends of each laser line are extracted, and the boundary points are divided into multiple sets of boundary points according to the distance relationship between the points.

[0114] Specifically, the point cloud is grouped according to the laser line number. For each laser line, the boundary points at both ends are extracted and saved separately. Based on the change in distance between points, the boundary points are divided into four groups: Group 1: left side of the upper boundary; Group 2: left side of the lower boundary; Group 3: right side of the lower boundary; Group 4: right side of the upper boundary. Straight lines are fitted to each of the four groups of point clouds using the random sample consensus algorithm.

[0115] S312, a fitting algorithm is used to fit the set of boundary points into a straight line. Based on the property that straight lines intersect pairwise in space, the nearest point between the straight lines is calculated, thereby determining the four corner points of the set of boundary points. The coordinates of the center point are calculated based on the corner points.

[0116] Specifically, since two straight lines in three-dimensional space generally do not intersect completely, the midpoint of the nearest point of the two straight lines is taken as their "intersection point".

[0117] Suppose the two lines are represented by parameters as follows: ;

[0118] Minimum distance parameter satisfy: ;

[0119] Its solution satisfies: ;

[0120] Where P1 is the line L1 and The coordinates of the intersection of the axes; d1 is the slope of line L1, and P2 is the coordinates of the intersection of line L2 and the axis. The coordinates of the intersection of the axes; d2 is the slope of the line L2.

[0121] Find the minimum distance parameter Afterwards, the closest points of the two lines are: ;

[0122] The midpoint of the intersection of two straight lines is taken as the point of intersection. ;

[0123] Calculate the four corner points:

[0124] ;

[0125] Calculate the center point: ;

[0126] in, The first corner point; The second corner point; It is the third corner point; It is the fourth corner point.

[0127] For details, please refer to the attached instruction manual. Figure 3 As shown. Figure 3 This is a schematic diagram of the center point of the computationally rotating multi-line radar in the embodiments of this application; Figure 3 In the diagram, the four purple lines represent the laser lines that hit the calibration plate, the four colored lines represent the four fitted straight lines, and the green dot represents the center point of the calculation.

[0128] In other embodiments, the method for calculating the center point of a hybrid solid-state radar specifically includes the following sub-steps:

[0129] S321, within the range of the stereo point cloud, voxel filtering is performed on the fused point cloud data to reduce the point cloud density.

[0130] S322, calculate the average value of all point cloud points in the filtered point cloud data to obtain the coordinates of the center point.

[0131] Specifically, the point cloud of this type of radar is relatively dense after multi-frame fusion, so only the fused point cloud needs to be processed. Voxel filtering is performed to reduce point cloud density and improve computational efficiency. The filtered point cloud... The coordinates of the center point are obtained by averaging the coordinates of all points.

[0132] ;

[0133] Where N is the number of points in the point cloud. These are the coordinates of the i-th point.

[0134] For details, please refer to the attached instruction manual. Figure 4 As shown. Figure 4 This is a schematic diagram illustrating the calculation of the center point of the hybrid solid-state multi-line radar in an embodiment of this application. Figure 4 In the diagram, the purple line represents the radar line hitting the calibration board, and the red dot in the middle represents the calculated center point.

[0135] Based on the above embodiments, this application discloses another embodiment of a multi-lidar extrinsic parameter calibration method, wherein step S4 specifically includes the following sub-steps:

[0136] S41, Establish the rigid body transformation matrix between the coordinates of the first center point and the coordinates of the second center point.

[0137] Specifically, input two sets of corresponding point cloud data: take the center point of the first lidar A as the source point cloud, and take the center point of the second lidar B as the target point cloud.

[0138] A rigid body transformation estimation method based on Singular Value Decomposition (SVD) is used to solve for a transformation matrix. , including rotation matrix Translation vector This makes the transformed source point cloud as close as possible to the target point cloud.

[0139] Transformation relationship: ;

[0140] The objective is estimated to be minimized as follows: ;

[0141] in, The first point cloud data within the first stereo point cloud range; This refers to the second point cloud data within the range of the second 3D point cloud; R is the transformation matrix; R is the rotation matrix; t is the translation vector.

[0142] S42, Solve the rigid body transformation matrix using the singular value decomposition method.

[0143] S43, obtain the rotation matrix and translation vector between the first coordinate system of the first lidar and the second coordinate system of the second lidar.

[0144] Specifically, calculate the transformation matrix and then calculate the centroids of the two corresponding point clouds:

[0145] ;

[0146] Calculate the central point:

[0147] ;

[0148] Construct the covariance matrix of the corresponding points:

[0149] ;

[0150] in, The first point cloud data within the first stereo point cloud range; This refers to the second point cloud data within the range of the second 3D point cloud; The coordinates of the first center point; The coordinates of the second center point; This is the first point cloud data after centralization; This is the centralized second point cloud data;

[0151] Perform singular value decomposition on H: ;

[0152] Calculate the rotation matrix: ;

[0153] Where U is the left singular vector; V is the right singular vector; V T It is the transpose of the right singular vector.

[0154] like Then correct it: ;

[0155] Calculate the translation vector: ;

[0156] Combined transformation matrix: ;

[0157] Transform the point cloud and calculate the error, then use the transformation matrix to transform the source point cloud to the target point cloud coordinate system:

[0158] ;

[0159] in, The source point cloud coordinate system; Establish the target point cloud coordinate system; The transformation matrix;

[0160] Optionally, in another embodiment of this example, the root mean square error (RMSE) is calculated using the following formula to measure the transformation effect:

[0161] ;

[0162] If the root mean square error is less than the threshold, the calibration is correct, and the calibration result is output. If the root mean square error is greater than the threshold, the calibration is repeated.

[0163] Based on the same concept, this application also discloses a multi-LiDAR extrinsic parameter calibration system. The system is used to implement the steps described in any of the above method embodiments. Specifically, one embodiment of the multi-LiDAR extrinsic parameter calibration system of this application includes:

[0164] The data acquisition module is used to simultaneously illuminate a calibration board with reflective stickers using a first lidar and a second lidar, and determine the spatial pose of the calibration board based on the point cloud data detected by the first lidar and the second lidar, respectively.

[0165] The data processing module is used to define the range of the stereo point cloud based on the spatial pose. It extracts the coordinates of the first and second center points of the point cloud data corresponding to the first and second lidars within the stereo point cloud range.

[0166] The calibration calculation module is used to calculate the rigid body transformation matrix between the first lidar and the second lidar based on the relative positional relationship between the coordinates of the first center point and the coordinates of the second center point, so as to realize the external parameter calibration of the two.

[0167] The multi-laser radar extrinsic parameter calibration method and system of this application have the same technical concept, and the technical details of the embodiments of the two are applicable to each other. In order to reduce repetition, they will not be repeated here.

[0168] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of program modules is merely an example. In practical applications, the above functions can be assigned to different program modules as needed, that is, the internal structure of the device can be divided into different program units or modules to complete all or part of the functions described above. The program modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one processing unit. The integrated unit can be implemented in hardware or as a software program unit. Furthermore, the specific names of the program modules are only for easy differentiation and are not intended to limit the scope of protection of this application.

[0169] Although preferred embodiments of this application have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of this application.

Claims

1. A method for calibrating extrinsic parameters of a multi-laser radar, characterized in that, Includes the following steps: The calibration board with reflective stickers is illuminated simultaneously by a first lidar and a second lidar, and the spatial pose of the calibration board is determined based on the point cloud data detected by the first lidar and the second lidar, respectively. Based on the spatial pose, the range of the three-dimensional point cloud is defined; according to the different types and characteristics of lidar, different calculation methods are used to extract the coordinates of the first center point and the second center point of the point cloud data corresponding to the first lidar and the second lidar within the range of the three-dimensional point cloud. Based on the relative positional relationship between the coordinates of the first center point and the coordinates of the second center point, the rigid body transformation matrix between the first lidar and the second lidar is calculated, thereby realizing the external parameter calibration of the two. Specifically, defining the range of the three-dimensional point cloud based on the spatial pose includes: Based on the spatial pose of the calibration board, the two-dimensional bounding box of the calibration board is extracted; in a direction perpendicular to the fitting plane of the calibration board, the two-dimensional bounding box is extended by the same preset distance in front of and behind the fitting plane to obtain a three-dimensional bounding box. The spatial range defined by the three-dimensional border is taken as the range of the stereo point cloud.

2. The multi-laser radar extrinsic parameter calibration method as described in claim 1, characterized in that, The method of simultaneously illuminating a calibration board with a reflective sticker using a first lidar and a second lidar, and determining the spatial pose of the calibration board based on point cloud data detected by the first lidar and the second lidar respectively, includes: The first point cloud data is obtained by using the first lidar detection, and the first prediction result of the spatial pose is calculated and obtained based on the first point cloud data. The second point cloud data is obtained by using the second lidar, and the second prediction result of the spatial pose is calculated based on the second point cloud data.

3. The multi-laser radar extrinsic parameter calibration method as described in claim 2, characterized in that, Based on the spatial pose, the process involves defining a 3D point cloud range; extracting the coordinates of the first and second center points of the point cloud data corresponding to the first and second lidars within the 3D point cloud range; specifically including: Based on the first prediction result of the spatial pose, the range of the first stereo point cloud is defined; the coordinates of the first center point of the first point cloud data within the range of the first stereo point cloud are extracted. Based on the second prediction result of the spatial pose, the range of the second stereo point cloud is defined; the coordinates of the second center point of the second point cloud data within the range of the second stereo point cloud are extracted.

4. The multi-laser radar extrinsic parameter calibration method as described in claim 2, characterized in that, The determination of the spatial pose of the calibration board based on point cloud data detected by the first lidar and the second lidar respectively includes: The first point cloud data and the second point cloud data are subjected to time-series frame fusion processing respectively. By registering and fusing multiple frames of point cloud data from adjacent time series, fused point cloud data is generated. Target point cloud data with point cloud intensity higher than a preset intensity threshold are selected from the fused point cloud data, and the largest point cloud cluster in the target point cloud data is extracted using a clustering method; Calculate the three-dimensional centroid of the largest point cloud cluster; The largest point cloud cluster is fitted with a plane to obtain a fitted plane, and the yaw angle of the fitted plane is calculated; the spatial pose of the calibration board is determined based on the three-dimensional centroid and the yaw angle.

5. The multi-laser radar extrinsic parameter calibration method as described in claim 1, characterized in that, The extraction of the first and second center point coordinates of the point cloud data corresponding to the first and second lidars within the 3D point cloud range specifically includes: Within the range of the three-dimensional point cloud, the point cloud data is grouped according to the laser line number, the boundary points at both ends of each laser line are extracted, and the boundary points are divided into multiple sets of boundary points according to the distance between the points. The boundary point set is fitted to a straight line using a fitting algorithm. Based on the property that straight lines intersect each other in space, the nearest point between the straight lines is calculated, thereby determining the four corner points of the boundary point set. The coordinates of the center point are then calculated based on the four corner points.

6. The multi-laser radar extrinsic parameter calibration method as described in claim 1, characterized in that, The extraction of the first and second center point coordinates of the point cloud data corresponding to the first and second lidars within the 3D point cloud range specifically includes: Within the range of the 3D point cloud, the point cloud data is subjected to voxel filtering to reduce the point cloud density; The average value of all point cloud points in the filtered point cloud data is calculated to obtain the coordinates of the center point.

7. A method for calibrating extrinsic parameters of a multi-laser radar as described in claim 5 or 6, characterized in that, The first lidar and the second lidar are respectively a mechanical rotating lidar and / or a hybrid solid-state lidar.

8. The multi-laser radar extrinsic parameter calibration method as described in claim 1, characterized in that, The method of calculating the rigid body transformation matrix between the first and second lidars based on the relative positional relationship between the coordinates of the first and second center points, thereby achieving extrinsic parameter calibration of both, specifically includes: Establish a rigid body transformation matrix between the coordinates of the first center point and the coordinates of the second center point; solve the rigid body transformation matrix using the singular value decomposition method; obtain the rotation matrix and translation vector between the first coordinate system of the first lidar and the second coordinate system of the second lidar.

9. A multi-laser radar extrinsic parameter calibration system, characterized in that, The system is used to execute a computer program to implement the steps of the multi-laser radar extrinsic parameter calibration method according to any one of claims 1-8.

Images