Coarse-to-fine target-free external parameter calibration method and system for cross-field-of-view multi-laser system

Abstract

The invention relates to the technical field of laser radars, and provides a coarse-to-fine cross-view-field multi-laser system target-free external parameter calibration method and system, and the method comprises the steps: carrying out the collection and preprocessing of calibration data of a multi-laser system, and obtaining a plurality of reference radar and target radar point cloud pairs with view field overlapping; performing external parameter initialization through an external parameter initialization method based on the microstructure diagram and multi-scale voxel downsampling until a successful convergence criterion is met; performing fusion optimization on the multiple groups of initial external parameters through an efficient fusion optimization method of the multi-source initial external parameters to obtain an optimal initial external parameter matrix; and optimizing the optimal initial external parameter matrix through an external parameter optimization method based on multi-scene point-to-point and point-to-surface residual adaptive weighting to obtain an optimal external parameter. According to the multi-laser external reference calibration method and device, external reference calibration among multiple lasers is accurately completed for a system without view field overlapping or with view field overlapping under the condition that an artificial target and an external reference initial value provided manually do not exist.

Description

Technical Field

[0001] The present invention relates to the technical field of lidar, and in particular, to a method and system for calibrating the extrinsic parameters of a cross-field multi-laser system from coarse to fine without a target. Background Art

[0002] A multi-lidar system can enable a robot or an intelligent vehicle to maximize its environmental perception ability and obtain sufficient high-quality measurement data. The effective operation of a multi-lidar system depends on accurate extrinsic parameter calibration, that is, converting the point cloud data of multiple lidars into a unified coordinate system. Extrinsic parameter calibration not only directly affects the perception accuracy but also is a prerequisite for the system operation. The extrinsic parameter calibration of a multi-lidar system essentially belongs to the problem of point cloud registration, that is, determining the optimal six-degree-of-freedom rigid body transformation between two point clouds, but requires higher accuracy and success rate. Point cloud registration has a wide range of applications, including scene reconstruction, robotic arm grasping, and three-dimensional object detection.

[0003] Most of the existing methods rely on known initial extrinsic parameters, which increases the usage difficulty and is difficult to ensure the accuracy of the initial extrinsic parameters, thus affecting the result of extrinsic parameter optimization. There are usually two implementation approaches for the initialization of multi-lidar extrinsic parameters: direct registration and indirect registration. Direct registration is based on two point clouds, and extracts corresponding point sets through various strategies to solve the transformation relationship; indirect registration is based on the self-motion estimation of the sensor and combines methods such as hand-eye calibration to estimate the relative pose of the sensors. The indirect registration method is not only limited to being used as an initialization method, but essentially relies on motion estimation, so it is inevitably affected by motion estimation errors and requires rich motion excitation, which limits its accuracy, robustness, and practicality. Therefore, it is a problem worthy of research to construct a globally overlapping point cloud for the problem of multi-lidar extrinsic parameter calibration, and improve the success rate and accuracy of the direct registration method to initialize the multi-lidar extrinsic parameters. The existing methods are usually only applicable to systems with overlapping fields of view, or only support the calibration using data of a single scene, and the performance drops severely when the scene constraints are insufficient. In addition, some calibration methods for non-overlapping fields of view optimize the laser pose and extrinsic parameters together, which reduces the robustness of the system; the calibration methods based on targets or bridging devices rely on additional hardware, and the calibration process is cumbersome and inconvenient to use in practice. Although using a target as the object achieves accurate calibration and reduces the computational complexity, it is time-consuming and laborious and inconvenient, and is more challenging when there is no overlap in the fields of view of the lasers, which usually requires a panoramic target to construct a globally visible object. Therefore, the existing multi-lidar extrinsic parameter calibration methods either rely on targets, or only adapt to multi-lidar systems with overlapping fields of view, or only support the calibration using data of a single scene to complete the calibration. How to accurately and robustly complete the extrinsic parameter calibration between multiple lasers at any time and anywhere for systems with or without overlapping fields of view without artificial targets and artificially provided initial extrinsic parameters is a technical problem to be solved urgently. Summary of the Invention

[0004] The present invention aims to solve at least one of the technical problems existing in the related art. To this end, the present invention provides a method and system for calibrating the external parameters of a cross-field multi-laser system from coarse to fine without a target, which can accurately and robustly complete the calibration of the external parameters between multiple lasers at any time and place for systems with or without field-of-view overlap in the absence of artificial targets.

[0005] The present invention provides a method for calibrating the external parameters of a cross-field multi-laser system from coarse to fine without a target, including: S1: By collecting and preprocessing the calibration data of the multi-laser system, multiple pairs of reference radar and target radar point cloud pairs with field-of-view overlap are obtained; S2: By using an external parameter initialization method based on a microstructure diagram and multi-scale voxel downsampling to initialize the external parameters of multiple pairs of reference radar and target radar point cloud pairs with field-of-view overlap, multiple sets of initial external parameters are obtained; S3: By using an efficient fusion and optimization method for multi-source initial external parameters to fuse and optimize multiple sets of initial external parameters, an optimal initial external parameter matrix is obtained; S4: By using an external parameter optimization method based on adaptive weighting of point-to-point and point-to-plane residuals in multiple scenarios to optimize the optimal initial external parameter matrix, the optimal external parameters are obtained.

[0006] Further, in step S1, S11: When there is field-of-view overlap in the multi-laser system, multiple pairs of reference radar and target radar point cloud pairs with field-of-view overlap are obtained through static data collection; S12: When there is no field-of-view overlap in the multi-laser system, by gradually rotating 360° and accumulating data at multiple positions while stationary, taking the pose provided by the laser odometer as the initial value, and using a coarse-to-fine bundle adjustment optimization to generate a global point cloud to construct field-of-view overlap, and through static data collection, reference radar and target radar point cloud pairs with field-of-view overlap are obtained.

[0007] Further, in step S2, for each pair of reference radar and target radar point cloud pairs with field-of-view overlap, the external parameters are initialized by using an external parameter initialization method based on a microstructure diagram and multi-scale voxel downsampling. The external parameter initialization method based on a microstructure diagram and multi-scale voxel downsampling includes: S21: Obtain the initial voxels of the reference radar and target radar point cloud pairs with field-of-view overlap; S22: Construct a microstructure diagram by gradually reducing the voxel scale, and initialize the point cloud pair by using a rough external parameter initialization method based on the microstructure diagram to obtain rough initial external parameters; S23: Perform fine optimization on the basis of the rough initial external parameters by using an iterative closest point algorithm based on adaptive weighting of point-to-point and point-to-plane residuals to obtain initial external parameters to be judged; S24: Determine whether the initial external parameters to be determined converge successfully through the successful convergence criterion. If the convergence fails, reduce the voxel size and repeat steps S22 to S24. If the convergence is successful, obtain the initial external parameters.

[0008] Furthermore, in step S22, the rough initialization of the external parameters based on the micro-structure diagram includes: S221: Perform voxel downsampling on the reference radar and target radar point cloud pairs with field of view overlap based on the octree to obtain the downsampled reference point set and the downsampled target point set. S222: Extract the key points of the downsampled reference point set and the downsampled target point set through the key point extraction algorithm, extract the descriptors corresponding to the key points, and obtain the corresponding set through descriptor matching. S223: Construct a micro-structure diagram according to the corresponding set, and remove the outliers in the micro-structure diagram through a graph-based hierarchical strategy to obtain the maximum consensus set. S224: Solve the rough initial external parameters based on the maximum consensus set.

[0009] Furthermore, in step S23, perform fine optimization on the basis of the rough initial external parameters through the iterative closest point algorithm based on the adaptive weighting of point-to-point and point-to-plane residuals. The optimization objective function is: Where, is the initial external parameter matrix, is the rough initial external parameter matrix, is the number of corresponding point pairs for calculating the residual by applying the point-to-plane error metric, is the number of corresponding point pairs for calculating the residual by applying the point-to-point error metric, is the th point-to-plane residual in the scene, is the th point-to-point residual in the scene, , is the corresponding adaptive weight of the point-to-plane and point-to-point residuals, According to and ratio is adjusted, is the value of the independent variable that makes the function obtain the minimum value, is the square of the Euclidean distance.

[0010] Furthermore, in step S24, the calculation expression of the successful convergence criterion is: Where, is the successful convergence criterion, is the number of matching point pairs, is the minimum threshold of the number of matching point pairs, is the registration average distance, is the maximum threshold of the registration average distance, is the extrinsic translation modulus length, is the maximum threshold of the extrinsic translation modulus length.

[0011] Furthermore, in step S3, for each group of initial external parameters, fusion optimization is performed through an efficient fusion optimization method for multi-source initial external parameters. The efficient fusion optimization method for multi-source initial external parameters includes: Eliminate outliers through the Random Sample Consensus algorithm to obtain the maximum consistency set of initial external parameters; Construct a fusion and optimization objective function for multi-source initial external parameters based on the initial external parameters in the maximum consistency set of initial external parameters; Solve the fusion and optimization objective function for multi-source initial external parameters through the Levenberg-Marquardt method to obtain the optimal initial external parameter matrix.

[0012] Furthermore, the fusion and optimization objective function for multi-source initial external parameters is: where, is the optimal initial external parameter matrix, is the initial external parameter matrix, is the th initial external parameter, is the six-dimensional Lie algebra residual vector, is the Cauchy robust kernel function scale parameter that controls the weight descent rate, is the initial external parameter inlier set, is the pose of the point cloud pair in three-dimensional space, is the value of the independent variable that makes the function obtain the minimum value, is the square of the Euclidean distance.

[0013] Furthermore, in step S4, an optimization objective function for the optimal initial external parameter matrix is constructed through an external parameter optimization method based on adaptive weighting of point-to-point and point-to-plane residuals for multi-scenes. The optimization objective function for the optimal initial external parameter matrix is: where, is the optimal external parameter matrix, is the number of point cloud pairs with overlapping fields of view between the reference radar and the target radar, is the th number of corresponding point pairs applying point-to-plane error in the th scene, is the th number of corresponding point pairs applying point-to-point error in the th scene. is the residual from the th point to the plane in the th scenario, is the residual from the th point to the point in the th scenario, is the adaptive weight of the point-to-point and point-to-plane residuals corresponding to the th scenario, , and is dynamically adjusted according to the and ratio, is the square of the Euclidean distance; Linearize the point-to-plane residual and the point-to-point residual, and solve the optimal extrinsic parameter matrix by the least squares method to obtain the optimal extrinsic parameters.

[0014] The present invention also provides a coarse-to-fine cross-field multi-laser system targetless extrinsic parameter calibration system for performing the above-mentioned coarse-to-fine cross-field multi-laser system targetless extrinsic parameter calibration method, including: A point cloud pair acquisition module, which acquires multiple pairs of reference radar and target radar point cloud pairs with overlapping fields of view by collecting and preprocessing the calibration data of the multi-laser system; An initial extrinsic parameter acquisition module, which initializes the extrinsic parameters of multiple pairs of reference radar and target radar point cloud pairs with overlapping fields of view by an extrinsic parameter initialization method based on a microstructure diagram and multi-scale voxel downsampling to obtain multiple groups of initial extrinsic parameters; An optimal initial extrinsic parameter matrix acquisition module, which fuses and optimizes multiple groups of initial extrinsic parameters by an efficient fusion optimization method for multi-source initial extrinsic parameters to obtain an optimal initial extrinsic parameter matrix; An optimal extrinsic parameter acquisition module, which optimizes the optimal initial extrinsic parameter matrix by an extrinsic parameter optimization method based on the adaptive weighting of point-to-point and point-to-plane residuals in multiple scenarios to obtain the optimal extrinsic parameters.

[0015] One or more of the above technical solutions in the embodiments of the present invention have at least one of the following technical effects: Through the external parameter initialization method based on the microstructure diagram and multi-scale voxel downsampling and the initialization success evaluation criterion, high-precision external parameter initialization is achieved by combining the two; through the efficient fusion optimization method of multi-source initial external parameters, the optimal initial external parameters can be obtained in the case of a large number of outliers in multiple initialization results. Through the external parameter optimization method based on the adaptive weighting of point-to-point and point-to-plane residuals in multiple scenarios, the constraints provided by multi-scenario data can be better utilized, the accuracy and robustness of external parameter calibration can be improved, and the generalization ability of the calibration scenario can be greatly enhanced. Without artificial targets, for systems with or without field-of-view overlap, the external parameters between multiple lasers can be accurately and robustly calibrated anytime and anywhere.

[0016] Additional aspects and advantages of the present invention will be given in part in the following description, become apparent in part from the following description, or be learned through the practice of the present invention. Brief Description of the Drawings

[0017] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the drawings in the following description are some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained based on these drawings.

[0018] Figure 1 It is a schematic flowchart of a method for calibrating the external parameters of a multi-laser system without targets across fields of view from coarse to fine provided by the present invention.

[0019] Figure 2 It is a schematic structural diagram of a system for calibrating the external parameters of a multi-laser system without targets across fields of view from coarse to fine provided by the present invention.

[0020] Reference Signs: 101, point cloud pair acquisition module; 102, initial external parameter acquisition module; 103, optimal initial external parameter matrix acquisition module; 104, optimal external parameter acquisition module. Detailed Embodiments

[0021] To make the objectives, technical solutions, and advantages of the present invention clearer, the technical solutions in the present invention will be clearly and completely described below. Obviously, the described embodiments are some, but not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts belong to the scope of protection of the present invention. The following embodiments are used to illustrate the present invention, but cannot be used to limit the scope of the present invention.

[0022] In the description of this specification, the descriptions referring to terms such as "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. mean that the specific features, structures, materials, or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the embodiments of the present invention. In this specification, the schematic expressions of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described can be combined in a suitable manner in any one or more embodiments or examples. In addition, without contradiction, those skilled in the art can combine and combine the different embodiments or examples described in this specification and the features of different embodiments or examples.

[0023] The following combines Figures 1 to 2 to describe a method and system for calibrating the extrinsic parameters of a cross-field multi-laser system from coarse to fine without a target.

[0024] As Figure 1 shown, a method for calibrating the extrinsic parameters of a cross-field multi-laser system from coarse to fine without a target includes: S1: Through collecting and preprocessing the calibration data of the multi-laser system, multiple pairs of reference radar and target radar point clouds with overlapping fields of view are obtained; For multi-laser radar systems with and without overlapping fields of view, different data collection and preprocessing strategies are designed to construct the overlapping fields of view between different lidars. S11: When there is an overlapping field of view in the multi-laser system, multiple pairs of reference radar and target radar point clouds with overlapping fields of view are obtained through static data collection; S12: When there is no overlapping field of view in the multi-laser system, by gradually rotating 360° and accumulating data at multiple positions while stationary, using the pose provided by the laser odometer as the initial value, and adopting a coarse-to-fine bundle adjustment optimization to generate a high-quality global point cloud to construct the overlapping field of view, and through static data collection, pairs of reference radar and target radar point clouds with overlapping fields of view are obtained; When there is no overlapping field of view in the multi-laser system, there are two methods without a target and without bridging lasers that can complete the extrinsic parameter calibration: Construct the overlapping field of view through motion and then achieve extrinsic parameter calibration through feature matching; Optimize the extrinsic parameters based on motion information and hand-eye calibration. Relatively speaking, the former has a higher upper limit of calibration accuracy.

[0025] Select the lidar with the largest field of view as the reference radar, denoted as , and other lidars as target radars, denoted as ; The goal of multi-laser extrinsic parameter calibration is to solve and The external parameter transformation matrix between; Taking the reference radar as the reference system, denote the th laser point in the target radar coordinate system as , and the th laser point in the reference radar coordinate system as . For each target radar point cloud, the target radar point cloud can be transformed into the reference radar coordinate system through the rough initial external parameter matrix , that is . Denote the horizontal field of view angle of the reference radar as . The calculation expression of the number of times that the platform needs to be stationary when rotating 360° step by step is: Among them, is the scaling factor to ensure that there is enough overlap area between the point clouds accumulated during two consecutive stationary times. Larger is meaningless for accuracy improvement and may increase the cumulative error.

[0026] In some specific embodiments of the present invention, Take 5 10.

[0027] The multi-lidar platform starts to rotate after being stationary for 3 5s. Denote the reference radar coordinate system at this stationary position as the global coordinate system. After that, every time it rotates , it will be stationary for 3 5s to accumulate data to ensure the quality of the point cloud at each position until the platform rotates one week.

[0028] For the reference radar, during the rotation process, record the point cloud data at each stationary time and obtain its real-time pose through the laser odometer. Denote the pose at the th stationary position as , . Among them, is the pose of the point cloud pair in three-dimensional space, is the rotation matrix at the th stationary position, , represents the Special Orthogonal Group in three dimensions, which is the rotation group in three-dimensional space and contains all rotation matrices. These matrices satisfy the conditions of orthogonality and determinant being 1. is a 3D translation vector. The set of all poses , .

[0029] For the target radar, just record the point cloud data at each stationary pose.

[0030] For each frame of point cloud, transform it to the global coordinate system, and extract planes by the graph adaptive voxel method. The point cloud is sliced into voxels of a fixed size. If the points in a voxel are not in a plane, continue to divide it by octree to obtain planes of different sizes.

[0031] Denote the th plane feature parameter as , where is the th plane normal vector, and is an arbitrary point on the th plane, all represented in the global coordinate system. Denote all plane features as , , is the position of the th point in the th plane feature measured at the th static position in the global coordinate system. The calculation expression is: where is the position of the th measurement point in the th plane feature at the th static position in the reference radar coordinate system.

[0032] Since it is impossible to extract good edge points, adding edges will instead reduce the accuracy. Therefore, only plane features are used to represent the scene, and the poses of all static positions are further optimized through efficient and consistent Bundle Adjustment (BA) , improving the consistency of the global map and making the reconstructed global point cloud as consistent as possible with the lidar point cloud measurements. The problem of optimizing the global point cloud map of the reference radar can be formulated as the following BA problem, that is, directly minimizing the Euclidean distance from each original measurement point in the point cloud accumulated from each static position to its corresponding geometric feature (the plane where it is located): where is the pose after BA optimization, is the pose, is the total number of plane features, is the th lidar pose, is the number of lidar points of the th plane observed under all lidar poses, is the total number of points in the point cloud of the is the square of the Euclidean distance from a point to a plane. Denote , where is the covariance matrix obtained by taking the outer product of the deviations of all measurement points (the differences between the points and the average point ), , , is the transpose of the matrix. The above BA problem can be transformed into a problem of minimizing the eigenvalue of the covariance matrix of all points: where is the third largest eigenvalue of the matrix .

[0033] The global optimal is obtained by optimizing and solving this equation through second-order expansion using Newton's second-order method.

[0034] To further improve the robustness of the algorithm to the initial pose of the reference radar obtained by the laser odometer, a coarse-to-fine strategy is adopted for global BA optimization. First, in the rough optimization stage, by using a larger voxel size and loose plane extraction conditions, it is ensured that the layered planes are correctly identified as the same plane and preliminary optimization is carried out. During the optimization process, the plane layering problem is gradually corrected. Since the relatively loose conditions will introduce some incorrect plane features, in the refinement stage, a smaller voxel size and more strict plane feature extraction conditions are used for fine plane optimization to improve the optimization accuracy. Through this coarse-to-fine optimization strategy, not only can the situation with a poor initial pose be effectively handled, but also a higher optimization accuracy can be obtained in the final stage. Specifically, the present invention performs three-level optimization with voxel sizes of 3m, 2m, and 1m to ensure the robustness and accuracy of global BA optimization, thereby obtaining a globally consistent reference radar point cloud. This strategy effectively avoids the coupling problem of optimizing the global point cloud when optimizing the external parameters subsequently.

[0035] Data is collected and preprocessed in multiple scenarios through the above method, and multiple pairs of reference radar-target radar point clouds with overlapping fields of view are obtained.

[0036] When there is a field of view overlap between lidars, static data collection is the best strategy. This approach avoids the effects of motion distortion and time offset between different lasers. By collecting data at multiple scenarios or different positions within the same scenario, the diversity and comprehensiveness of the data can be ensured, thereby enhancing the robustness and accuracy of calibration. For each pair of lidar point cloud data, the voxel downsampling method is used to divide the space into cubic grids, and the point closest to the center of each grid is selected as a representative to ensure that the point cloud data has the same resolution. This method updates the voxel representation by deleting the original points and inserting new points. Although the computational complexity is high, it can better retain the spatial distribution characteristics of the point cloud. The final output point cloud positions are close to the voxel centers, thus maintaining the spatial structure details of the original data. Specifically, downsampling is performed with a voxel size of 1 cm, which can ensure that the point cloud has approximately the same resolution and maintain the density of the point cloud.

[0037] S2: Initialize the extrinsic parameters of multiple pairs of reference radar and target radar point clouds with field of view overlap through the extrinsic parameter initialization method based on the microstructure diagram and multi-scale voxel downsampling to obtain multiple sets of initial extrinsic parameters; For each pair of reference radar and target radar point clouds with field of view overlap, perform extrinsic parameter initialization through the extrinsic parameter initialization method based on the microstructure diagram and multi-scale voxel downsampling. The extrinsic parameter initialization method based on the microstructure diagram and multi-scale voxel downsampling includes: S21: Obtain the initial voxels of the reference radar and target radar point cloud pair with field of view overlap; The voxel size during the solution process can be calculated by the following formula: During the actual optimization process, the initial voxel is set to 0.5 m, and the minimum voxel size is set to 0.05 m. This parameter selection has strong adaptability and is applicable to various scenarios and data.

[0038] S22: Construct a microstructure diagram by gradually reducing the voxel scale, and initialize the point cloud pair through the rough extrinsic parameter initialization method based on the microstructure diagram to obtain rough initial extrinsic parameters; The rough extrinsic parameter initialization based on the microstructure diagram includes: S221: Perform voxel downsampling on the reference radar and target radar point cloud pair with field of view overlap based on the octree to obtain the downsampled reference point set and the downsampled target point set, Perform downsampling on the point cloud with the current voxel size to achieve fast search and indexing, and obtain the downsampled reference point set and the downsampled target point set; Perform downsampling on the original point cloud set with a resolution of based on the octree, and the downsampled reference point set obtained after downsampling and the downsampled target point set .

[0039] S222: Extract the key points of the downsampled reference point set and the downsampled target point set through the key point extraction algorithm, extract the descriptors corresponding to the key points, and obtain the corresponding set through descriptor matching; Extract respectively through the key point extraction algorithm and corresponding key points, and then extract the descriptors corresponding to the key points (Fast point feature histograms for 3D registration, abbreviated as FPFH), and then obtain the corresponding set through descriptor matching .

[0040] S223: Construct a microstructure graph according to the corresponding set, and remove the outliers of the microstructure graph through a graph-based hierarchical strategy to obtain the maximum consensus set; Based on the corresponding set Construct an undirected graph , is a connected graph, where, is the node of the graph, is the edge connecting the nodes, , is the microstructure of the node; the voxel corresponding to the node is considered as the microstructure, representing a tight spatial region. In this way, construct the point cloud map corresponding to the reference radar and the point cloud map corresponding to the target radar ; Due to problems such as noise and external interference in the point cloud data, descriptor matching is prone to errors, resulting in a large number of outliers in the pairing. To improve the accuracy of the pairing, remove the outliers through a graph-based hierarchical strategy to optimize the initial corresponding relationship set and obtain the maximum consensus set.

[0041] S224: Solve the rough initial external parameters based on the maximum consensus set.

[0042] Through the above steps, the rough initial external parameters are obtained. Since only the point cloud data in the maximum consensus set is used, the accuracy of the initial external parameters is limited and needs to be further optimized to improve its accuracy.

[0043] S23: Perform fine optimization on the basis of the rough initial external parameters through the iterative closest point algorithm with adaptive weighting of point-to-point and point-to-plane residuals to obtain the initial external parameters, Perform fine optimization on the basis of the rough initial external parameters through the iterative closest point algorithm with adaptive weighting of point-to-point and point-to-plane residuals, and consider both point-to-point and point-to-plane residuals to improve the accuracy and robustness of the registration. The optimization objective function is: in, is the initial external parameter, is a rough initial external parameter, The number of corresponding pairs of points for computing the residuals using the point-to-plane error metric, The number of corresponding pairs of points for which the residual is computed using the point-to-point error metric, For the scene The residual from a point to the plane is For the scene point-to-point residuals, , are the corresponding point-to-surface and point-to-point residual adaptive weights, according to and to adjust the ratio, The value of the independent variable for which the function achieves its minimum value.

[0044] In order to improve the accuracy of the algorithm, optimization is performed directly on the original point cloud instead of on the downsampled point cloud.

[0045] For multi-laser extrinsic parameter initialization, initialization success (i.e., the deviation of the extrinsic parameter from the true value is within a small range) is the most important, which directly affects the convergence of the subsequent extrinsic parameter optimization. Under the premise of ensuring successful initialization, further improving the accuracy and efficiency of optimization will help accelerate the convergence process of extrinsic parameter optimization. Through the extrinsic parameter initialization method based on microstructure map and multi-scale voxel downsampling and the initialization success evaluation criterion, a high success rate initialization of multi-laser extrinsic parameters is achieved, while taking into account both accuracy and efficiency.

[0046] S24: Determine whether the initial external parameters have converged successfully through the successful convergence criterion. If convergence fails, the voxel is reduced and steps S22 to S24 are repeated; If convergence is successful, the initial external parameters are obtained.

[0047] Since the subsequent optimization process relies on the multi-scenario iterative closest point algorithm to finely adjust the extrinsic parameters, and good initial extrinsic parameters are a prerequisite for ensuring the correct convergence of the optimization algorithm, it is particularly important to determine whether the initialization result of the single-scenario data is successful. A good initialization result can significantly improve the stability and convergence speed of the optimization process, while inappropriate initial extrinsic parameters may lead the optimization process to fall into a local optimal solution or even diverge completely. Therefore, the successful convergence criterion aims to evaluate the reliability and rationality of the initial extrinsic parameters. Specifically, a comprehensive evaluation criterion based on three core indicators is proposed. This criterion not only considers the quality and quantity of the matching points, but also combines the rationality of the extrinsic translation, comprehensively measuring the accuracy and feasibility of the initialization result. This evaluation criterion provides a strong initial guarantee for the extrinsic parameter optimization, ensuring the smooth progress of the optimization process.

[0048] The successful convergence criterion is determined according to the registration score of the reference point cloud and the target point cloud based on the nearest neighbor search , the number of matching point pairs in the maximum consensus set, and the magnitude of the extrinsic translation vector.

[0049] The registration score of the reference point cloud and the target point cloud based on the nearest neighbor search . By calculating the distance between each target radar cloud point and the nearest neighbor point in the reference radar point cloud, and then calculating the average registration distance to evaluate the point cloud registration quality, the calculation expression of the registration score of the reference point cloud and the target point cloud based on the nearest neighbor search is: where is the -th point of the target point cloud, is the nearest neighbor point of the -th point in the reference point cloud, is the number of points in the target point cloud, is the Euclidean distance between the target cloud point and the corresponding nearest neighbor point in the reference point cloud, is the minimum function.

[0050] The number of matching point pairs in the maximum consensus set directly determines the performance of the rough initialization of the extrinsic parameters, and thus determines the convergence of the iterative closest point algorithm. For the initialization of the extrinsic parameters, ensuring a sufficient number and high quality of matching point pairs is the key to ensuring the successful convergence of the iterative closest point algorithm and reaching the global optimal solution. Therefore, the number of matching point pairs is used as an evaluation index for the success of the extrinsic parameter initialization. The number of matching point pairs in the maximum consensus set directly determines the initial value of the optimization, and thus determines the convergence of the iterative closest point algorithm. Therefore, it is also an important index for evaluating the success of the initialization.

[0051] The external rotation parameters between multiple lidar sensors are usually difficult to directly measure, while the magnitude of the external translation is necessarily within a limited range. For autonomous vehicles, the magnitude of the external translation between multiple lidar sensors generally remains within which makes the magnitude of the external translation can be used as one of the effective judgment indicators for successful initialization of the external parameters. If the initialization of the external parameters fails, the external translation will necessarily diverge. Therefore, the outliers of the magnitude of the external translation can be judged as the failure of the external parameter initialization. If the translation magnitude is within a limited small range, it can usually be regarded as the successful initialization of the external parameters, and provide a basis for subsequent rotation and other parameter optimizations, ensuring that the translation component of the external parameter initialization is reasonable and does not deviate far from the actual value.

[0052] The calculation expression of the successful convergence criterion is: where is the successful convergence criterion, is the number of matching point pairs, is the minimum threshold of the number of matching point pairs, is the average registration distance, is the maximum threshold of the average registration distance, is the magnitude of the external translation, is the maximum threshold of the magnitude of the external translation; The joint judgment of the three indicators ensures the registration quality of the reference lidar point cloud and the target lidar point cloud, the accuracy of the rough initial external parameters, and the rationality of the external translation, thus improving the success rate of initialization.

[0053] For each pair of reference lidar - target lidar point clouds, a set of initial external parameters is obtained, S3: Optimize and fuse multiple sets of initial external parameters through an efficient fusion and optimization method for multi-source initial external parameters to obtain the optimal initial external parameter matrix; For each set of initial external parameters, fuse and optimize them through an efficient fusion and optimization method for multi-source initial external parameters, The efficient fusion and optimization method for multi-source initial external parameters includes: Eliminate outliers through the Random Sample Consensus algorithm to obtain the maximum consistency set of the initial external parameters, Construct a fusion and optimization objective function for multi-source initial external parameters based on the initial external parameters in the maximum consistency set of the initial external parameters; Solve the fusion and optimization objective function for multi-source initial external parameters through the Levenberg-Marquardt method to obtain the optimal initial external parameter matrix.

[0054] First, denote the multiple sets of initial external parameters to be fused as where . The initial external parameter matrix is , the problem of fusing and optimizing multi-source initial extrinsic parameters can be formulated as a weighted least squares optimization problem, is the th translation vector, is the th translation vector direction component, is the th translation vector direction component, is the th translation vector direction component, where is the optimal initial extrinsic parameter matrix, is the initial extrinsic parameter matrix, is the th initial extrinsic parameter, is a six-dimensional Lie algebra residual vector, is the Cauchy kernel function scale parameter that controls the weight descent rate; is the set of inliers of the initial extrinsic parameters, is the pose of the point cloud pair in three-dimensional space.

[0055] Before optimization , the Random Sample Consensus (RANSAC) algorithm is used to remove outliers. When the extrinsic parameter initialization fails, usually the external rotation deviates significantly from the true value, which will inevitably lead to the divergence of the extrinsic parameter translation. Based on this, when the extrinsic parameter initialization is successful, the translation components of the extrinsic parameters in each scenario should be basically the same. Therefore, the external translations of each rotation axis are independently modeled, , where is the horizontal axis of the Cartesian coordinate system, is the vertical axis of the Cartesian coordinate system, is the vertical axis of the Cartesian coordinate system as the horizontal axis, and the optimal translation amount is estimated as: where is the median function, and the translation component of the correct extrinsic parameter should fluctuate within a small range around . The screening condition for inliers of the initial extrinsic parameters is: where is the residual threshold, set to 5% of , is the maximum function, is the minimum function, is the th translation amount, indicating that the subsequent conditions are satisfied for all directions.

[0056] The maximum consistency set of the initial extrinsic parameters was obtained through multiple iterations of RANSAC optimization. Then, the Levenberg-Marquardt (LM) method was used to solve the optimization objective function to obtain the optimal initial extrinsic parameter matrix .

[0057] The fusion of multi-source initial extrinsic parameters and the optimization objective function are as follows: where, is the optimal initial extrinsic parameter matrix, is the initial extrinsic parameter matrix, is the th initial extrinsic parameter, is the six-dimensional Lie algebra residual vector, is the Cauchy robust kernel function scale parameter that controls the weight descent rate; is the initial extrinsic parameter inlier screening condition, is the pose of the point cloud pair in three-dimensional space.

[0058] S4: Optimize the optimal initial extrinsic parameter matrix through an extrinsic parameter optimization method based on adaptive weighting of point-to-point and point-to-plane residuals in multiple scenarios to obtain the optimal extrinsic parameters.

[0059] Extrinsic parameter optimization based on adaptive weighting of point-to-point and point-to-plane residuals in multiple scenarios: On the basis of the initial extrinsic parameters, make full use of the multi-scenario data information, apply an environment-robust adaptive weight to calculate the point-to-point and point-to-plane residuals for each single-scenario data, and finally jointly optimize and align multiple pairs of reference radar-target radar point cloud pairs to obtain the optimal extrinsic parameters; Extrinsic Parameter Optimization Based on Adaptive Weighting of Point-to-Point and Point-to-Plane Residuals in Multiple Scenarios: To further improve the accuracy of multi-lidar extrinsic parameter calibration and make full use of multi-scenario data information, an extrinsic parameter optimization method based on adaptive weighting of point-to-point and point-to-plane residuals in multiple scenarios is proposed. Through the complementarity of multi-scenario constraints, and by applying an environment-robust adaptive weight to calculate the point-to-point and point-to-plane residuals for each single-scenario data, finally, a multi-pair of reference radar-target radar point clouds are jointly optimized and aligned to obtain the optimal extrinsic parameters, achieving high adaptability of extrinsic parameter calibration to different environments. Compared with methods based on point-to-point residuals or point-to-plane residuals that rely on a single error metric, it utilizes point-to-point residuals and point-to-plane residuals in a complementary manner; compared with methods that include point-to-point and point-to-plane residual metrics in a probabilistic framework and assume that the environment is locally planar, it solves the significant approximation errors caused by this assumption and thus improves performance; compared with the iterative closest point algorithm based on single-scenario data, it can better utilize the constraints provided by multi-scenario data, improve the accuracy and robustness of extrinsic parameter calibration, and greatly enhance the generalization ability for the calibration scenario. The extrinsic parameter optimization based on adaptive weighting of point-to-point and point-to-plane residuals in multiple scenarios can be reformulated as the following optimization problem: where, is the optimal extrinsic parameter matrix, is the number of point cloud pairs with overlapping fields of view between the reference radar and the target radar, is the th number of corresponding point pairs applying the point-to-plane error in the th scenario, is the th number of corresponding point pairs applying the point-to-point error in the th scenario, is the th point-to-plane residual of the th point in the th scenario, is the th point-to-point residual of the th point in the and ratio for dynamic adjustment; Linearize the point-to-plane residual and the point-to-point residual in the above formula, and then transform it into a linear least squares problem, construct a linear system to iteratively solve the pose transformation until the convergence condition is met to obtain the optimal extrinsic parameters.

[0060] As Figure 2As shown in the figure, a targetless external parameter calibration system for a coarse-to-fine cross-field multi-laser system is used to execute a targetless external parameter calibration method for a coarse-to-fine cross-field multi-laser system, including: The point cloud pair acquisition module 101 obtains multiple pairs of reference radar and target radar point cloud pairs with overlapping fields of view through the collection and preprocessing of the calibration data of the multi-laser system; The initial external parameter acquisition module 102 performs external parameter initialization on multiple pairs of reference radar and target radar point cloud pairs with overlapping fields of view through an external parameter initialization method based on a microstructure diagram and multi-scale voxel downsampling, and obtains multiple groups of initial external parameters; The optimal initial external parameter matrix acquisition module 103 performs fusion optimization on multiple groups of initial external parameters through an efficient fusion optimization method for multi-source initial external parameters, and obtains the optimal initial external parameter matrix; The optimal external parameter acquisition module 104 optimizes the optimal initial external parameter matrix through an external parameter optimization method based on the adaptive weighting of point-to-point and point-to-plane residuals in multiple scenarios, and obtains the optimal external parameters.

[0061] The collaborative work of the above modules realizes high-precision external parameter initialization through an external parameter initialization method based on a microstructure diagram and multi-scale voxel downsampling and an initialization success evaluation criterion, and combines the two. Through an efficient fusion optimization method for multi-source initial external parameters, the optimal initial external parameters can be obtained in the case of a large number of outliers in multiple initialization results. Through an external parameter optimization method based on the adaptive weighting of point-to-point and point-to-plane residuals in multiple scenarios, the constraints provided by multi-scenario data can be better utilized, the accuracy and robustness of external parameter calibration can be improved, and the generalization ability of the calibration scenario can be greatly enhanced.

[0062] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements on some of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for calibrating a multi-laser system across a field of view without a target external parameter from coarse to fine, characterized in that: include: S1: By collecting and preprocessing the calibration data of the multi-laser system, multiple pairs of reference radar and target radar point cloud pairs with overlapping fields of view are obtained; S2: Initialize the external parameters of multiple pairs of reference radar and target radar point cloud pairs with overlapping fields of view by using an external parameter initialization method based on microstructure map and multi-scale voxel downsampling to obtain multiple sets of initial external parameters; S3: Fusion optimization of multiple groups of initial external parameters is performed through an efficient fusion optimization method of multi-source initial external parameters to obtain the optimal initial external parameter matrix; S4: Optimize the optimal initial extrinsic parameter matrix through the extrinsic parameter optimization method based on multi-scenario point-to-point and point-to-surface residual adaptive weighting to obtain the optimal extrinsic parameters.

2. According to claim 1, a method for non-target external parameter calibration of a multi-laser system across a field of view from coarse to fine is characterized in that: In step S1, S11: When the multi-laser system has overlapping fields of view, multiple reference radar and target radar point cloud pairs with overlapping fields of view are obtained through static data collection; S12: When there is no field of view overlap in the multi-laser system, the system gradually rotates 360° and accumulates data at multiple locations. The pose provided by the laser odometer is used as the initial value, and a coarse-to-fine bundle adjustment optimization is adopted to generate a global point cloud to construct a field of view overlap. Through static data collection, the reference radar and target radar point cloud pairs with field of view overlap are obtained.

3. According to claim 1, a method for non-target external parameter calibration of a multi-laser system across a field of view from coarse to fine is characterized in that: In step S2, for each pair of reference radar and target radar point cloud pairs with overlapping fields of view, external parameters are initialized by an external parameter initialization method based on a microstructure map and multi-scale voxel downsampling. The external parameter initialization method based on a microstructure map and multi-scale voxel downsampling includes: S21: Acquire initial voxels of the reference radar and target radar point cloud pairs with overlapping fields of view; S22: construct a microstructure map by gradually reducing the voxel scale, initialize the point cloud pair by a rough initialization method of external parameters based on the microstructure map, and obtain rough initial external parameters; S23: performing fine optimization on the basis of the rough initial external parameters by using an iterative closest point algorithm based on adaptive weighting of point-to-point and point-to-plane residuals to obtain the initial external parameters to be determined; S24: judging whether the initial external parameter to be judged has converged successfully through the successful convergence criterion, If convergence fails, the voxel is reduced and steps S22 to S24 are repeated; If convergence is successful, the initial external parameters are obtained.

4. According to claim 3, a method for non-target external parameter calibration of a multi-laser system across a field of view from coarse to fine is characterized in that: In step S22, the external parameters based on the microstructure image are roughly initialized, including: S221: performing voxel downsampling on the reference radar and target radar point cloud pairs with overlapping fields of view based on the octree to obtain a downsampled reference point set and a downsampled target point set; S222: extracting key points of the downsampled reference point set and the downsampled target point set by using a key point extraction algorithm, extracting descriptors corresponding to the key points, and obtaining corresponding sets by descriptor matching; S223: construct a microstructure graph according to the corresponding set, remove outliers from the microstructure graph through a graph-based hierarchical strategy, and obtain the maximum consensus set; S224: Solve the rough initial external parameters based on the maximum consensus set.

5. According to claim 3, a method for non-target external parameter calibration of a multi-laser system across a field of view from coarse to fine is characterized in that: In step S23, the iterative closest point algorithm based on adaptive weighting of point-to-point and point-to-surface residuals is used to perform fine optimization on the basis of the rough initial external parameters, and the optimization objective function is: in, is the initial external parameter matrix, is the rough initial extrinsic parameter matrix, The number of corresponding pairs of points for computing the residuals using the point-to-plane error metric, The number of corresponding pairs of points for which the residual is computed using the point-to-point error metric, For the scene The residual from a point to the plane is For the scene point-to-point residuals, , are the corresponding point-to-surface and point-to-point residual adaptive weights, according to and to adjust the ratio, The value of the independent variable that makes the function reach the minimum value, is the square of the Euclidean distance.

6. The method for non-target external parameter calibration of a multi-laser system across a field of view from coarse to fine according to claim 3 is characterized in that: In step S24, the calculation expression of the successful convergence criterion is: in, is the successful convergence criterion, is the number of matching point pairs, is the minimum threshold of the number of matching points, is the average registration distance, is the maximum threshold of the average registration distance, is the external translation modulus length, is the maximum threshold of the extrinsic translation modulus.

7. The method for non-target external parameter calibration of a multi-laser system across a field of view from coarse to fine according to claim 1 is characterized in that: In step S3, for each set of initial external parameters, fusion optimization is performed through an efficient fusion optimization method of multi-source initial external parameters. The efficient fusion optimization method of multi-source initial external parameters includes: Outliers are eliminated through the random sample consensus algorithm to obtain the maximum consistent set of initial external parameters; Constructing the fusion and optimization objective function of multi-source initial external parameters according to the initial external parameters in the maximum consistency set of initial external parameters; The Levenberg-Marquardt method is used to solve the fusion of multi-source initial extrinsic parameters and optimize the objective function to obtain the optimal initial extrinsic parameter matrix.

8. The method for non-target external parameter calibration of a multi-laser system across a field of view from coarse to fine according to claim 7 is characterized in that: The fusion and optimization objective function of multi-source initial external parameters is: in, is the optimal initial extrinsic parameter matrix, is the initial external parameter matrix, For the Initial external parameters, is the six-dimensional Lie algebra residual vector, is the scale parameter of the Cauchy robust kernel function that controls the rate at which the weights decrease, is the initial external parameter internal point set, is the pose of the point cloud in three-dimensional space, The value of the independent variable that makes the function reach the minimum value, is the square of the Euclidean distance.

9. The method for non-target external parameter calibration of a multi-laser system across a field of view from coarse to fine according to claim 1 is characterized in that: In step S4, the optimal initial external parameter matrix optimization objective function is constructed by using an external parameter optimization method based on adaptive weighting of multi-scenario point-to-point and point-to-surface residuals. The optimal initial external parameter matrix optimization objective function is: in, is the optimal external parameter matrix, is the number of point cloud pairs with overlapping fields of view between the reference radar and the target radar, For the The number of corresponding point pairs in the scene to which the point-to-plane error is applied, For the The number of corresponding point pairs in the scene to which the point-to-point error is applied, For the In the scene The residual from a point to the plane is For the In the scene point-to-point residuals, For the The point-to-point and point-to-surface residual adaptive weights corresponding to the scene, ,according to and The ratio is adjusted dynamically. is the square of the Euclidean distance; The point-to-surface residuals and point-to-point residuals are linearized, and the optimal extrinsic parameter matrix is ​​solved by the least squares method to obtain the optimal extrinsic parameters.

10. A coarse-to-fine cross-field multi-laser system non-target external parameter calibration system, characterized in that: The method for performing a coarse-to-fine cross-field multi-laser system non-target external parameter calibration method as claimed in any one of claims 1 to 9 comprises: A point cloud pair acquisition module, wherein the point cloud pair acquisition module obtains multiple pairs of reference radar and target radar point cloud pairs with overlapping fields of view by collecting and preprocessing calibration data of multiple laser systems; An initial external parameter acquisition module, wherein the initial external parameter acquisition module performs external parameter initialization on multiple pairs of reference radar and target radar point cloud pairs with overlapping fields of view by using an external parameter initialization method based on a microstructure map and multi-scale voxel downsampling to obtain multiple groups of initial external parameters; An optimal initial external parameter matrix acquisition module, wherein the optimal initial external parameter matrix acquisition module performs fusion optimization on multiple groups of initial external parameters through an efficient fusion optimization method of multi-source initial external parameters to obtain an optimal initial external parameter matrix; The optimal external parameter acquisition module optimizes the optimal initial external parameter matrix by using an external parameter optimization method based on multi-scenario point-to-point and point-to-surface residual adaptive weighting to obtain the optimal external parameters.

Images