A laser radar calibration method and device

Abstract

The present disclosure belongs to the field of autonomous driving technology, and specifically relates to a laser radar calibration method and device. The method comprises: spherically sampling the laser point clouds of the target area respectively acquired by multiple laser radars and representing them into the same vector space, generating Tylenol vectors, and determining the rotation parameters suitable for aligning the laser point clouds based on the distance of the Tylenol vectors; constructing a local map of the laser radar based on the posture of the laser point cloud, obtaining the external parameters of the laser radar based on the local map, optimizing the posture based on the external parameters, and repeating the above steps, wherein the external parameters at the initial moment are determined based on the rotation parameters; when the external parameters still do not converge after executing a preset number of cycles, it is determined that the calibration has failed. If they converge successfully, it is determined whether the target area is a degraded scene. If it is a degraded scene, the target area is replaced and recalibrated. If it is not a degraded scene, it is determined that the calibration is successful.

Description

Technical Field

[0001] The present disclosure belongs to the field of autonomous driving technology, and specifically relates to a laser radar calibration method and device. Background Art

[0002] With the growing demand for enhanced perception system performance, equipping intelligent unmanned systems such as autonomous vehicles and robots with various types of lidar has become a significant trend. These include using traditional mechanical lidar to achieve a 360° horizontal field of view, ensuring a uniform spatial point cloud distribution, or using solid-state lasers to obtain denser point clouds within a limited field of view at a relatively low cost.

[0003] To effectively leverage the advantages of multiple lidars, sensor calibration is required to identify the position transformation between different lidar coordinate systems to align and synchronize the data from each sensor. Existing lidar calibration methods typically employ a coarse-to-fine framework to gradually solve for calibration parameters. This involves first finding appropriate initial values ​​by leveraging hand-eye calibration or scene-specific prior knowledge. Calibration refinement is then performed to further improve the accuracy of the calibration parameters by identifying elements that can be correlated between lidar data, such as similar geometric structures in point clouds or similar motion trajectories.

[0004] However, this coarse-to-fine calibration framework encounters increasing difficulties when calibrating different types of lidar. First, the fields of view of different lidar types vary significantly, and the point cloud distributions generated by different lidars in the same area are also different. The overlap between two lidars may also be very limited, making it difficult to find suitable initial calibration values. In addition, problems such as point cloud noise and environmental degradation make it difficult to find similar geometric structures or similar motion trajectories in the point cloud, making it impossible to perform calibration refinement. Summary of the Invention

[0005] The disclosed embodiments propose a laser radar calibration solution to solve the problem that existing laser radar calibration methods have poor calibration effects on different types of laser radars.

[0006] A first aspect of the embodiments of the present disclosure provides a laser radar calibration method, including:

[0007] Spherically sampling laser point clouds of a target area acquired by multiple laser radars and representing them in the same vector space to generate Tylenol vectors, and determining rotation parameters suitable for aligning the laser point clouds based on distances between the Tylenol vectors, wherein the relative positions of the laser radars remain unchanged;

[0008] Constructing a local map of the laser radar based on the pose of the laser point cloud, obtaining extrinsic parameters of the laser radar based on the local map, optimizing the pose based on the extrinsic parameters, and repeating the above steps, wherein the local map is the accumulation of the laser point cloud pose transformation up to the target time, and the extrinsic parameters at the initial time are determined based on the rotation parameters;

[0009] When the external parameters have not converged after executing a preset number of frames, the calibration is determined to have failed. If convergence is successful, it is determined whether the target area is a degraded scene. If it is a degraded scene, the target area is replaced and recalibrated. If it is not a degraded scene, the calibration is determined to be successful.

[0010] In some embodiments of the present disclosure, performing spherical sampling on the laser point cloud and representing it in the same vector space to generate the Tyranno vector includes:

[0011] Use the Fibonacci grid sampling method to uniformly sample N points on the sphere to obtain a spherical point cloud consisting of N points, each of which covers the same area on the sphere, where N is a preset natural number;

[0012] Projecting each point of the laser point cloud onto the sphere of the spherical point cloud, and searching for the nearest point corresponding to each laser point cloud point in the spherical point cloud through the KD tree of the spherical point cloud;

[0013] The distance of the laser point cloud point corresponding to each point in the spherical point cloud to the laser radar origin is formed into a vector with a length of N in the order of the spherical point cloud points.

[0014] In some embodiments of the present disclosure, determining a rotation parameter suitable for aligning the laser point cloud based on the distance of the Tylenol vector includes:

[0015] When the first laser point cloud acquired by the first laser radar does not overlap with the second laser point cloud acquired by the second laser radar, traverse the rotation space with the first step length until an overlap is detected;

[0016] After detecting the existence of overlap, searching all overlapping first rotations in the rotation space with a second step size, calculating the Tylenol distance between the first laser point cloud and the second laser point cloud based on the Tylenol vector and the overlap state of the first laser point cloud and the second laser point cloud for all the first rotations, and selecting a preset number of first rotations with the smallest Tylenol distance;

[0017] For each of the preset number of first rotations with the smallest Tylenol distance, move along a path with decreasing Tylenol distance with a third step length until a second rotation with the smallest Tylenol distance is found among the first rotations, and use the minimum value of all the second rotations as a rotation parameter suitable for aligning the first laser point cloud and the second laser point cloud, wherein the third step length is not greater than the second step length, and the second step length is not greater than the first step length.

[0018] In some embodiments of the present disclosure, detecting the presence of overlap includes:

[0019] When the Tylenol mask lengths of the first laser point cloud and the second laser point cloud are not zero, it is determined that the first laser point cloud and the second laser point cloud overlap, and the Tylenol mask is:

[0020]

[0021] in, and Represent the Terminology vectors of the first laser point cloud respectively The Tylenol vector of the second laser point cloud The i-th position of , d1 and d2 are preset parameters, where d1 is used to filter distant landmarks in the lidar to reduce the interference caused by translation between lidars, and d2 is adjusted according to the actual distance between lidars;

[0022] The calculating the Tylenol distance between the first laser point cloud and the second laser point cloud based on the Tylenol vector and the overlapping state of the first laser point cloud and the second laser point cloud includes:

[0023]

[0024] Wherein, D(F1, F2) is the Tylenol distance between the first laser point cloud and the second laser point cloud, is the Terminology vector of the first laser point cloud Taino vector with the second laser point cloud The difference at the corresponding position, To use the Tylenol mask M to filter the data involved in the distance calculation, ||·||1 is the 1-norm of the calculation vector, and α is set to 2.

[0025] In some embodiments of the present disclosure, constructing a local map of the laser radar based on the pose of the laser point cloud includes:

[0026] The accumulation of the product of each frame of point cloud data of the laser point cloud and the corresponding posture up to the target time is used as the local map of the laser radar, wherein the posture of the first frame is used as the local map of the first frame.

[0027] In some embodiments of the present disclosure, acquiring the external parameters of the laser radar based on the local map includes:

[0028] The position of the second laser point cloud and the initial values ​​of the external parameters of the first laser radar and the second laser radar are determined based on the calculation formula of the external parameters when the local map distance between the first laser radar and the second laser radar is minimized at the target time. The calculation formula is:

[0029]

[0030] in, t C * are the initial values ​​of the external parameters of the first laser radar and the second laser radar at the target time (time t), are the local maps of the first laser radar and the second laser radar at the target time (t), T is the position of the second laser point cloud, and d(·) represents the distance.

[0031] In some embodiments of the present disclosure, optimizing the posture based on the external parameter includes:

[0032] The pose of the first laser point cloud at the target time is determined based on the point cloud data of the first laser point cloud and the second laser point cloud at the target time and the initial value of the extrinsic parameter at the previous time. The formula is as follows:

[0033]

[0034] in, is the pose of the first laser point cloud at the target time (time t), are the local maps of the first laser radar and the second laser radar at the previous moment ((t-1)), and are the point cloud data of the first laser radar and the second laser radar at the target time (time t), t-1 C * Refers to the initial value of the external parameter at the previous moment ((t-1) moment), where, 0 C * The rotation part uses the rotation parameters, the translation part is set to (0,0,0), T is the position of the second laser point cloud, and d(·) represents the distance.

[0035] In some embodiments of the present disclosure, determining whether the extrinsic parameter converges includes:

[0036] If | t C* t-1 C -1 |<θ c , then the external parameter C at time t converges, where θc is the preset threshold.

[0037] In some embodiments of the present disclosure, determining whether the target area is a degraded scene includes:

[0038] Calculate the Jacobian matrix J of the calibration residual and the information matrix J t J, obtain the maximum eigenvalue λ of the information matrix max With the minimum eigenvalue λ min ,if Then the target area is determined to be a degraded scene, where θ d To preset the threshold, the calculation formula of the Jacobian matrix J is as follows:

[0039]

[0040] in, are the local maps of the first laser radar and the second laser radar at the target time (t), C is the external parameter of the first laser radar and the second laser radar, T is the position of the second laser point cloud, and d(·) represents the distance.

[0041] A second aspect of the embodiments of the present disclosure provides a laser radar calibration device, comprising:

[0042] A rotation initialization module is configured to perform spherical sampling on the laser point clouds of the target area acquired by the multiple laser radars and represent them in the same vector space, generate Tylenol vectors, and determine rotation parameters suitable for aligning the laser point clouds based on the distance between the Tylenol vectors, wherein the relative positions of the laser radars remain unchanged;

[0043] a joint calibration module, configured to construct a local map of the laser radar based on the pose of the laser point cloud, obtain extrinsic parameters of the laser radar based on the local map, and optimize the pose based on the extrinsic parameters, repeating the above steps, wherein the local map is the accumulation of the pose transformation of the laser point cloud up to the target time;

[0044] The evaluation module is configured to determine that the calibration has failed if the external parameters have not converged after executing a preset number of frames. If convergence is successful, the evaluation module determines whether the target area is a degraded scene. If so, the evaluation module changes the target area and recalibrates. If not, the evaluation module determines that the calibration has succeeded.

[0045] In summary, the lidar calibration methods and devices provided by various embodiments of the present disclosure, by proposing a spherical TERRA descriptor that is sensitive to point cloud rotation and designing a reasonable distance function and rotation search method, can initialize the rotation of two point clouds without any prior knowledge of lidar placement or point cloud characteristics. This solves the problem of initialization failure due to the inherent properties of the lidar point cloud. Furthermore, by simultaneously solving for both pose and extrinsic parameters, the problem of low calibration parameter accuracy caused by point cloud noise and environmental degradation is overcome. Furthermore, the reliability of the calibration parameters is evaluated through quantitative evaluation methods. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] The features and advantages of the present disclosure will be more clearly understood by referring to the accompanying drawings, which are schematic and should not be construed as limiting the present disclosure in any way. In the accompanying drawings:

[0047] Figure 1 is a schematic diagram of a computer system to which the present disclosure is applicable;

[0048] Figure 2 This is laser point cloud data obtained based on a 32-line laser radar and an AVIA laser radar for the same target area in one embodiment of the present disclosure;

[0049] Figure 3 is a flowchart of a laser radar calibration method according to some embodiments of the present disclosure;

[0050] Figure 4 This is an example of generating TERRA descriptors for a 32-line laser point cloud and an AVIA laser point cloud, respectively, according to one embodiment of the present disclosure;

[0051] Figure 5 According to one embodiment of the present disclosure, by rotating TERRA, searching for alignment Figure 2 Examples of optimal rotations for the 32-line laser point cloud and the AVIA laser point cloud shown;

[0052] Figure 6 According to an embodiment of the present disclosure Figure 2 Schematic diagram of iterative pose-extrinsic parameter joint estimation of 32-line laser point cloud and AVIA laser point cloud;

[0053] Figure 7 According to an embodiment of the present disclosure Figure 2 The diagram shows the effect of aligning one frame of 32-line laser point cloud data with one frame of AVIA laser point cloud data.

[0054] Figure 8 is based on Figure 3The calibration method shown performs more calibration tasks and displays the calibration results;

[0055] Figure 9 This is a schematic diagram of a laser radar calibration device according to some embodiments of the present disclosure. DETAILED DESCRIPTION

[0056] In the detailed description that follows, many specific details of the present disclosure are set forth by way of example in order to provide a thorough understanding of the relevant disclosure. However, it will be apparent to one of ordinary skill in the art that the present disclosure can be implemented without these details. It should be understood that the use of the terms "system," "device," "unit," and / or "module" in the present disclosure is a method for distinguishing between different parts, elements, parts, or assemblies at different levels in a sequential arrangement. However, these terms may be replaced by other expressions if they can achieve the same purpose.

[0057] It should be understood that when a device, unit, or module is referred to as being "on," "connected to," or "coupled to" another device, unit, or module, it may be directly on, connected to, coupled to, or in communication with the other device, unit, or module, or there may be intervening devices, units, or modules, unless the context clearly indicates an exception. For example, the term "and / or" as used in this disclosure includes any and all combinations of one or more of the associated listed items.

[0058] The terms used in this disclosure are only for describing specific embodiments and are not intended to limit the scope of this disclosure. As shown in the specification and claims of this disclosure, unless the context clearly indicates an exception, the words "a", "an", "a kind" and / or "the" do not specifically refer to the singular and may also include the plural. Generally speaking, the terms "include" and "comprise" only indicate the inclusion of clearly identified features, wholes, steps, operations, elements and / or components, and such expressions do not constitute an exclusive list, and other features, wholes, steps, operations, elements and / or components may also be included.

[0059] These and other features and characteristics of the present disclosure, as well as the methods of operation, the functions of the related elements of the structure, the combination of parts, and the economy of manufacture may be better understood with reference to the following description and accompanying drawings, which form a part of this specification. However, it is to be expressly understood that the drawings are for illustration and description purposes only and are not intended to limit the scope of protection of the present disclosure. It is to be understood that the drawings are not drawn to scale.

[0060] Various structural diagrams are used in this disclosure to illustrate various variations of the embodiments of the present disclosure. It should be understood that the preceding or following structures are not intended to limit the present disclosure. The scope of protection of the present disclosure is subject to the claims.

[0061] Figure 1 is a schematic diagram of a computer system to which the present disclosure is applicable. Figure 1 The computer system shown includes a calibration server connected to multiple laser radar data, and the multiple laser radars respectively obtain point cloud data of the target area. The calibration server calibrates the laser radar based on the point cloud data.

[0062] The laser radar can be the same type of laser radar that obtains the same type of point cloud data, or it can be different types of laser radar that obtain different types of point cloud data, such as a mechanical laser radar for obtaining a 360° horizontal viewing angle to ensure a uniform spatial point cloud distribution or a solid-state laser radar for obtaining a denser point cloud within a limited field of view. In some embodiments of the present disclosure, a plurality of the laser radars are deployed together on an autonomous driving vehicle to respectively obtain different types of point cloud data of the target area, and perform data fusion to utilize the unique advantages of different types of laser radars to perform autonomous driving tasks. When two or more laser radars are deployed together on an autonomous driving vehicle, generally speaking, the relative positions between the laser radars remain unchanged.

[0063] The calibration server calibrates the laser radar based on point cloud data of the target area acquired by different laser radars. The calibration server can be any of a stand-alone, cluster or distributed server.

[0064] LiDAR calibration refers to the process of finding the position transformation relationship between different LiDAR coordinate systems. Only after calibration can the point cloud data acquired by multiple LiDARs be aligned and synchronized.

[0065] Existing lidar calibration methods typically employ a coarse-to-fine framework to gradually solve for calibration parameters. Finding suitable initial values ​​is a prerequisite for obtaining accurate parameters, which can be accomplished by leveraging hand-eye calibration or prior knowledge of specific scenarios. Following the initialization step, a calibration refinement process is typically performed to further improve the accuracy of the calibration parameters. Specifically, the basic principle of the refinement step is to identify elements that can be associated between lidar data, such as similar geometric structures in the point cloud or their similar motion trajectories. Therefore, the refinement step is often modeled as a point cloud registration problem or a trajectory alignment problem.

[0066] Although multi-lidar calibration has been extensively studied, calibration between different types of lidars is becoming increasingly challenging. First, the fields of view of different lidars vary greatly. For example, the field of view of Livox's Tele is 14.5°*16.2°, while the field of view of a mechanical lidar is 360°*30°. In addition, due to the different scanning methods of lidars, the point cloud distribution generated by different lidars in the same area is also different. Finally, because the main motivation for using multiple lidars is to expand the field of view of unmanned intelligent systems, the overlap between two lidars may be limited, Figure 2 This is the laser point cloud data obtained for the same target area using a 32-line laser radar and an AVIA laser radar. The distribution and density of the laser point clouds obtained by the different types of laser radars vary significantly. This diverse distribution and density of the point clouds makes it challenging to develop a universal initialization method that can consistently generate qualified initial values ​​for any two laser radars. Furthermore, issues such as point cloud noise and environmental degradation make it difficult to find similar geometric structures or similar motion trajectories within the point clouds, making it impossible to perform calibration refinement.

[0067] To effectively address the aforementioned challenges, this disclosure proposes a calibration method applicable to multiple lidars of any type. Specifically, it solves the extrinsic parameters between two lidars, including translation and rotation. To achieve this, initialization is crucial. On a single autonomous vehicle, the translation distance between lidars is limited by vehicle size. This disclosure emphasizes the importance of initializing rotation parameters rather than translation parameters. This disclosure introduces a point cloud descriptor for rotation initialization, named "TERRA," which robustly aligns the three-degree-of-freedom rotations between lidar point clouds. By carefully designing the distance function between multiple TERRAs, the effects of translation on rotation estimation are overcome. After initialization, to address the poor registration accuracy caused by point cloud sparseness and scene degradation, this disclosure proposes a method for jointly estimating lidar extrinsic parameters and the unmanned system's pose. This method integrates geometric and motion information to obtain optimized, usable extrinsic parameters.

[0068] Figure 3 is a flow chart of a laser radar calibration method according to some embodiments of the present disclosure. In some embodiments, the laser radar calibration method is composed of Figure 1 The calibration server shown in the figure executes the laser radar calibration method, which includes the following steps:

[0069] S310, performing spherical sampling on the laser point clouds of the target area respectively acquired by multiple laser radars and representing them into the same vector space, generating Tylenol vectors, and determining rotation parameters suitable for aligning the laser point clouds based on the distance of the Tylenol vectors, wherein the relative positions of the laser radars remain unchanged.

[0070] First, generate the Terminology vector for the laser point cloud:

[0071] Use the Fibonacci grid sampling method to uniformly sample N points on the sphere to obtain a spherical point cloud consisting of N points Each point covers the same area on the sphere. Then, use Encode any LiDAR point cloud frame: First, normalize the LiDAR point cloud, that is, project each point onto On the unit sphere of the point cloud; then by KD-Tree, search for each lidar point in The nearest point in . The closest distance from the lidar origin to the radar point corresponding to each point in the image is saved, and finally the distance is calculated according to The order of the point clouds forms a vector of length N.

[0072] Figure 4 In one embodiment of the present disclosure, Figure 2 The following are examples of generating TERRA descriptors using a 32-line laser point cloud obtained by a 32-line laser radar and an AVIA laser point cloud obtained by an AVIA laser radar.

[0073] After calculating the TERRA descriptor for each LiDAR point cloud, a rotation operation needs to be designed to rotate the TERRA to find the optimal rotation that can align the two TERRAs. The specific operation of rotating the TERRA is as follows: The rotation point cloud obtained by rotating a certain rotation R is expressed as use KD-Tree in Rotated Point Cloud and original A one-to-one correspondence is established between pairs of points in . In this case, each pair consists of two points with different indices. A rotated TERRA is created by taking values ​​from the original TERRA index and filling them into the index positions of the corresponding points.

[0074] By evaluating the distance between TERRA, the optimal rotation to align the two TERRA can be found. A small distance between TERRA indicates that the point cloud projected onto the sphere also has the correct correspondence, which means that the correct rotation has been identified. Specifically, the present disclosure defines a mask To indicate which positions are suitable for two and Distance calculation. m i Defined as:

[0075]

[0076] in, and Represents the TERRA vector of the first laser point cloud TERRA vector with the second laser point cloud The i-th position of the LiDAR is selected, and d1 and d2 are preset parameters. d1 is used to filter distant landmarks from the LiDARs to reduce interference caused by translation between LiDARs. In practice, d1 is set to 20 meters. d2 is used to select locations that may belong to the same object for subsequent calculations. d2 can be adjusted based on the actual distance between LiDARs. In some embodiments of the present disclosure, it is set to 5 meters based on the typical size of a vehicle. Finally, the distance between two TERRAs is defined as:

[0077]

[0078] Where D(F1, F2) is the TERRA distance between the first laser point cloud and the second laser point cloud, is the TERRA vector of the first laser point cloud TERRA vector with the second laser point cloud The difference at the corresponding position, To use the TERRA mask M to filter the data involved in the distance calculation, ||·||1 is the 1-norm of the calculation vector, and α is set to 2.

[0079] Then search for the initial rotation value suitable for aligning the laser point cloud based on the TERRA distance:

[0080] Because the rotation space is too large, it takes an unacceptable amount of time to traverse all feasible rotations. Therefore, the present disclosure first checks whether the initial rotation will cause an overlap between the two point clouds based on whether the length of the mask M is not zero. If there is no overlap, the rotation space will be traversed with a relatively large step size (for example, 30°) until a suitable initial rotation that produces an overlapping area is found. This step is very time-saving. Starting from this initial rotation, the surrounding rotation space is explored with a large step size (for example, 10°), sampling in all rotations that result in overlapping areas, and scoring each rotation using a distance formula. The top ten rotations with the smallest distance are input to the next stage. In the second stage, based on the first stage, a heuristic search is used to find improved solutions near each previously identified discrete rotation. Specifically, in each area, exploration is performed with a step size of 1°, moving along a path with decreasing distance calculated by the distance formula until the local optimal rotation parameter is reached.

[0081] Figure 5 According to one embodiment of the present disclosure, by rotating TERRA, searching for alignment Figure 2 Examples of optimal rotations for the 32-line laser point cloud and the AVIA laser point cloud are shown.

[0082] S320, constructing a local map of the laser radar based on the posture of the laser point cloud, obtaining external parameters of the laser radar based on the local map, optimizing the posture based on the external parameters, and repeating the above steps, wherein the local map is the accumulation of the laser point cloud posture transformation up to the target moment, and the external parameters at the initial moment are determined based on the rotation parameters.

[0083] For the two lidars to be calibrated, only their point cloud frames at the same time are used. and The external parameters can be solved through registration.

[0084] However, since only a single frame of data provides constraints for estimating extrinsic parameters, the effects of point cloud noise and scene degradation become significant. To mitigate these shortcomings, this paper integrates knowledge from hand-eye calibration and uses a sequence of point cloud frames from two lidars to obtain extrinsic parameters with optimized rotation and translation.

[0085] The present disclosure iteratively executes the extrinsic parameter estimation task and the pose optimization task, and ultimately achieves the joint estimation of extrinsic parameters and pose.

[0086] First, build a local map of the lidar based on the pose of the laser point cloud:

[0087] The local map obtained by the LiDAR refers to the accumulation of the product of each frame of the laser point cloud data and the corresponding pose up to the target time. That is:

[0088]

[0089] in and are the local maps of the first laser radar and the second laser radar at the target time (t), t P a and t P b are the poses of the first laser point cloud and the second laser point cloud at the target time (t), and The first laser point cloud and the second laser point cloud are point cloud data of the target area acquired by the first laser radar and the second laser radar at the target time (t).

[0090] Then the external parameters are estimated based on the local map:

[0091] In the external parameter estimation stage, the present disclosure uses the local map formed by the first laser and the second laser radar to solve the external parameters, and the objective function used is as follows:

[0092]

[0093] in, t C * is the initial value of the external parameters of the first laser radar and the second laser radar at the target time (time t), are the local maps of the first and second laser radars at the target time (t), T is the position of the second laser point cloud, and d(·) represents the distance.

[0094] In this way, the position of the second laser point cloud and the initial values ​​of the external parameters of the first laser radar and the second laser radar at the target time when the local map distance between the first laser radar and the second laser radar is minimum can be determined.

[0095] Then, based on the point cloud data of the first laser point cloud and the second laser point cloud at the target moment and the initial value of the external parameter at the previous moment, the position and posture of the first laser point cloud at the target moment are optimized:

[0096]

[0097] in, is the pose of the first laser point cloud at the target time (time t), are the local maps of the first laser radar and the second laser radar at the previous moment ((t-1)), and are the point cloud data of the first laser radar and the second laser radar at the target time (time t), t-1 C * Refers to the initial value of the external parameter at the previous moment ((t-1) moment), where, 0 C * The rotation part uses the rotation parameters obtained by S310, the translation part is set to (0,0,0), d(·) represents the distance, and T is the position of the second laser point cloud.

[0098] Iterate the two processes of external parameter estimation and pose optimization until t C * and Stable and convergent. In this process, when t C * Inaccurate due to The position is incorrect. The estimates are also difficult to converge.

[0099] Since the present disclosure uses a map for the external parameter calibration task, it introduces more data to provide more sufficient constraints for the registration problem, making the registration result more accurate than using only a single frame.

[0100] Figure 6 According to an embodiment of the present disclosure Figure 2The diagram shows the iterative joint estimation of pose and external parameters of 32-line laser point cloud and AVIA laser point cloud. The alignment effect is as follows Figure 7 shown.

[0101] S330: When the external parameters have not converged after executing a preset number of loops, the calibration is determined to have failed. If convergence is successful, it is determined whether the target area is a degraded scene. If so, the target area is replaced and recalibrated. If not, the calibration is determined to have succeeded.

[0102] The evaluation of the results is divided into two parts: the first is the evaluation of the convergence degree of the calibration process, and the second is the evaluation of the reliability of the calibration environment on the calibration results. Specifically, when estimating the external parameters, when | t C* t-1 C -1 |<θ c (θ c When t is the specified threshold, the external parameter C at time t is considered to have converged and the calibration process can be stopped. On the other hand, for the external parameter estimation, the Jacobian matrix J of the calibration residual is calculated as follows:

[0103]

[0104] And calculate the information matrix J t J. Note λ max ,λ min are the maximum and minimum eigenvalues ​​of the information matrix respectively, if This indicates that the environment is a degraded scenario and the calibration environment needs to be replaced and the complete calibration process needs to be repeated. In actual use, based on experience, θ c Set to 1 mm, θ d Set to 2.

[0105] We conduct extensive experiments on eight autonomous vehicle sensor setups, testing a total of 16 different types of lidars and dozens of calibration tasks, such as Figure 8 In various challenging tasks, the calibration error can be controlled within 5cm and 1°.

[0106] Figure 9 FIG. 1 is a schematic diagram of a laser radar calibration device according to some embodiments of the present disclosure. Figure 9 As shown, the laser radar calibration device 900 includes a rotation initialization module 910, a joint calibration module 920, and an evaluation module 930. In some embodiments of the present disclosure, the laser radar calibration function is performed by Figure 1 The calibration server is executed as shown.

[0107] A rotation initialization module 910 is configured to perform spherical sampling on the laser point clouds of the target area acquired by the plurality of laser radars and represent them in the same vector space, generate Tylenol vectors, and determine rotation parameters suitable for aligning the laser point clouds based on the distance between the Tylenol vectors, wherein the relative positions of the laser radars remain unchanged;

[0108] A joint calibration module 920 is configured to construct a local map of the laser radar based on the pose of the laser point cloud, obtain extrinsic parameters of the laser radar based on the local map, and optimize the pose based on the extrinsic parameters, repeating the above steps, wherein the local map is the accumulation of the pose transformation of the laser point cloud up to the target time;

[0109] Evaluation module 930 is used to determine that calibration has failed if the external parameters have not converged after executing a preset number of frames. If convergence is successful, it is used to determine whether the target area is a degraded scene. If so, the target area is replaced and recalibrated. If not, the calibration is determined to be successful.

[0110] In summary, the lidar calibration methods and devices provided by various embodiments of the present disclosure, by proposing a spherical TERRA descriptor that is sensitive to point cloud rotation and designing a reasonable distance function and rotation search method, can initialize the rotation of two point clouds without any prior knowledge of the lidar placement or point cloud characteristics, thus resolving the issue of initialization failure due to the inherent properties of the lidar point cloud. Furthermore, by simultaneously solving for both pose and extrinsic parameters, the problem of low calibration parameter accuracy due to point cloud noise and environmental degradation is overcome. Furthermore, the reliability of the calibration parameters is evaluated through quantitative evaluation methods.

[0111] Although the subject matter described herein is provided in the general context of being executed in conjunction with the execution of an operating system and application programs on a computer system, those skilled in the art will recognize that other implementations may also be performed in conjunction with other types of program modules. Generally speaking, program modules include routines, programs, components, data structures, and other types of structures that perform specific tasks or implement specific abstract data types. Those skilled in the art will appreciate that the subject matter described herein may be practiced using other computer system configurations, including handheld devices, multiprocessor systems, microprocessor-based or programmable consumer electronics, minicomputers, mainframe computers, and the like, and may also be used in distributed computing environments in which tasks are performed by remote processing devices connected via a communication network. In a distributed computing environment, program modules may be located in both local and remote memory storage devices.

[0112] Those skilled in the art will appreciate that the units and method steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this disclosure.

[0113] It should be understood that the above-described specific embodiments of the present disclosure are merely illustrative of or explanation of the principles of the present disclosure and do not constitute limitations on the present disclosure. Therefore, any modifications, equivalent substitutions, improvements, etc. made without departing from the spirit and scope of the present disclosure shall be included within the scope of protection of the present disclosure. In addition, the claims appended to the present disclosure are intended to cover all variations and modifications that fall within the scope and metes and bounds of the appended claims, or equivalents of such scope and metes and bounds.

Claims

1. A laser radar calibration method, characterized in that: include: Spherically sampling laser point clouds of a target area acquired by multiple laser radars and representing them in the same vector space to generate Tylenol vectors, and determining rotation parameters suitable for aligning the laser point clouds based on distances between the Tylenol vectors, wherein the relative positions of the laser radars remain unchanged; Constructing a local map of the laser radar based on the pose of the laser point cloud, obtaining extrinsic parameters of the laser radar based on the local map, optimizing the pose based on the extrinsic parameters, and repeating the above steps, wherein the local map is the accumulation of the laser point cloud pose transformation up to the target time, and the extrinsic parameters at the initial time are determined based on the rotation parameters; If the extrinsic parameters still do not converge after executing a preset number of loops, the calibration is determined to have failed. If convergence is successful, it is determined whether the target area is a degraded scene. If so, the target area is changed and recalibrated. If not, the calibration is determined to have succeeded. Among them, the laser point cloud is spherically sampled and represented in the same vector space to generate the Tylenol vector, which includes: Use the Fibonacci grid sampling method to uniformly sample N points on the sphere to obtain a spherical point cloud consisting of N points, each of which covers the same area on the sphere, where N is a preset natural number; Projecting each point of the laser point cloud onto the sphere of the spherical point cloud, and searching for the nearest point corresponding to each laser point cloud point in the spherical point cloud through the KD tree of the spherical point cloud; The distance of the laser point cloud point corresponding to each point in the spherical point cloud to the laser radar origin is formed into a vector with a length of N in the order of the spherical point cloud points.

2. The method according to claim 1, characterized in that Determining the rotation parameters suitable for aligning the laser point cloud based on the distance of the Tylenol vector includes: When the first laser point cloud acquired by the first laser radar does not overlap with the second laser point cloud acquired by the second laser radar, traverse the rotation space with the first step length until an overlap is detected; After detecting the existence of overlap, searching all overlapping first rotations in the rotation space with a second step size, calculating the Tylenol distance between the first laser point cloud and the second laser point cloud based on the Tylenol vector and the overlap state of the first laser point cloud and the second laser point cloud for all the first rotations, and selecting a preset number of first rotations with the smallest Tylenol distance; For each of the preset number of first rotations with the smallest Tylenol distance, move along a path with decreasing Tylenol distance with a third step length until a second rotation with the smallest Tylenol distance is found among the first rotations, and use the minimum value of all the second rotations as a rotation parameter suitable for aligning the first laser point cloud and the second laser point cloud, wherein the third step length is not greater than the second step length, and the second step length is not greater than the first step length.

3. The method according to claim 2, characterized in that The detecting that there is overlap includes: When the Tylenol mask lengths of the first laser point cloud and the second laser point cloud are not zero, it is determined that the first laser point cloud and the second laser point cloud overlap, and the Tylenol mask is: ， in, and Represent the Terminology vectors of the first laser point cloud respectively The Tylenol vector of the second laser point cloud The i-th position of and are preset parameters, where Used to filter distant landmarks in lidar to reduce interference caused by translation between lidars. Adjust based on the actual distance between the lidars; The calculating the Tylenol distance between the first laser point cloud and the second laser point cloud based on the Tylenol vector and the overlapping state of the first laser point cloud and the second laser point cloud includes: in, is the Tylenol distance between the first laser point cloud and the second laser point cloud, is the Terminology vector of the first laser point cloud Taino vector with the second laser point cloud The difference at the corresponding position, For use with Tylenol mask Filter the data involved in distance calculation, To calculate the 1-norm of a vector, Set to 2.

4. The method according to claim 3, wherein: The step of constructing a local map of the laser radar based on the position of the laser point cloud includes: The accumulation of the product of each frame of point cloud data of the laser point cloud and the corresponding posture up to the target time is used as the local map of the laser radar, wherein the posture of the first frame is used as the local map of the first frame.

5. The method according to claim 4, characterized in that: The acquiring of the external parameters of the laser radar based on the local map includes: The position of the second laser point cloud and the initial values ​​of the external parameters of the first laser radar and the second laser radar are determined based on the calculation formula of the external parameters when the local map distance between the first laser radar and the second laser radar is minimized at the target time. The calculation formula is: in, are the initial values ​​of the external parameters of the first laser radar and the second laser radar at the target time, , are the local maps of the first laser radar and the second laser radar at the target moment, is the pose of the second laser point cloud, d ( ) indicates distance.

6. The method according to claim 5, characterized in that Optimizing the posture based on the external parameter includes: The pose of the first laser point cloud at the target time is determined based on the point cloud data of the first laser point cloud and the second laser point cloud at the target time and the initial value of the extrinsic parameter at the previous time. The formula is as follows: in, is the pose of the first laser point cloud at the target moment, 、 are the local maps of the first laser radar and the second laser radar at the previous moment, and are the point cloud data of the first laser radar and the second laser radar at the target moment, Refers to the initial value of the external parameter at the previous moment, where The rotation part uses the rotation parameters, and the translation part is set to (0,0,0). is the pose of the second laser point cloud, d ( ) indicates distance.

7. The method according to claim 6, characterized in that Determining whether the external parameter converges includes: if , then the external parameter C at time t converges, where is the preset threshold.

8. The method according to claim 7, characterized in that: The determining whether the target area is a degraded scene includes: Calculate the Jacobian matrix of the calibration residual , and the information matrix , obtain the maximum eigenvalue of the information matrix With the minimum eigenvalue ,if The target area is determined to be a degraded scene, wherein: is the preset threshold, the Jacobian matrix The calculation formula is as follows: in, , are the local maps of the first laser radar and the second laser radar at the target time, C is the external parameter of the first laser radar and the second laser radar, is the pose of the second laser point cloud, d ( ) indicates distance.

9. A laser radar calibration device, characterized in that: include: A rotation initialization module is configured to perform spherical sampling on the laser point clouds of the target area acquired by the multiple laser radars and represent them in the same vector space, generate Tylenol vectors, and determine rotation parameters suitable for aligning the laser point clouds based on the distance between the Tylenol vectors, wherein the relative positions of the laser radars remain unchanged; a joint calibration module, configured to construct a local map of the laser radar based on the pose of the laser point cloud, obtain extrinsic parameters of the laser radar based on the local map, and optimize the pose based on the extrinsic parameters, repeating the above steps, wherein the local map is the accumulation of the pose transformation of the laser point cloud up to the target time; an evaluation module, configured to determine that the calibration has failed if the extrinsic parameters have not converged after executing a preset number of frames; and if converged successfully, to determine whether the target area is a degraded scene; if so, to change the target area and recalibrate; and if not, to determine that the calibration has succeeded; Among them, the laser point cloud is spherically sampled and represented in the same vector space to generate the Tylenol vector, which includes: Use the Fibonacci grid sampling method to uniformly sample N points on the sphere to obtain a spherical point cloud consisting of N points, each of which covers the same area on the sphere, where N is a preset natural number; Projecting each point of the laser point cloud onto the sphere of the spherical point cloud, and searching for the nearest point corresponding to each laser point cloud point in the spherical point cloud through the KD tree of the spherical point cloud; The distance of the laser point cloud point corresponding to each point in the spherical point cloud to the laser radar origin is formed into a vector with a length of N in the order of the spherical point cloud points.

Images