Laser radar calibration method and device

Abstract

The invention belongs to the technical field of automatic driving, and particularly relates to a laser radar calibration method and device. The method comprises the following steps: performing spherical sampling on laser point clouds of a target area acquired by a plurality of laser radars respectively, representing the laser point clouds to the same vector space, generating a Tyronoi vector, and determining rotation parameters suitable for aligning the laser point clouds based on the distance of the Tyronoi vector; a local map of the laser radar is constructed based on the pose of the laser point cloud, external parameters of the laser radar are obtained based on the local map, the pose is optimized based on the external parameters, the steps are repeated, and the external parameters at the initial moment are determined based on the rotation parameters; and when the external parameters are still not converged when a preset frame number of cycles is executed, determining that calibration fails, if convergence succeeds, determining whether the target area is a degradation scene, if yes, replacing the target area for re-calibration, and if not, determining that calibration succeeds.

Description

Technical Field

[0001] The present disclosure belongs to the field of autonomous driving technology, and specifically relates to a laser radar calibration method and device. Background Art

[0002] As the demand for enhanced perception system performance continues to grow, it has become a significant trend to equip intelligent unmanned systems such as autonomous vehicles and robots with various types of LiDARs, such as using traditional mechanical LiDARs to obtain a 360° horizontal field of view to ensure uniform spatial point cloud distribution, or using solid-state lasers to obtain denser point clouds within a limited field of view at a relatively low cost.

[0003] In order to effectively utilize the advantages of multiple lidars, it is necessary to find the position transformation relationship between different lidar coordinate systems based on sensor calibration to align and synchronize the data of each sensor. Existing lidar calibration methods usually use a coarse-to-fine framework to gradually solve the calibration parameters. That is, first find the appropriate initial value by using hand-eye calibration or prior knowledge of a specific scene. Then, by identifying elements that can be associated between radar data, such as similar geometric structures in point clouds or their similar motion trajectories, a calibration refinement process is performed to further improve the accuracy of the calibration parameters.

[0004] However, this coarse-to-fine calibration framework encounters more and more difficulties when calibrating different types of LiDARs. First, the field of view of different types of LiDARs varies greatly, and the point cloud distribution generated by different LiDARs in the same area is also different. The overlapping area between two LiDARs may also be very limited, which makes it difficult to find a suitable initial calibration value. In addition, due to problems such as point cloud noise and environmental degradation, it is difficult to find similar geometric structures or similar motion trajectories in the point cloud, making it impossible to perform calibration refinement. Summary of the invention

[0005] The disclosed embodiment proposes a laser radar calibration solution to solve the problem that the existing laser radar calibration method has poor calibration effect on different types of laser radars.

[0006] A first aspect of an embodiment of the present disclosure provides a laser radar calibration method, including:

[0007] Spherical sampling is performed on the laser point clouds of the target area respectively acquired by multiple laser radars and represented into the same vector space to generate Tylenol vectors, and based on the distance of the Tylenol vectors, a rotation parameter suitable for aligning the laser point clouds is determined, wherein the relative positions of the laser radars remain unchanged;

[0008] Constructing a local map of the laser radar based on the pose of the laser point cloud, obtaining external parameters of the laser radar based on the local map, optimizing the pose based on the external parameters, and repeating the above steps, wherein the local map is the accumulation of the laser point cloud pose transformation up to the target time, and the external parameters at the initial time are determined based on the rotation parameters;

[0009] When the external parameters still do not converge after executing a preset number of frames, the calibration is determined to have failed. If converged successfully, it is determined whether the target area is a degraded scene. If it is a degraded scene, the target area is replaced and recalibrated. If it is not a degraded scene, the calibration is determined to be successful.

[0010] In some embodiments of the present disclosure, spherical sampling is performed on the laser point cloud and represented in the same vector space, and the generation of the Tyno vector includes:

[0011] Use the Fibonacci grid sampling method to uniformly sample N points on the sphere to obtain a spherical point cloud consisting of N points, each point covers the same area on the sphere, where N is a preset natural number;

[0012] Projecting each point of the laser point cloud onto the sphere of the spherical point cloud, and searching for the nearest point corresponding to each laser point cloud point in the spherical point cloud through the KD tree of the spherical point cloud;

[0013] The distance of the laser point cloud point corresponding to each point in the spherical point cloud to the laser radar origin is formed into a vector with a length of N in the order of the spherical point cloud points.

[0014] In some embodiments of the present disclosure, determining the rotation parameters suitable for aligning the laser point cloud based on the distance of the Tylenol vector includes:

[0015] When the first laser point cloud acquired by the first laser radar does not overlap with the second laser point cloud acquired by the second laser radar, traverse the rotation space with the first step length until an overlap is detected;

[0016] After detecting the existence of overlap, all the first rotations with overlap are retrieved in the rotation space with a second step length, for all the first rotations, the Tylenol distance between the first laser point cloud and the second laser point cloud is calculated based on the Tylenol vector and the overlap state of the first laser point cloud and the second laser point cloud, and a preset number of first rotations with the smallest Tylenol distance are selected;

[0017] For each first rotation among a preset number of first rotations with the smallest Tylenol distance, move along a path with decreasing Tylenol distance with a third step length until a second rotation with the smallest Tylenol distance is found among the first rotations, and use the minimum value of all the second rotations as a rotation parameter suitable for aligning the first laser point cloud with the second laser point cloud, wherein the third step length is not greater than the second step length, and the second step length is not greater than the first step length.

[0018] In some embodiments of the present disclosure, detecting the existence of overlap includes:

[0019] When the length of the Tylenol mask of the first laser point cloud and the second laser point cloud is not zero, it is determined that the first laser point cloud and the second laser point cloud overlap, and the Tylenol mask is:

[0020]

[0021] in, and Respectively represent the Tyno vectors of the first laser point cloud The Terminology vector of the second laser point cloud The i-th position of , d1 and d2 are preset parameters, where d1 is used to filter distant landmarks in the lidar to reduce the interference caused by translation between lidars, and d2 is adjusted according to the actual distance between lidars;

[0022] The calculating the Tylenol distance between the first laser point cloud and the second laser point cloud based on the Tylenol vector and the overlapping state of the first laser point cloud and the second laser point cloud comprises:

[0023]

[0024] Wherein, D(F1, F2) is the Tylenol distance between the first laser point cloud and the second laser point cloud, is the Terminology vector of the first laser point cloud Taino vector with the second laser point cloud The difference in the corresponding position, To use the Tyno mask M to filter the data involved in the distance calculation, ||·||1 is the 1-norm of the calculation vector, and α is set to 2.

[0025] In some embodiments of the present disclosure, constructing a local map of the laser radar based on the position and posture of the laser point cloud includes:

[0026] The accumulation of the product of each frame of point cloud data of the laser point cloud and the corresponding posture as of the target time is used as the local map of the laser radar, wherein the posture of the first frame is used as the local map of the first frame.

[0027] In some embodiments of the present disclosure, acquiring the external parameters of the laser radar based on the local map includes:

[0028] The position of the second laser point cloud and the initial values ​​of the external parameters of the first laser radar and the second laser radar are determined based on the calculation formula of the external parameters when the local map distance between the first laser radar and the second laser radar is the smallest at the target time. The calculation formula is:

[0029]

[0030] in, t C * are the initial values ​​of the external parameters of the first laser radar and the second laser radar at the target time (time t), are the local maps of the first laser radar and the second laser radar at the target time (time t), T is the position of the second laser point cloud, and d(·) represents the distance.

[0031] In some embodiments of the present disclosure, optimizing the posture based on the external parameter includes:

[0032] The position and posture of the first laser point cloud at the target time are determined based on the point cloud data of the first laser point cloud and the second laser point cloud at the target time and the initial value of the external parameter at the previous time. The formula is as follows:

[0033]

[0034] in, is the position and posture of the first laser point cloud at the target time (time t), are the local maps of the first laser radar and the second laser radar at the previous moment ((t-1) moment), and are the point cloud data of the first laser radar and the second laser radar at the target time (time t), t-1 C * Refers to the initial value of the external parameter at the previous moment ((t-1) moment), where, 0 C * The rotation part of uses the rotation parameters, the translation part is set to (0,0,0), T is the pose of the second laser point cloud, and d(·) represents the distance.

[0035] In some embodiments of the present disclosure, determining whether the external parameter converges includes:

[0036] If | t C* t-1 C -1 |<θ c , then the external parameter C at time t converges, where θc is the preset threshold.

[0037] In some embodiments of the present disclosure, determining whether the target area is a degraded scene includes:

[0038] Calculate the Jacobian matrix J of the calibration residual and the information matrix J t J, obtain the maximum eigenvalue λ of the information matrix max With the minimum eigenvalue λ min ,if Then the target area is determined to be a degraded scene, where θ d To preset the threshold, the calculation formula of the Jacobian matrix J is as follows:

[0039]

[0040] in, are the local maps of the first laser radar and the second laser radar at the target time (time t), C is the external parameter of the first laser radar and the second laser radar, T is the position of the second laser point cloud, and d(·) represents the distance.

[0041] A second aspect of an embodiment of the present disclosure provides a laser radar calibration device, including:

[0042] A rotation initialization module is used to perform spherical sampling on the laser point clouds of the target area respectively acquired by multiple laser radars and represent them into the same vector space, generate Tylenol vectors, and determine rotation parameters suitable for aligning the laser point clouds based on the distance of the Tylenol vectors, wherein the relative positions of the laser radars remain unchanged;

[0043] A joint calibration module is used to construct a local map of the laser radar based on the pose of the laser point cloud, obtain external parameters of the laser radar based on the local map, optimize the pose based on the external parameters, and repeat the above steps, wherein the local map is the accumulation of the pose transformation of the laser point cloud up to the target time;

[0044] The evaluation module is used to determine that the calibration has failed when the external parameters have not converged after executing a preset number of frames. If converged successfully, it is determined whether the target area is a degraded scene. If it is a degraded scene, the target area is replaced and recalibrated. If it is not a degraded scene, the calibration is determined to be successful.

[0045] In summary, the laser radar calibration method and device provided in each embodiment of the present disclosure, by proposing a spherical TERRA descriptor that is sensitive to point cloud rotation, and designing a reasonable distance function and rotation search method, can initialize the rotation of two frames of point clouds without any prior knowledge about the placement of the laser radar and the characteristics of the point cloud. The problem of being unable to initialize due to the properties of the laser radar point cloud itself is solved. In addition, since the pose and external parameters are solved at the same time, the problem of low accuracy of calibration parameters caused by point cloud noise and environmental degradation is overcome; and the reliability of the calibration parameters is evaluated by a quantitative evaluation method. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] The features and advantages of the present disclosure will be more clearly understood by referring to the accompanying drawings, which are schematic and should not be construed as limiting the present disclosure in any way. In the accompanying drawings:

[0047] Figure 1 is a schematic diagram of a computer system to which the present disclosure is applicable;

[0048] Figure 2 It is the laser point cloud data obtained based on 32-line laser radar and AVIA laser radar for the same target area in one embodiment of the present disclosure;

[0049] Figure 3 is a flow chart of a laser radar calibration method according to some embodiments of the present disclosure;

[0050] Figure 4 This is an example of generating TERRA descriptors for 32-line laser point cloud and AVIA laser point cloud respectively according to an embodiment of the present disclosure;

[0051] Figure 5 According to an embodiment of the present disclosure, by rotating TERRA, searching for alignment Figure 2 Examples of optimal rotations of the 32-line laser point cloud and the AVIA laser point cloud shown;

[0052] Figure 6 According to an embodiment of the present disclosure, Figure 2 Schematic diagram of iterative joint estimation of pose and external parameters of 32-line laser point cloud and AVIA laser point cloud shown;

[0053] Figure 7 According to an embodiment of the present disclosure, Figure 2 The schematic diagram of the effect after aligning one frame of data of 32-line laser point cloud and one frame of data of AVIA laser point cloud;

[0054] Figure 8 is based on Figure 3The calibration method shown performs more calibration tasks and displays the calibration results;

[0055] Fig. 9 It is a schematic diagram of a laser radar calibration device according to some embodiments of the present disclosure. DETAILED DESCRIPTION

[0056] In the detailed description below, many specific details of the present disclosure are set forth by way of example in order to provide a thorough understanding of the relevant disclosure. However, it will be apparent to one of ordinary skill in the art that the present disclosure can be implemented without these details. It should be understood that the use of the terms "system," "device," "unit," and / or "module" in the present disclosure is a method for distinguishing different parts, elements, parts, or assemblies at different levels in a sequential arrangement. However, these terms may be replaced by other expressions if other expressions can achieve the same purpose.

[0057] It should be understood that when a device, unit or module is referred to as being "on," "connected to," or "coupled to" another device, unit or module, it may be directly on, connected to, coupled to, or communicating with other devices, units or modules, or there may be intermediate devices, units or modules, unless the context clearly indicates an exception. For example, the term "and / or" as used in this disclosure includes any and all combinations of one or more of the relevant listed items.

[0058] The terms used in this disclosure are only for describing specific embodiments and are not intended to limit the scope of this disclosure. As shown in the specification and claims of this disclosure, unless the context clearly indicates an exception, the words "one", "a kind" and / or "the" do not specifically refer to the singular, but may also include the plural. Generally speaking, the terms "include" and "comprise" only indicate the inclusion of clearly identified features, wholes, steps, operations, elements and / or components, and such expressions do not constitute an exclusive list, and other features, wholes, steps, operations, elements and / or components may also be included.

[0059] These and other features and characteristics, methods of operation, functions of related elements of the structure, combinations of parts, and economies of manufacture of the present disclosure may be better understood with reference to the following description and drawings, which form a part of the specification. However, it is to be clearly understood that the drawings are for illustration and description purposes only and are not intended to limit the scope of protection of the present disclosure. It is to be understood that the drawings are not drawn to scale.

[0060] Various structural diagrams are used in this disclosure to illustrate various variations of the embodiments of the present disclosure. It should be understood that the above or below structures are not intended to limit the present disclosure. The scope of protection of the present disclosure shall be subject to the claims.

[0061] Figure 1 is a schematic diagram of a computer system to which the present disclosure is applicable. Figure 1 The computer system shown includes a calibration server connected to multiple laser radar data, and the multiple laser radars respectively obtain point cloud data of the target area. The calibration server calibrates the laser radar based on the point cloud data.

[0062] The laser radar can be the same type of laser radar that obtains the same type of point cloud data, or it can be different types of laser radars that obtain different types of point cloud data, such as a mechanical laser radar for obtaining a 360° horizontal viewing angle to ensure a uniform spatial point cloud distribution or a solid-state laser radar for obtaining a denser point cloud within a limited field of view. In some embodiments of the present disclosure, a plurality of the laser radars are deployed together in an autonomous driving vehicle to respectively obtain different types of point cloud data of the target area, and perform data fusion to utilize the unique advantages of different types of laser radars to perform autonomous driving tasks. When two or more laser radars are deployed together in an autonomous driving vehicle, generally speaking, the relative positions between the laser radars remain unchanged.

[0063] The calibration server calibrates the laser radar based on the point cloud data of the target area acquired by different laser radars. The calibration server can be any of a stand-alone server, a cluster server or a distributed server.

[0064] LiDAR calibration refers to the process of finding the position conversion relationship between different LiDAR coordinate systems. Only after calibration, multiple LiDARs can obtain multiple point cloud data and perform data alignment and synchronization.

[0065] Existing lidar calibration methods usually adopt a coarse-to-fine framework to gradually solve the calibration parameters. Finding suitable initial values ​​is a prerequisite for obtaining accurate parameters, which can be done by leveraging hand-eye calibration or prior knowledge of specific scenes. After the initialization step, a calibration refinement process is usually performed to further improve the accuracy of the calibration parameters. Specifically, the basic principle of the refinement step is to identify elements that can be associated between radar data, such as similar geometric structures in the point cloud or their similar motion trajectories. Therefore, the refinement step is usually modeled as a point cloud registration problem or a trajectory alignment problem.

[0066] Although multi-lidar calibration has been widely studied, calibration between different types of lidars has become increasingly challenging. First, the field of view of different lidars varies greatly. For example, the field of view of Livox's Tele is 14.5°*16.2°, while the field of view of mechanical lidar is 360°*30°. In addition, due to the different scanning methods of lidars, the point cloud distribution generated by different lidars in the same area is also different. Finally, because the main motivation for using multiple lidars is to expand the field of view of unmanned intelligent systems, the overlap between two lidars may be limited, Figure 2 The laser point cloud data obtained by 32-line laser radar and AVIA laser radar for the same target area shows that the distribution and density of laser point clouds obtained by different types of laser radars are very different. The different distribution and density of point clouds make it challenging to develop a universal initialization method that can continuously generate qualified initial values ​​for any two laser radars; at the same time, problems such as point cloud noise and environmental degradation also make it difficult to find similar geometric structures or similar motion trajectories in the point cloud, making it impossible to perform calibration refinement.

[0067] In order to effectively solve the above challenges, the present disclosure proposes a calibration method applicable to multiple lidars of any type, that is, solving the external parameters between two lidars, including translation and rotation. To this end, the initialization problem needs to be solved first, because on a single autonomous driving vehicle, the translation distance between lidars will be limited by the size of the vehicle. The present disclosure emphasizes the importance of initializing rotation parameters rather than translation parameters. The present disclosure introduces a point cloud descriptor for rotation initialization, named "TERRA", which is used to robustly align the three-degree-of-freedom rotation between lidar point clouds. By carefully designing the distance function between multiple TERRAs, the influence of translation on rotation estimation is overcome. After initialization, in order to solve the problem of poor registration accuracy caused by sparse point clouds and scene degradation, the present disclosure proposes a joint estimation method of lidar external parameters and unmanned system pose, which integrates geometric and motion information to obtain optimized and usable external parameters.

[0068] Figure 3 is a flow chart of a laser radar calibration method according to some embodiments of the present disclosure. In some embodiments, the laser radar calibration method is Figure 1 The calibration server shown is executed, and the laser radar calibration method includes the following steps:

[0069] S310, spherical sampling is performed on the laser point clouds of the target area respectively acquired by multiple laser radars and represented into the same vector space, generating Tylenol vectors, and based on the distance of the Tylenol vectors, determining rotation parameters suitable for aligning the laser point clouds, wherein the relative positions of the laser radars remain unchanged.

[0070] First, generate the Terminology vector for the laser point cloud:

[0071] Use the Fibonacci grid sampling method to uniformly sample N points on the sphere to obtain a spherical point cloud consisting of N points Each point covers the same area on the sphere. Then, use Encode any LiDAR point cloud frame: First, normalize the LiDAR point cloud, that is, project each point onto The point cloud is on the unit sphere; then through KD-Tree, search for each lidar point in The nearest point in . The closest distance to the laser radar origin among the radar points corresponding to each point in the image is saved, and finally The order of the point cloud forms a vector of length N.

[0072] Figure 4 In one embodiment of the present disclosure, Figure 2 The shown examples are examples of generating TERRA descriptors using a 32-line laser point cloud obtained by a 32-line laser radar and an AVIA laser point cloud obtained by an AVIA laser radar.

[0073] After calculating the TERRA descriptor of each lidar point cloud, a rotation operation needs to be designed to rotate the TERRA to find the optimal rotation that can align the two TERRAs. The specific operation of rotating the TERRA is as follows: The rotated point cloud obtained by rotating a certain rotation R is expressed as use KD-Tree on Rotated Point Cloud and original A one-to-one correspondence is established between the point pairs in . In this case, each point pair consists of two points with different indices. A rotated TERRA is created by taking the values ​​from the original index of TERRA and filling them into the index position of the corresponding point.

[0074] By evaluating the distance between TERRA, the optimal rotation to align the two TERRA can be found. A small distance between TERRA indicates that the point cloud projected onto the sphere also has the correct correspondence, which means that the correct rotation has been identified. Specifically, the present disclosure defines a mask to indicate which positions are suitable for two and Distance calculation. m i Defined as:

[0075]

[0076] in, and Represents the TERRA vector of the first laser point cloud TERRA vector with the second laser point cloud The i-th position of , d1 and d2 are preset parameters, where d1 is used to filter distant landmarks in the LiDAR to reduce the interference caused by translation between LiDARs. In practice, d1 is set to 20m. d2 is used to select positions that may belong to the same object for subsequent calculations. d2 can be adjusted according to the actual distance between LiDARs. In some embodiments of the present disclosure, it is set to 5m according to the typical size of the vehicle. Finally, the distance between two TERRAs is defined as:

[0077]

[0078] Where D(F1, F2) is the TERRA distance between the first laser point cloud and the second laser point cloud. is the TERRA vector of the first laser point cloud TERRA vector with the second laser point cloud The difference in the corresponding position, To use the TERRA mask M to filter the data involved in the distance calculation, ||·||1 is the 1-norm of the calculation vector, and α is set to 2.

[0079] Then search for the initial rotation value suitable for aligning the laser point cloud based on the TERRA distance:

[0080] Because the rotation space is too large, it takes an unacceptable time to traverse all feasible rotations. Therefore, the present disclosure first checks whether the initial rotation will cause overlap between the two point clouds based on whether the length of the mask M is not zero. If there is no overlap, the rotation space will be traversed with a relatively large step size (e.g., 30°) until a suitable initial rotation that produces an overlapping area is found. This step is very time-saving. Starting from this initial rotation, explore the surrounding rotation space with a large step size (e.g., 10°), sample all rotations that lead to overlapping areas, and score each rotation using a distance formula. The top ten rotations with the smallest distance will be input to the next stage. In the second stage, based on the first stage, a heuristic search is used to find improved solutions near each previously identified discrete rotation. Specifically, in each area, exploration is performed with a step size of 1°, moving along a path with decreasing distance calculated by the distance formula until the local optimal rotation parameter is reached.

[0081] Figure 5 According to an embodiment of the present disclosure, by rotating TERRA, searching for alignment Figure 2 Examples of optimal rotations for the 32-line laser point cloud and the AVIA laser point cloud shown.

[0082] S320, constructing a local map of the laser radar based on the posture of the laser point cloud, obtaining external parameters of the laser radar based on the local map, optimizing the posture based on the external parameters, and repeating the above steps, wherein the local map is the accumulation of the laser point cloud posture transformation up to the target moment, and the external parameters at the initial moment are determined based on the rotation parameters.

[0083] For the two laser radars to be calibrated, only their point cloud frames at the same time are used. and The external parameters can be solved through registration.

[0084] However, since only a single frame of data provides constraints for estimating extrinsic parameters, the effects of point cloud noise and scene degradation become significant. To alleviate these shortcomings, the present disclosure integrates the knowledge of hand-eye calibration and uses a sequence of point cloud frames from two lidars to obtain extrinsic parameters with optimized rotation and translation.

[0085] The present invention iteratively executes the extrinsic parameter estimation task and the posture optimization task, and finally realizes the joint estimation of the extrinsic parameter and the posture.

[0086] First, build a local map of the lidar based on the position of the laser point cloud:

[0087] The local map obtained by the laser radar refers to the accumulation of the product of each frame of the laser point cloud data and the corresponding posture up to the target time. That is:

[0088]

[0089] in and are the local maps of the first laser radar and the second laser radar at the target time (time t), t P a and t P b are the positions of the first laser point cloud and the second laser point cloud at the target time (time t), and They are respectively the point cloud data of the first laser radar and the second laser radar at the target time (time t), and the first laser point cloud and the second laser point cloud are respectively the point cloud data acquired by the first laser radar and the second laser radar for the target area.

[0090] Then the external parameters are estimated based on the local map:

[0091] In the external parameter estimation stage, the present disclosure uses the local map formed by the first laser and the second laser radar to solve the external parameters, and the objective function used is as follows:

[0092]

[0093] in, t C * is the initial value of the external parameters of the first laser radar and the second laser radar at the target time (time t), are the local maps of the first laser radar and the second laser radar at the target time (time t), T is the position of the second laser point cloud, and d(·) represents the distance.

[0094] In this way, the position and posture of the second laser point cloud and the initial values ​​of the external parameters of the first laser radar and the second laser radar at the target time when the local map distance between the first laser radar and the second laser radar is the smallest can be determined.

[0095] Then, based on the point cloud data of the first laser point cloud and the second laser point cloud at the target moment and the initial value of the external parameter at the previous moment, the position and posture of the first laser point cloud at the target moment are optimized:

[0096]

[0097] in, is the position of the first laser point cloud at the target time (time t), are the local maps of the first laser radar and the second laser radar at the previous moment ((t-1) moment), and are the point cloud data of the first laser radar and the second laser radar at the target time (time t), t-1 C * Refers to the initial value of the external parameter at the previous moment ((t-1) moment), where, 0 C * The rotation part uses the rotation parameters obtained by S310, the translation part is set to (0,0,0), d(·) represents the distance, and T is the pose of the second laser point cloud.

[0098] Iterate the two processes of external parameter estimation and pose optimization until t C * and Stable and convergent. In this process, when t C * Inaccurate, due to The position is incorrect. The estimates of are also difficult to converge.

[0099] Since the present disclosure uses a map to perform an external parameter calibration task, more data is introduced to provide more sufficient constraints for the registration problem, making the registration result more accurate than using only a single frame.

[0100] Figure 6 According to an embodiment of the present disclosure, Figure 2The diagram shows the iterative joint estimation of pose and external parameters of the 32-line laser point cloud and AVIA laser point cloud. Figure 7 shown.

[0101] S330, when the external parameters still do not converge after executing the loop of the preset number of frames, it is determined that the calibration has failed. If it has converged successfully, it is determined whether the target area is a degraded scene. If it is a degraded scene, the target area is replaced and recalibrated. If it is not a degraded scene, it is determined that the calibration is successful.

[0102] The evaluation of the results is divided into two parts: the first is the evaluation of the convergence degree of the calibration process, and the second is the evaluation of the reliability of the calibration results by the calibration environment. Specifically, when estimating the external parameters, when | t C* t-1 C -1 |<θ c (θ c is the specified threshold), the external parameter C at time t is considered to have converged and the calibration process can be stopped. On the other hand, for external parameter estimation, the Jacobian matrix J of the calibration residual is calculated as follows:

[0103]

[0104] And calculate the information matrix J t J. Note max ,λ min are the maximum and minimum eigenvalues ​​of the information matrix respectively. If This indicates that the environment is a degraded scenario and the calibration environment needs to be changed and the calibration process needs to be repeated. c Set to 1 mm, θ d Set to 2.

[0105] We conduct extensive experiments on eight autonomous vehicle sensor setups, testing a total of 16 different types of lidar and dozens of calibration tasks, such as Figure 8 In various challenging tasks, the calibration error can be controlled within 5cm and 1°.

[0106] Fig. 9 is a schematic diagram of a laser radar calibration device according to some embodiments of the present disclosure. Fig. 9 As shown, the laser radar calibration device 900 includes a rotation initialization module 910, a joint calibration module 920, and an evaluation module 930. In some embodiments of the present disclosure, the laser radar calibration function is Figure 1 The calibration server is executed as shown.

[0107] A rotation initialization module 910 is used to perform spherical sampling on the laser point clouds of the target area respectively acquired by multiple laser radars and represent them in the same vector space, generate Tylenol vectors, and determine rotation parameters suitable for aligning the laser point clouds based on the distance of the Tylenol vectors, wherein the relative positions of the laser radars remain unchanged;

[0108] A joint calibration module 920 is used to construct a local map of the laser radar based on the pose of the laser point cloud, obtain external parameters of the laser radar based on the local map, optimize the pose based on the external parameters, and repeat the above steps, wherein the local map is the accumulation of the pose transformation of the laser point cloud up to the target time;

[0109] The evaluation module 930 is used to determine that the calibration has failed when the external parameters have not converged after executing a preset number of frames. If converged successfully, determine whether the target area is a degraded scene. If it is a degraded scene, replace the target area and re-calibrate. If it is not a degraded scene, determine that the calibration is successful.

[0110] In summary, the laser radar calibration method and device provided by each embodiment of the present disclosure, by proposing a spherical TERRA descriptor that is sensitive to point cloud rotation, and designing a reasonable distance function and rotation search method, can initialize the rotation of two frames of point clouds without any prior knowledge about the placement of the laser radar and the characteristics of the point cloud, thus solving the problem of being unable to initialize due to the properties of the laser radar point cloud itself. In addition, since the pose and external parameters are solved at the same time, the problem of low accuracy of calibration parameters caused by point cloud noise and environmental degradation is overcome; and the reliability of the calibration parameters is evaluated by a quantitative evaluation method.

[0111] Although the subject matter described herein is provided in the general context of being executed in conjunction with the execution of operating systems and application programs on computer systems, it will be appreciated by those skilled in the art that other implementations may also be performed in conjunction with other types of program modules. In general, program modules include routines, programs, components, data structures, and other types of structures that perform specific tasks or implement specific abstract data types. It will be appreciated by those skilled in the art that the subject matter described herein may be practiced using other computer system configurations, including handheld devices, multiprocessor systems, microprocessor-based or programmable consumer electronics, minicomputers, mainframe computers, etc., and may also be used in a distributed computing environment in which tasks are performed by remote processing devices connected via a communication network. In a distributed computing environment, program modules may be located in both local and remote memory storage devices.

[0112] Those of ordinary skill in the art will appreciate that the units and method steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this disclosure.

[0113] It should be understood that the above specific embodiments of the present disclosure are only used to illustrate or explain the principles of the present disclosure, and do not constitute a limitation of the present disclosure. Therefore, any modifications, equivalent substitutions, improvements, etc. made without departing from the spirit and scope of the present disclosure should be included in the protection scope of the present disclosure. In addition, the claims attached to the present disclosure are intended to cover all changes and modifications that fall within the scope and boundaries of the attached claims, or the equivalent forms of such scope and boundaries.

Claims

1. A laser radar calibration method, characterized in that: include: Spherical sampling is performed on the laser point clouds of the target area respectively acquired by multiple laser radars and represented into the same vector space to generate Tylenol vectors, and based on the distance of the Tylenol vectors, a rotation parameter suitable for aligning the laser point clouds is determined, wherein the relative positions of the laser radars remain unchanged; Constructing a local map of the laser radar based on the pose of the laser point cloud, obtaining external parameters of the laser radar based on the local map, optimizing the pose based on the external parameters, and repeating the above steps, wherein the local map is the accumulation of the laser point cloud pose transformation up to the target time, and the external parameters at the initial time are determined based on the rotation parameters; When the external parameters still do not converge after executing a preset number of frames, the calibration is determined to have failed. If converged successfully, it is determined whether the target area is a degraded scene. If it is a degraded scene, the target area is replaced and recalibrated. If it is not a degraded scene, the calibration is determined to be successful.

2. The method according to claim 1, characterized in that: Spherical sampling of the laser point cloud is performed and represented in the same vector space to generate the Tyno vector including: Use the Fibonacci grid sampling method to uniformly sample N points on the sphere to obtain a spherical point cloud consisting of N points, each point covers the same area on the sphere, where N is a preset natural number; Projecting each point of the laser point cloud onto the sphere of the spherical point cloud, and searching for the nearest point corresponding to each laser point cloud point in the spherical point cloud through the KD tree of the spherical point cloud; The distance of the laser point cloud point corresponding to each point in the spherical point cloud to the laser radar origin is formed into a vector with a length of N in the order of the spherical point cloud points.

3. The method according to claim 2, characterized in that: The determining of the rotation parameters suitable for aligning the laser point cloud based on the distance of the Tylenol vector comprises: When the first laser point cloud acquired by the first laser radar does not overlap with the second laser point cloud acquired by the second laser radar, traverse the rotation space with the first step length until an overlap is detected; After detecting the existence of overlap, all the first rotations with overlap are retrieved in the rotation space with a second step length, for all the first rotations, the Tylenol distance between the first laser point cloud and the second laser point cloud is calculated based on the Tylenol vector and the overlap state of the first laser point cloud and the second laser point cloud, and a preset number of first rotations with the smallest Tylenol distance are selected; For each first rotation among a preset number of first rotations with the smallest Tylenol distance, move along a path with decreasing Tylenol distance with a third step length until a second rotation with the smallest Tylenol distance is found among the first rotations, and use the minimum value of all the second rotations as a rotation parameter suitable for aligning the first laser point cloud with the second laser point cloud, wherein the third step length is not greater than the second step length, and the second step length is not greater than the first step length.

4. The method according to claim 3, characterized in that: The detecting that there is overlap comprises: When the length of the Tylenol mask of the first laser point cloud and the second laser point cloud is not zero, it is determined that the first laser point cloud and the second laser point cloud overlap, and the Tylenol mask is: in, and Respectively represent the Terminology vectors of the first laser point cloud The Terminology vector of the second laser point cloud The i-th position of , d1 and d2 are preset parameters, where d1 is used to filter distant landmarks in the lidar to reduce the interference caused by translation between lidars, and d2 is adjusted according to the actual distance between lidars; The calculating the Tylenol distance between the first laser point cloud and the second laser point cloud based on the Tylenol vector and the overlapping state of the first laser point cloud and the second laser point cloud comprises: Wherein, D(F1, F2) is the Tylenol distance between the first laser point cloud and the second laser point cloud, is the Terminology vector of the first laser point cloud Taino vector with the second laser point cloud The difference in the corresponding position, To use the Tyno mask M to filter the data involved in the distance calculation, ||·||1 is the 1-norm of the calculation vector, and α is set to 2.

5. The method according to claim 4, characterized in that: The step of constructing a local map of the laser radar based on the position and posture of the laser point cloud comprises: The accumulation of the product of each frame of point cloud data of the laser point cloud and the corresponding posture as of the target time is used as the local map of the laser radar, wherein the posture of the first frame is used as the local map of the first frame.

6. The method according to claim 5, characterized in that: The acquiring of the external parameters of the laser radar based on the local map comprises: The position of the second laser point cloud and the initial values ​​of the external parameters of the first laser radar and the second laser radar are determined based on the calculation formula of the external parameters when the local map distance between the first laser radar and the second laser radar is the smallest at the target time. The calculation formula is: in, t C * are the initial values ​​of the external parameters of the first laser radar and the second laser radar at the target time (time t), are the local maps of the first laser radar and the second laser radar at the target time (time t), T is the position of the second laser point cloud, and d(·) represents the distance.

7. The method according to claim 6, characterized in that: Optimizing the posture based on the external parameter includes: The position and posture of the first laser point cloud at the target time are determined based on the point cloud data of the first laser point cloud and the second laser point cloud at the target time and the initial value of the external parameter at the previous time. The formula is as follows: in, is the position and posture of the first laser point cloud at the target time (time t), are the local maps of the first laser radar and the second laser radar at the previous moment ((t-1) moment), and are the point cloud data of the first laser radar and the second laser radar at the target time (time t), t-1 C * Refers to the initial value of the external parameter at the previous moment ((t-1) moment), where, 0 C * The rotation part of uses the rotation parameters, the translation part is set to (0,0,0), T is the pose of the second laser point cloud, and d(·) represents the distance.

8. The method according to claim 7, characterized in that: Determining whether the external parameter converges includes: If | t C *t-1 C -1 |<θ c , then the external parameter C at time t converges, where θ c is the preset threshold.

9. The method according to claim 8, characterized in that: The determining whether the target area is a degraded scene includes: Calculate the Jacobian matrix J of the calibration residual and the information matrix J t J, obtain the maximum eigenvalue λ of the information matrix max With the minimum eigenvalue λ min ,if Then the target area is determined to be a degraded scene, where θ d To preset the threshold, the calculation formula of the Jacobian matrix j is as follows: in, are the local maps of the first laser radar and the second laser radar at the target time (time t), C is the external parameter of the first laser radar and the second laser radar, T is the position of the second laser point cloud, and d(·) represents the distance.

10. A laser radar calibration device, characterized in that: include: A rotation initialization module is used to perform spherical sampling on the laser point clouds of the target area respectively acquired by multiple laser radars and represent them into the same vector space, generate Tylenol vectors, and determine rotation parameters suitable for aligning the laser point clouds based on the distance of the Tylenol vectors, wherein the relative positions of the laser radars remain unchanged; A joint calibration module is used to construct a local map of the laser radar based on the pose of the laser point cloud, obtain external parameters of the laser radar based on the local map, optimize the pose based on the external parameters, and repeat the above steps, wherein the local map is the accumulation of the pose transformation of the laser point cloud up to the target time; The evaluation module is used to determine that the calibration has failed when the external parameters have not converged after executing a preset number of frames. If converged successfully, it is determined whether the target area is a degraded scene. If it is a degraded scene, the target area is replaced and recalibrated. If it is not a degraded scene, the calibration is determined to be successful.

Images