An external parameter calibration device

Abstract

This invention discloses an extrinsic parameter calibration device, comprising: a data acquisition unit for acquiring at least a first point cloud sequence of a moving or rotating first lidar and a second point cloud sequence of a second lidar; a registration unit for obtaining, based on the first and second point cloud sequences, a first point cloud map of the first lidar and a second point cloud map of the second lidar at a preset time, wherein the second point cloud map partially overlaps with the first point cloud map; and a calibration unit for applying a point cloud registration algorithm to calculate the transformation relationship from the second point cloud map to the first point cloud map, and obtaining a calibration result. Therefore, this invention enables multiple lidars with insufficient overlapping areas to undergo extrinsic parameter calibration using a general point cloud registration algorithm.

Description

[0001] This application is a divisional application of Chinese application No. 201980002761.1, the foregoing contents of which are incorporated herein by reference. Technical Field

[0002] This invention relates to the field of lidar technology, and more specifically to an external parameter calibration device. Background Technology

[0003] LiDAR (Light Detection and Ranging) is an optical remote sensing technology that measures parameters such as distance to a target by illuminating it with a pulsed laser beam. The set of three-dimensional measurement points obtained by LiDAR is large and dense, and can be called a point cloud. Point cloud registration involves finding the spatial transformation relationship between two sets of three-dimensional data points from different coordinate systems, so that the two sets can be unified into the same coordinate system. Extrinsic parameter calibration is used to determine the rotation and translation relationships between multiple sensor coordinate systems, aiming to represent data from multiple sensors in a unified coordinate system.

[0004] Point cloud registration algorithms rely on the premise that two point clouds have overlapping areas, such as the portion of an object illuminated by two LiDAR sensors. In autonomous driving applications, there are situations where the point clouds of multiple radars have little or no overlap. For example, one LiDAR might be mounted at the front of the vehicle, and another at the rear; due to the vehicle body's obstruction, the two radars have no overlapping area. In such cases, conventional registration methods cannot be directly applied. Summary of the Invention

[0005] In view of the above problems, embodiments of the present invention provide an external parameter calibration method, apparatus, computing device, and computer storage medium that overcomes or at least partially solves the above problems.

[0006] According to one aspect of the present invention, an external parameter calibration method is provided, comprising: acquiring at least a first point cloud sequence of a moving or rotating first lidar and a second point cloud sequence of a second lidar, wherein the first point cloud sequence is located in the body coordinate system of the first lidar and the second point cloud sequence is located in the body coordinate system of the second lidar; obtaining a first point cloud map of the first lidar and a second point cloud map of the second lidar at a preset time based on the first point cloud sequence and the second point cloud sequence, respectively; applying a point cloud registration algorithm to calculate the transformation relationship from the second point cloud map to the first point cloud map, and obtaining a calibration result.

[0007] Optionally, the first and second lidars, which can be moved or rotated, are located in a site with structured features.

[0008] Optionally, obtaining the first point cloud map of the first LiDAR at a preset time based on the first point cloud sequence includes: applying a simultaneous localization and mapping algorithm to calculate the transformation relationship between adjacent point clouds in the first point cloud sequence; transforming the first point cloud sequence to the body coordinate system of the first LiDAR at the preset time based on the transformation relationship between adjacent point clouds to form a new first point cloud sequence; merging the new first point cloud sequences to obtain the first point cloud map based on the body coordinate system of the first LiDAR at the preset time; obtaining the second point cloud map of the second LiDAR at a preset time based on the second point cloud sequence includes: applying a simultaneous localization and mapping algorithm to calculate the transformation relationship between adjacent point clouds in the second point cloud sequence; transforming the second point cloud sequence to the body coordinate system of the second LiDAR at the preset time based on the transformation relationship between adjacent point clouds to form a new second point cloud sequence; merging the new second point cloud sequences to obtain the second point cloud map based on the body coordinate system of the second LiDAR at the preset time, wherein the second point cloud map partially overlaps with the first point cloud map.

[0009] Optionally, the simultaneous localization and mapping algorithm is applied to calculate the transformation relationship between adjacent point clouds in the first point cloud sequence, including: traversing the first point cloud sequence, applying a point cloud registration algorithm to calculate the transformation relationship between any adjacent point cloud in the first point cloud sequence; and using a general graph optimization algorithm to optimize the transformation relationship between any adjacent point cloud in the first point cloud sequence. The simultaneous localization and mapping algorithm is also applied to calculate the transformation relationship between adjacent point clouds in the second point cloud sequence, including: traversing the second point cloud sequence, applying a point cloud registration algorithm to calculate the transformation relationship between any adjacent point cloud in the second point cloud sequence; and using a general graph optimization algorithm to optimize the transformation relationship between any adjacent point cloud in the second point cloud sequence.

[0010] Optionally, the new first point cloud sequence satisfies the following relationship:

[0011] P i =T1T2…T i P i ,

[0012] Among them, P i Let ' be the coordinates of the i-th point in the new first point cloud sequence, where i is a positive integer, and P i Let T be the coordinates of the i-th point cloud in the first point cloud sequence. i For the adjacent point clouds P in the first point cloud sequence i-1 and P i The transformation relationship, T i =F(P i-1 ,P i);

[0013] The new second point cloud sequence satisfies the following relationship:

[0014] P j =T1T2…T j P j ,

[0015] Among them, P j Let ′ be the coordinates of the j-th point in the new second point cloud sequence, where j is a positive integer, and P j Let T be the coordinates of the j-th point cloud in the second point cloud sequence. j For the adjacent point cloud P in the second point cloud sequence j-1 and P j The transformation relationship, T j =F(P j-1 ,P j ).

[0016] Optionally, before applying the point cloud registration algorithm to calculate the transformation relationship between the second point cloud map and the first point cloud map and obtain the calibration result, the method includes: performing filtering operations on the first point cloud map and the second point cloud map respectively.

[0017] Optionally, the point cloud registration algorithm includes an iterative nearest point algorithm or a normal distribution transformation algorithm.

[0018] According to another aspect of the present invention, an extrinsic parameter calibration device is provided, the extrinsic parameter calibration device comprising: a data acquisition unit, configured to acquire at least a first point cloud sequence of a moving or rotating first lidar and a second point cloud sequence of a second lidar, wherein the first point cloud sequence is located in the body coordinate system of the first lidar and the second point cloud sequence is located in the body coordinate system of the second lidar; a first registration unit, configured to obtain a first point cloud map of the first lidar and a second point cloud map of the second lidar at a preset time based on the first point cloud sequence and the second point cloud sequence, respectively; and a calibration unit, configured to apply a point cloud registration algorithm to calculate the transformation relationship between the second point cloud map and the first point cloud map, and obtain a calibration result.

[0019] According to another aspect of the present invention, a computing device is provided, comprising: a processor, a memory, a communication interface, and a communication bus, wherein the processor, the memory, and the communication interface communicate with each other via the communication bus; the memory is used to store at least one executable instruction, the executable instruction causing the processor to perform the steps of the aforementioned external parameter calibration method.

[0020] According to another aspect of the present invention, a computer storage medium is provided, the storage medium storing at least one executable instruction that causes a processor to perform the steps of the aforementioned external parameter calibration method.

[0021] In an embodiment of the present invention, the extrinsic parameter calibration method includes: acquiring at least a first point cloud sequence of a moving or rotating first lidar and a second point cloud sequence of a second lidar, wherein the first point cloud sequence is located in the body coordinate system of the first lidar and the second point cloud sequence is located in the body coordinate system of the second lidar; obtaining a first point cloud map of the first lidar and a second point cloud map of the second lidar at a preset time based on the first point cloud sequence and the second point cloud sequence, respectively; and applying a point cloud registration algorithm to calculate the transformation relationship from the second point cloud map to the first point cloud map to obtain a calibration result. Therefore, by moving or rotating the carrier carrying the lidar, the field of view of a single lidar is expanded, point clouds in different coordinate systems at different times are transformed to the same coordinate system, time registration of multiple lidars is performed, and point cloud maps represented in their respective body coordinate systems at the same time are used to calculate the relative coordinate transformation relationship between the point cloud maps, thereby achieving extrinsic parameter calibration of the lidars. This allows multiple lidars with insufficient overlapping areas to use a general point cloud registration algorithm for extrinsic parameter calibration. Attached Figure Description

[0022] One or more embodiments are illustrated by way of example with reference numerals in the accompanying drawings. These illustrations do not constitute a limitation on the embodiments. Elements with the same reference numerals in the drawings are denoted as similar elements. Unless otherwise stated, the figures in the drawings are not to be limited by scale.

[0023] Figure 1 A flowchart illustrating an external parameter calibration method according to an embodiment of the present invention is shown;

[0024] Figure 2 A flowchart illustrating another external parameter calibration method according to an embodiment of the present invention is shown;

[0025] Figure 3 This diagram shows a point cloud before calibration, according to an embodiment of the present invention, of an external parameter calibration method.

[0026] Figure 4 This diagram illustrates point clouds of two lidar sensors after calibration, according to an embodiment of the present invention, which employs an external parameter calibration method.

[0027] Figure 5 The diagram shows point clouds of three lidar sensors after calibration, according to an embodiment of the present invention, using an external parameter calibration method.

[0028] Figure 6A schematic diagram of the structure of another external parameter calibration device according to an embodiment of the present invention is shown;

[0029] Figure 7 A schematic diagram of the structure of a computing device according to an embodiment of the present invention is shown. Detailed Implementation

[0030] Exemplary embodiments of the present disclosure will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.

[0031] Figure 1 A flowchart illustrating an external parameter calibration method according to an embodiment of the present invention is shown. Figure 1 As shown, the external parameter calibration method includes:

[0032] Step S11: At least acquire a first point cloud sequence of a moving or rotating first lidar and a second point cloud sequence of a second lidar, wherein the first point cloud sequence is located in the body coordinate system of the first lidar and the second point cloud sequence is located in the body coordinate system of the second lidar.

[0033] In step S11, the moving or rotating first and second lidars are located in a site with structured features. In this embodiment of the invention, a site with abundant structured features is selected; a carrier carrying at least the first and second lidars moves or rotates on the site; and the first point cloud sequence of the first lidar and the second point cloud sequence of the second lidar are simultaneously acquired. Taking two lidars as an example, a site rich in walls and rods is selected, and the carrier carrying the first and second lidars moves forward in a straight line at a constant speed of 5 km / h for 10 meters, then moves in a figure-eight pattern, while simultaneously acquiring the point cloud sequences of the two lidars. The first point cloud sequence is located in the body coordinate system of the first lidar. Preferably, the body coordinate system of the first lidar can be a coordinate system with the center of the first lidar as the origin, and the first point cloud sequence is represented as L1 = {P0, P1, ... P}. n}, where n is a positive integer. The second point cloud sequence is located in the body coordinate system of the second lidar. Preferably, the body coordinate system of the second lidar can be a coordinate system with the center of the second lidar as the origin. The second point cloud sequence is represented as L2 = {P0, P1, ..., P}. m}, where m is a positive integer.

[0034] Selecting a site with abundant structured features, such as walls and poles, facilitates subsequent mapping. In this embodiment of the invention, multiple lidars can be rigidly mounted on a carrier that moves at a low, constant speed, with trajectories including straight lines or curves, while simultaneously acquiring point cloud sequences from each lidar.

[0035] Step S12: Obtain the first point cloud map of the first lidar and the second point cloud map of the second lidar at preset times based on the first point cloud sequence and the second point cloud sequence, respectively.

[0036] The preset time can be the initial time of the first point cloud sequence and the second point cloud sequence, or any other time during the acquisition process of the selected first point cloud sequence and the second point cloud sequence, without any restrictions.

[0037] The first point cloud sequence and the second point cloud sequence will be described below. In this embodiment of the invention, for the first point cloud sequence, as follows: Figure 2 As shown, step S12 includes:

[0038] Step S121: Apply the simultaneous localization and mapping algorithm to calculate the transformation relationship between adjacent point clouds in the first point cloud sequence.

[0039] Specifically, the first point cloud sequence is traversed, and a point cloud registration algorithm is applied to calculate the transformation relationship between any two adjacent point clouds in the first point cloud sequence. A general graph optimization algorithm is then used to optimize the transformation relationship between any two adjacent point clouds in the first point cloud sequence. The transformation relationship between adjacent point clouds can be a simple coordinate transformation relationship or other transformation relationships, such as those that have undergone scaling. For the first point cloud sequence L1, from 1 to n, a Simultaneous Localization and Mapping (SLAM) algorithm is used. For example, the Normal Distributions Transform (NDT) algorithm or the Iterative Closest Point (ICP) algorithm is first used, followed by optimization using the General Graph Optimization (G2O) algorithm to calculate the coordinate transformation relationship T between adjacent point clouds in the first point cloud sequence L1. i =F(P i-1 P i ), where P i-1 Let P be the coordinates of the (i-1)th point in the first point cloud sequence. i Let T be the coordinates of the i-th point in the first point cloud sequence. i For adjacent point clouds P in the first point cloud sequence i-1 and P iThe transformation relationship.

[0040] Step S122: Transform the first point cloud sequence to the body coordinate system of the first lidar at the preset time according to the coordinate transformation relationship of adjacent point clouds, to form a new first point cloud sequence.

[0041] In this embodiment of the invention, the new first point cloud sequence satisfies the following relationship:

[0042] P i =T1T2…T i P i ,

[0043] Among them, P i Let ' be the coordinates of the i-th point in the new first point cloud sequence, where i is a positive integer, and P i Let T be the coordinates of the i-th point cloud in the first point cloud sequence. i For the adjacent point clouds P in the first point cloud sequence i-1 and P i The transformation relationship, T i =F(P i-1 ,P i ).

[0044] like Figure 3 As shown, the new first point cloud sequence before calibration is represented as L'1={P'0,P'1,…P' n The new second point cloud sequence is represented as L'2 = {P'0, P'1, ..., P'}. m}

[0045] In this embodiment of the invention, preferably, a starting time is selected, and the first point cloud sequence and the second point cloud sequence are transformed to the body coordinate system of the corresponding lidar at the starting time. That is, the first point cloud sequence is transformed to the body coordinate system of the first lidar at the starting time, and the second point cloud sequence is transformed to the body coordinate system of the second lidar at the starting time.

[0046] Step S123: Merge the new first point cloud sequence to obtain the first point cloud map based on the body coordinate system of the first lidar at the preset time.

[0047] For the second point cloud sequence, a simultaneous localization and mapping (SLAM) algorithm is applied to calculate the transformation relationship between adjacent point clouds in the second point cloud sequence. Specifically, the second point cloud sequence is traversed, and a point cloud registration algorithm is applied to calculate the transformation relationship between any two adjacent point clouds in the second point cloud sequence; a general graph optimization algorithm is used to optimize the transformation relationship between any two adjacent point clouds in the second point cloud sequence. For the second point cloud sequence L2, from 1 to m, the coordinate transformation relationship T between adjacent point clouds in the first point cloud sequence L2 is calculated using the SLAM algorithm.j =F(P j-1 P j ), where P j-1 Let P be the coordinates of the (j-1)th point in the first point cloud sequence. j Let T be the coordinates of the j-th point in the first point cloud sequence. j For adjacent point clouds P in the first point cloud sequence j-1 and P j The transformation relationship.

[0048] Then, based on the transformation relationship between adjacent point clouds, the second point cloud sequence is transformed to the body coordinate system of the second lidar at the preset time, forming a new second point cloud sequence; subsequently, the new second point cloud sequences are merged to obtain a second point cloud map based on the body coordinate system of the second lidar at the preset time, wherein the second point cloud map partially overlaps with the first point cloud map. The new second point cloud sequence satisfies the following relationship:

[0049] P j =T1T2…T j P j ,

[0050] Among them, P j Let ′ be the coordinates of the j-th point in the new second point cloud sequence, where j is a positive integer, and P j Let T be the coordinates of the j-th point cloud in the second point cloud sequence. j For the adjacent point cloud P in the second point cloud sequence j-1 and P j The transformation relationship, T j =F(P j-1 ,P j The new second point cloud sequence before calibration is represented as L'2 = {P'0, P'1, ..., P'}. m The single-frame point clouds in the new first point cloud sequence L'1 before calibration and the single-frame point clouds in the new second point cloud sequence L'2 before calibration are as follows: Figure 3 As shown.

[0051] In this embodiment of the invention, the first point cloud map and the second point cloud map are respectively based on different body coordinate systems at the same preset time. The n point clouds in the new first point cloud sequence before calibration are merged to obtain the first point cloud map M1 based on the body coordinate system of the first lidar at the previously selected preset time before calibration. The first point cloud map M1 is equivalent to the superposition of the n point clouds in the first point cloud sequence, and its shape is similar to... Figure 3Similar to L'1 in the previous example, but with a denser point density. Correspondingly, the m point clouds in the new second point cloud sequence are merged to obtain the second point cloud map M2 in the body coordinate system of the second lidar before calibration, based on the previously selected preset time. The second point cloud map M2 is equivalent to the superposition of the m point clouds in the second point cloud sequence, and its shape is similar to... Figure 3 Similar to L'2, but with a denser point density. The first point cloud map M1 and the second point cloud map M2 partially overlap.

[0052] Step S13: Apply the point cloud registration algorithm to calculate the transformation relationship between the second point cloud map and the first point cloud map, and obtain the calibration result.

[0053] Specifically, a point cloud registration algorithm is applied to calculate the transformation relationship from the second point cloud map M2 to the first point cloud map M1, i.e., the transformation matrix T = F(M1, M2), which is the final calibration matrix. The point cloud registration algorithm includes the Iterative Closest Point (ICP) algorithm or the Normal Distribution Transform (NDT) algorithm. In other embodiments of this invention, other point cloud registration algorithms can also be applied, and there are no limitations here. In this embodiment of the invention, the transformation relationship from the first point cloud map M1 to the second point cloud map M2 can also be calculated. Based on the calibration results, the second point cloud map M2 and the first point cloud map M1 are calibrated to obtain a calibrated point cloud map. The calibration process is equivalent to transforming the first point cloud map M1 and the second point cloud map M2 to the same coordinate system. Thus, the calibrated point cloud map obtained after calibrating the first point cloud map M1 and the second point cloud map M2 is equivalent to offsetting the first point cloud map M1 or the second point cloud map M2 by a certain distance and / or rotating it by a certain angle relative to another point cloud map to make them overlap as much as possible.

[0054] To accelerate computation, filtering operations are performed on the obtained first and second point cloud maps before step S13. Specifically, filtering operations such as thinning, denoising, and feature point extraction are performed on the first and second point cloud maps to reduce the amount of data and improve data quality, facilitating subsequent calibration. The thinning algorithm includes grid downsampling algorithms, such as the octree grid thinning algorithm.

[0055] In this embodiment of the invention, after obtaining the calibration structure of the second point cloud map M2 relative to the first point cloud map M1, i.e., the calibration matrix T, the calibration matrix T can be directly applied to calibrate the second point cloud sequence L2 of the second lidar and the first point cloud sequence L1 of the first lidar. Figure 4 This refers to the point cloud after calibration of the second point cloud sequence L2 of the second lidar and the first point cloud sequence L1 of the first lidar.

[0056] It should be noted that when there are multiple lidars, the calibration matrix of each lidar's point cloud sequence relative to the point cloud sequence of one of the lidars can be obtained first, and then each calibration matrix can be calibrated. For example, for point cloud sequences L1, L2, and L3 of three lidars, the calibration matrix of the second lidar's point cloud sequence L2 relative to the first lidar's point cloud sequence L1, and the calibration matrix of the third lidar's point cloud sequence L3 relative to the first lidar's point cloud sequence L1 can be obtained first, and then the point cloud sequences of these three lidars can be calibrated to obtain the calibrated point cloud. Alternatively, the calibration matrices of any two lidar point cloud sequences can be obtained first, and then each calibration matrix can be calibrated. For example, given point cloud sequences L1, L2, and L3 from three lidar sensors, first obtain the calibration matrix of the second lidar sensor's point cloud sequence L2 relative to the first lidar sensor's point cloud sequence L1, and the calibration matrix of the third lidar sensor's point cloud sequence L3 relative to the second lidar sensor's point cloud sequence L2. Then, calibrate the point cloud sequences of the three lidar sensors based on these two calibration matrices to obtain the calibrated point cloud. Both methods ultimately yield the same calibrated point cloud. Figure 5 This is the point cloud after calibration of the point cloud sequences L1, L2, and L3 of three lidar sensors.

[0057] In an embodiment of the present invention, the extrinsic parameter calibration method includes: acquiring at least a first point cloud sequence of a moving or rotating first lidar and a second point cloud sequence of a second lidar, wherein the first point cloud sequence is located in the body coordinate system of the first lidar and the second point cloud sequence is located in the body coordinate system of the second lidar; obtaining a first point cloud map of the first lidar and a second point cloud map of the second lidar at a preset time based on the first point cloud sequence and the second point cloud sequence, respectively; and applying a point cloud registration algorithm to calculate the transformation relationship from the second point cloud map to the first point cloud map to obtain a calibration result. Therefore, by moving or rotating the carrier carrying the lidar, the field of view of a single lidar is expanded, point clouds in different coordinate systems at different times are transformed to the same coordinate system, time registration of multiple lidars is performed, and point cloud maps represented in their respective body coordinate systems at the same time are used to calculate the relative coordinate transformation relationship between the point cloud maps, thereby achieving extrinsic parameter calibration of the lidars. This allows multiple lidars with insufficient overlapping areas to use the point cloud registration algorithm for extrinsic parameter calibration.

[0058] Figure 6 A schematic diagram of the external parameter calibration device according to an embodiment of the present invention is shown. Figure 6 As shown, the external parameter calibration device includes: a data acquisition unit 601, a registration unit 602, and a calibration unit 603.

[0059] in:

[0060] The data acquisition unit 601 is used to acquire at least a first point cloud sequence of a moving or rotating first lidar and a second point cloud sequence of a second lidar, wherein the first point cloud sequence is located in the body coordinate system of the first lidar and the second point cloud sequence is located in the body coordinate system of the second lidar; the registration unit 602 is used to obtain a first point cloud map of the first lidar and a second point cloud map of the second lidar at a preset time based on the first point cloud sequence and the second point cloud sequence, respectively; the calibration unit 603 is used to apply a point cloud registration algorithm to calculate the transformation relationship from the second point cloud map to the first point cloud map and obtain a calibration result.

[0061] In one alternative approach, the first and second lidars, which are moved or rotated, are located in a site with structured features.

[0062] In one optional embodiment, the registration unit 602 is configured to: apply a simultaneous localization and mapping (SLT) algorithm to calculate the transformation relationship between adjacent point clouds in the first point cloud sequence; select a preset time and transform the first point cloud sequence to the body coordinate system of the first lidar at the preset time according to the coordinate transformation relationship between adjacent point clouds, forming a new first point cloud sequence; merge the new first point cloud sequences to obtain a first point cloud map based on the body coordinate system of the first lidar at the preset time; apply a SLT algorithm to calculate the transformation relationship between adjacent point clouds in the second point cloud sequence; transform the second point cloud sequence to the body coordinate system of the second lidar at the preset time according to the transformation relationship between adjacent point clouds, forming a new second point cloud sequence; merge the new second point cloud sequences to obtain a second point cloud map based on the body coordinate system of the second lidar at the preset time, wherein the second point cloud map partially overlaps with the first point cloud map.

[0063] In an optional manner, the registration unit 602 is further configured to: traverse the first point cloud sequence, apply a point cloud registration algorithm to calculate the transformation relationship of any adjacent point cloud in the first point cloud sequence; optimize the transformation relationship of any adjacent point cloud in the first point cloud sequence using a general graph optimization algorithm; traverse the second point cloud sequence, apply a point cloud registration algorithm to calculate the transformation relationship of any adjacent point cloud in the second point cloud sequence; and optimize the transformation relationship of any adjacent point cloud in the second point cloud sequence using a general graph optimization algorithm.

[0064] In one alternative approach, the new first point cloud sequence satisfies the following relation:

[0065] P i =T1T2…T i P i ,

[0066] Among them, P i Let ' be the coordinates of the i-th point in the new first point cloud sequence, where i is a positive integer, and P i Let T be the coordinates of the i-th point cloud in the first point cloud sequence. i For the adjacent point clouds P in the first point cloud sequence i-1 and P i The transformation relationship, T i =F(P i-1 ,P i );

[0067] The new second point cloud sequence satisfies the following relationship:

[0068] P j =T1T2…T j P j ,

[0069] Among them, P j Let ′ be the coordinates of the j-th point in the new second point cloud sequence, where j is a positive integer, and P j Let T be the coordinates of the j-th point cloud in the second point cloud sequence. j For the adjacent point cloud P in the second point cloud sequence j-1 and P j The transformation relationship, T j =F(P j-1 ,P j ).

[0070] In an alternative embodiment, the registration unit 602 is further configured to perform filtering operations on the first point cloud map and the second point cloud map, respectively.

[0071] In one alternative approach, the point cloud registration algorithm includes an iterative nearest-point algorithm or a normal distribution transformation algorithm.

[0072] In an embodiment of the present invention, the extrinsic parameter calibration method includes: acquiring at least a first point cloud sequence of a moving or rotating first lidar and a second point cloud sequence of a second lidar, wherein the first point cloud sequence is located in the body coordinate system of the first lidar and the second point cloud sequence is located in the body coordinate system of the second lidar; obtaining a first point cloud map of the first lidar and a second point cloud map of the second lidar at a preset time based on the first point cloud sequence and the second point cloud sequence, respectively; and applying a point cloud registration algorithm to calculate the transformation relationship from the second point cloud map to the first point cloud map to obtain a calibration result. Therefore, by moving or rotating the carrier carrying the lidar, the field of view of a single lidar is expanded, point clouds in different coordinate systems at different times are transformed to the same coordinate system, time registration of multiple lidars is performed, and point cloud maps represented in their respective body coordinate systems at the same time are used to calculate the relative coordinate transformation relationship between the point cloud maps, thereby achieving extrinsic parameter calibration of the lidars. This allows multiple lidars with insufficient overlapping areas to use a general point cloud registration algorithm for extrinsic parameter calibration.

[0073] This invention provides a non-volatile computer storage medium storing at least one executable instruction that can execute the external parameter calibration method in any of the above method embodiments.

[0074] Executable instructions can specifically be used to cause the processor to perform the following operations:

[0075] At least a first point cloud sequence from a first moving or rotating lidar and a second point cloud sequence from a second lidar are acquired, wherein the first point cloud sequence is located in the body coordinate system of the first lidar and the second point cloud sequence is located in the body coordinate system of the second lidar.

[0076] The first point cloud map of the first lidar and the second point cloud map of the second lidar are obtained at preset times based on the first point cloud sequence and the second point cloud sequence, respectively.

[0077] The point cloud registration algorithm is applied to calculate the transformation relationship between the second point cloud map and the first point cloud map to obtain the calibration result.

[0078] In one alternative approach, the first and second lidars, which are moved or rotated, are located in a site with structured features.

[0079] In one alternative approach, the executable instructions can specifically be used to cause the processor to perform the following operations:

[0080] Simultaneous localization and mapping algorithm is applied to calculate the transformation relationship between adjacent point clouds in the first point cloud sequence;

[0081] The first point cloud sequence is transformed to the body coordinate system of the first lidar at the preset time according to the coordinate transformation relationship of adjacent point clouds, forming a new first point cloud sequence.

[0082] The new first point cloud sequence is merged to obtain the first point cloud map based on the body coordinate system of the first lidar at the preset time.

[0083] Simultaneous localization and mapping algorithm is applied to calculate the transformation relationship between adjacent point clouds in the second point cloud sequence;

[0084] The second point cloud sequence is transformed to the body coordinate system of the second lidar at the preset time according to the transformation relationship between adjacent point clouds, forming a new second point cloud sequence;

[0085] The new second point cloud sequence is merged to obtain the second point cloud map based on the body coordinate system of the second lidar at the preset time, wherein the second point cloud map partially overlaps with the first point cloud map.

[0086] In one alternative approach, the executable instructions can specifically be used to cause the processor to perform the following operations:

[0087] Traverse the first point cloud sequence and apply the point cloud registration algorithm to calculate the transformation relationship between any two adjacent point clouds in the first point cloud sequence;

[0088] The transformation relationship between any two adjacent point clouds in the first point cloud sequence and the second point cloud sequence is optimized using a general graph optimization algorithm.

[0089] Traverse the second point cloud sequence and apply the point cloud registration algorithm to calculate the transformation relationship between any two adjacent point clouds in the second point cloud sequence;

[0090] The transformation relationship between any two adjacent point clouds in the second point cloud sequence is optimized using a general graph optimization algorithm.

[0091] In one alternative approach, the new first point cloud sequence satisfies the following relation:

[0092] P i =T1T2…T i P i ,

[0093] Among them, P i Let ' be the coordinates of the i-th point in the new first point cloud sequence, where i is a positive integer, and P i Let T be the coordinates of the i-th point cloud in the first point cloud sequence. i For the adjacent point clouds P in the first point cloud sequence i-1 and P iThe transformation relationship, T i =F(P i-1 ,P i );

[0094] The new second point cloud sequence satisfies the following relationship:

[0095] P j =T1T2…T j P j ,

[0096] Among them, P j Let ′ be the coordinates of the j-th point in the new second point cloud sequence, where j is a positive integer, and P j Let T be the coordinates of the j-th point cloud in the second point cloud sequence. j For the adjacent point cloud P in the second point cloud sequence j-1 and P j The transformation relationship, T j =F(P j-1 ,P j ).

[0097] In one alternative approach, the executable instructions can specifically be used to cause the processor to perform the following operations:

[0098] Filtering operations are performed on the first point cloud map and the second point cloud map respectively.

[0099] In one alternative approach, the point cloud registration algorithm includes an iterative nearest-point algorithm or a normal distribution transformation algorithm.

[0100] In an embodiment of the present invention, the extrinsic parameter calibration method includes: acquiring at least a first point cloud sequence of a moving or rotating first lidar and a second point cloud sequence of a second lidar, wherein the first point cloud sequence is located in the body coordinate system of the first lidar and the second point cloud sequence is located in the body coordinate system of the second lidar; obtaining a first point cloud map of the first lidar and a second point cloud map of the second lidar at a preset time based on the first point cloud sequence and the second point cloud sequence, respectively; and applying a point cloud registration algorithm to calculate the transformation relationship from the second point cloud map to the first point cloud map to obtain a calibration result. Therefore, by moving or rotating the carrier carrying the lidar, the field of view of a single lidar is expanded, point clouds in different coordinate systems at different times are transformed to the same coordinate system, time registration of multiple lidars is performed, and point cloud maps represented in their respective body coordinate systems at the same time are used to calculate the relative coordinate transformation relationship between the point cloud maps, thereby achieving extrinsic parameter calibration of the lidars. This allows multiple lidars with insufficient overlapping areas to use a general point cloud registration algorithm for extrinsic parameter calibration.

[0101] This invention provides a computer program product, which includes a computer program stored on a computer storage medium. The computer program includes program instructions, which, when executed by a computer, cause the computer to perform the external parameter calibration method in any of the above method embodiments.

[0102] Executable instructions can specifically be used to cause the processor to perform the following operations:

[0103] At least a first point cloud sequence from a first moving or rotating lidar and a second point cloud sequence from a second lidar are acquired, wherein the first point cloud sequence is located in the body coordinate system of the first lidar and the second point cloud sequence is located in the body coordinate system of the second lidar.

[0104] The first point cloud map of the first lidar and the second point cloud map of the second lidar are obtained at preset times based on the first point cloud sequence and the second point cloud sequence, respectively.

[0105] The point cloud registration algorithm is applied to calculate the transformation relationship between the second point cloud map and the first point cloud map to obtain the calibration result.

[0106] In one alternative approach, the first and second lidars, which are moved or rotated, are located in a site with structured features.

[0107] In one alternative approach, the executable instructions can specifically be used to cause the processor to perform the following operations:

[0108] Simultaneous localization and mapping algorithm is applied to calculate the transformation relationship between adjacent point clouds in the first point cloud sequence;

[0109] The first point cloud sequence is transformed to the body coordinate system of the first lidar at the preset time according to the coordinate transformation relationship of adjacent point clouds, forming a new first point cloud sequence.

[0110] The new first point cloud sequence is merged to obtain the first point cloud map based on the body coordinate system of the first lidar at the preset time.

[0111] Simultaneous localization and mapping algorithm is applied to calculate the transformation relationship between adjacent point clouds in the second point cloud sequence;

[0112] The second point cloud sequence is transformed to the body coordinate system of the second lidar at the preset time according to the transformation relationship between adjacent point clouds, forming a new second point cloud sequence;

[0113] The new second point cloud sequence is merged to obtain the second point cloud map based on the body coordinate system of the second lidar at the preset time, wherein the second point cloud map partially overlaps with the first point cloud map.

[0114] In one alternative approach, the executable instructions can specifically be used to cause the processor to perform the following operations:

[0115] Traverse the first point cloud sequence and apply the point cloud registration algorithm to calculate the transformation relationship between any two adjacent point clouds in the first point cloud sequence;

[0116] The transformation relationship between any two adjacent point clouds in the first point cloud sequence is optimized using a general graph optimization algorithm.

[0117] Traverse the second point cloud sequence and apply the point cloud registration algorithm to calculate the transformation relationship between any two adjacent point clouds in the second point cloud sequence;

[0118] The transformation relationship between any two adjacent point clouds in the second point cloud sequence is optimized using a general graph optimization algorithm.

[0119] In one alternative approach, the new first point cloud sequence satisfies the following relation:

[0120] P i =T1T2…T i P i ,

[0121] Among them, P i Let ' be the coordinates of the i-th point in the new first point cloud sequence, where i is a positive integer, and P i Let T be the coordinates of the i-th point cloud in the first point cloud sequence. i For the adjacent point clouds P in the first point cloud sequence i-1 and P i The transformation relationship, T i =F(P i-1 ,P i );

[0122] The new second point cloud sequence satisfies the following relationship:

[0123] P j =T1T2…T j P j ,

[0124] Among them, P j Let ′ be the coordinates of the j-th point in the new second point cloud sequence, where j is a positive integer, and P j Let T be the coordinates of the j-th point cloud in the second point cloud sequence. j For the adjacent point cloud P in the second point cloud sequence j-1 and Pj The transformation relationship, T j =F(P j-1 ,P j ).

[0125] In one alternative approach, the executable instructions can specifically be used to cause the processor to perform the following operations:

[0126] Filtering operations are performed on the first point cloud map and the second point cloud map respectively.

[0127] In one alternative approach, the point cloud registration algorithm includes an iterative nearest-point algorithm or a normal distribution transformation algorithm.

[0128] In an embodiment of the present invention, the extrinsic parameter calibration method includes: acquiring at least a first point cloud sequence of a moving or rotating first lidar and a second point cloud sequence of a second lidar, wherein the first point cloud sequence is located in the body coordinate system of the first lidar and the second point cloud sequence is located in the body coordinate system of the second lidar; obtaining a first point cloud map of the first lidar and a second point cloud map of the second lidar at a preset time based on the first point cloud sequence and the second point cloud sequence, respectively; and applying a point cloud registration algorithm to calculate the transformation relationship from the second point cloud map to the first point cloud map to obtain a calibration result. Therefore, by moving or rotating the carrier carrying the lidar, the field of view of a single lidar is expanded, point clouds in different coordinate systems at different times are transformed to the same coordinate system, time registration of multiple lidars is performed, and point cloud maps represented in their respective body coordinate systems at the same time are used to calculate the relative coordinate transformation relationship between the point cloud maps, thereby achieving extrinsic parameter calibration of the lidars. This allows multiple lidars with insufficient overlapping areas to use a general point cloud registration algorithm for extrinsic parameter calibration.

[0129] Figure 7 The diagram shows a structural schematic of an embodiment of the device of the present invention. The specific embodiments of the present invention do not limit the specific implementation of the device.

[0130] like Figure 7 As shown, the device may include: a processor 702, a communications interface 704, a memory 706, and a communications bus 708.

[0131] The processor 702, communication interface 704, and memory 706 communicate with each other via communication bus 708. Communication interface 704 is used to communicate with other network elements such as clients or other servers. The processor 702 executes program 710, specifically performing the relevant steps in the above-described external parameter calibration method embodiment.

[0132] Specifically, program 710 may include program code that includes computer operation instructions.

[0133] The processor 702 may be a central processing unit (CPU), an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement embodiments of the present invention. The device may include one or more processors of the same type, such as one or more CPUs; or it may include processors of different types, such as one or more CPUs and one or more ASICs.

[0134] Memory 706 is used to store program 710. Memory 706 may include high-speed RAM memory, and may also include non-volatile memory, such as at least one disk storage device.

[0135] Specifically, program 710 can be used to cause processor 702 to perform the following operations:

[0136] At least a first point cloud sequence from a first moving or rotating lidar and a second point cloud sequence from a second lidar are acquired, wherein the first point cloud sequence is located in the body coordinate system of the first lidar and the second point cloud sequence is located in the body coordinate system of the second lidar.

[0137] The first point cloud map of the first lidar and the second point cloud map of the second lidar are obtained at preset times based on the first point cloud sequence and the second point cloud sequence, respectively.

[0138] The point cloud registration algorithm is applied to calculate the transformation relationship between the second point cloud map and the first point cloud map to obtain the calibration result.

[0139] In one alternative approach, the first and second lidars, which are moved or rotated, are located in a site with structured features.

[0140] In an alternative approach, program 710 can specifically be used to cause processor 702 to perform the following operations:

[0141] Simultaneous localization and mapping algorithm is applied to calculate the transformation relationship between adjacent point clouds in the first point cloud sequence;

[0142] The first point cloud sequence is transformed to the body coordinate system of the first lidar at the preset time according to the coordinate transformation relationship of adjacent point clouds, forming a new first point cloud sequence.

[0143] The new first point cloud sequence is merged to obtain the first point cloud map based on the body coordinate system of the first lidar at the preset time.

[0144] Simultaneous localization and mapping algorithm is applied to calculate the transformation relationship between adjacent point clouds in the second point cloud sequence;

[0145] The second point cloud sequence is transformed to the body coordinate system of the second lidar at the preset time according to the transformation relationship between adjacent point clouds, forming a new second point cloud sequence;

[0146] The new second point cloud sequence is merged to obtain the second point cloud map based on the body coordinate system of the second lidar at the preset time, wherein the second point cloud map partially overlaps with the first point cloud map.

[0147] In an alternative approach, program 710 can specifically be used to cause processor 702 to perform the following operations:

[0148] Traverse the first point cloud sequence and apply the point cloud registration algorithm to calculate the transformation relationship between any two adjacent point clouds in the first point cloud sequence;

[0149] The transformation relationship between any two adjacent point clouds in the first point cloud sequence is optimized using a general graph optimization algorithm.

[0150] Traverse the second point cloud sequence and apply the point cloud registration algorithm to calculate the transformation relationship between any two adjacent point clouds in the second point cloud sequence;

[0151] The transformation relationship between any two adjacent point clouds in the second point cloud sequence is optimized using a general graph optimization algorithm.

[0152] In one alternative approach, the new first point cloud sequence satisfies the following relation:

[0153] P i =T1T2…T i P i ,

[0154] Among them, P i Let ' be the coordinates of the i-th point in the new first point cloud sequence, where i is a positive integer, and P i Let T be the coordinates of the i-th point cloud in the first point cloud sequence. i For the adjacent point clouds P in the first point cloud sequence i-1 and P i The transformation relationship, T i =F(P i-1 ,P i );

[0155] The new second point cloud sequence satisfies the following relationship:

[0156] P j =T1T2…T j P j ,

[0157] Among them, P j Let ′ be the coordinates of the j-th point in the new second point cloud sequence, where j is a positive integer, and P j Let T be the coordinates of the j-th point cloud in the second point cloud sequence. j For the adjacent point cloud P in the second point cloud sequence j-1 and P j The transformation relationship, T j =F(P j-1 ,P j ).

[0158] In an alternative approach, program 710 can specifically be used to cause processor 702 to perform the following operations:

[0159] Filtering operations are performed on the first point cloud map and the second point cloud map respectively.

[0160] In one alternative approach, the point cloud registration algorithm includes an iterative nearest-point algorithm or a normal distribution transformation algorithm.

[0161] In an embodiment of the present invention, the extrinsic parameter calibration method includes: acquiring at least a first point cloud sequence of a moving or rotating first lidar and a second point cloud sequence of a second lidar, wherein the first point cloud sequence is located in the body coordinate system of the first lidar and the second point cloud sequence is located in the body coordinate system of the second lidar; obtaining a first point cloud map of the first lidar and a second point cloud map of the second lidar at a preset time based on the first point cloud sequence and the second point cloud sequence, respectively; and applying a point cloud registration algorithm to calculate the transformation relationship from the second point cloud map to the first point cloud map to obtain a calibration result. Therefore, by moving or rotating the carrier carrying the lidar, the field of view of a single lidar is expanded, point clouds in different coordinate systems at different times are transformed to the same coordinate system, time registration of multiple lidars is performed, and point cloud maps represented in their respective body coordinate systems at the same time are used to calculate the relative coordinate transformation relationship between the point cloud maps, thereby achieving extrinsic parameter calibration of the lidars. This allows multiple lidars with insufficient overlapping areas to use a general point cloud registration algorithm for extrinsic parameter calibration.

[0162] The algorithms or displays provided herein are not inherently related to any particular computer, virtual system, or other device. Various general-purpose systems can also be used in conjunction with the teachings herein. The required structure for constructing such systems is apparent from the above description. Furthermore, the embodiments of the present invention are not directed to any particular programming language. It should be understood that the content of the invention described herein can be implemented using various programming languages, and the above description of specific languages ​​is for the purpose of disclosing the best mode of implementation of the invention.

[0163] Numerous specific details are set forth in the specification provided herein. However, it will be understood that embodiments of the invention may be practiced without these specific details. In some instances, well-known methods, structures, and techniques have not been shown in detail so as not to obscure the understanding of this specification.

[0164] Similarly, it should be understood that, in order to simplify this disclosure and aid in understanding one or more of the various aspects of the invention, in the above description of exemplary embodiments of the invention, various features of the invention are sometimes grouped together in a single embodiment, figure, or description thereof. However, this method of disclosure should not be construed as reflecting an intention that the claimed invention requires more features than are expressly recited in each claim. Rather, as reflected in the following claims, inventive aspects lie in fewer than all features of a single foregoing disclosed embodiment. Therefore, the claims following the detailed description are hereby expressly incorporated into this detailed description, wherein each claim itself is a separate embodiment of the invention.

[0165] Those skilled in the art will understand that modules in the device of the embodiments can be adaptively changed and placed in one or more devices different from that embodiment. Modules, units, or components in the embodiments can be combined into a single module, unit, or component, and further, they can be divided into multiple sub-modules, sub-units, or sub-components. Except where at least some of such features and / or processes or units are mutually exclusive, any combination can be used to combine all features disclosed in this specification (including the accompanying claims, abstract, and drawings) and all processes or units of any method or device so disclosed. Unless expressly stated otherwise, each feature disclosed in this specification (including the accompanying claims, abstract, and drawings) may be replaced by an alternative feature that serves the same, equivalent, or similar purpose.

[0166] Furthermore, those skilled in the art will understand that although some embodiments described herein include certain features but not others included in other embodiments, combinations of features from different embodiments are intended to be within the scope of the invention and form different embodiments. For example, in the following claims, any of the claimed embodiments can be used in any combination.

[0167] It should be noted that the above embodiments are illustrative of the invention and not restrictive, and that those skilled in the art can devise alternative embodiments without departing from the scope of the appended claims. In the claims, any reference signs placed between parentheses should not be construed as limiting the claims. The word "comprising" does not exclude the presence of elements or steps not listed in the claims. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. The invention can be implemented by means of hardware comprising several different elements and by means of a suitably programmed computer. In the unit claims enumerating several means, several of these means may be embodied by the same item of hardware. The use of the words first, second, and third, etc., does not indicate any order. These words can be interpreted as names.

Claims

1. An external parameter calibration device, characterized in that, The external parameter calibration device includes: A data acquisition unit is used to acquire at least a first point cloud sequence of a moving or rotating first lidar and a second point cloud sequence of a second lidar, wherein the first point cloud sequence is located in the body coordinate system of the first lidar and the second point cloud sequence is located in the body coordinate system of the second lidar. The registration unit is used to obtain, based on the first point cloud sequence and the second point cloud sequence, a first point cloud map of the first lidar and a second point cloud map of the second lidar at a preset time, respectively, wherein the second point cloud map partially overlaps with the first point cloud map. The calibration unit is used to apply a point cloud registration algorithm to calculate the transformation relationship between the second point cloud map and the first point cloud map, and obtain the calibration result; The step of obtaining the first point cloud map of the first lidar and the second point cloud map of the second lidar at a preset time based on the first point cloud sequence and the second point cloud sequence respectively includes transforming the first point cloud sequence to the body coordinate system of the first lidar at the preset time to obtain the first point cloud map; and transforming the second point cloud sequence to the body coordinate system of the second lidar at the preset time to obtain the second point cloud map.

2. The external parameter calibration device as described in claim 1, characterized in that, The first and second lidars, which are moving or rotating, are located in a site with structured features.

3. The external parameter calibration device as described in claim 1, characterized in that, The registration unit is used to obtain a first point cloud map of the first lidar at a preset time based on the first point cloud sequence, including: Simultaneous localization and mapping algorithm is applied to calculate the transformation relationship between adjacent point clouds in the first point cloud sequence; The first point cloud sequence is transformed to the body coordinate system of the first lidar at the preset time according to the transformation relationship between adjacent point clouds, forming a new first point cloud sequence. The new first point cloud sequence is merged to obtain the first point cloud map based on the body coordinate system of the first lidar at the preset time. The step of obtaining the second point cloud map of the second lidar at a preset time based on the second point cloud sequence includes: Simultaneous localization and mapping algorithm is applied to calculate the transformation relationship between adjacent point clouds in the second point cloud sequence; The second point cloud sequence is transformed to the body coordinate system of the second lidar at the preset time according to the transformation relationship between adjacent point clouds, forming a new second point cloud sequence; The new second point cloud sequence is merged to obtain the second point cloud map based on the body coordinate system of the second lidar at the preset time.

4. The external parameter calibration device as described in claim 3, characterized in that, The application simultaneously uses a localization and mapping algorithm to calculate the transformation relationship between adjacent point clouds in the first point cloud sequence, including: Traverse the first point cloud sequence and apply the point cloud registration algorithm to calculate the transformation relationship between any two adjacent point clouds in the first point cloud sequence; The transformation relationship between any two adjacent point clouds in the first point cloud sequence is optimized using a general graph optimization algorithm. The application simultaneously uses a localization and mapping algorithm to calculate the transformation relationship between adjacent point clouds in the second point cloud sequence, including: Traverse the second point cloud sequence and apply the point cloud registration algorithm to calculate the transformation relationship between any two adjacent point clouds in the second point cloud sequence; The transformation relationship between any two adjacent point clouds in the second point cloud sequence is optimized using a general graph optimization algorithm.

5. The external parameter calibration device as described in claim 3, characterized in that, The new first point cloud sequence satisfies the following relationship: P′ i =T1T2…T i P i , Among them, P′ i Let P be the coordinates of the i-th point in the new first point cloud sequence, where i is a positive integer. i Let T be the coordinates of the i-th point cloud in the first point cloud sequence. i For the first point cloud sequence, the adjacent point cloud P i-1 and P i The transformation relationship, T i =F(P i-1 ,P i ); The new second point cloud sequence satisfies the following relationship: P′ j =T1T2…T j P j , Among them, P′ j Let P be the coordinates of the j-th point in the new second point cloud sequence, where j is a positive integer. j Let T be the coordinates of the j-th point cloud in the second point cloud sequence. j For the adjacent point cloud P in the second point cloud sequence j-1 and P j The transformation relationship, T j =F(P j-1 ,P j ).

6. The external parameter calibration device as described in claim 1, characterized in that, Before applying the point cloud registration algorithm to calculate the transformation relationship from the second point cloud map to the first point cloud map and obtain the calibration result, the following steps are included: Filtering operations are performed on the first point cloud map and the second point cloud map respectively.

7. The external parameter calibration device according to any one of claims 1-6, characterized in that, The point cloud registration algorithm includes the iterative nearest point algorithm or the normal distribution transformation algorithm.

Images