A method, device and equipment for extrinsic parameter calibration of laser radar and IMU

Abstract

The present application discloses a method, device, and apparatus for extrinsic parameter calibration of a laser radar and an inertial measurement unit (IMU), which is used to address the problem in the prior art that the inter-frame matching algorithm is not suitable for extrinsic parameter calibration of tilted laser radars. The calibration apparatus converts the point cloud data collected by the laser radar into the IMU coordinate system based on the extrinsic parameter value to be optimized; converts the point cloud data from the IMU coordinate system to the reference coordinate system based on the relative conversion relationship between the IMU coordinate system and the reference coordinate system; and iteratively adjusts the extrinsic parameter value to be optimized based on the point cloud data in the reference coordinate system to obtain the target extrinsic parameter value, thereby preventing the extrinsic parameter calibration result from being affected by the installation angle of the laser radar, achieving automated calibration, and improving the efficiency of extrinsic parameter calibration.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of autonomous driving technology, and in particular to a method, device, and equipment for calibrating external parameters of a lidar and an IMU. Background Art

[0002] High-precision maps are the foundation and essential geographic information for lane-level navigation and monitoring of autonomous vehicles. These maps rely primarily on sensors such as inertial measurement units (IMUs), global navigation satellite systems (GNSS), and lidar for data collection and production. To represent data from different sensors in the same coordinate system, calibration of the extrinsic parameters between the sensors is necessary.

[0003] Currently, the LiDAR on a vehicle is typically mounted parallel to the vehicle's roof. During calibration, the LiDAR uses point cloud data from common features between two consecutive frames to calculate the vehicle's pose change using an inter-frame matching algorithm, such as the iterative closest point (ICP) algorithm. This yields the LiDAR's trajectory, which is then combined with the integrated navigation system to produce the IMU's trajectory. The LiDAR's extrinsic parameters relative to the IMU are then calibrated using the least squares method.

[0004] However, in order to improve the accuracy of ground feature recognition, the radar in the collection vehicle can be installed at an angle when collecting high-precision maps. However, the range of features scanned by the tilted lidar per unit time is reduced. The features that can be scanned are mainly features on the ground, on both sides of the vehicle, and diagonally above the roof, while features in front of the collection vehicle cannot be scanned, resulting in a reduction in the features scanned between adjacent frames, making the inter-frame matching error larger, and affecting the accuracy of the external parameter calibration. Summary of the Invention

[0005] The embodiments of the present application provide a method, device and equipment for extrinsic parameter calibration of a laser radar and an IMU, which are used to solve the problem in the prior art that the inter-frame matching algorithm is not suitable for extrinsic parameter calibration of a laser radar installed at an angle.

[0006] In a first aspect, an embodiment of the present application provides an extrinsic parameter calibration method for a laser radar and an IMU, which is applied to a calibration device, wherein the laser radar and the IMU are both fixedly mounted on a vehicle to be calibrated, and the method includes:

[0007] Acquire first point cloud data collected by a laser radar, where the first point cloud data is used to represent positions of feature objects around the vehicle to be calibrated collected when the vehicle to be calibrated is traveling on a target path in a laser radar coordinate system;

[0008] In the current adjustment, the first point cloud data is converted from the lidar coordinate system to the IMU coordinate system to obtain second point cloud data according to the extrinsic parameter value to be optimized, wherein the extrinsic parameter value to be optimized is used to indicate a first relative conversion relationship between the lidar coordinate system and the IMU coordinate system;

[0009] The second point cloud data is converted into the reference coordinate system according to a second relative transformation relationship between the IMU coordinate system and the reference coordinate system to obtain third point cloud data; the second relative transformation relationship is obtained based on the position and posture of the vehicle to be calibrated collected by the IMU during the process of the vehicle to be calibrated traveling on the target path;

[0010] According to the third point cloud data, the extrinsic parameter value to be optimized used in the current adjustment is determined so that when the laser radar travels to different positions to collect the same feature object in the reference coordinate system, the position is the same or the position difference meets the preset conditions, the current extrinsic parameter value to be optimized is used as the target extrinsic parameter value; otherwise, the extrinsic parameter value to be optimized is adjusted, and the adjusted extrinsic parameter value is used as the extrinsic parameter value to be optimized in the next adjustment.

[0011] Through the above design, the calibration device converts the point cloud data collected by the laser radar from the laser radar coordinate system to the reference coordinate system, and iteratively optimizes the external parameter value to be optimized according to the point cloud data in the reference coordinate system, until the external parameter value to be optimized used for the current adjustment makes the position of the same feature object collected by the laser radar at different positions in the reference coordinate system the same or the position difference meets the preset conditions. The current external parameter value to be optimized is used as the target external parameter value to avoid the influence of the installation angle of the laser radar, which leads to low accuracy and efficiency of the external parameter calibration result, realizes automated external parameter calibration, and improves the efficiency and accuracy of external parameter calibration.

[0012] In one possible design, the first point cloud data is collected by the laser radar at N first collection moments, and the second relative transformation relationship between the IMU coordinate system and the reference coordinate system is obtained, including: obtaining measurement data collected by the IMU at M second collection moments, the measurement data including the linear acceleration and angular velocity of the vehicle to be calibrated collected by the IMU during the process of the vehicle to be calibrated traveling on the target path; according to the measurement data, the second relative transformation relationship at the M second collection moments is obtained, and the second relative transformation relationship at the second collection moment is used to characterize the relative transformation relationship between the IMU coordinate system and the reference coordinate system at the second collection moment.

[0013] Through the above design, the calibration device calibrates the vehicle's linear acceleration and angular velocity collected by the IMU, obtains the relative transformation relationship between the IMU coordinate system and the reference coordinate system at M second collection moments, and improves the accuracy of external parameter calibration.

[0014] In one possible design, according to the second relative transformation relationship between the IMU coordinate system and the reference coordinate system, the second point cloud data is converted to the reference coordinate system to obtain the third point cloud data, including: according to the second relative transformation relationship at the M second acquisition moments, respectively obtaining the third relative transformation relationship at the N first acquisition moments, the third relative transformation relationship at the first acquisition moment is used to characterize the relative transformation relationship between the IMU coordinate system and the reference coordinate system at the first acquisition moment; according to the third relative transformation relationship at the i-th first acquisition moment, respectively converting the second point cloud data at the i-th first acquisition moment to the reference coordinate system to obtain the point cloud data at the i-th first acquisition moment, where i is a positive integer less than or equal to N, so as to obtain the point cloud data at the N first acquisition moments to constitute the third point cloud data.

[0015] Through the above design, the calibration device can obtain the relative conversion relationship between the IMU coordinate system and the reference coordinate system at N first acquisition moments based on the relative conversion relationship between the IMU coordinate system and the reference coordinate system at M second acquisition moments, thereby realizing the conversion of the second point cloud data at N first acquisition moments from the IMU coordinate system to the reference coordinate system, thereby improving the efficiency and accuracy of the external parameter calibration of the lidar and IMU.

[0016] In one possible design, the third point cloud data is used to represent the three-dimensional coordinates of X features in the reference coordinate system, where X is a positive integer;

[0017] Determining, based on the third point cloud data, an extrinsic parameter value to be optimized for use in the current adjustment so that the positions of the same feature object collected by the laser radar at different locations in the reference coordinate system are the same or the position difference satisfies a preset condition, including: if the difference between the sum of the error parameters of the X feature objects in the current adjustment and the sum of the error parameters of the X feature objects in the previous adjustment is less than a first threshold, determining the extrinsic parameter value to be optimized for use in the current adjustment so that the positions of the same feature object collected by the laser radar at different locations in the reference coordinate system are the same or the position difference satisfies a preset condition; wherein the error parameter of the first feature object is the sum of the variances corresponding to the coordinates of the first feature object in three dimensions of the reference coordinate system, respectively, and the first feature object is any one of the X features.

[0018] Through the above design, based on the sum of the error parameters of the X feature objects, it is determined whether the extrinsic parameter value to be optimized can make the position of the same feature object collected by the lidar at different locations in the reference coordinate system the same or the position difference meet the preset conditions. It can also be understood that the extrinsic parameter value to be optimized can make the same feature object collected by the lidar at different locations overlap or almost overlap in the reference coordinate system, thereby improving the efficiency of the extrinsic parameter calibration. At the same time, it is insensitive to the first extrinsic parameter value to be optimized (i.e., the initial extrinsic parameter value) and has good convergence.

[0019] In one possible design, the third point cloud data is used to represent the three-dimensional coordinates of X features in the reference coordinate system, where X is a positive integer;

[0020] Determining, based on the third point cloud data, an extrinsic parameter value to be optimized for use in the current adjustment so that the positions of the same feature object collected by the laser radar at different locations in the reference coordinate system are the same or the position difference satisfies a preset condition, including: if the difference between the error parameters of the X feature objects in the current adjustment and the error parameters of the X feature objects in the previous adjustment is less than a second threshold, determining the extrinsic parameter value to be optimized for use in the current adjustment so that the positions of the same feature object collected by the laser radar at different locations in the reference coordinate system are the same or the position difference satisfies a preset condition; wherein the error parameter of the first feature object is the sum of the variances corresponding to the coordinates of the first feature object in three dimensions of the reference coordinate system, and the first feature object is any one of the X features.

[0021] Through the above design, based on the error parameters of X feature objects, it is determined whether the extrinsic parameter value to be optimized can make the position of the same feature object collected by the lidar at different locations in the reference coordinate system the same or the position difference meets the preset conditions, thereby improving the accuracy of the extrinsic parameter calibration. In addition, it is insensitive to the initial extrinsic parameter value, has good convergence, and improves the efficiency of the extrinsic parameter calibration.

[0022] In one possible design, the third point cloud data is used to represent the three-dimensional coordinates of X features in the reference coordinate system, where X is a positive integer;

[0023] Determining, based on the third point cloud data, the extrinsic parameter value to be optimized used in the current adjustment so that the positions of the same feature object collected by the laser radar at different positions in the reference coordinate system are the same or the position difference satisfies a preset condition, including: if the difference between the sum of the error parameters of the X feature objects in the current adjustment and the sum of the error parameters of the X feature objects in the previous adjustment is less than a first threshold, and the difference between the extrinsic parameter value to be optimized used in the current adjustment and the extrinsic parameter value to be optimized used in the previous adjustment is less than a third threshold, then determining the extrinsic parameter value to be optimized used in the current adjustment so that the positions of the same feature object collected by the laser radar at different positions in the reference coordinate system are the same or the position difference satisfies a preset condition; wherein the error parameter of the first feature object is the sum of the variances corresponding to the coordinates of the first feature object in three dimensions of the reference coordinate system, and the first feature object is any one of the X features.

[0024] Through the above design, based on the sum of the error parameters of the X feature objects and the extrinsic parameter values ​​to be optimized in two adjacent adjustments, the extrinsic parameter value to be optimized used in the current adjustment is determined so that the position of the same feature object collected by the laser radar at different locations in the reference coordinate system is the same or the position difference meets the preset conditions, while ensuring the accuracy and efficiency of the extrinsic parameter calibration.

[0025] In one possible design, the third point cloud data is used to represent the three-dimensional coordinates of X features in the reference coordinate system, where X is a positive integer;

[0026] Determining, based on the third point cloud data, the extrinsic parameter value to be optimized used in the current adjustment so that the positions of the same feature object collected by the laser radar at different positions in the reference coordinate system are the same or the position difference satisfies a preset condition, including: if the difference between the error parameters of the X feature objects in the current adjustment and the error parameters of the X feature objects in the previous adjustment is less than a second threshold, and the difference between the extrinsic parameter value to be optimized used in the current adjustment and the extrinsic parameter value to be optimized used in the previous adjustment is less than a third threshold, then determining the extrinsic parameter value to be optimized used in the current adjustment so that the positions of the same feature object collected by the laser radar at different positions in the reference coordinate system are the same or the position difference satisfies a preset condition; wherein the error parameter of the first feature object is the sum of the variances corresponding to the coordinates of the first feature object in three dimensions of the reference coordinate system, and the first feature object is any one of the X features.

[0027] Through the above design, based on the error parameters of the X feature objects and the extrinsic parameter values ​​to be optimized in two adjacent adjustments, the extrinsic parameter value to be optimized for the current adjustment is determined so that the position of the same feature object collected by the lidar at different locations in the reference coordinate system is the same or the position difference meets the preset conditions, while ensuring the accuracy and efficiency of the extrinsic parameter calibration.

[0028] In one possible design, according to the second relative transformation relationship between the IMU coordinate system and the reference coordinate system, the second point cloud data is converted to the reference coordinate system to obtain third point cloud data, including: according to the second relative transformation relationship between the IMU coordinate system and the reference coordinate system, the second point cloud data is converted from the IMU coordinate system to the reference coordinate system to obtain fourth point cloud data; the motion-compensated fourth point cloud data is used as the third point cloud data, and the motion-compensated fourth point cloud data is motion-compensated based on the position, posture and speed of the vehicle to be calibrated collected by the IMU during the process of the vehicle to be calibrated traveling on the target path.

[0029] Through the above design, motion compensation is performed on the point cloud data according to the posture and speed of the vehicle to be calibrated, eliminating the motion error caused by the vehicle movement and improving the accuracy of the external parameter calibration.

[0030] In a second aspect, an embodiment of the present application provides an external parameter calibration method for a laser radar and an IMU, which is applied to a calibration device, wherein the laser radar and the IMU are both fixedly mounted on a vehicle to be calibrated. The method includes:

[0031] Acquire first point cloud data collected by a laser radar, where the first point cloud data is used to represent positions of feature objects around the vehicle to be calibrated collected when the vehicle to be calibrated is traveling on a target path in a laser radar coordinate system;

[0032] In the current adjustment, the first point cloud data is converted from the lidar coordinate system to the IMU coordinate system to obtain second point cloud data according to the extrinsic parameter value to be optimized, wherein the extrinsic parameter value to be optimized is used to indicate a first relative conversion relationship between the lidar coordinate system and the IMU coordinate system;

[0033] According to the second point cloud data, the extrinsic parameter value to be optimized used in the current adjustment is determined so that when the position of the same feature object collected by the laser radar at different positions is the same in the IMU coordinate system or the position difference meets the preset conditions, the extrinsic parameter value to be optimized in the current time is used as the target extrinsic parameter value; otherwise, the extrinsic parameter value to be optimized is adjusted, and the adjusted extrinsic parameter value is used as the extrinsic parameter value to be optimized in the next adjustment.

[0034] Through the above design, the calibration device can convert the point cloud data collected by the lidar from the lidar coordinate system to the IMU coordinate system after obtaining the point cloud data, and iteratively adjust the external parameter value to be optimized directly according to the point cloud data in the IMU coordinate system, until the external parameter value to be optimized used in the current adjustment can make the same feature object collected by the lidar when traveling to different positions overlap or partially overlap in the IMU coordinate system, thereby avoiding the external parameter calibration result being affected by the installation angle of the lidar, and improving the external parameter calibration efficiency and accuracy of the lidar.

[0035] In one possible design, the second point cloud data is used to represent the three-dimensional coordinates of X features in the IMU coordinate system, where X is a positive integer;

[0036] Determining, based on the second point cloud data, an extrinsic parameter value to be optimized for use in a current adjustment so that the same feature object collected by the lidar at different locations has the same position in the IMU coordinate system or the position difference satisfies a preset condition, including: if a difference between the sum of the error parameters of the X feature objects in the current adjustment and the sum of the error parameters of the X feature objects in the previous adjustment is less than a first threshold, determining the extrinsic parameter value to be optimized for use in the current adjustment so that the same feature object collected by the lidar at different locations has the same position in the IMU coordinate system or the position difference satisfies a preset condition; wherein the error parameter of the first feature object is the sum of the variances corresponding to the coordinates of the first feature object in three dimensions in the IMU coordinate system, respectively, and the first feature object is any one of the X feature objects.

[0037] Through the above design, based on the sum of the error parameters of X feature objects, it is determined whether the extrinsic parameter value to be optimized can make the position of the same feature object collected by the lidar at different locations in the reference coordinate system the same or the position difference meets the preset conditions, thereby improving the efficiency of extrinsic parameter calibration.

[0038] In one possible design, the second point cloud data is used to represent the three-dimensional coordinates of X features in the IMU coordinate system, where X is a positive integer;

[0039] Determining, based on the second point cloud data, an extrinsic parameter value to be optimized for use in a current adjustment so that the same feature object collected by the lidar at different locations has the same position in the IMU coordinate system or the position difference satisfies a preset condition, including: if the difference between the error parameters of the X feature objects in the current adjustment and the error parameters of the X feature objects in the previous adjustment is less than a second threshold, determining the extrinsic parameter value to be optimized for use in the current adjustment so that the same feature object collected by the lidar at different locations has the same position in the IMU coordinate system or the position difference satisfies a preset condition; wherein the error parameter of the first feature object is the sum of the variances corresponding to the coordinates of the first feature object in three dimensions in the IMU coordinate system, and the first feature object is any one of the X features.

[0040] Through the above design, based on the error parameters of X feature objects, it is determined whether the extrinsic parameter value to be optimized can make the position of the same feature object collected by the lidar at different locations in the reference coordinate system the same or the position difference meet the preset conditions, thereby improving the accuracy of the extrinsic parameter calibration.

[0041] In one possible design, the second point cloud data is used to represent the three-dimensional coordinates of X features in the IMU coordinate system, where X is a positive integer;

[0042] Determining, based on the second point cloud data, an extrinsic parameter value to be optimized used in the current adjustment so that the positions of the same feature object collected by the laser radar at different locations in the IMU coordinate system are the same or the position difference satisfies a preset condition, including: if the difference between the sum of the error parameters of the X feature objects in the current adjustment and the sum of the error parameters of the X feature objects in the previous adjustment is less than a first threshold, and the difference between the extrinsic parameter value to be optimized used in the current adjustment and the extrinsic parameter value to be optimized used in the previous adjustment is less than a third threshold, then determining the extrinsic parameter value to be optimized used in the current adjustment so that the positions of the same feature object collected by the laser radar at different locations in the IMU coordinate system are the same or the position difference satisfies a preset condition; wherein the error parameter of the first feature object is the sum of the variances corresponding to the coordinates of the first feature object in three dimensions in the IMU coordinate system, respectively, and the first feature object is any one of the X features.

[0043] Through the above design, based on the sum of the error parameters of the X feature objects and the extrinsic parameter values ​​to be optimized in two adjacent adjustments, the extrinsic parameter value to be optimized for the current adjustment is determined so that the position of the same feature object collected by the lidar at different locations in the reference coordinate system is the same or the position difference meets the preset conditions, while ensuring the accuracy and efficiency of the extrinsic parameter calibration.

[0044] In one possible design, the second point cloud data is used to represent the three-dimensional coordinates of X features in the IMU coordinate system, where X is a positive integer;

[0045] Determining, based on the second point cloud data, the extrinsic parameter values ​​to be optimized for the current adjustment so that the positions of the same feature object collected by the laser radar at different locations in the IMU coordinate system are the same or the position difference satisfies a preset condition, including: if the differences between the error parameters of the X feature objects in the current adjustment and the error parameters of the X feature objects in the previous adjustment are all less than a second threshold, and the difference between the extrinsic parameter values ​​to be optimized used in the current adjustment and the extrinsic parameter values ​​to be optimized used in the previous adjustment is less than a third threshold, then determining the extrinsic parameter values ​​to be optimized for the current adjustment so that the positions of the same feature object collected by the laser radar at different locations in the IMU coordinate system are the same or the position difference satisfies a preset condition; wherein the error parameter of the first feature object is the sum of the variances corresponding to the coordinates of the first feature object in three dimensions in the IMU coordinate system, and the first feature object is any one of the X features.

[0046] Through the above design, based on the error parameters of the X feature objects and the extrinsic parameter values ​​to be optimized in two adjacent adjustments, the extrinsic parameter value to be optimized for the current adjustment is determined so that the position of the same feature object collected by the lidar at different locations in the reference coordinate system is the same or the position difference meets the preset conditions, while ensuring the accuracy and efficiency of the extrinsic parameter calibration.

[0047] In the third aspect, an embodiment of the present application provides an external parameter calibration method for a laser radar and an IMU, which is applied to a calibration device, wherein the laser radar and the IMU are both fixedly mounted on a vehicle to be calibrated. The method comprises: obtaining first point cloud data collected by the laser radar, the first point cloud data being used to characterize the positions of feature objects around the vehicle to be calibrated collected when the vehicle to be calibrated is traveling on the target path in the laser radar coordinate system; in the current adjustment, according to the external parameter value to be optimized, and according to the second relative transformation relationship between the IMU coordinate system and the reference coordinate system, a fourth relative transformation relationship between the laser radar coordinate system and the reference coordinate system is obtained, the external parameter value to be optimized being used to indicate the first relative transformation relationship between the laser radar coordinate system and the IMU coordinate system, and the second relative transformation relationship between the IMU coordinate system and the reference coordinate system being obtained according to the process of the vehicle to be calibrated traveling on the target path. The position and posture of the vehicle to be calibrated collected by the IMU are obtained; according to the fourth relative transformation relationship between the laser radar coordinate system and the reference coordinate system, the first point cloud data is converted from the laser radar coordinate system to the reference coordinate system to obtain the second point cloud data; according to the second point cloud data, the external parameter value to be optimized used in the current adjustment is determined so that when the position of the same feature object collected by the laser radar when traveling to different positions in the IMU coordinate system is the same or the position difference meets the preset conditions, the current external parameter value to be optimized is used as the target external parameter value; otherwise, the external parameter value to be optimized is adjusted, and the adjusted external parameter value is used as the external parameter value to be optimized in the next adjustment.

[0048] Through the above design, after obtaining the point cloud data collected by the lidar, the calibration device can directly convert the point cloud data from the lidar coordinate system to the reference coordinate system, and iteratively adjust the external parameter value to be optimized according to the point cloud data in the reference coordinate system, until the external parameter value to be optimized used in the current adjustment can make the same feature object collected by the lidar when traveling to different positions overlap or partially overlap in the IMU coordinate system, thereby avoiding the external parameter calibration result being affected by the installation angle of the lidar, and improving the external parameter calibration efficiency and accuracy of the lidar.

[0049] In a fourth aspect, an embodiment of the present application provides a method for calibrating external parameters of a lidar and an IMU. The method can be executed by a calibration device or a chip or chip system in the calibration device to implement the method in any possible implementation method executed by the calibration device in the first, second or third aspects above.

[0050] In a fifth aspect, the present application provides an external parameter calibration device for a lidar and an IMU, the calibration device comprising modules / units that execute the method of any possible implementation of the first, second, or third aspects described above. These modules / units can be implemented in hardware, or the corresponding software can be implemented in hardware.

[0051] In a sixth aspect, the present application provides a calibration device comprising a processor and a memory, wherein the memory is used to store one or more computer programs; when the one or more computer programs stored in the memory are executed by the processor, the calibration device is able to implement the method in any possible implementation of the first aspect, second aspect or third aspect mentioned above.

[0052] In a seventh aspect, the present application provides a computer program, which, when executed on a computer, enables the computer to execute the method in any possible implementation of the first, second or third aspects above.

[0053] In an eighth aspect, the present application provides a computer-readable storage medium, in which a computer program is stored. When the computer program is executed by a computer, the computer executes the method in any possible implementation of the first aspect, the second aspect or the third aspect.

[0054] In a ninth aspect, the present application provides a chip, which is used to read a computer program stored in a memory and execute a method in any possible implementation of the first aspect, the second aspect or the third aspect.

[0055] In a tenth aspect, an embodiment of the present application further provides a chip system, which includes a processor for supporting a computer device to implement the method in any possible implementation of the first, second, or third aspects above. In one possible design, the chip system also includes a memory for storing the necessary programs and data for the computer device. The chip system can be composed of a chip, or it can include a chip and other discrete devices.

[0056] For the technical effects that can be achieved by any possible technical solution in the above-mentioned fourth to tenth aspects, please refer to the description of the technical effects that can be achieved by the method in any possible implementation method in the above-mentioned first, second or third aspects, and no further details will be given here. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] Figure 1 A schematic diagram of a possible calibration scenario provided in an embodiment of the present application;

[0058] Figure 2A schematic diagram of a possible position between a laser radar and a collection vehicle provided in an embodiment of the present application;

[0059] Figure 3A A schematic diagram of a possible calibration site provided in an embodiment of the present application;

[0060] Figure 3B A schematic diagram of another possible calibration site provided in an embodiment of the present application;

[0061] Figure 4 A schematic diagram of a possible collection route provided in an embodiment of the present application;

[0062] Figure 5 This is a flow chart of a first possible method for extrinsic calibration of a lidar and an IMU provided in an embodiment of the present application;

[0063] Figure 6 A flowchart of a possible extrinsic parameter calibration method for a lidar and an IMU provided in an embodiment of the present application;

[0064] Figure 7 A flowchart of another possible extrinsic parameter calibration method for a lidar and an IMU provided in an embodiment of the present application;

[0065] Figure 8 Schematic diagram of the flow of a second possible method for extrinsic calibration of a lidar and an IMU provided in an embodiment of the present application;

[0066] Figure 9 This is a flow chart of a third possible method for extrinsic calibration of a lidar and an IMU provided in an embodiment of the present application;

[0067] Figure 10 A schematic structural diagram of a possible calibration device provided in an embodiment of the present application;

[0068] Figure 11 This is a schematic structural diagram of another possible calibration device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0069] In order to make this application easier to understand, some basic concepts involved in the embodiments of this application are first explained below. It should be noted that these explanations are intended to make the embodiments of this application easier to understand and should not be regarded as limiting the scope of protection claimed in this application.

[0070] 1. Public coordinate system.

[0071] A common coordinate system, also known as a world coordinate system or global coordinate system, has its origin at a fixed point in space. A common coordinate system is an absolute coordinate system; all objects in space can be positioned relative to the common coordinate system. For example, a common coordinate system can be a world coordinate system with east, north, and zenith as the X, Y, and Z axes.

[0072] 2. External reference.

[0073] Generally, there are two types of parameters that affect LiDAR performance: intrinsic and extrinsic. Internal parameters are determined during LiDAR manufacturing and may include, but are not limited to, the horizontal and vertical angles of each laser beam and distance correction values. Extrinsic parameters refer to the LiDAR's offset distance and angle relative to the IMU. For example, the offset distance of the lidar relative to the IMU means that if the lidar is regarded as a point mass, the coordinates (x, y, z) of the point mass in the IMU coordinate system O-x1y1z1 can indicate the offset distance of the lidar relative to the IMU; for another example, the lidar coordinate system O-x2y2z2 is established with the center of mass of the lidar as the origin. Assuming that the lidar coordinate system O-x2y2z2 is rotated around the x2 axis, y2 axis and z2 axis by three angles of θ1, θ2 and θ3 respectively, the x1 axis and the x2 axis have the same direction in the three-dimensional space, the y1 axis and the y2 axis have the same direction in the three-dimensional space, and the z1 axis and the z2 axis have the same direction in the three-dimensional space, then θ1, θ2 and θ3 can be regarded as the offset angles of the lidar relative to the IMU.

[0074] Figure 1 This is a schematic diagram of a possible calibration scenario provided in an embodiment of the present application, including a collection vehicle (the collection vehicle may also be referred to as a vehicle to be calibrated), a lidar, and an IMU, both of which are fixedly mounted on the collection vehicle. While the collection vehicle is traveling along the collection route, the lidar may obtain point cloud data of feature objects around the collection route, and the IMU may obtain the linear acceleration and angular velocity of the collection vehicle. Exemplarily, the collection vehicle also includes a global navigation satellite system (GNSS), which is used to achieve clock synchronization between the lidar and the IMU, and to provide the three-dimensional position of the collection vehicle in a common coordinate system.

[0075] Figure 2This is a possible schematic diagram of the position between the laser radar and the collection vehicle provided in an embodiment of the present application. The tilt angle between the laser radar and the collection vehicle is a set angle. As an example, the laser radar is installed obliquely on the collection vehicle, that is, the tilt angle is a first set angle, for example, the first set angle can be 32°. In the process of collecting high-precision maps, when the collection vehicle travels along the collection route, the reflection intensity of the obliquely installed laser radar is higher, and the acquired road point cloud is dense, thereby improving the collection efficiency and accuracy of high-precision road maps. As another example, the laser radar can also be installed parallel to the collection vehicle, that is, the tilt angle is a second set angle, for example, the second set angle can be 1°.

[0076] The data collection requirements in the embodiments of the present application are described below with reference to the accompanying drawings.

[0077] 1) Calibration site requirements: See Figure 3A 、 Figure 3B As shown, with the satellite navigation antenna installed on the top of the collection vehicle as the center, a site with a radius of R meters, an elevation angle of more than α degrees, and no obstructions is used as the calibration site. For example, with the satellite navigation antenna installed on the top of the vehicle as the center, a site with a radius of 20 meters, an elevation angle of more than 45 degrees, and no obstructions is used as the calibration site. Exemplarily, the number of satellites received by the satellite navigation receiving device is greater than 30, and the positioning precision dilution factor (PDOP) value is less than 1.5, so as to ensure that the acquired position of the collection vehicle in the public coordinate system is more accurate and improve the measurement accuracy. Among them, the satellite navigation antenna and the satellite navigation receiving device are both devices in GNSS.

[0078] 2) Feature Requirements: Features include plane features and high-altitude features. A plane feature can be a flat surface of a set area. For example, a plane feature is a 1-square-meter area of ​​relatively flat ground with no uphill slopes or potholes. High-altitude features are objects that reach a set height above the ground. For example, a high-altitude feature can be a street lamp head 3 meters above the ground, or a sign 3 meters above the ground.

[0079] 3) Collection route requirements: For example, the collection vehicle can pass the same feature in both forward and reverse directions while driving. Another example is that the laser radar can scan the feature multiple times while the collection vehicle is driving. The collection route can also be called the target path. Figure 4 As shown, Figure 4The data includes two features: the ground and the lamp head of a street lamp. L1, L2, L3, and L4 are different driving routes. The collection route of the collection vehicle can be A→L1→B→L2→C→A→L3→B→L4→C→L4→B→L3→A→C→L2→B→L1→A. Furthermore, to improve the precision and accuracy of the external parameter calibration, when the collection vehicle collects data along the collection route, the collection vehicle's driving speed is less than a set threshold, for example, the collection vehicle's driving speed is less than 10 km / h.

[0080] When the acquisition vehicle travels in the calibration site according to the acquisition route, the laser radar collects point cloud data of the feature object, and the IMU obtains the linear acceleration and angular velocity of the acquisition vehicle. When performing external parameter calibration, the calibration device can convert the point cloud data collected by the laser radar to the IMU coordinate system according to the external parameter value to be optimized; obtain the relative conversion relationship between the IMU coordinate system and the reference coordinate system based on the linear acceleration and angular velocity of the acquisition vehicle collected by the IMU; convert the point cloud data from the IMU coordinate system to the reference coordinate system based on the relative conversion relationship between the IMU coordinate system and the reference coordinate system; adjust the external parameter value to be optimized based on the point cloud data of the reference coordinate system, so as to obtain the target external parameter value. By converting the point cloud data collected by the laser radar to the reference coordinate system, the external parameter value to be optimized is optimized and adjusted based on the point cloud data in the reference coordinate system, automatic calibration is achieved and the efficiency of external parameter calibration is improved.

[0081] The technical solutions provided by the embodiments of the present application are described below with reference to the accompanying drawings.

[0082] See Figure 5 As shown, Figure 5 This is a possible extrinsic parameter calibration method for a lidar and an IMU provided in an embodiment of the present application. This method can be performed by a calibration device or by a chip or chip system in the calibration device. Hereinafter, the execution subject of S501-S504 is taken as an example, using the calibration device as an example.

[0083] S501: The calibration device obtains first point cloud data collected by a laser radar. The first point cloud data is used to represent the position of a feature collected by a vehicle traveling on a target path in a laser radar coordinate system. Hereinafter, the laser radar coordinate system is referred to as the radar coordinate system.

[0084] The first point cloud data includes N frames of point cloud data collected by the laser radar at N collection moments.

[0085] The first point cloud data can be obtained based on the laser radar data collected by the laser radar during the data acquisition phase, and the laser radar data includes any one or more of the reflection intensity, scanning angle and scanning direction of the laser radar. In the process of collecting data from the vehicle, the laser radar collects the laser radar data. When performing external parameter calibration of the laser radar and the IMU, the calibration device can parse the laser radar data according to the internal parameters to obtain the first point cloud data collected by the laser radar. Exemplarily, the first point cloud data can be stored in a point cloud data (PCD) format. Exemplarily, during the data acquisition process, after the laser radar collects the laser radar data, the collected laser radar data can be stored in a storage area. Furthermore, when performing external parameter calibration, the calibration device can obtain the laser radar data from the storage area.

[0086] S502. During the current adjustment, the calibration device converts the first point cloud data from the lidar coordinate system to the IMU coordinate system according to the extrinsic parameter value to be optimized to obtain the second point cloud data. The extrinsic parameter value to be optimized is used to indicate the first relative conversion relationship between the lidar coordinate system and the IMU coordinate system.

[0087] The extrinsic parameters to be optimized during the first adjustment are also called initial extrinsic parameters. These initial extrinsic parameters include the offset distance and / or the offset angle between the radar and IMU coordinate systems. These initial extrinsic parameters can be estimated based on the installation positions of the IMU and lidar on the acquisition vehicle.

[0088] The calibration device can perform a rectangular coordinate transformation on the first point cloud data based on the initial extrinsic parameter values, converting the first point cloud data from the radar coordinate system to the IMU coordinate system. Exemplarily, the calibration device performs a displacement transformation on the first point cloud data based on an offset distance included in the initial extrinsic parameter values, and performs a rotational transformation on the first point cloud data based on an offset angle included in the initial extrinsic parameter values, to obtain second point cloud data in the IMU coordinate system.

[0089] S503. The calibration device converts the second point cloud data into the reference coordinate system to obtain third point cloud data based on a second relative transformation relationship between the IMU coordinate system and the reference coordinate system. The relative transformation relationship between the IMU coordinate system and the reference coordinate system is obtained based on the position and posture of the acquisition vehicle captured by the IMU while the acquisition vehicle is traveling on the target path.

[0090] Among them, the second relative transformation relationship between the IMU coordinate system and the reference coordinate system can be obtained by the calibration device in the following manner: the calibration device obtains measurement data collected by the IMU at M second collection moments, and the measurement data includes the linear acceleration and angular velocity of the collection vehicle collected by the IMU during the process of the collection vehicle traveling on the target path; according to the measurement data, the second relative transformation relationship at the M second collection moments is obtained respectively, and the second relative transformation relationship at the second collection moment is used to characterize the relative transformation relationship between the IMU coordinate system and the reference coordinate system at the second collection moment.

[0091] The calibration device obtains the third relative transformation relationship at the N first acquisition moments based on the second relative transformation relationship at the M second acquisition moments. The third relative transformation relationship at the first acquisition moment is used to characterize the relative transformation relationship between the IMU coordinate system and the reference coordinate system at the first acquisition moment; according to the third relative transformation relationship at the i-th first acquisition moment, the second point cloud data at the i-th first acquisition moment is converted to the reference coordinate system to obtain the point cloud data at the i-th first acquisition moment, where i is a positive integer less than or equal to N, so as to obtain the point cloud data at the N first acquisition moments to constitute the said third point cloud data.

[0092] Exemplarily, in the process of obtaining the third relative transformation relationship at N first acquisition moments based on the second relative transformation relationship at M second acquisition moments, taking single-frame point cloud data as an example, the single-frame point cloud data is any frame of point cloud data in the second point cloud data. As a possible scenario, there is an acquisition moment of single-frame point cloud data among the M second acquisition moments, and based on the relative transformation relationship between the IMU coordinate system and the reference coordinate system at the M second acquisition moments, the third relative transformation relationship at the acquisition moment of the single-frame point cloud data is directly obtained, that is, the relative transformation relationship between the IMU coordinate system and the reference coordinate system at the acquisition moment of the single-frame point cloud data is obtained.

[0093] As another possible scenario, if no single-frame point cloud data acquisition moment exists among the M second acquisition moments, the calibration device can employ an interpolation algorithm based on the second relative transformation relationships at each of the M second acquisition moments to obtain a third relative transformation relationship for the single-frame point cloud data acquisition moment. The interpolation algorithm can be any of linear, parabolic, Lagrange, and Newton interpolation. The interpolation algorithm can be selected based on one or more of the following: the intensity of the motion changes of the acquisition vehicle, the desired interpolation accuracy, and the desired real-time computational performance.

[0094] As a possible implementation, the reference coordinate system may be a public coordinate system. When the reference coordinate system is a public coordinate system, the method for determining the second relative transformation relationship between the IMU coordinate system and the public coordinate system is described in detail in Example 1.

[0095] As another possible implementation, the reference coordinate system may also refer to a local coordinate system, which includes a radar coordinate system at a fixed time, an IMU coordinate system at a fixed time, or a custom coordinate system, such as a radar coordinate system at time 0 seconds (s), or an IMU coordinate system at time 0 seconds. When the reference coordinate system is a local coordinate system, the method for determining the second relative transformation relationship between the IMU coordinate system and the local coordinate system is specifically described in Example 2.

[0096] Exemplarily, the collection time can be expressed in GNSS time. Before the collection vehicle collects data, the IMU and the lidar can be clock-synchronized via GNSS. Exemplarily, the GNSS can input the GNSS pulse-second signal and the global positioning system (GPS) information (GPRMC) data frame with the recommended minimum data volume into the corresponding data interface of the lidar. After the lidar receives the GNSS pulse-second signal, it calibrates the lidar clock according to the standard time contained in the GPRMC data frame. After completing the time synchronization between the IMU and the lidar, both the IMU and the lidar can convert the data collection time into GNSS time.

[0097] S504. The calibration device determines, based on the third point cloud data, the extrinsic parameter value to be optimized used in the current adjustment so that the positions of the same feature object collected by the laser radar at different positions in the reference coordinate system are the same or the position difference meets the preset conditions. The extrinsic parameter value to be optimized is used as the target extrinsic parameter value; otherwise, the extrinsic parameter value to be optimized is adjusted, and the adjusted extrinsic parameter value is used as the extrinsic parameter value to be optimized in the next adjustment.

[0098] As one possible implementation, the third point cloud data is used to represent the three-dimensional coordinates of X features in the reference coordinate system, where X is a positive integer. Based directly on the third point cloud data, the calibration device determines the extrinsic parameter values ​​to be optimized for the current adjustment. If this ensures that the same feature captured by the lidar at different locations has the same position in the reference coordinate system, or the position difference satisfies a preset condition, the current extrinsic parameter values ​​to be optimized are used as the target extrinsic parameter values. Otherwise, the extrinsic parameter values ​​to be optimized are adjusted and used as the extrinsic parameter values ​​to be optimized in the next adjustment.

[0099] As another possible implementation, to improve calibration efficiency and accuracy, after the calibration device acquires the third point cloud data, it can also extract features from the third point cloud data. Furthermore, based on the feature point cloud set extracted for each feature, the calibration device determines the extrinsic parameter value to be optimized for the current adjustment. If this ensures that the same feature collected by the lidar at different locations has the same position in the reference coordinate system, or the position difference satisfies a preset condition, the current extrinsic parameter value to be optimized is used as the target extrinsic parameter value. Otherwise, the extrinsic parameter value to be optimized is adjusted, and the adjusted extrinsic parameter value is used as the extrinsic parameter value to be optimized in the next adjustment.

[0100] Exemplarily, if there are X features in the calibration site, the calibration device extracts X feature point cloud sets from the third point cloud data based on X reference coordinate ranges, where the X reference coordinate ranges are the reference coordinate ranges of the X features in the reference coordinate system. As an example, the calibration device first performs point cloud stitching on the third point cloud data in the reference coordinate system, and then performs feature object extraction. Exemplarily, the calibration device performs point cloud stitching on N frames of point cloud data contained in the third point cloud data in the reference coordinate system to obtain stitched third point cloud data; then performs feature object extraction on the stitched third point cloud data to obtain feature point cloud sets for the X features. Point cloud stitching refers to the process of unifying data collected from different angles and time points into the same coordinate system. For example, the third point cloud data contains point cloud data 1, point cloud data 2, and point cloud data 3. The calibration device performs point cloud stitching on point cloud data 1, point cloud data 2, and point cloud data 3 to obtain stitched third point cloud data. The calibration site includes a feature object A. According to the reference coordinate range of the feature object A, a feature point cloud set of the feature object A is extracted from the spliced ​​third point cloud data.

[0101] As another example, the calibration device first extracts features from the third point cloud data in the reference coordinate system, and then performs point cloud splicing. Exemplarily, the calibration device extracts features from the N frames of point cloud data contained in the third point cloud data in the reference coordinate system, and obtains feature data of each frame of X feature objects; and performs point cloud splicing on the corresponding N frames of feature data for each of the X feature objects, to obtain the spliced ​​third point cloud data. Taking point cloud data 1 contained in the third point cloud data as an example, the calibration device can extract features from a single frame of point cloud data based on the reference coordinate range of each feature object, and the measurement accuracy of the reference coordinate range can be at the meter level. The calibration site contains feature object A, and the calibration device extracts the point cloud data within the reference coordinate range of feature object A from point cloud data 1 in the reference coordinate system based on the reference coordinate range of feature object A, as a frame of feature point cloud of feature object A. In the reference coordinate system, the feature object extraction method in the above-mentioned single-frame point cloud data is used to extract features from the N frames of point cloud data contained in the third point cloud data, so as to obtain N frames of feature point clouds of feature object A. Point clouds of the N frames of feature point clouds of feature object A are spliced ​​to obtain a feature point cloud set of feature object A.

[0102] Exemplarily, the calibration device may determine the extrinsic parameter value to be optimized for the current adjustment so that the positions of the same feature object collected by the laser radar at different locations in the reference coordinate system are the same or the position difference satisfies a preset condition by, but not limited to, the following two possible implementation methods:

[0103] The first possible implementation method: When the calibration device determines that the first iteration stop condition is met, it determines the external parameter value to be optimized used in the current adjustment so that the position of the same feature object collected by the laser radar at different positions in the reference coordinate system is the same or the position difference meets the preset condition.

[0104] Exemplarily, the calibration device employs an iterative optimization algorithm based on the extracted feature point cloud set for each feature object to adjust the extrinsic parameter values ​​to be optimized until a first iterative stopping condition is satisfied. The extrinsic parameter values ​​to be optimized for the current adjustment are then determined to ensure that the same feature object captured by the lidar at different locations has the same position in the reference coordinate system, or a position difference that satisfies a preset condition. The iterative optimization algorithm may employ, but is not limited to, any of the following: the Gauss-Newton method, the conjugate gradient method, and the gradient descent method.

[0105] When the calibration device uses an iterative optimization algorithm, the error parameter is used as the objective function and the extrinsic parameter to be optimized is used as the optimization variable. The calibration device can calculate the error parameter of each feature based on the feature point cloud set. The error parameter of a feature represents the sum of the variances corresponding to the coordinates of the feature in the three dimensions of the reference coordinate system.

[0106] The first iteration stopping condition can be, but is not limited to, any of the following: a fixed number of iterations, a fixed time, or a forward-backward difference method. The fixed number of iterations method stops iterations when the number of iterations reaches a threshold. The fixed time method stops iterations when the iteration duration reaches a threshold.

[0107] When the forward and backward difference method is used, as an example, the first iteration stopping condition may be that the differences between the error parameters of the X features in the current iteration and the error parameters of the X features in the previous iteration are all smaller than the second threshold.

[0108] Taking two feature objects, feature object A and feature object B as examples, the calibration device adopts an iterative optimization algorithm. In the process of iterative optimization of the external parameter values ​​to be optimized, in the reference coordinate system O-xyz, the variance 1 of the feature point cloud set of feature object A on the x-axis, the variance 2 on the y-axis, and the variance 3 on the z-axis are statistically calculated. According to the statistically obtained variance 1, variance 2, and variance 3, the sum of the variances of the feature point cloud set of feature object A on each axis of the reference coordinate system is obtained, that is, the error parameter of feature object A is obtained. In the reference coordinate system O-xyz, the calibration device calculates the variance 4 of the feature point cloud set of feature object B on the x-axis, the variance 5 on the y-axis, and the variance 6 on the z-axis. According to the statistically obtained variance 4, variance 5, and variance 6, the sum of the variances of the feature point cloud set of feature object B on each axis of the reference coordinate system is obtained, that is, the error parameter of feature object B is obtained. When the error parameter of feature object A is less than the second threshold and the error parameter of feature object B is less than the second threshold, the iteration is stopped and the adjusted extrinsic parameter value is output.

[0109] As another example, the first iteration stopping condition may also be that the difference between the sum of the error parameters of the X feature objects in the current iteration and the sum of the error parameters of the X feature objects in the previous iteration is less than a first threshold.

[0110] Taking feature A and feature B as an example, the calibration device uses an iterative optimization algorithm to iteratively optimize the extrinsic parameter values ​​to be optimized. In the reference coordinate system O-xyz, the device calculates the variance 1 on the x-axis, the variance 2 on the y-axis, and the variance 3 on the z-axis of feature A's feature point cloud set. This yields the sum of the variances of feature A's feature point cloud set along each axis of the reference coordinate system, i.e., the error parameter of feature A. In the reference coordinate system O-xyz, the device calculates the variance 4 on the x-axis, the variance 5 on the y-axis, and the variance 6 on the z-axis of feature B's feature point cloud set. This yields the sum of the variances of feature B's feature point cloud set along each axis of the reference coordinate system, i.e., the error parameter of feature B. The calibration device then sums the error parameters of feature A and feature B to obtain the sum of the error parameters of feature A and feature B. When the sum of the error parameters of feature A and feature B is less than a first threshold, the iteration stops and the adjusted extrinsic parameter value is output.

[0111] Using an iterative optimization algorithm, if the first iteration stopping condition is not met during the iterative optimization process of the extrinsic parameter to be optimized, the extrinsic parameter to be optimized is adjusted according to the set step size. The set step size includes the step size of the offset distance and the step size of the offset angle. For example, the step size of the offset distance can be 0.1 cm, and the step size of the offset angle can be 0.01°.

[0112] As a second possible implementation method, when the calibration device determines that the first iteration stop condition and the second iteration stop condition are met, it determines that the external parameter value to be optimized used in the current adjustment makes the position of the same feature object collected by the laser radar at different positions in the reference coordinate system the same or the position difference meets the preset conditions.

[0113] The second iteration stopping condition may also adopt, but is not limited to, any one of a fixed number of iterations method, a fixed time method, and a front-back difference method.

[0114] Taking the second iteration stopping condition using the previous and next difference method as an example, the second iteration stopping condition may be that the difference between the extrinsic parameter value to be optimized used in the current adjustment and the extrinsic parameter value to be optimized used in the previous adjustment is less than the third threshold.

[0115] Exemplarily, the third threshold includes a distance threshold and an angle threshold. The calibration device determines whether the difference in offset distance between the external parameter value used in the current adjustment and the external parameter value to be optimized used in the previous adjustment is less than the distance threshold, and determines whether the difference in offset angle between the external parameter value to be optimized used in the current adjustment and the external parameter value to be optimized used in the previous adjustment is less than the angle threshold.

[0116] If the difference in offset distance between the extrinsic parameter value to be optimized used in the current adjustment and the extrinsic parameter value to be optimized used in the previous adjustment is less than the distance threshold, and the offset angle between the extrinsic parameter value to be optimized used in the current adjustment and the extrinsic parameter value to be optimized used in the previous adjustment is less than the angle threshold, then the extrinsic parameter value to be optimized used in the current adjustment is used as the target extrinsic parameter value.

[0117] Furthermore, to eliminate point cloud ghosting errors caused by vehicle motion, the calibration device can also perform motion compensation on the first, second, or third point cloud data based on the position, attitude, and velocity of the acquisition vehicle as captured by the IMU before extracting features from the third point cloud data. Motion compensation is a method of describing the differences between adjacent frames to eliminate the effects of motion caused by any of the acquisition vehicle's velocity, linear acceleration, or angular velocity.

[0118] Example 1: Take the reference coordinate system as a public coordinate system as an example.

[0119] Figure 6 A possible external parameter calibration method for lidar and IMU is provided in an embodiment of the present application. The method can be executed by a calibration device or by a chip or chip system in the calibration device.

[0120] S601: The calibration device obtains measurement data collected by the IMU and, based on the measurement data collected by the IMU, obtains a set of IMU pose information in a public coordinate system. The pose information includes one or more of three-dimensional position, three-dimensional velocity, and three-dimensional attitude angle. Hereinafter, the three-dimensional position, three-dimensional velocity, and three-dimensional attitude angle are referred to as position, velocity, and attitude angle. The pose information of the IMU in the public coordinate system is used to describe the relative transformation relationship between the IMU coordinate system and the public coordinate system.

[0121] The pose information set includes the pose information collected by the IMU at M collection moments.

[0122] The measurement information collected by the IMU includes linear acceleration and angular velocity. As the vehicle travels along the acquisition route, the IMU can collect the vehicle's linear acceleration and angular velocity at N acquisition moments. Furthermore, after collecting the measurement information, the IMU can store it in a storage area. For example, the storage area can be a hard drive in the acquisition vehicle, a portable notebook, or the like. During external parameter calibration between the LiDAR and the IMU, the calibration device can retrieve the stored IMU-collected measurement information from the storage area.

[0123] Exemplarily, the collection vehicle may be configured with an inertial navigation system (INS), which includes an IMU. After the calibration device obtains the measurement information of the IMU, it performs inertial navigation calculations on the measurement information collected by the IMU through the inertial navigation system (INS) to obtain INS navigation information, which includes one or more of the three-dimensional position, three-dimensional velocity, and three-dimensional attitude angle of the collection vehicle in the IMU coordinate system. The collection vehicle is also configured with a GNSS. When the collection vehicle is traveling on the collection route, the GNSS can collect GNSS measurement information, which includes one or more of the three-dimensional position, three-dimensional velocity, and three-dimensional attitude angle of the collection vehicle in a public coordinate system. The calibration device uses an inertial navigation algorithm and a fusion filtering algorithm to fuse the INS navigation information and the GNSS measurement information, and can obtain a set of IMU position and posture information in a public coordinate system.

[0124] S602: The calibration device obtains point cloud data collected by the laser radar. The point cloud data collected by the laser radar includes N frames of point cloud data collected by the laser radar at N collection moments.

[0125] S603: The calibration device converts the point cloud data from the radar coordinate system to the IMU coordinate system based on the Mth set of extrinsic parameter values ​​to obtain point cloud data in the IMU coordinate system. M is a natural number.

[0126] When the value of M is 0, the zeroth group of extrinsic parameter values ​​can also be called the initial extrinsic parameter values, which include the offset distance and offset angle between the radar coordinate system and the IMU coordinate system.

[0127] S604. The calibration device projects the point cloud data in the IMU coordinate system to the public coordinate system according to the pose information corresponding to the acquisition time of the point cloud data to obtain the point cloud data in the public coordinate system.

[0128] Taking single-frame point cloud data as an example, single-frame point cloud data is any frame of point cloud data in the IMU coordinate system.

[0129] As a possible scenario, the acquisition time of the single-frame point cloud data is the same as the acquisition time of one of the pose information in the pose information set. The calibration device can determine that the pose information corresponding to the acquisition time of the single-frame point cloud data is the one of the pose information.

[0130] As another possible situation, the collection time of the single-frame point cloud data is different from the collection time of the pose information in the pose information set. The calibration device can use an interpolation algorithm to calculate the pose information corresponding to the collection time of the single-frame point cloud data as the target pose information. The target pose information is used to represent the pose information corresponding to the collection time of the single-frame point cloud data. Furthermore, the calibration device projects the single-frame point cloud data in the IMU coordinate system to the public coordinate system based on the target pose information. For example, the GNSS time of the single-frame point cloud data is the 15th second, the GNSS time of the pose information 1 is the 14th second, and the GNSS time of the pose information 2 is the 16th second. Using the linear interpolation algorithm, the pose information at the 15th second is calculated as the average of the pose information 1 and the pose information 2, which is used as the target pose information.

[0131] Furthermore, in order to eliminate the point cloud ghosting error caused by the movement of the collected vehicle, the calibration device projects the point cloud data in the IMU coordinate system to the public coordinate system, and then performs motion compensation on the point cloud data in the public coordinate system based on the posture information and / or the measurement data of the IMU.

[0132] S605 , the calibration device extracts features from the point cloud data in the public coordinate system to obtain a feature point cloud set of each feature object.

[0133] S606 , the calibration device uses an iterative optimization algorithm to iteratively optimize the Mth group of extrinsic parameter values ​​until the first iteration stop condition is met, thereby obtaining the M+1th group of extrinsic parameter values. For details, see S504 .

[0134] When the calibration device employs an iterative optimization algorithm, the error parameter serves as the objective function, and the Mth set of extrinsic parameter values ​​serves as the optimization variable. The calibration device can calculate the error parameter for each feature object based on the feature point cloud set. For example, using the forward-backward difference method as the first iteration stopping condition, the first iteration stopping condition can be that the difference between the error parameter for each feature object in the current iteration and the error parameter for each feature object in the previous iteration is less than a first error threshold.

[0135] S607: The calibration device determines whether the second iteration stop condition is met. If so, execute S609; otherwise, execute S608.

[0136] Taking the front-to-back difference method as an example of the second iteration stop condition, the calibration device determines whether the difference in offset distance between the M+1th group of extrinsic parameter values ​​and the Mth group of extrinsic parameter values ​​is less than the distance threshold, and determines whether the difference in offset angle between the M+1th group of extrinsic parameter values ​​and the Mth group of extrinsic parameter values ​​is less than the angle threshold.

[0137] When the difference in offset distance between the M+1th group of extrinsic parameter values ​​and the Mth group of extrinsic parameter values ​​is less than the distance threshold, and the offset angle of the relative deflection between the M+1th group of extrinsic parameter values ​​and the Mth group of extrinsic parameter values ​​is less than the angle threshold, execute S609, that is, the M+1th group of extrinsic parameter values ​​is used as the target extrinsic parameter values, otherwise execute S608.

[0138] S608: The calibration device replaces the Mth group of extrinsic parameter values ​​with the M+1th group of extrinsic parameter values, and executes S603.

[0139] S609: The calibration device uses the Mth group of extrinsic parameter values ​​as target extrinsic parameter values.

[0140] Example 2: Take the reference coordinate system as the local coordinate system as an example. Figure 7 This is a possible extrinsic parameter calibration method for a lidar and an IMU provided in an embodiment of the present application. This method can be performed by a calibration device or by a chip or chip system in the calibration device. Hereinafter, the execution subject of steps S701-S709 is taken as an example, using a calibration device.

[0141] S701: The calibration device obtains measurement data collected by the IMU, and obtains the relative transformation relationship between the IMU coordinate system and the local coordinate system at M collection moments based on the measurement data collected by the IMU.

[0142] As an example, the local coordinate system is the IMU coordinate system at a fixed time, for example, the IMU coordinate system at a fixed time is the IMU coordinate system at time 0s. Taking the IMU coordinate system at a fixed time as the IMU coordinate system at time 0s as an example, the measurement information collected by the IMU includes the linear acceleration and angular velocity of the vehicle collected by the IMU at M collection times. After the calibration device obtains the measurement information collected by the IMU, it can obtain the relative conversion relationship between the IMU coordinate system at the M collection times and the IMU coordinate system at time 0s through integral calculation based on the linear acceleration and angular velocity collected at the M collection times. For example, the measurement data collected by the IMU includes the linear acceleration and angular velocity collected by the IMU at time 0s, and the linear acceleration and angular velocity collected by the IMU at time 1s. The calibration device can obtain the relative conversion relationship between the IMU coordinate system at time 1s and the IMU coordinate system at time 0s through integral calculation based on the linear acceleration and angular velocity collected at time 0s and time 1s.

[0143] As another example, the local coordinate system is a radar coordinate system at a fixed time, for example, the IMU coordinate system at a fixed time is the radar coordinate system at time 0s. Taking the IMU coordinate system at a fixed time as the radar coordinate system at time 0s as an example, after the calibration device obtains the measurement information collected by the IMU, based on the linear acceleration and angular velocity collected at M collection times, through integral calculation, it can obtain the relative transformation relationship between the IMU coordinate system at M collection times and the IMU coordinate system at time 0s. Based on the initial external parameter value, the calibration device can obtain the relative transformation relationship between the IMU coordinate system at M collection times and the radar coordinate system at time 0s.

[0144] S702: The calibration device obtains point cloud data collected by the laser radar. For details, see S501.

[0145] S703. The calibration device converts the point cloud data from the radar coordinate system to the IMU coordinate system according to the Mth group of extrinsic parameter values ​​to obtain the point cloud data in the IMU coordinate system. For details, see S502.

[0146] S704. The calibration device converts the point cloud data in the IMU coordinate system to the local coordinate system according to the relative conversion relationship between the IMU coordinate system and the local coordinate system to obtain the point cloud data in the local coordinate system. For details, see S503.

[0147] S705 , the calibration device extracts features from the point cloud data in the local coordinate system to obtain a feature point cloud set of each feature object. For details, see S504 .

[0148] S706 , the calibration device uses an iterative optimization algorithm to iteratively optimize the Mth group of extrinsic parameter values ​​until the first iteration stop condition is met, thereby obtaining the M+1th group of extrinsic parameter values. For details, see S504 .

[0149] S707: The calibration device determines whether the second iteration stop condition is met. If so, execute S709; otherwise, execute S708.

[0150] S708. The calibration device replaces the Mth group of extrinsic parameter values ​​with the M+1th group of extrinsic parameter values, and executes S703.

[0151] S709: The calibration device uses the Mth group of extrinsic parameter values ​​as target extrinsic parameter values.

[0152] The above method converts the point cloud data collected by the LiDAR into a local coordinate system based on the extrinsic parameter values ​​to be optimized and the relative transformation relationship between the IMU and the local coordinate system. Feature objects are then extracted from the point cloud data in the local coordinate system to obtain feature point cloud data for each feature object. The extrinsic parameter values ​​to be optimized are then iteratively optimized based on the feature point cloud data for each feature object, achieving automated calibration between the LiDAR and IMU coordinate systems. This improves calibration efficiency and mitigates the impact of the LiDAR installation angle on calibration accuracy.

[0153] Based on the same concept, a second possible method for calibrating the external parameters of the laser radar and the IMU is provided in an embodiment of the present application. In the embodiment of the present application, the calibration device obtains the first point cloud data collected by the laser radar; in the current adjustment, the calibration device obtains the fourth relative transformation relationship between the laser radar coordinate system and the reference coordinate system according to the external parameter value to be optimized and the second relative transformation relationship between the IMU coordinate system and the reference coordinate system. The external parameter value to be optimized is used to indicate the first relative transformation relationship between the laser radar coordinate system and the IMU coordinate system; wherein, the method for determining the second relative transformation relationship between the IMU coordinate system and the reference coordinate system is shown in Example 1 and Example 2; the calibration device obtains the fourth relative transformation relationship between the laser radar coordinate system and the reference coordinate system according to the second relative transformation relationship between the laser radar coordinate system and the reference coordinate system. Four relative transformation relationships, converting the first point cloud data from the laser radar coordinate system to the reference coordinate system to obtain the second point cloud data; adjusting the extrinsic parameter value to be optimized according to the second point cloud data; the calibration device determines the extrinsic parameter value to be optimized used in the current adjustment based on the second point cloud data so that the position of the same feature object collected by the laser radar at different positions in the IMU coordinate system is the same or the position difference meets the preset conditions, and the current extrinsic parameter value to be optimized is used as the target extrinsic parameter value; otherwise, the extrinsic parameter value to be optimized is adjusted, and the adjusted extrinsic parameter value is used as the extrinsic parameter value to be optimized in the next adjustment.

[0154] Taking the reference coordinate system as the local coordinate system as an example, Figure 8 This is the second possible extrinsic parameter calibration method for a lidar and an IMU provided in the embodiments of this application. This method can be performed by a calibration device or by a chip or chip system in the calibration device. Below, the execution subject of S801-S808 is taken as an example of a calibration device.

[0155] S801: The calibration device obtains measurement data collected by the IMU and, based on the measurement data collected by the IMU, determines the relative transformation relationship between the IMU coordinate system and the local coordinate system at M collection moments. The relative transformation relationship between the IMU coordinate system and the local coordinate system at M collection moments is determined as described in S701.

[0156] S802: The calibration device obtains point cloud data collected by the laser radar. For details, see S501.

[0157] S803. The calibration device obtains the relative transformation relationship between the radar coordinate system and the local coordinate system at N acquisition moments based on the Mth group of extrinsic parameter values ​​and the relative transformation relationship between the IMU coordinate system and the local coordinate system at M acquisition moments.

[0158] Exemplarily, based on the relative transformation relationship between the IMU coordinate system and the local coordinate system at M acquisition moments, an interpolation algorithm is used to obtain the relative transformation relationship between the IMU coordinate system and the local coordinate system at N acquisition moments; based on the Mth group of external parameter values ​​and the relative transformation relationship between the IMU coordinate system and the local coordinate system at N acquisition moments, the relative transformation relationship between the radar coordinate system and the local coordinate system at N acquisition moments can be obtained.

[0159] S804. The calibration device converts the point cloud data in the radar coordinate system to the local coordinate system according to the relative conversion relationship between the radar coordinate system and the local coordinate system at N acquisition moments, thereby obtaining the point cloud data in the local coordinate system.

[0160] S805 , the calibration device extracts features from the point cloud data in the local coordinate system to obtain a feature point cloud set of each feature object. For details, see S504 .

[0161] S806 , the calibration device uses an iterative optimization algorithm to adjust the Mth group of extrinsic parameter values ​​until the first iteration stop condition is met, thereby obtaining the M+1th group of extrinsic parameter values. For details, see S504 .

[0162] S807: The calibration device determines whether the second iteration convergence condition is met. If so, execute S809; otherwise, execute S808.

[0163] S808. The calibration device replaces the Mth group of extrinsic parameter values ​​with the M+1th group of extrinsic parameter values, and executes S803.

[0164] S809: The calibration device uses the M+1th group of extrinsic parameter values ​​as target extrinsic parameter values.

[0165] Based on the same concept, a third possible external parameter calibration method for a laser radar and an IMU is provided in an embodiment of the present application. In an embodiment of the present application, a calibration device obtains first point cloud data collected by a laser radar, and the first point cloud data is used to characterize the position of the feature objects around the vehicle to be calibrated collected when the vehicle to be calibrated is traveling on the target path in the laser radar coordinate system; in the current adjustment, the calibration device converts the first point cloud data from the laser radar coordinate system to the IMU coordinate system according to the external parameter value to be optimized to obtain second point cloud data, and the external parameter value to be optimized is used to indicate the first relative transformation relationship between the laser radar coordinate system and the IMU coordinate system; the calibration device determines the external parameter value to be optimized used in the current adjustment based on the second point cloud data so that the position of the same feature object collected by the laser radar at different positions in the IMU coordinate system is the same or the position difference meets a preset condition, and the current external parameter value to be optimized is used as the target external parameter value; otherwise, the external parameter value to be optimized is adjusted, and the adjusted external parameter value is used as the external parameter value to be optimized in the next adjustment.

[0166] See Figure 9 As shown, Figure 9 This is the third possible external parameter calibration method for lidar and IMU provided in the embodiments of the present application. This method can be executed by a calibration device or by a chip or chip system in the calibration device.

[0167] S901. The calibration device obtains point cloud data collected by the laser radar. For details, see S501.

[0168] S902. The calibration device converts the point cloud data from the radar coordinate system to the IMU coordinate system according to the Mth group of extrinsic parameter values ​​to obtain the point cloud data in the IMU coordinate system. For details, see S502.

[0169] S903: The calibration device extracts features from the point cloud data in the IMU coordinate system to obtain a feature point cloud set for each feature. The process of extracting features from the point cloud data in the IMU coordinate system is similar to the process of extracting features from the third point cloud data in S504 and will not be repeated here.

[0170] S904: The calibration device uses an iterative optimization algorithm to iteratively optimize the Mth group of extrinsic parameter values ​​until the first iteration stop condition is met, thereby obtaining the M+1th group of extrinsic parameter values.

[0171] S905 , the calibration device determines whether the second iteration convergence condition is met, if so, execute S907 , otherwise, execute S906 .

[0172] S906. The calibration device replaces the Mth group of extrinsic parameter values ​​with the M+1th group of extrinsic parameter values, and executes S902.

[0173] S907: The calibration device uses the Mth group of extrinsic parameter values ​​as target extrinsic parameter values.

[0174] Through the above method, after converting the point cloud data collected by the lidar from the radar coordinate system to the IMU coordinate system according to the external parameter value to be optimized, the external parameter value to be optimized can be directly optimized according to the point cloud data in the IMU coordinate system, thereby improving the efficiency of the external parameter calibration and avoiding the influence of the installation angle of the lidar on the calibration accuracy.

[0175] Based on the above embodiment, the embodiment of the present application also provides a method for calibrating the external parameters between multiple laser radars. When calibrating the external parameters between multiple laser radars, the method can be based on Figure 5-Figure 9 Any possible method is used to obtain the external parameter values ​​between each laser radar and the IMU respectively, and the external parameter values ​​between multiple laser radars are obtained according to the external parameter values ​​between each laser radar and the IMU.

[0176] Taking the extrinsic parameter calibration process between the first lidar and the second lidar as an example, see S501-S504. The first extrinsic parameter value between the first lidar and the IMU, and the second extrinsic parameter value between the second lidar and the IMU can be obtained. Based on the relative relationship between the first and second extrinsic parameter values, the extrinsic parameter value between the first and second lidars is obtained.

[0177] Taking the extrinsic parameter calibration process between the first, second, and third laser radars as an example, see S501-S504. The first extrinsic parameter value between the first laser radar and the IMU, the second extrinsic parameter value between the second laser radar and the IMU, and the third extrinsic parameter value between the third laser radar and the IMU can be obtained. Based on the relative relationship between the first and second extrinsic parameter values, the extrinsic parameter value between the first and second laser radars can be obtained. Based on the relative relationship between the first and third extrinsic parameter values, the extrinsic parameter value between the first and third laser radars can be obtained. Based on the relative relationship between the second and third extrinsic parameter values, the extrinsic parameter value between the second and third laser radars can be obtained.

[0178] Based on the same technical concept, this application also provides an external parameter calibration device for laser radar and IMU, the structure of which is as follows: Figure 10 As shown, it includes a communication unit 1001 and a processing unit 1002. The calibration device 1000 can be applied to Figure 5-Figure 9 The functions of each unit in the calibration device 1000 are introduced below.

[0179] The communication unit 1001 is used to receive and send data.

[0180] When the calibration device 1000 is applied to a calibration device, the communication unit 601 can also be called a physical interface, a communication module, a communication interface, or an input / output interface.

[0181] In a possible application scenario, the calibration device 1000 includes a communication unit 1001 and a processing unit 1002.

[0182] The communication unit 1001 is configured to obtain first point cloud data collected by a laser radar, where the first point cloud data is used to represent positions of features around the vehicle to be calibrated collected when the vehicle to be calibrated is traveling on a target path in a laser radar coordinate system;

[0183] The processing unit 1002 is configured to, in the current adjustment, convert the first point cloud data from the lidar coordinate system to the IMU coordinate system according to the extrinsic parameter value to be optimized to obtain second point cloud data, where the extrinsic parameter value to be optimized is used to indicate a first relative transformation relationship between the lidar coordinate system and the IMU coordinate system; convert the second point cloud data to the reference coordinate system according to a second relative transformation relationship between the IMU coordinate system and the reference coordinate system to obtain third point cloud data; the second relative transformation relationship is obtained based on the position and posture of the vehicle to be calibrated collected by the IMU during the process of the vehicle to be calibrated traveling on the target path; and, based on the third point cloud data, determine the extrinsic parameter value to be optimized used in the current adjustment so that when the positions of the same feature object collected by the lidar at different positions in the reference coordinate system are the same or the position difference satisfies a preset condition, use the current extrinsic parameter value to be optimized as the target extrinsic parameter value; otherwise, adjust the extrinsic parameter value to be optimized, and use the adjusted extrinsic parameter value as the extrinsic parameter value to be optimized in the next adjustment.

[0184] In one possible design, the first point cloud data is collected by the laser radar at N first collection moments, and the communication unit 1001 is further used to obtain measurement data collected by the IMU at M second collection moments, where the measurement data includes the linear acceleration and angular velocity of the vehicle to be calibrated collected by the IMU while the vehicle to be calibrated is traveling on the target path;

[0185] When obtaining the second relative transformation relationship between the IMU coordinate system and the reference coordinate system, the processing unit 1002 is used to: obtain the second relative transformation relationship at M second acquisition moments respectively according to the measurement data, and the second relative transformation relationship at the second acquisition moment is used to characterize the relative transformation relationship between the IMU coordinate system and the reference coordinate system at the second acquisition moment.

[0186] In one possible design, when converting the second point cloud data into the reference coordinate system to obtain third point cloud data according to the second relative transformation relationship between the IMU coordinate system and the reference coordinate system, the processing unit 1002 is configured to:

[0187] Obtaining, according to the second relative transformation relationships at the M second acquisition moments, third relative transformation relationships at the N first acquisition moments, wherein the third relative transformation relationships at the first acquisition moments are used to represent the relative transformation relationship between the IMU coordinate system and the reference coordinate system at the first acquisition moment;

[0188] According to the third relative transformation relationship of the i-th first acquisition moment, the second point cloud data of the i-th first acquisition moment is respectively transformed into the reference coordinate system to obtain the point cloud data of the i-th first acquisition moment, where i is a positive integer less than or equal to N to obtain the point cloud data of N first acquisition moments to constitute the third point cloud data.

[0189] In one possible design, the third point cloud data is used to represent the three-dimensional coordinates of X features in the reference coordinate system, where X is a positive integer;

[0190] When it is determined based on the third point cloud data that the extrinsic parameter value to be optimized used in the current adjustment makes the positions of the same feature object collected by the laser radar at different locations in the reference coordinate system the same or the position difference satisfies a preset condition, the processing unit 1002 is configured to:

[0191] If the difference between the sum of the error parameters of the X feature objects in the current adjustment and the sum of the error parameters of the X feature objects in the previous adjustment is less than a first threshold, then the extrinsic parameter value to be optimized for the current adjustment is determined so that the position of the same feature object collected by the lidar at different locations in the reference coordinate system is the same or the position difference satisfies a preset condition; wherein the error parameter of the first feature object is the sum of the variances corresponding to the coordinates of the first feature object in three dimensions of the reference coordinate system, respectively, and the first feature object is any one of the X features.

[0192] In one possible design, the third point cloud data is used to represent the three-dimensional coordinates of X features in the reference coordinate system, where X is a positive integer;

[0193] When it is determined based on the third point cloud data that the extrinsic parameter value to be optimized used in the current adjustment makes the positions of the same feature object collected by the laser radar at different locations in the reference coordinate system the same or the position difference satisfies a preset condition, the processing unit 1002 is configured to:

[0194] If the differences between the error parameters of the X feature objects in the current adjustment and the error parameters of the X feature objects in the previous adjustment are all less than a second threshold, then the extrinsic parameter value to be optimized for the current adjustment is determined so that the position of the same feature object collected by the laser radar at different locations in the reference coordinate system is the same or the position difference satisfies a preset condition; wherein the error parameter of the first feature object is the sum of the variances corresponding to the coordinates of the first feature object in three dimensions of the reference coordinate system, and the first feature object is any one of the X features.

[0195] In one possible design, the third point cloud data is used to represent the three-dimensional coordinates of X features in the reference coordinate system, where X is a positive integer;

[0196] When it is determined based on the third point cloud data that the extrinsic parameter value to be optimized used in the current adjustment makes the positions of the same feature object collected by the laser radar at different locations in the reference coordinate system the same or the position difference satisfies a preset condition, the processing unit 1002 is configured to:

[0197] If the difference between the sum of the error parameters of the X feature objects in the current adjustment and the sum of the error parameters of the X feature objects in the previous adjustment is less than a first threshold, and the difference between the extrinsic parameter value to be optimized used in the current adjustment and the extrinsic parameter value to be optimized used in the previous adjustment is less than a third threshold, then the extrinsic parameter value to be optimized used in the current adjustment is determined so that the positions of the same feature object collected by the laser radar at different positions in the reference coordinate system are the same or the position difference satisfies a preset condition; wherein the error parameter of the first feature object is the sum of the variances corresponding to the coordinates of the first feature object in three dimensions of the reference coordinate system, and the first feature object is any one of the X features.

[0198] In one possible design, the third point cloud data is used to represent the three-dimensional coordinates of X features in the reference coordinate system, where X is a positive integer;

[0199] When it is determined based on the third point cloud data that the extrinsic parameter value to be optimized used in the current adjustment makes the positions of the same feature object collected by the laser radar at different locations in the reference coordinate system the same or the position difference satisfies a preset condition, the processing unit 1002 is configured to:

[0200] If the differences between the error parameters of the X feature objects in the current adjustment and the error parameters of the X feature objects in the previous adjustment are all less than the second threshold, and the difference between the extrinsic parameter value to be optimized used in the current adjustment and the extrinsic parameter value to be optimized used in the previous adjustment is less than the third threshold, then the extrinsic parameter value to be optimized used in the current adjustment is determined so that the positions of the same feature object collected by the laser radar at different positions in the reference coordinate system are the same or the position difference satisfies a preset condition; wherein the error parameter of the first feature object is the sum of the variances corresponding to the coordinates of the first feature object in three dimensions of the reference coordinate system, and the first feature object is any one of the X features.

[0201] In one possible design, when converting the second point cloud data into the reference coordinate system to obtain third point cloud data according to the second relative transformation relationship between the IMU coordinate system and the reference coordinate system, the processing unit 1002 is configured to:

[0202] Converting the second point cloud data from the IMU coordinate system to the reference coordinate system according to a second relative conversion relationship between the IMU coordinate system and the reference coordinate system to obtain fourth point cloud data;

[0203] The fourth point cloud data after motion compensation is used as the third point cloud data. The fourth point cloud data after motion compensation is motion compensated based on the position, posture and speed of the vehicle to be calibrated collected by the IMU during the process of the vehicle to be calibrated traveling on the target path.

[0204] In another possible application scenario, the calibration device 1000 includes a communication unit 1001 and a processing unit 1002.

[0205] The communication unit 1001 is configured to obtain first point cloud data collected by a laser radar, where the first point cloud data is used to represent positions of features around the vehicle to be calibrated collected when the vehicle to be calibrated is traveling on a target path in a laser radar coordinate system;

[0206] The processing unit 1002 is used to convert the first point cloud data from the laser radar coordinate system to the IMU coordinate system to obtain second point cloud data according to the external parameter value to be optimized in the current adjustment, and the external parameter value to be optimized is used to indicate a first relative conversion relationship between the laser radar coordinate system and the IMU coordinate system; and according to the second point cloud data, determine the external parameter value to be optimized used in the current adjustment so that when the position of the same feature object collected by the laser radar at different positions is the same in the IMU coordinate system or the position difference meets a preset condition, the current external parameter value to be optimized is used as the target external parameter value; otherwise, the external parameter value to be optimized is adjusted, and the adjusted external parameter value is used as the external parameter value to be optimized in the next adjustment.

[0207] In one possible design, the second point cloud data is used to represent the three-dimensional coordinates of X features in the IMU coordinate system, where X is a positive integer;

[0208] When it is determined based on the second point cloud data that the extrinsic parameter value to be optimized used in the current adjustment makes the positions of the same feature object collected by the laser radar at different locations in the IMU coordinate system the same or the position difference satisfies a preset condition, the processing unit 1002 is configured to:

[0209] If the difference between the sum of the error parameters of the X feature objects in the current adjustment and the sum of the error parameters of the X feature objects in the previous adjustment is less than a first threshold, then the extrinsic parameter value to be optimized for the current adjustment is determined so that the position of the same feature object collected by the lidar at different locations in the IMU coordinate system is the same or the position difference satisfies a preset condition; wherein the error parameter of the first feature object is the sum of the variances corresponding to the coordinates of the first feature object in three dimensions in the IMU coordinate system, respectively, and the first feature object is any one of the X feature objects.

[0210] In one possible design, the second point cloud data is used to represent the three-dimensional coordinates of X features in the IMU coordinate system, where X is a positive integer;

[0211] When it is determined based on the second point cloud data that the extrinsic parameter value to be optimized used in the current adjustment makes the positions of the same feature object collected by the laser radar at different locations in the IMU coordinate system the same or the position difference satisfies a preset condition, the processing unit 1002 is configured to:

[0212] If the difference between the sum of the error parameters of the X feature objects in the current adjustment and the sum of the error parameters of the X feature objects in the previous adjustment is less than a first threshold, then the extrinsic parameter value to be optimized for the current adjustment is determined so that the position of the same feature object collected by the lidar at different locations in the IMU coordinate system is the same or the position difference satisfies a preset condition; wherein the error parameter of the first feature object is the sum of the variances corresponding to the coordinates of the first feature object in three dimensions in the IMU coordinate system, respectively, and the first feature object is any one of the X feature objects.

[0213] In one possible design, the second point cloud data is used to represent the three-dimensional coordinates of X features in the IMU coordinate system, where X is a positive integer;

[0214] When it is determined based on the second point cloud data that the extrinsic parameter value to be optimized used in the current adjustment makes the positions of the same feature object collected by the laser radar at different locations in the IMU coordinate system the same or the position difference satisfies a preset condition, the processing unit 1002 is configured to:

[0215] If the differences between the error parameters of the X feature objects in the current adjustment and the error parameters of the X feature objects in the previous adjustment are all less than a second threshold, then the extrinsic parameter value to be optimized for the current adjustment is determined so that the position of the same feature object collected by the lidar at different locations in the IMU coordinate system is the same or the position difference satisfies a preset condition; wherein the error parameter of the first feature object is the sum of the variances corresponding to the coordinates of the first feature object in three dimensions in the IMU coordinate system, and the first feature object is any one of the X features.

[0216] In one possible design, the second point cloud data is used to represent the three-dimensional coordinates of X features in the IMU coordinate system, where X is a positive integer;

[0217] When it is determined based on the second point cloud data that the extrinsic parameter value to be optimized used in the current adjustment makes the positions of the same feature object collected by the laser radar at different locations in the IMU coordinate system the same or the position difference satisfies a preset condition, the processing unit 1002 is configured to:

[0218] If the difference between the sum of the error parameters of the X feature objects in the current adjustment and the sum of the error parameters of the X feature objects in the previous adjustment is less than a first threshold, and the difference between the extrinsic parameter value to be optimized used in the current adjustment and the extrinsic parameter value to be optimized used in the previous adjustment is less than a third threshold, then the extrinsic parameter value to be optimized used in the current adjustment is determined so that the positions of the same feature object collected by the lidar at different positions in the IMU coordinate system are the same or the position difference satisfies a preset condition; wherein the error parameter of the first feature object is the sum of the variances corresponding to the coordinates of the first feature object in three dimensions in the IMU coordinate system, and the first feature object is any one of the X feature objects.

[0219] In one possible design, the second point cloud data is used to represent the three-dimensional coordinates of X features in the IMU coordinate system, where X is a positive integer;

[0220] When it is determined based on the second point cloud data that the extrinsic parameter value to be optimized used in the current adjustment makes the positions of the same feature object collected by the laser radar at different locations in the IMU coordinate system the same or the position difference satisfies a preset condition, the processing unit 1002 is configured to:

[0221] If the differences between the error parameters of the X feature objects in the current adjustment and the error parameters of the X feature objects in the previous adjustment are all less than the second threshold, and the difference between the extrinsic parameter value to be optimized used in the current adjustment and the extrinsic parameter value to be optimized used in the previous adjustment is less than the third threshold, then the extrinsic parameter value to be optimized used in the current adjustment is determined so that the positions of the same feature object collected by the laser radar at different positions in the IMU coordinate system are the same or the position difference satisfies a preset condition; wherein the error parameter of the first feature object is the sum of the variances corresponding to the coordinates of the first feature object in three dimensions of the IMU coordinate system, and the first feature object is any one of the X feature objects.

[0222] In another possible application scenario, the calibration device 1000 includes a communication unit 1001 and a processing unit 1002.

[0223] The communication unit 1001 is configured to obtain first point cloud data collected by a laser radar, where the first point cloud data is used to represent positions of features around the vehicle to be calibrated collected when the vehicle to be calibrated is traveling on a target path in a laser radar coordinate system;

[0224] The processing unit 1002 is used to obtain, in the current adjustment, a fourth relative transformation relationship between the lidar coordinate system and the reference coordinate system according to the external parameter value to be optimized and the second relative transformation relationship between the IMU coordinate system and the reference coordinate system, wherein the external parameter value to be optimized is used to indicate the first relative transformation relationship between the lidar coordinate system and the IMU coordinate system, and the second relative transformation relationship between the IMU coordinate system and the reference coordinate system is obtained according to the position and posture of the vehicle to be calibrated collected by the IMU during the process of the vehicle to be calibrated traveling on the target path; according to the lidar coordinate system The fourth relative transformation relationship between the system and the reference coordinate system is established, and the first point cloud data is transformed from the lidar coordinate system to the reference coordinate system to obtain the second point cloud data; and, based on the second point cloud data, the extrinsic parameter value to be optimized used in the current adjustment is determined so that when the positions of the same feature object collected by the lidar when traveling to different positions are the same in the IMU coordinate system or the position difference meets the preset conditions, the current extrinsic parameter value to be optimized is used as the target extrinsic parameter value; otherwise, the extrinsic parameter value to be optimized is adjusted, and the adjusted extrinsic parameter value is used as the extrinsic parameter value to be optimized in the next adjustment.

[0225] Based on the same technical concept, this application also provides an external parameter calibration device for laser radar and IMU, and the calibration device 1100 can achieve Figure 5-Figure 9 The function of the calibration device in the calibration method shown. Figure 11As shown, the calibration device 1100 includes: a transceiver 1101, a processor 1102 and a memory 1103. The transceiver 1101, the processor 1102 and the memory 1103 are interconnected. For example, the processor 1102 can be used to execute Figure 5-Figure 9 In any of the embodiments shown, all operations except the acquisition operation are performed by the calibration device.

[0226] Exemplarily, the transceiver 1101, the processor 1102, and the memory 1103 are interconnected via a bus 1104. The bus 1104 may be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus. The bus may be divided into an address bus, a data bus, a control bus, and the like. For ease of representation, Figure 11 Only one thick line is used in the diagram, but this does not mean that there is only one bus or one type of bus.

[0227] The transceiver 1101 is used to receive and send data to achieve communication interaction with other devices.

[0228] It is understood that this application Figure 11The memory in the memory may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct RAM bus random access memory (DR RAM). It should be noted that the memory of the systems and methods described herein is intended to include, but is not limited to, these and any other suitable types of memory.

[0229] Based on the above embodiments, the present application also provides a computer program, which, when executed on a computer, enables the computer to execute Figure 5-Figure 9 The illustrated embodiment provides a calibration method.

[0230] Based on the above embodiments, the present application also provides a computer-readable storage medium in which a computer program is stored. When the computer program is executed by a computer, the computer executes Figure 5-Figure 9 The embodiment shown provides a calibration method. The storage medium may be any available medium that can be accessed by a computer. By way of example and not limitation, computer-readable media may include RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage media or other magnetic storage devices, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer.

[0231] Based on the above embodiments, the present application also provides a chip, which is used to read the computer program stored in the memory to implement Figure 5-Figure 9 The illustrated embodiment provides a calibration method.

[0232] Based on the above embodiments, the present invention provides a chip system, which includes a processor for supporting a computer device to implement Figure 5-Figure 9 The functions involved in the calibration device in the embodiment shown. In one possible design, the chip system also includes a memory for storing the necessary programs and data for the computer device. The chip system can be composed of a chip or include a chip and other discrete devices.

[0233] Those skilled in the art will appreciate that the embodiments of the present application can be provided as methods, systems, or computer program products. Therefore, the present application can adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application can adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0234] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the present application. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0235] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0236] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0237] Obviously, those skilled in the art may make various modifications and variations to this application without departing from the scope of protection of this application. Thus, if these modifications and variations of this application fall within the scope of the claims of this application and their equivalents, this application is intended to include these modifications and variations.

Claims

1. A method for calibrating external parameters of a laser radar and an inertial navigation unit (IMU), applied to a calibration device, characterized in that: The laser radar and the IMU are both fixedly installed on the vehicle to be calibrated, including: Acquire first point cloud data collected by a laser radar, the first point cloud data being used to represent positions of characteristic objects around the vehicle to be calibrated collected while the vehicle to be calibrated is traveling on a target path in a laser radar coordinate system; wherein the vehicle to be calibrated passes the same characteristic object multiple times while traveling on the target path; In the current adjustment, the first point cloud data is converted from the lidar coordinate system to the IMU coordinate system to obtain second point cloud data according to the extrinsic parameter value to be optimized, wherein the extrinsic parameter value to be optimized is used to indicate a first relative conversion relationship between the lidar coordinate system and the IMU coordinate system; The second point cloud data is converted into the reference coordinate system according to a second relative transformation relationship between the IMU coordinate system and the reference coordinate system to obtain third point cloud data; the second relative transformation relationship is obtained based on the position and posture of the vehicle to be calibrated collected by the IMU during the process of the vehicle to be calibrated traveling on the target path; According to the third point cloud data, the extrinsic parameter value to be optimized used in the current adjustment is determined so that when the positions of the same feature object collected by the laser radar at different times in the reference coordinate system are the same or the position difference meets the preset conditions, the extrinsic parameter value to be optimized in the current time is used as the target extrinsic parameter value; otherwise, the extrinsic parameter value to be optimized is adjusted, and the adjusted extrinsic parameter value is used as the extrinsic parameter value to be optimized in the next adjustment.

2. The method according to claim 1, wherein The first point cloud data is collected by the laser radar at N first collection moments, and the second relative transformation relationship between the IMU coordinate system and the reference coordinate system is obtained by: Acquire measurement data collected by the IMU at M second collection moments, the measurement data including linear acceleration and angular velocity of the vehicle to be calibrated collected by the IMU while the vehicle to be calibrated is traveling on the target path; According to the measurement data, a second relative transformation relationship at each of the M second acquisition moments is obtained, where the second relative transformation relationship at the second acquisition moment is used to characterize the relative transformation relationship between the IMU coordinate system and the reference coordinate system at the second acquisition moment.

3. The method according to claim 2, wherein The method further comprises: converting the second point cloud data into the reference coordinate system according to a second relative transformation relationship between the IMU coordinate system and the reference coordinate system to obtain third point cloud data, comprising: Obtaining, according to the second relative transformation relationships at the M second acquisition moments, third relative transformation relationships at the N first acquisition moments, wherein the third relative transformation relationships at the first acquisition moments are used to represent the relative transformation relationship between the IMU coordinate system and the reference coordinate system at the first acquisition moment; According to the third relative transformation relationship of the i-th first acquisition moment, the second point cloud data of the i-th first acquisition moment is respectively transformed into the reference coordinate system to obtain the point cloud data of the i-th first acquisition moment, where i is a positive integer less than or equal to N to obtain the point cloud data of N first acquisition moments to constitute the third point cloud data.

4. The method according to any one of claims 1 to 3, wherein The third point cloud data is used to represent the three-dimensional coordinates of X features in the reference coordinate system, where X is a positive integer; Determining, based on the third point cloud data, the extrinsic parameter values ​​to be optimized for the current adjustment so that the positions of the same feature object collected by the laser radar at different times in the reference coordinate system are the same or the position difference satisfies a preset condition, including: If the difference between the sum of the error parameters of the X feature objects in the current adjustment and the sum of the error parameters of the X feature objects in the previous adjustment is less than a first threshold, then determining the extrinsic parameter value to be optimized for the current adjustment so that the positions of the same feature object collected by the lidar at different times in the reference coordinate system are the same or the position difference meets a preset condition; The error parameter of the first feature object is the sum of the variances corresponding to the coordinates of the first feature object in three dimensions in the reference coordinate system, and the first feature object is any one of the X feature objects.

5. The method according to any one of claims 1 to 3, wherein The third point cloud data is used to represent the three-dimensional coordinates of X features in the reference coordinate system, where X is a positive integer; Determining, based on the third point cloud data, the extrinsic parameter values ​​to be optimized for the current adjustment so that the positions of the same feature object collected by the laser radar at different times in the reference coordinate system are the same or the position difference satisfies a preset condition, including: If the differences between the error parameters of the X feature objects in the current adjustment and the error parameters of the X feature objects in the previous adjustment are all less than a second threshold, determining the extrinsic parameter values ​​to be optimized for the current adjustment so that the positions of the same feature objects collected by the lidar at different locations in the reference coordinate system are the same or the positions differ by a preset condition; The error parameter of the first feature object is the sum of the variances corresponding to the coordinates of the first feature object in three dimensions in the reference coordinate system, and the first feature object is any one of the X feature objects.

6. The method according to any one of claims 1 to 3, wherein: The third point cloud data is used to represent the three-dimensional coordinates of X features in the reference coordinate system, where X is a positive integer; Determining, based on the third point cloud data, the extrinsic parameter values ​​to be optimized for the current adjustment so that the positions of the same feature object collected by the laser radar at different times in the reference coordinate system are the same or the position difference satisfies a preset condition, including: If the difference between the sum of the error parameters of the X feature objects in the current adjustment and the sum of the error parameters of the X feature objects in the previous adjustment is less than a first threshold, and the difference between the extrinsic parameter value to be optimized used in the current adjustment and the extrinsic parameter value to be optimized used in the previous adjustment is less than a third threshold, then it is determined that the extrinsic parameter value to be optimized used in the current adjustment makes the positions of the same feature object collected by the laser radar at different times in the reference coordinate system the same or the position difference meets a preset condition; The error parameter of the first feature object is the sum of the variances corresponding to the coordinates of the first feature object in three dimensions in the reference coordinate system, and the first feature object is any one of the X feature objects.

7. The method according to any one of claims 1 to 3, wherein: The third point cloud data is used to represent the three-dimensional coordinates of X features in the reference coordinate system, where X is a positive integer; Determining, based on the third point cloud data, the extrinsic parameter values ​​to be optimized for the current adjustment so that the positions of the same feature object collected by the laser radar at different times in the reference coordinate system are the same or the position difference satisfies a preset condition, including: If the differences between the error parameters of the X feature objects in the current adjustment and the error parameters of the X feature objects in the previous adjustment are all less than the second threshold, and the difference between the extrinsic parameter value to be optimized used in the current adjustment and the extrinsic parameter value to be optimized used in the previous adjustment is less than the third threshold, then it is determined that the extrinsic parameter value to be optimized used in the current adjustment makes the positions of the same feature object collected by the laser radar at different times in the reference coordinate system the same or the position difference meets a preset condition; The error parameter of the first feature object is the sum of the variances corresponding to the coordinates of the first feature object in three dimensions in the reference coordinate system, and the first feature object is any one of the X feature objects.

8. The method according to any one of claims 1 to 7, wherein: The method further comprises: converting the second point cloud data into the reference coordinate system according to a second relative transformation relationship between the IMU coordinate system and the reference coordinate system to obtain third point cloud data, comprising: Converting the second point cloud data from the IMU coordinate system to the reference coordinate system according to a second relative conversion relationship between the IMU coordinate system and the reference coordinate system to obtain fourth point cloud data; The fourth point cloud data after motion compensation is used as the third point cloud data. The fourth point cloud data after motion compensation is motion compensated based on the position, posture and speed of the vehicle to be calibrated collected by the IMU during the process of the vehicle to be calibrated traveling on the target path.

9. A method for calibrating external parameters of a laser radar and an IMU, applied to a calibration device, characterized in that: The laser radar and the IMU are both fixedly installed on the vehicle to be calibrated, including: Acquire first point cloud data collected by a laser radar, the first point cloud data being used to represent positions of characteristic objects around the vehicle to be calibrated collected while the vehicle to be calibrated is traveling on a target path in a laser radar coordinate system; wherein the vehicle to be calibrated passes the same characteristic object multiple times while traveling on the target path; In the current adjustment, the first point cloud data is converted from the lidar coordinate system to the IMU coordinate system to obtain second point cloud data according to the extrinsic parameter value to be optimized, wherein the extrinsic parameter value to be optimized is used to indicate a first relative conversion relationship between the lidar coordinate system and the IMU coordinate system; According to the second point cloud data, the extrinsic parameter value to be optimized used in the current adjustment is determined so that when the positions of the same feature object collected by the laser radar at different times in the IMU coordinate system are the same or the position difference meets the preset conditions, the extrinsic parameter value to be optimized in the current time is used as the target extrinsic parameter value; otherwise, the extrinsic parameter value to be optimized is adjusted, and the adjusted extrinsic parameter value is used as the extrinsic parameter value to be optimized in the next adjustment.

10. The method according to claim 9, wherein The second point cloud data is used to represent the three-dimensional coordinates of X feature objects in the IMU coordinate system, where X is a positive integer; Determining, based on the second point cloud data, the extrinsic parameter values ​​to be optimized for the current adjustment so that the positions of the same feature object collected by the lidar at different times in the IMU coordinate system are the same or the position difference satisfies a preset condition, including: If the difference between the sum of the error parameters of the X feature objects in the current adjustment and the sum of the error parameters of the X feature objects in the previous adjustment is less than a first threshold, then determining the extrinsic parameter value to be optimized for the current adjustment so that the positions of the same feature object collected by the lidar at different times in the IMU coordinate system are the same or the position difference meets a preset condition; The error parameter of the first feature object is the sum of the variances corresponding to the coordinates of the first feature object in three dimensions in the IMU coordinate system, and the first feature object is any one of the X feature objects.

11. The method according to claim 9, wherein The second point cloud data is used to represent the three-dimensional coordinates of X feature objects in the IMU coordinate system, where X is a positive integer; Determining, based on the second point cloud data, the extrinsic parameter values ​​to be optimized for the current adjustment so that the positions of the same feature object collected by the lidar at different times in the IMU coordinate system are the same or the position difference satisfies a preset condition, including: If the differences between the error parameters of the X feature objects in the current adjustment and the error parameters of the X feature objects in the previous adjustment are all less than the second threshold, then determining the extrinsic parameter values ​​to be optimized for the current adjustment so that the positions of the same feature object collected by the lidar at different times in the IMU coordinate system are the same or the position difference meets a preset condition; The error parameter of the first feature object is the sum of the variances corresponding to the coordinates of the first feature object in three dimensions in the IMU coordinate system, and the first feature object is any one of the X feature objects.

12. The method according to claim 9, wherein The second point cloud data is used to represent the three-dimensional coordinates of X feature objects in the IMU coordinate system, where X is a positive integer; Determining, based on the second point cloud data, the extrinsic parameter values ​​to be optimized for the current adjustment so that the positions of the same feature object collected by the lidar at different times in the IMU coordinate system are the same or the position difference satisfies a preset condition, including: If the difference between the sum of the error parameters of the X feature objects in the current adjustment and the sum of the error parameters of the X feature objects in the previous adjustment is less than a first threshold, and the difference between the extrinsic parameter value to be optimized used in the current adjustment and the extrinsic parameter value to be optimized used in the previous adjustment is less than a third threshold, then it is determined that the extrinsic parameter value to be optimized used in the current adjustment makes the positions of the same feature object collected by the lidar at different times in the IMU coordinate system the same or the position difference meets a preset condition; The error parameter of the first feature object is the sum of the variances corresponding to the coordinates of the first feature object in three dimensions in the IMU coordinate system, and the first feature object is any one of the X feature objects.

13. The method according to claim 9, wherein The second point cloud data is used to represent the three-dimensional coordinates of X feature objects in the IMU coordinate system, where X is a positive integer; Determining, based on the second point cloud data, the extrinsic parameter values ​​to be optimized for the current adjustment so that the positions of the same feature object collected by the lidar at different times in the IMU coordinate system are the same or the position difference satisfies a preset condition, including: If the difference between the error parameters of the X feature objects in the current adjustment and the error parameters of the X feature objects in the previous adjustment is less than the second threshold, and the difference between the extrinsic parameter value to be optimized used in the current adjustment and the extrinsic parameter value to be optimized used in the previous adjustment is less than the third threshold, then it is determined that the extrinsic parameter value to be optimized used in the current adjustment makes the positions of the same feature object collected by the lidar at different times in the IMU coordinate system the same or the position difference meets a preset condition; The error parameter of the first feature object is the sum of the variances corresponding to the coordinates of the first feature object in three dimensions in the IMU coordinate system, and the first feature object is any one of the X feature objects.

14. A method for calibrating external parameters of a laser radar and an IMU, applied to a calibration device, characterized in that: The laser radar and the IMU are both fixedly installed on the vehicle to be calibrated, including: Acquire first point cloud data collected by a laser radar, the first point cloud data being used to represent positions of characteristic objects around the vehicle to be calibrated collected while the vehicle to be calibrated is traveling on a target path in a laser radar coordinate system; wherein the vehicle to be calibrated passes the same characteristic object multiple times while traveling on the target path; In the current adjustment, a fourth relative transformation relationship between the lidar coordinate system and the reference coordinate system is obtained according to the extrinsic parameter value to be optimized and the second relative transformation relationship between the IMU coordinate system and the reference coordinate system, wherein the extrinsic parameter value to be optimized is used to indicate the first relative transformation relationship between the lidar coordinate system and the IMU coordinate system, and the second relative transformation relationship between the IMU coordinate system and the reference coordinate system is obtained according to the position and posture of the vehicle to be calibrated collected by the IMU during the process of the vehicle to be calibrated traveling on the target path; According to a fourth relative transformation relationship between the laser radar coordinate system and the reference coordinate system, the first point cloud data is transformed from the laser radar coordinate system to the reference coordinate system to obtain second point cloud data; According to the second point cloud data, the extrinsic parameter value to be optimized used in the current adjustment is determined so that when the positions of the same feature object collected by the laser radar at different times in the reference coordinate system are the same or the position difference meets the preset conditions, the extrinsic parameter value to be optimized in the current time is used as the target extrinsic parameter value; otherwise, the extrinsic parameter value to be optimized is adjusted, and the adjusted extrinsic parameter value is used as the extrinsic parameter value to be optimized in the next adjustment.

15. A laser radar and inertial navigation unit IMU external parameter calibration device, characterized in that: The laser radar and the IMU are both fixedly installed on the vehicle to be calibrated, including: a communication unit configured to obtain first point cloud data collected by a laser radar, wherein the first point cloud data is used to represent positions of features around the vehicle to be calibrated collected while the vehicle to be calibrated is traveling on a target path, in a laser radar coordinate system; wherein the vehicle to be calibrated passes the same feature multiple times while traveling on the target path; A processing unit is configured to, in a current adjustment, convert the first point cloud data from the lidar coordinate system to the IMU coordinate system to obtain second point cloud data according to the extrinsic parameter value to be optimized, wherein the extrinsic parameter value to be optimized is used to indicate a first relative transformation relationship between the lidar coordinate system and the IMU coordinate system; convert the second point cloud data to the reference coordinate system to obtain third point cloud data according to a second relative transformation relationship between the IMU coordinate system and the reference coordinate system; the second relative transformation relationship is obtained based on the position and posture of the vehicle to be calibrated collected by the IMU during the process of the vehicle to be calibrated traveling on the target path; and, based on the third point cloud data, determine the extrinsic parameter value to be optimized used in the current adjustment so that when the positions of the same feature object collected by the lidar at different times in the reference coordinate system are the same or the position difference meets a preset condition, the current extrinsic parameter value to be optimized is used as the target extrinsic parameter value; otherwise, the extrinsic parameter value to be optimized is adjusted, and the adjusted extrinsic parameter value is used as the extrinsic parameter value to be optimized in the next adjustment.

16. The device according to claim 15, characterized in that The first point cloud data is collected by the laser radar at N first collection moments, and the communication unit is further used to obtain measurement data collected by the IMU at M second collection moments, the measurement data including the linear acceleration and angular velocity of the vehicle to be calibrated collected by the IMU during the process of the vehicle to be calibrated traveling on the target path; When acquiring a second relative transformation relationship between the IMU coordinate system and the reference coordinate system, the processing unit is configured to: According to the measurement data, a second relative transformation relationship at each of the M second acquisition moments is obtained, where the second relative transformation relationship at the second acquisition moment is used to characterize the relative transformation relationship between the IMU coordinate system and the reference coordinate system at the second acquisition moment.

17. The device according to claim 16, wherein When converting the second point cloud data into the reference coordinate system to obtain third point cloud data according to a second relative conversion relationship between the IMU coordinate system and the reference coordinate system, the processing unit is configured to: Obtaining, according to the second relative transformation relationships at the M second acquisition moments, third relative transformation relationships at the N first acquisition moments, wherein the third relative transformation relationships at the first acquisition moments are used to represent the relative transformation relationship between the IMU coordinate system and the reference coordinate system at the first acquisition moment; According to the third relative transformation relationship of the i-th first acquisition moment, the second point cloud data of the i-th first acquisition moment is respectively transformed into the reference coordinate system to obtain the point cloud data of the i-th first acquisition moment, where i is a positive integer less than or equal to N to obtain the point cloud data of N first acquisition moments to constitute the third point cloud data.

18. The device according to any one of claims 15 to 17, characterized in that The third point cloud data is used to represent the three-dimensional coordinates of X features in the reference coordinate system, where X is a positive integer; When it is determined based on the third point cloud data that the extrinsic parameter value to be optimized used in the current adjustment makes the positions of the same feature object collected by the laser radar at different times in the reference coordinate system the same or the position difference satisfies a preset condition, the processing unit is configured to: If the difference between the sum of the error parameters of the X feature objects in the current adjustment and the sum of the error parameters of the X feature objects in the previous adjustment is less than a first threshold, then determining the extrinsic parameter value to be optimized for the current adjustment so that the positions of the same feature object collected by the lidar at different times in the reference coordinate system are the same or the position difference meets a preset condition; The error parameter of the first feature object is the sum of the variances corresponding to the coordinates of the first feature object in three dimensions in the reference coordinate system, and the first feature object is any one of the X feature objects.

19. The device according to any one of claims 15 to 17, characterized in that The third point cloud data is used to represent the three-dimensional coordinates of X features in the reference coordinate system, where X is a positive integer; When it is determined based on the third point cloud data that the extrinsic parameter value to be optimized used in the current adjustment makes the positions of the same feature object collected by the laser radar at different times in the reference coordinate system the same or the position difference satisfies a preset condition, the processing unit is configured to: If the differences between the error parameters of the X feature objects in the current adjustment and the error parameters of the X feature objects in the previous adjustment are all less than a second threshold, determining the extrinsic parameter values ​​to be optimized for the current adjustment so that the positions of the same feature objects collected by the lidar at different locations in the reference coordinate system are the same or the positions differ by a preset condition; The error parameter of the first feature object is the sum of the variances corresponding to the coordinates of the first feature object in three dimensions in the reference coordinate system, and the first feature object is any one of the X feature objects.

20. The device according to any one of claims 15 to 17, characterized in that The third point cloud data is used to represent the three-dimensional coordinates of X features in the reference coordinate system, where X is a positive integer; When it is determined based on the third point cloud data that the extrinsic parameter value to be optimized used in the current adjustment makes the positions of the same feature object collected by the laser radar at different times in the reference coordinate system the same or the position difference satisfies a preset condition, the processing unit is configured to: If the difference between the sum of the error parameters of the X feature objects in the current adjustment and the sum of the error parameters of the X feature objects in the previous adjustment is less than a first threshold, and the difference between the extrinsic parameter value to be optimized used in the current adjustment and the extrinsic parameter value to be optimized used in the previous adjustment is less than a third threshold, then it is determined that the extrinsic parameter value to be optimized used in the current adjustment makes the positions of the same feature object collected by the laser radar at different times in the reference coordinate system the same or the position difference meets a preset condition; The error parameter of the first feature object is the sum of the variances corresponding to the coordinates of the first feature object in three dimensions in the reference coordinate system, and the first feature object is any one of the X feature objects.

21. The device according to any one of claims 15 to 17, characterized in that The third point cloud data is used to represent the three-dimensional coordinates of X features in the reference coordinate system, where X is a positive integer; When it is determined based on the third point cloud data that the extrinsic parameter value to be optimized used in the current adjustment makes the positions of the same feature object collected by the laser radar at different times in the reference coordinate system the same or the position difference satisfies a preset condition, the processing unit is configured to: If the differences between the error parameters of the X feature objects in the current adjustment and the error parameters of the X feature objects in the previous adjustment are all less than the second threshold, and the difference between the extrinsic parameter value to be optimized used in the current adjustment and the extrinsic parameter value to be optimized used in the previous adjustment is less than the third threshold, then it is determined that the extrinsic parameter value to be optimized used in the current adjustment makes the positions of the same feature object collected by the laser radar at different times in the reference coordinate system the same or the position difference meets a preset condition; The error parameter of the first feature object is the sum of the variances corresponding to the coordinates of the first feature object in three dimensions in the reference coordinate system, and the first feature object is any one of the X feature objects.

22. The device according to any one of claims 15 to 21, characterized in that When converting the second point cloud data into the reference coordinate system to obtain third point cloud data according to a second relative conversion relationship between the IMU coordinate system and the reference coordinate system, the processing unit is configured to: Converting the second point cloud data from the IMU coordinate system to the reference coordinate system according to a second relative conversion relationship between the IMU coordinate system and the reference coordinate system to obtain fourth point cloud data; The fourth point cloud data after motion compensation is used as the third point cloud data. The fourth point cloud data after motion compensation is motion compensated based on the position, posture and speed of the vehicle to be calibrated collected by the IMU during the process of the vehicle to be calibrated traveling on the target path.

23. A laser radar and IMU external parameter calibration device, characterized in that: The laser radar and the IMU are both fixedly installed on the vehicle to be calibrated, including: a communication unit configured to obtain first point cloud data collected by a laser radar, wherein the first point cloud data is used to represent positions of features around the vehicle to be calibrated collected while the vehicle to be calibrated is traveling on a target path, in a laser radar coordinate system; wherein the vehicle to be calibrated passes the same feature multiple times while traveling on the target path; A processing unit is used to convert the first point cloud data from the lidar coordinate system to the IMU coordinate system to obtain second point cloud data according to the extrinsic parameter value to be optimized in the current adjustment, wherein the extrinsic parameter value to be optimized is used to indicate a first relative conversion relationship between the lidar coordinate system and the IMU coordinate system; and, based on the second point cloud data, determine the extrinsic parameter value to be optimized used in the current adjustment so that when the positions of the same feature object collected by the lidar at different times in the IMU coordinate system are the same or the position difference meets a preset condition, the current extrinsic parameter value to be optimized is used as the target extrinsic parameter value; otherwise, the extrinsic parameter value to be optimized is adjusted, and the adjusted extrinsic parameter value is used as the extrinsic parameter value to be optimized in the next adjustment.

24. The device according to claim 23, wherein The second point cloud data is used to represent the three-dimensional coordinates of X feature objects in the IMU coordinate system, where X is a positive integer; When it is determined based on the second point cloud data that the extrinsic parameter value to be optimized used in the current adjustment makes the positions of the same feature object collected by the laser radar at different positions in the IMU coordinate system the same or the position difference satisfies a preset condition, the processing unit is configured to: If the difference between the sum of the error parameters of the X feature objects in the current adjustment and the sum of the error parameters of the X feature objects in the previous adjustment is less than a first threshold, then determining the extrinsic parameter value to be optimized for the current adjustment so that the positions of the same feature object collected by the lidar at different times in the IMU coordinate system are the same or the position difference meets a preset condition; The error parameter of the first feature object is the sum of the variances corresponding to the coordinates of the first feature object in three dimensions in the IMU coordinate system, and the first feature object is any one of the X feature objects.

25. The device according to claim 24, wherein The second point cloud data is used to represent the three-dimensional coordinates of X feature objects in the IMU coordinate system, where X is a positive integer; When determining, based on the second point cloud data, the extrinsic parameter value to be optimized used in the current adjustment so that the positions of the same feature object collected by the laser radar at different times in the IMU coordinate system are the same or the position difference satisfies a preset condition, the processing unit is configured to: If the difference between the sum of the error parameters of the X feature objects in the current adjustment and the sum of the error parameters of the X feature objects in the previous adjustment is less than a first threshold, then determining the extrinsic parameter value to be optimized for the current adjustment so that the positions of the same feature object collected by the lidar at different times in the IMU coordinate system are the same or the position difference meets a preset condition; The error parameter of the first feature object is the sum of the variances corresponding to the coordinates of the first feature object in three dimensions in the IMU coordinate system, and the first feature object is any one of the X feature objects.

26. The device according to claim 24, wherein The second point cloud data is used to represent the three-dimensional coordinates of X feature objects in the IMU coordinate system, where X is a positive integer; When determining, based on the second point cloud data, the extrinsic parameter value to be optimized used in the current adjustment so that the positions of the same feature object collected by the laser radar at different times in the IMU coordinate system are the same or the position difference satisfies a preset condition, the processing unit is configured to: If the difference between the sum of the error parameters of the X feature objects in the current adjustment and the sum of the error parameters of the X feature objects in the previous adjustment is less than a first threshold, and the difference between the extrinsic parameter value to be optimized used in the current adjustment and the extrinsic parameter value to be optimized used in the previous adjustment is less than a third threshold, then it is determined that the extrinsic parameter value to be optimized used in the current adjustment makes the positions of the same feature object collected by the lidar at different times in the IMU coordinate system the same or the position difference meets a preset condition; The error parameter of the first feature object is the sum of the variances corresponding to the coordinates of the first feature object in three dimensions in the IMU coordinate system, and the first feature object is any one of the X feature objects.

27. The device according to claim 24, wherein The second point cloud data is used to represent the three-dimensional coordinates of X feature objects in the IMU coordinate system, where X is a positive integer; When determining, based on the second point cloud data, the extrinsic parameter value to be optimized used in the current adjustment so that the positions of the same feature object collected by the laser radar at different times in the IMU coordinate system are the same or the position difference satisfies a preset condition, the processing unit is configured to: If the difference between the error parameters of the X feature objects in the current adjustment and the error parameters of the X feature objects in the previous adjustment is less than the second threshold, and the difference between the extrinsic parameter value to be optimized used in the current adjustment and the extrinsic parameter value to be optimized used in the previous adjustment is less than the third threshold, then it is determined that the extrinsic parameter value to be optimized used in the current adjustment makes the positions of the same feature object collected by the lidar at different times in the IMU coordinate system the same or the position difference meets a preset condition; The error parameter of the first feature object is the sum of the variances corresponding to the coordinates of the first feature object in three dimensions in the IMU coordinate system, and the first feature object is any one of the X feature objects.

28. A laser radar and IMU external parameter calibration device, characterized in that: The laser radar and the IMU are both fixedly installed on the vehicle to be calibrated, including: a communication unit configured to obtain first point cloud data collected by a laser radar, wherein the first point cloud data is used to represent positions of features around the vehicle to be calibrated collected while the vehicle to be calibrated is traveling on a target path, in a laser radar coordinate system; wherein the vehicle to be calibrated passes the same feature multiple times while traveling on the target path; A processing unit is configured to obtain, in a current adjustment, a fourth relative transformation relationship between a lidar coordinate system and a reference coordinate system based on an extrinsic parameter value to be optimized and a second relative transformation relationship between an IMU coordinate system and a reference coordinate system, wherein the extrinsic parameter value to be optimized is used to indicate a first relative transformation relationship between the lidar coordinate system and the IMU coordinate system, and the second relative transformation relationship between the IMU coordinate system and the reference coordinate system is obtained based on the position and posture of the vehicle to be calibrated collected by the IMU during the process of the vehicle to be calibrated traveling on the target path; convert the first point cloud data from the lidar coordinate system to the reference coordinate system to obtain second point cloud data based on the fourth relative transformation relationship between the lidar coordinate system and the reference coordinate system; and determine, based on the second point cloud data, the extrinsic parameter value to be optimized for use in the current adjustment so that when the positions of the same feature object collected by the lidar at different times in the reference coordinate system are the same or the position difference satisfies a preset condition, the current extrinsic parameter value to be optimized is used as the target extrinsic parameter value; otherwise, the extrinsic parameter value to be optimized is adjusted, and the adjusted extrinsic parameter value is used as the extrinsic parameter value to be optimized in the next adjustment.

29. A calibration device, characterized in that: It comprises a memory and one or more processors; wherein the memory is used to store computer program code, and the computer program code includes computer instructions; when the computer instructions are executed by the processor, the calibration device executes the method as described in any one of claims 1 to 8, or executes the method as described in any one of claims 9 to 13, or executes the method as claimed in claim 14.

30. A computer-readable storage medium, characterized in that The method comprises computer instructions, which, when executed in a calibration device, cause the calibration device to execute the method according to any one of claims 1 to 8, or the method according to any one of claims 9 to 13, or the method according to claim 14.

31. A computer program product, characterized in that When the computer program product is run on a processor, the processor is enabled to execute the method according to any one of claims 1 to 8, or the method according to any one of claims 9 to 13, or the method according to claim 14.

Images