METHOD, DEVICE, CONTROL UNIT AND PRODUCT FOR DETERMINING SENSOR CALIBRATION PARAMETERS

Abstract

The present disclosure relates to a method, a device, a control unit, and a computer program for determining the calibration parameters of sensors. The method comprises determining odometer data from several first sensors based on the data from several first sensors. The method further comprises determining the calibration parameters of several first sensors and a second sensor based on the odometer data of the several first sensors and the data of the second sensor, wherein the first sensors and the second sensor are different sensor types. In this way, the simultaneous calibration of several different sensor types is enabled, thereby not only saving time but also improving the accuracy and consistency of the data obtained from these sensors.

Description

AREA OF INVENTION

[0001] The present disclosure relates to the technical field of sensors and relates in particular to a method, a device, an apparatus and a computer program product for determining the calibration parameters of sensors. STATE OF THE ART

[0002] Sensor technology plays a key role in the realization of intelligent and automated systems. Both in the process control of industrial manufacturing and in smart devices of everyday life, high-quality sensors are essential for obtaining precise information. To improve the measurement accuracy of sensors, high-performance sensors can be selected, rigorously calibrated and verified, the installation and operating environment optimized, suitable signal processing and filtering techniques employed to reduce interference, and measures implemented to validate and correct the measurement data.

[0003] Sensor calibration not only improves measurement accuracy but also enables the establishment of uniform measurement standards for sensor output, the optimization of potential performance issues, adaptation to different environments, increased reliability of measurement results, and easier integration of sensors with other system components. After manufacturing and installation, sensors typically require calibration through testing to ensure they meet design specifications and guarantee the accuracy of their readings. REVELATION OF THE INVENTION

[0004] In the embodiments of the present disclosure, a method, a device, a apparatus and a computer program product for determining the calibration parameters of sensors are proposed.

[0005] In a first aspect of the present disclosure, a method for determining the calibration parameters of sensors is provided. The method comprises determining odometer data from several first sensors based on the data from the several first sensors. The method further comprises determining the calibration parameters of the several first sensors and a second sensor based on the odometer data from the several first sensors and the data from the second sensor, wherein the first sensors and the second sensor are different sensor types.

[0006] In a second aspect of the present disclosure, an apparatus for determining the calibration parameters of sensors is provided. The apparatus comprises an odometer data determination unit configured to determine the odometer data of the first several sensors based on the data of the first several sensors. The apparatus further comprises a calibration parameter determination unit configured to determine the calibration parameters of the first several sensors and the second sensor based on the odometer data of the first several sensors and the data of a second sensor, wherein the first several sensors and the second sensor are different sensor types.

[0007] According to a third aspect of the present disclosure, a control unit is provided. This control unit comprises one or more processors and a storage device for storing one or more programs. When the one or more programs are executed by the one or more processors, the one or more processors are caused to implement a method for determining the calibration parameters of sensors. The method comprises determining odometer data from several first sensors based on the data from the several first sensors. The method further comprises determining the calibration parameters of several first sensors and a second sensor based on the odometer data from the several first sensors and the data from the second sensor, wherein the first sensors and the second sensor are different sensor types.

[0008] According to a fourth aspect of the present disclosure, a computer program product is provided. The computer program product is physically stored on a non-volatile, machine-readable medium and comprises machine-executable instructions, wherein, upon execution, these machine-executable instructions cause the machine to implement a method for determining the calibration parameters of sensors. The method comprises determining odometer data of several first sensors based on the data of several first sensors. The method further comprises determining the calibration parameters of several first sensors and a second sensor based on the odometer data of the several first sensors and the data of the second sensor, wherein the first sensors and the second sensor are different sensor types.

[0009] It should be understood that the content described in the disclosure of the invention is neither intended to define the essential or important features of the embodiments of the present disclosure, nor to limit the scope of the present disclosure. Other features of the present disclosure will be readily understood from the following description. DESCRIPTION OF THE FIGURES

[0010] The aforementioned and other features, advantages, and aspects of various embodiments of the present disclosure are illustrated by the following detailed description in conjunction with the figures. In the figures, identical or similar reference symbols represent identical or similar elements, wherein: Fig. 1 shows a schematic representation of an example environment in which some embodiments of the present disclosure can be implemented; Fig. 2 shows a flowchart of a method for determining the calibration parameters of sensors according to some embodiments of the present disclosure; Fig. Figure 3 shows a schematic representation of an example for determining the calibration parameters of sensors according to some embodiments of the present disclosure; Fig. 4 shows a schematic representation of a device for determining the calibration parameters of sensors according to some embodiments of the present disclosure; Fig. Figure 5 shows a schematic representation of a control unit according to some embodiments of the present disclosure.

[0011] In all figures, the same or similar reference symbols stand for the same or similar elements. DETAILED DESCRIPTION OF THE EXECUTION FORMS

[0012] The embodiments of this disclosure are described in more detail below with reference to the figures. Although some embodiments of this disclosure are illustrated in the figures, it is understood that this disclosure can be implemented in various forms and should not be interpreted as being limited to the embodiments presented here. Rather, these embodiments are provided to enable a thorough and complete understanding of this disclosure. It should be understood that the figures and embodiments of this disclosure serve only as examples and are not intended to limit the scope of protection of the disclosure.

[0013] In the description of the embodiments of this disclosure, the term "comprise" and similar terms should be understood as encompassing in an open manner, i.e., as "comprise but not be limited to." The term "based on" should be understood as "at least partially based on." The terms "an embodiment" or "this embodiment" should be understood as "at least one embodiment." The terms "first," "second," etc., may refer to different or the same objects. Although the terms "first" and "second" are used in this document to describe the elements, these elements are not intended to be limited by these terms. These terms serve to distinguish one element from another.For example, a first element can be called a second element and likewise a second element can be called a first element, without deviating from the scope of the exemplary embodiments.

[0014] As used in this document, the term "or" encompasses all possible combinations unless expressly stated otherwise or this is not practicable. For example, if it is stated that a component may comprise A or B, the component may comprise A, B, or A and B unless expressly stated otherwise or this is not practicable. In a second example, if it is stated that a component may comprise A, B, or C, the component may comprise A, B, C, A and B, A and C, B and C, or A, B, and C unless expressly stated otherwise or this is not practicable.

[0015] As described above, sensors must be calibrated after being installed on a platform (such as a vehicle, robot, or aircraft). Furthermore, during the platform's movement, vibrations or other factors can cause the sensor's position to deviate from its original position, necessitating periodic recalibration. Therefore, sensor calibration technology can be divided into two main categories: static calibration and dynamic calibration. Static calibration is performed when the sensor is at rest and primarily calibrates basic parameters such as position, angle, and zero point. Dynamic calibration, on the other hand, is performed while the sensor is in operation, simulating real-world scenarios to comprehensively evaluate and adjust the sensor's performance parameters.The accuracy of sensor calibration determines the performance of perception and forms the basis of all perceptual tasks. For example, autonomous vehicles, robots, and the like use different sensor types simultaneously to enable both environmental and self-perception, and the stability and accuracy of the sensors are of crucial importance, especially in the perception process of autonomous vehicles and robots.

[0016] Some existing techniques for calibrating extrinsic parameters of multiple sensors (where these parameters mainly describe the attitude and orientation of the sensors relative to an external reference coordinate system) use a sequential calibration procedure, in which the extrinsic calibration is performed for two of the multiple sensors at a time. For example, sensor A and sensor B are calibrated first, then sensor A and sensor C, and finally sensor B and sensor C. Since there is an error in each individual calibration, the errors accumulate over the course of the multiple calibrations, so that the relative position of the multiple sensors to each other can no longer be determined exactly. This results in a calibration process that is both time-consuming and does not allow for consistent integration of spatial data.Furthermore, these existing techniques typically require a static calibration environment and a target calibration plate, and struggle to manage calibration drift that occurs over time, limiting their applicability in dynamic real-world scenarios. Moreover, the calibration technologies in question are often limited to specific types of laser radars, reducing their universality. Additionally, existing calibration technologies make it difficult to calibrate temporal and spatial extrinsic parameters simultaneously, further increasing the time required for sensor calibration.

[0017] Aside from the aforementioned disadvantages, common calibration techniques for calibrating the extrinsic parameters of multiple sensors, when dealing with multiple laser radars, require the use of objects in the common field of view to solve an optimization equation for the extrinsic parameters in order to obtain them. With multiple laser radars having only a small overlap in the field of view, corresponding calibration procedures using target calibration plates are often difficult and ineffective.

[0018] Therefore, in embodiments of this disclosure, a solution for determining the calibration parameters of sensors is proposed. In these embodiments, this solution can determine the odometer data of several first sensors based on their data. The solution further comprises determining the calibration parameters of the several first sensors and a second sensor based on the odometer data of the several first sensors and the data of the second sensor, in order to provide them for downstream tasks.

[0019] This enables the simultaneous calibration of several different sensor types, saving time and improving the accuracy and consistency of the data acquired by these sensors. Furthermore, using odometer data avoids the problem that arises when calibrating multiple sensors, including multiple laser radars, where effective calibration of extrinsic parameters is impossible if the overlapping field of view of the laser radars is too small.

[0020] Fig. Figure 1 shows a schematic representation of an example environment 100 in which some embodiments of the present disclosure can be implemented. As in Fig. As shown in Figure 1, the environment 100 comprises a first sensor data module 106, a second sensor 108, and a calibration parameter determination module 110. Within the environment 100, the first sensor data module 106 includes several first sensors of the same type (102-1, 102-2, ..., 102-N) as well as several odometers of the same type (104-1, 104-2, ..., 104-N). Although the figure shows multiple first sensors, the number of first sensors is not limited to this, and the number of first sensors can also be one or two.

[0021] In Fig. 1 The first sensor 102 and the second sensor 108 are sensors of different types, wherein the sensors can be, for example, a radar sensor, a camera, a laser radar, an ultrasonic sensor, a GPS or an IMU, and the odometer 104 can be a method or device that estimates the change in position of an object over time based on data obtained from a moving sensor, for example, a long-range radar odometer, a camera odometer, a laser radar odometer, an ultrasonic sensor odometer, a GPS odometer or an IMU odometer.

[0022] In the first sensor data module 106, the data from each of the multiple first sensors 102 (102-1, 102-2, ..., 102-N) are transferred to the respective corresponding odometer 104 from the multiple odometers (104-1, 104-2, ..., 104-N). For example, the data from the first sensor 102-1 are transferred to the odometer 104-1, the data from the first sensor 102-2 are transferred to the odometer 104-2, and the data from the first sensor 102-N are transferred to the odometer 104-N.

[0023] After receiving the data from the multiple first sensors (102-1, 102-2, ..., 102-N), the multiple odometers (104-1, 104-2, ..., 104-N) calculate the odometer data of the multiple first sensors (102-1, 102-2, ..., 102-N) based on this data. The odometer data of the multiple first sensors (102-1, 102-2, ..., 102-N) can include position data (for example, the coordinates of the first sensor 102 in three-dimensional space), velocity data (for example, the linear and angular velocity of the first sensor 102), attitude and orientation data (to describe the attitude and orientation, for example, pitch angle, roll angle, and yaw angle of the first sensor 102), displacement data (for example, information about the distance traveled and direction of movement by the first sensor 102 within a certain period of time), and timestamp data (to record the exact acquisition time of each data point for performing time series analyses and processing).

[0024] In environment 100, the odometer data of the several first sensors (102-1, 102-2, ..., 102-N) are transferred from the first sensor data module 106 to the calibration parameter determination module 110, and the data of the second sensor 108 are also transferred to the calibration parameter determination module 110. Based on the odometer data of the multiple first sensors (102-1, 102-2, ..., 102-N) and the data of the second sensor 108, the calibration parameter determination module 110 calculates the calibration parameters of the multiple first sensors (102-1, 102-2, ..., 102-N) and the second sensor 108, respectively. The calibration parameters can include multiple translation vectors describing the transformation of the multiple first sensors (102-1, 102-2, ..., 102-N) into the coordinate system of the second sensor 108 (representing the displacement in the x, y, and z directions), and multiple rotation matrices for converting the multiple first sensors (102-1, 102-2, ..., 102-N) into the coordinate system of the second sensor 108 (representable, for example, in the form of Euler angles or quaternions), calibration parameters with respect to the second sensor 108, the time offset of the second sensor 108 relative to the several first sensors (102-1, 102-2, ..., 102-N) (where the several first sensors (102-1, 102-2, ..., 102-N) are time synchronized with each other and there is no time offset), as well as the gravity vector.

[0025] In this way, the odometer data of the multiple first sensors (102-1, 102-2, ..., 102-N) are calculated by the multiple odometers (104-1, 104-2, ..., 104-N), and the calibration parameter determination module 110 calculates the calibration parameters of the multiple first sensors (102-1, 102-2, ..., 102-N) and the second sensor 108 based on the odometer data of the multiple first sensors (102-1, 102-2, ..., 102-N) and the second sensor 108, thus enabling the simultaneous calibration of several different sensor types, which not only reduces the time required for the calibration process but also improves the accuracy and consistency of the data obtained from the multiple sensors. Furthermore, by determining the time offset of the second sensor 108 relative to the several first sensors (102-1, 102-2, ..., 102-N) solved the problem that temporal and spatial extrinsic parameters cannot be calibrated simultaneously, thus saving calibration time.

[0026] Fig. Figure 2 shows a flowchart of a method 200 for determining the calibration parameters of sensors according to some embodiments of the present disclosure. In some embodiments, the method 200 can be described in the Fig. The method 200 can be executed in the example environment 100 shown in section 1. The method 200 may further include additional actions not shown and / or omit actions shown, the scope of this disclosure being unlimited in this respect.

[0027] In block 202, procedure 200 can determine the odometer data of several first sensors based on the data of the several first sensors. For example, in the Fig. 1 depicted environment 100 the multiple odometers (104-1, 104-2, ..., 104-N) the odometer data of the multiple first sensors (102-1, 102-2, ..., 102-N) based on the data of the multiple first sensors (102-1, 102-2, ..., 102-N).

[0028] In some embodiments, the multiple first sensors (102-1, 102-2, ..., 102-N) are multiple laser radars that emit laser beams into the environment and measure the time the laser beams take from emission to reflection from a target object and return, in order to calculate the distance between the laser radar and the target object. A series of point cloud data can be acquired by continuously scanning and measuring with the laser beams.

[0029] In some embodiments, the data from the multiple first sensors (102-1, 102-2, ..., 102-N) are point cloud data acquired by multiple laser radars at different times, and these point cloud data contain three-dimensional coordinate information for determining the position and shape of objects, information about the reflection intensity, which describes the strength of the laser beam reflected by the object after it hits it, and time information, which records the exact time of measurement of each individual point.

[0030] In some embodiments, the multiple distance counters (104-1, 104-2, ..., 104-N) are laser radar distance counters that determine information such as the displacement and orientation angles of the carrier at different times relative to a specific initial position by analyzing and processing the data acquired by the laser radars. The distance counter data of the multiple first sensors (102-1, 102-2, ..., 102-N) include the angular velocity, acceleration, and angular acceleration of the laser radars.

[0031] In block 204, method 200 can determine the calibration parameters of the multiple first sensors and the second sensor based on the odometer data of the multiple first sensors and the data of a second sensor, where the first sensors and the second sensor are different sensor types. For example, in the Fig. 1 depicted environment 100 the calibration parameter determination module 110 the calibration parameters of the several first sensors (102-1, 102-2, ..., 102-N) and the second sensor 108 based on the odometer data of the several first sensors (102-1, 102-2, ..., 102-N) which come from the first sensor data module 106, as well as the data from the second sensor 108.

[0032] In some embodiments, the second sensor 108 is an IMU. An IMU essentially consists of an accelerometer and a gyroscope. An accelerometer measures the linear acceleration of an object in three orthogonal directions, and the change in velocity and displacement of the object can be obtained by integrating the acceleration. A gyroscope measures the angular velocity of an object about three orthogonal axes, and the change in angle of the object can be obtained by integrating the angular velocity. As the object moves, the accelerometer and gyroscope operate continuously, generating corresponding signals for acceleration and angular velocity. These signals can be processed and computed to obtain real-time information about the object's attitude and orientation, as well as its velocity and position.

[0033] In some embodiments, the data of the second sensor 108 are the angular velocity, acceleration, and angular acceleration of the IMU. The calibration parameters of the multiple first sensors (102-1, 102-2, ..., 102-N) and the second sensor 108 include multiple translation matrices converted from the multiple laser radars to the IMU coordinate system, multiple rotation matrices converted from the multiple laser radars to the IMU coordinate system, a gyroscope bias of the IMU, an accelerometer offset of the IMU, a time offset of the IMU relative to the multiple laser radars, and the gravity vector.

[0034] This enables the simultaneous calibration of several different sensor types, thereby improving the accuracy and consistency of the data acquired from the multiple sensors and reducing the time required for the calibration process. Furthermore, by determining the time offset of the second sensor 108 relative to the multiple first sensors (102-1, 102-2, ..., 102-N), the problem of not being able to calibrate temporal and spatial extrinsic parameters simultaneously is overcome.

[0035] Fig. Figure 3 shows a schematic representation of an example 300 for determining the calibration parameters of sensors according to some embodiments of the present disclosure. As in Fig. As shown in Figure 3, the environment of Example 300 comprises a laser radar 302-1, a laser radar 302-2, an IMU 306, a first phase module 312, and a second phase module 318. The first phase module 312 comprises a LiDAR distance counter 304-1, a laser distance counter 304-2, a measurement noise reduction module 308, and a first parameter calibration module 310. The second phase module 318 comprises a time-space optimization module 314 and a second parameter calibration module 316.

[0036] In Example 300, the point cloud data acquired by laser radar 302-1 and laser radar 302-2 at different times are transferred to their respective LiDAR distance counters 304-1 and 304-2. These distance counters then calculate the angular velocity, acceleration, and angular acceleration of the laser radars 302-1 and 304-2. This approach eliminates the need to calibrate the extrinsic parameters even when the overlapping field of view of laser radar 302-1 and 302-2 is very narrow, because the distance counter data is calculated directly by the laser radar distance counters 304-1 and 304-2, thus avoiding the need to utilize the overlapping field of view. This improves practicality and applicability in various operating environments.

[0037] In example 300, the laser radar distance counters 304-1 and 304-2 transmit the distance counter data from laser radar 302-1 and laser radar 302-2 to the measurement noise reduction module 308. The measurement noise reduction module 308 reduces the noise component in the distance counter data (angular velocity, acceleration and angular acceleration of laser radar 302-1 and laser radar 302-2) by means of suitable noise reduction methods (for example, filtering or smoothing methods) in order to facilitate the subsequent calculations.

[0038] In Example 300, the first parameter calibration module 310 calculates the first calibration parameters using an optimization function to determine the first calibration parameters (represented in formula (1), to create the first calibration parameter based on the angular velocity in the distance counter data of several laser radars as well as the angular velocity and angular acceleration of the inertial measurement unit), the distance counter data provided by the measurement noise reduction module 308 as well as the data from the IMU (the already noise-reduced data, including the angular velocity, acceleration and angular acceleration of the IMU 306). These first calibration parameters include two rotation matrices for converting laser radar 302-1 and laser radar 302-2 into the coordinate system of the IMU 306.the gyroscope bias of IMU 306 and the time offset of the IMU relative to laser radar 302-1 and laser radar 302-2 (where laser radar 302-1 and laser radar 302-2 are time-synchronized and there is no time offset). This optimization function can be expressed as follows: argminRL0I,RL1I,bw,δt∑‖RL0IwL0+RL1IwL1+2bw−2wI−2δtΩI‖2

[0039] Here, "argmin" is an abbreviation for "argument minimal" (parameter for the minimum) and refers to the search for the parameters for which the function in expression (1) (the part to the right of the summation symbol (including the summation symbol)) obtains the minimum value, where RL0I and RL1I The rotation matrices of Laserradar 302-1 and Laserradar 302-2, which were converted into the coordinate system of the IMU 306, are represented. L0 and w L1 represent the angular velocities of laser radar 302-1 and laser radar 302-2, b ww1 represents the gyroscope bias of the IMU 306, w1 represents the angular velocity of the IMU 306, δ t represents the time offset of the IMU 306 relative to the laser radar 302-1 and 302-2 and Ω I represents the angular acceleration of the IMU 306.

[0040] To the parameters RL0I, RL1I, b w and δ t To determine which minimize the function in expression (1), the initial values ​​of RL0I and RL1I assigned to an identity matrix with 3 rows and 3 columns each, the initial value of b w a zero vector with 3 rows and 1 column, and the initial value of δ t is assigned 0, and using a solver used for extrinsic parameter optimization, the function in expression (1) is iteratively optimized to find the solution (RL0I,RL1I,bw,δt) to find a function that minimizes the function value of expression (1). The solver used to optimize extrinsic parameters can include, for example, the Ceres solver, the G2O solver (generic graph optimizer), the NLopt solver (nonlinear optimizer), the IPOPT solver (interior point optimizer), or an optimization module from the Eigen library, whereby a suitable solver can be selected depending on the application scenario.

[0041] In example 300, the first parameter calibration module 310 transfers the first calibration parameters obtained by solving the function in expression (1). (RL0I,RL1I,bw,δt), the distance counter data provided by the measurement noise reduction module 308 (i.e., the respective angular velocity, acceleration, and angular acceleration of laser radar 302-1 and laser radar 302-2) and the data from the IMU 306 (including the angular velocity, acceleration, and angular acceleration of the IMU 306) to the time-space optimization module 314 in the second phase module 318.

[0042] In some embodiments, the frame rate (amount of data received per second) of the data scanned by Laserradar 302-1 and Laserradar 302-2 is typically lower than the frame rate of the measurement data from the IMU 306. This results in the odometer data provided by the noise reduction module 308 per second not corresponding to the amount of data acquired by the IMU 306 per second. Therefore, the odometer data from the noise reduction module 308 must be processed accordingly so that the amount of data from the odometer data per second corresponds to the amount of data acquired by the IMU 306 per second. This enables temporal and spatial optimization of the data from multiple (identical or different) sensors and ensures data consistency.This processing can include interpolation optimization of the odometer data originating from the measurement noise reduction module 308, whereby interpolation methods such as linear interpolation (easy to calculate, but comparatively low accuracy), polynomial interpolation (higher accuracy), spline interpolation (allows good data fitting while maintaining a certain smoothness) or segment-wise interpolation (good local accuracy, avoids the accumulation of global errors) can be used, whereby a suitable interpolation method can be selected according to the characteristics of the data associated with laser radar 302-1 and laser radar 302-2.

[0043] In the second phase module 318, the second parameter calibration module 316 uses the second calibration parameters, an optimization function to determine the second calibration parameters (i.e., expression (2), where the function is obtained based on the angular velocity, angular acceleration, and acceleration of several laser radars, as well as several rotation matrices converted from several laser radars into the coordinate system of the inertial measurement unit), and the data provided by the time-space optimization module 314. The data provided by the time-space optimization module 314 include the first calibration parameters. (RL0I,RL1I,bw,δt), The respective angular velocity, acceleration, and angular acceleration of laser radar 302-1 and laser radar 302-2, as well as the acceleration of the IMU 306, are specified. The second calibration parameters comprise two translation matrices for converting laser radar 302-1 and laser radar 302-2 into the coordinate system of the IMU 306, the accelerometer offset of the IMU 306, and the gravity vector. This optimization function can be expressed as follows: argminpL0I,pL1I,ba,g∑‖RIL0(aI−ba)+RIL1(aI−ba)−aL0−aL 1−⌊wL0⌋∧2⋅PIL0−⌊wL1⌋∧2⋅PIL1−⌊ΩL0⌋∧⋅PIL0−⌊ΩL1⌋∧⋅PIL1‖2

[0044] This refers to PL0I and PL1I the translation matrices for converting Laserradar 302-1 and Laserradar 302-2 into the coordinate system of the IMU 306, b a denotes the accelerometer offset of the IMU 306, g the gravity vector, a I the acceleration of the IMU 306, a L0 and a L1the acceleration of Laserradar 302-1 and Laserradar 302-2, RIL0 and RIL1 the rotation matrices for converting the IMU 306 into the coordinate system of Laserradar 302-1 and Laserradar 302-2 (which consist of RL0I and RL1I (derive) PIL0 and PIL1 the translation matrices for converting the IMU 306 into the coordinate system of Laserradar 302-1 and Laserradar 302-2, and Ω L0 and Ω L0 the angular accelerations of Laserradar 302-1 and Laserradar 302-2.

[0045] To the parameter PL0I, PL1I, b a and to determine g, which cause the function in expression (2) (part to the right of the summation symbol (including the summation symbol)) to obtain the minimum value, the initial values ​​of PL0I and PL1I Each assigned to the 3-row, 3-column identity matrix, the initial value of b a The 3-row, 1-column zero vector is assigned, the initial value of g is assigned 0, and using a solver used for extrinsic parameter optimization, the function in expression (2) is iteratively optimized to find the solution. (PL0I,PL1I,ba,g) to find a solver that minimizes the function value in expression (2). The solver used to optimize extrinsic parameters can include, for example, the Ceres solver, the G2O solver, the NLopt solver, the IPOPT solver, or an optimization module from the Eigen library, whereby a suitable solver can be selected depending on the application scenario.

[0046] In some embodiments, multiple laser radars and multiple IMUs are present. If extrinsic parameter calibration has already been performed between the multiple IMUs, only the calibration of extrinsic parameters between the multiple laser radars and a single IMU needs to be carried out, and a calibration of extrinsic parameters between the multiple laser radars is unnecessary. This avoids the accumulation of calibration errors and allows for more precise calibration. A method for calibrating extrinsic parameters between multiple laser radars and a single IMU can be implemented by appropriately extending the method described above. For example, if three laser radars are present, a term is added to the square of expression (1). RL2IwL2, a term for the absolute value squared in expression (2) RIL2aI−aL2−⌊wL2⌋∧2⋅PIL2−⌊ΩL2⌋∧⋅ PIL2 added and so on. In this way, the method according to the embodiments of the present disclosure can be extended to scenarios with a larger number of sensors and therefore exhibits good applicability and general usability.

[0047] Fig. Figure 4 shows a schematic representation of a device 400 for determining the calibration parameters of sensors according to some embodiments of the present disclosure. As in Fig. As shown in Figure 4, the device 400 comprises an odometer data determination unit 402 and a calibration parameter determination unit 404, wherein the odometer data determination unit 402 is configured to determine the odometer data of the multiple first sensors based on the data of multiple first sensors, and the calibration parameter determination unit 404 is configured to determine the calibration parameters of the multiple first sensors and the second sensor based on the odometer data of the multiple first sensors and the data of a second sensor, wherein the first sensor and the second sensor are different sensor types.

[0048] In some embodiments, the odometer data determination unit 402 comprises multiple odometers of first sensors configured to acquire data from multiple first sensors and to calculate the odometer data of the multiple first sensors (for example, position, displacement, velocity, attitude and orientation).

[0049] In some embodiments, the calibration parameter determination unit 404 comprises a first parameter calibration unit configured to determine the first calibration parameters based on the odometer data of the multiple first sensors and the data of a second sensor. These first calibration parameters include multiple rotation matrices for converting the multiple first sensors into a second sensor coordinate system, a first calibration parameter associated with the second sensor, and the time offset of the second sensor relative to the multiple first sensors.

[0050] In some embodiments, the calibration parameter determination unit 404 further comprises a second parameter calibration unit configured to determine the second calibration parameters based on the first calibration parameters, the odometer data of the multiple first sensors, and the data of the second sensor. These second calibration parameters include multiple translation matrices for converting the multiple first sensors into the coordinate system of the second sensor, second calibration parameters with respect to the second sensor, and the gravity vector.

[0051] In some embodiments, the calibration parameter determination unit 404 may further comprise a measurement noise reduction unit and a time-space optimization unit. The measurement noise reduction unit is configured to apply noise reduction to the data before it arrives at the first parameter calibration unit. The time-space optimization unit is configured to perform consistency processing of the data before it arrives at the second parameter calibration unit. This consistency process includes interpolating the data so that the data from multiple sensors are aligned to facilitate computation.

[0052] In some embodiments, the first sensor is a laser radar, the second sensor is an IMU, the first sensor's odometer is a laser radar odometer, and the first calibration parameters include several rotation matrices for converting multiple laser radars into the IMU's coordinate system, the IMU's gyroscope bias, and the IMU's time offset relative to the multiple laser radars. The second calibration parameters include several translation matrices for converting multiple laser radars into the IMU's coordinate system, the IMU's accelerometer offset, and the gravity vector.

[0053] In some embodiments, the first and second calibration parameters determined by means of the embodiments of the present disclosure can be used for the three-dimensional reconstruction of images acquired by sensors (such as cameras, radar, or laser radars). For example, using the first and second calibration parameters, the data obtained from a laser radar (e.g., point cloud data) can be precisely fused with the data obtained from the IMU (e.g., acceleration, angular velocity) to obtain more accurate pose information and optimize point cloud registration, so that a three-dimensional model can be generated based on the pose data and the point cloud data.

[0054] Fig. Figure 5 shows a block diagram of a control unit 500, which can implement several embodiments of the present disclosure. The control unit 500 can, for example, be a device such as the one described in Fig. The environment 100 depicted is shown. As shown in the figure, the control unit 500 comprises a processor 501, which can execute various suitable actions and processes according to computer program instructions stored in a read-only memory (ROM) 502 and loaded into a random-access memory (RAM) 503. The RAM 503 can also store various programs and data required for the operation of the control unit 500. The processor 501, the ROM 502, and the RAM 503 are interconnected via a bus 504. An input / output (I / O) interface 505 is also connected to the bus 504.

[0055] The Processor 501 can be a variety of general-purpose and / or application-specific processing components with processing and computing capabilities. Examples of Processors 501 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various application-specific computing chips for artificial intelligence (AI), various computing units that execute algorithms for machine learning models, digital signal processors (DSPs), and any suitable processors, control units, and microcontrollers. The Processor 501 executes various of the described methods and processes, such as Method 200. In some embodiments, Method 200 can be implemented, for example, as a computer software program that is physically contained on a machine-readable medium.In some embodiments, the computer program can be loaded and / or installed wholly or partially onto the control unit 500 via the ROM 502. When the computer program is loaded into the RAM 503 and executed by the processor 501, one or more steps of the method 200 described above can be performed. Alternatively, in other embodiments, the processor 501 can be configured to execute the method 200 in any other suitable way (for example, by means of firmware).

[0056] The functions described above can be performed, at least partially, by one or more hardware logic components. Examples of hardware logic components that can be used include, but are not limited to: Field Programmable Gate Arrays (FPGAs), Application Specific Integrated Circuits (ASICs), Application Specific Standard Products (ASSPs), System-on-Chip (SOC), Programmable Logic Devices (CPLDs), etc.

[0057] The program code for implementing the method of this disclosure can be written in one or any combination of several programming languages. This program code can be provided to a processor or control unit of a general-purpose computer, an application-specific computer, or another programmable data processing device, such that when the program code is executed by the processor or control unit, the functions / operations specified in the flowcharts and / or block diagrams are performed. The program code can be executed entirely on one machine, partially on one machine, as a standalone software package, partially on one machine and partially on a remote machine, or entirely on a remote machine or server.

[0058] In the context of this disclosure, the machine-readable medium can be a physical medium containing or storing programs for use by, or in conjunction with, a system, device, or apparatus to execute instructions. The machine-readable medium can be a machine-readable signaling medium or a machine-readable storage medium. Machine-readable media can include, among others, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or apparatuses, or any suitable combination thereof.More specific examples of machine-readable storage media include electrical connections based on one or more wires, a portable computer drive, a hard disk drive, random-access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, a portable compact disc read-only storage device (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. Although the operations are shown in a particular order, this does not mean that these operations must be performed in the order shown or sequentially, or that all of the operations shown must be performed to achieve the desired results. In certain environments, multitasking and parallel processing may be advantageous.Although the above discussion includes several specific implementation details, these should not be interpreted as limiting the scope of this disclosure. Certain features described in the context of separate embodiments can also be implemented in combination within a single implementation. Conversely, various functions described in the context of a single implementation can also be implemented separately in multiple implementations or in any suitable subcombination.

[0059] Although the subject matter has been described using language specifically intended for the structural features and / or logical actions of the method, it should be understood that the subject matter defined by the attached claims is not limited to the specific features or actions described above. On the contrary, the specific features and actions described above merely represent exemplary embodiments of the claims.

Claims

[1] Method (200) for determining the calibration parameters of sensors, comprising: Determining (202) the odometer data of the multiple first sensors based on the data of the multiple first sensors; and Determine (204) the calibration parameters of the multiple first sensors and the second sensor based on the odometer data of the multiple first sensors and the data of the second sensor, wherein the first sensors and the second sensor are different sensor types. [2] Method (200) according to claim 1, wherein the multiple first sensors are multiple laser radars, the second sensor is an inertial measurement unit, and determining (204) the calibration parameters of the multiple laser radars and the inertial measurement unit comprises: Determining a first calibration parameter for the multiple laser radars and the inertial measurement unit; and Determining a second calibration parameter for the multiple laser radars and the inertial measurement unit; and Determine the calibration parameters based on the first calibration parameter and the second calibration parameter. [3] Method (200) according to claim 2, wherein determining the first calibration parameter of the multiple laser radars and the inertial measurement unit comprises the following: Determining a first calibration parameter based on the angular velocity in the distance counter data of the several laser radars as well as on the angular velocity and angular acceleration of the inertial measurement unit. [4] Method (200) according to claim 3, wherein determining the first calibration parameter comprises: Setting the initial value of the first calibration parameter; Substituting this initial value of the first calibration parameter into an optimization function, wherein the optimization function is based on the angular velocity in the distance odometer data of the multiple laser radars, as well as on the angular velocity and angular acceleration of the inertial measurement unit in conjunction with the first calibration parameter; and Determine the first calibration parameter iteratively, such that the optimization function takes on a minimum. [5] Method (200) according to claim 3, wherein determining the first calibration parameter comprises: Determining several rotation matrices to convert the multiple laser radars into the coordinate system of the inertial measurement unit; Determining the gyroscope bias of the inertial measurement unit; and Determining the time offset of the inertial measurement unit relative to the multiple laser radars. [6] Method (200) according to claim 2, wherein determining the second calibration parameter of the multiple laser radars and the inertial measurement unit comprises the following: Processing the odometer data from the multiple laser radars so that the amount of this odometer data per second corresponds to the amount of data transmitted by the inertial measurement unit per second; and Determining the second calibration parameter based on the processed distance counter data of the multiple laser radars, the acceleration of the inertial measurement unit, and a sub-parameter from the first calibration parameter. [7] Method (200) according to claim 6, wherein: the processed odometer data from the multiple laser radars include the angular velocity, angular acceleration, and acceleration of the multiple laser radars; and The part of the parameters from the first calibration parameters includes several rotation matrices for converting the multiple laser radars into the coordinate system of the inertial measurement unit. [8] Method (200) according to claim 6 or 7, wherein determining the second calibration parameter comprises: Setting an initial value for the second calibration parameter; Substituting this initial value of the second calibration parameter into an optimization function, where the optimization function is based on the angular velocity, based on the angular acceleration and the acceleration of the multiple laser radars, as well as on the multiple rotation matrices for converting the multiple laser radars into the coordinate system of the inertial measurement unit; and Determine the second calibration parameter iteratively so that the optimization function reaches a minimum. [9] Method (200) according to claim 7, wherein determining the second calibration parameter comprises: Determining several translation matrices to convert the multiple laser radars into the coordinate system of the inertial measurement unit; and Determining the accelerometer offset of the inertial measurement unit and the gravity vector. [10] Method (200) according to claim 1, further comprising: Performing a three-dimensional reconstruction of an image based on the calibration parameters of the multiple first sensors and the second sensor. [11] Device (400) for determining calibration parameters, comprising: a odometer data determination unit (402) that is configured, to determine the odometer data of the several first sensors based on the data from several first sensors; a calibration parameter determination unit (404) configured to determine the calibration parameters of the multiple first sensors and the second sensor based on the odometer data of the multiple first sensors and the data of a second sensor, wherein the first sensors and the second sensor are different sensor types. [12] Control unit (500), comprising: at least one processor (501); and a memory (502) coupled to the at least one processor and on which instructions are stored, wherein the instructions, when executed by the at least one processor (501), cause the control unit (500) to execute the method according to any one of claims 1 to 10. [13] Computer program product comprising a computer program wherein the computer program is executed by a processor to implement a method according to any one of claims 1 to 10.

Images