Multi-Sensor Extrinsic Calibration Method, Device, Electronic Equipment, and Storage Medium

Abstract

The present application discloses a multi-sensor extrinsic parameter calibration method, device, electronic device, and storage medium. The method includes calculating the coordinate positions of a plurality of corner points in a calibration board measured by a measuring instrument in the coordinate system of the measuring instrument according to the positional relationship of the plurality of corner points; determining a first pose relationship between a sensor to be calibrated on an autonomous driving vehicle and the calibration board; determining a second pose relationship of the calibration board relative to the vehicle body coordinate system according to the coordinate positions of the plurality of corner points in the coordinate system of the measuring instrument and the corresponding coordinate positions of the plurality of corner points; and determining a third pose relationship of the sensor to be calibrated in the vehicle body coordinate system according to the first pose relationship and the second pose relationship, thereby completing the extrinsic parameter calibration of the sensor to be calibrated. Through the present application, the calculation of the extrinsic parameters of the sensor in the vehicle body coordinate system is realized, so as to obtain more accurate sensor information under the vehicle body.

Description

Technical Field

[0001] This application relates to the field of autonomous driving technology, and particularly to a multi-sensor extrinsic parameter calibration method, device, electronic device, and storage medium. Background Art

[0002] With the development of the autonomous driving field, the requirement for environmental perception ability is getting higher and higher, and the calculation accuracy of the extrinsic parameters between the sensor and the vehicle body is a key indicator for improving the perception ability. Currently, common calibration methods include the calibration method of the physical installation position and the calculation method using the IMU as the vehicle body coordinate system. The goal is to reduce the error between the sensor and the vehicle body to achieve better results.

[0003] The key ability of autonomous driving perception is based on the vehicle body coordinate system. Although the pose relationship between the first sensor and the vehicle body can be determined through the installation structure, as the product ages, establishing the calibration relationship between the vehicle body and the sensor becomes a problem that must be convergent. Summary of the Invention

[0004] Embodiments of this application provide a multi-sensor extrinsic parameter calibration method, device, electronic device, and storage medium to improve the accuracy of the extrinsic parameter calibration results of multiple sensors on a vehicle.

[0005] Embodiments of this application adopt the following technical solutions:

[0006] In a first aspect, embodiments of this application provide a multi-sensor extrinsic parameter calibration method for an autonomous driving vehicle, where the method includes: calculating the coordinate positions of multiple corner points in the calibration board in the coordinate system of the measuring instrument according to the position relationship of the multiple corner points in the calibration board measured by the measuring instrument; determining the first pose relationship between the sensor to be calibrated on the autonomous driving vehicle and the calibration board; determining the second pose relationship of the calibration board relative to the vehicle body coordinate system according to the coordinate positions of the multiple corner points in the coordinate system of the measuring instrument and the corresponding coordinate positions of the multiple corner points; and determining the third pose relationship of the sensor to be calibrated in the vehicle body coordinate system according to the first pose relationship and the second pose relationship, thereby completing the extrinsic parameter calibration of the sensor to be calibrated.

[0007] In some embodiments, the pose relationship further includes: rotation parameters and translation parameters.

[0008] In some embodiments, the step of determining the third pose relationship of the sensor to be calibrated in the vehicle body coordinate system according to the first pose relationship and the second pose relationship, thereby completing the extrinsic parameter calibration of the sensor to be calibrated, includes:

[0009] Using the product of the rotation parameters in the first pose relationship and the second pose relationship as the rotation parameter for determining the third pose relationship of the sensor to be calibrated in the vehicle body coordinate system;

[0010] Determining the translation parameter of the third pose relationship of the sensor to be calibrated in the vehicle body coordinate system based on the sum of the products of the rotation parameters and translation parameters in the first pose relationship and the second pose relationship and the translation parameter in the second pose relationship.

[0011] In some embodiments, determining the first pose relationship between the sensor to be calibrated on the autonomous vehicle and the calibration board includes:

[0012] By using the sensor to be calibrated on the autonomous vehicle to collect the calibration board data, calculating the first pose relationship between the sensor to be calibrated and the calibration board, and the relationship between the sensor to be calibrated and the calibration board satisfies:

[0013] Ci = [R1, T1]Pi

[0014] where Pi is the coordinate of the point on the calibration board collected by the sensor to be calibrated, Ci is the coordinate of the midpoint of the calibration board, and [R1, T1] is the first pose relationship;

[0015] Determining the second pose relationship of the calibration board relative to the vehicle body coordinate system based on the coordinate positions of the multiple corner points in the coordinate system of the measuring instrument and the corresponding coordinate positions of the multiple corner points includes:

[0016] Using 3D-3D pose calculation to obtain the second pose relationship of the calibration board relative to the vehicle body coordinate system based on the coordinate positions of the multiple corner points in the coordinate system of the measuring instrument and the coordinate positions of the multiple corner points in the calibration board coordinate system.

[0017] In some embodiments, the vehicle body information of the autonomous vehicle includes the front wheelbase of the vehicle body, the rear wheelbase of the vehicle body, and the positional relationship of the vehicle body central axis. According to the vehicle body information, ground front axle, ground rear axle, and ground central axis marking lines are arranged in a preset calibration site. Before determining the first pose relationship between the sensor to be calibrated on the autonomous vehicle and the calibration board, it further includes:

[0018] Aligning the corresponding positions of the vehicle body of the autonomous vehicle with the ground preset marking lines in projection, where the vehicle body front axis is aligned with the ground front axis, and the vehicle body central axis is aligned with the ground central axis.

[0019] In some embodiments, calculating the coordinate positions of the multiple corner points in the coordinate system of the measuring instrument based on the positional relationships of the multiple corner points in the calibration board measured by the measuring instrument includes:

[0020] Place the measuring instrument at the intersection position of the front axle of the vehicle and the central axis of the ground, collect the positional relationships of at least three corner points, and calculate the coordinate positions of the multiple corner points in the coordinate system of the measuring instrument.

[0021] In some embodiments, the sensor to be calibrated includes at least one of the following: lidar, millimeter-wave radar, camera, IMU.

[0022] In a second aspect, an embodiment of the present application further provides a multi-sensor extrinsic parameter calibration device. The device includes: a corner point calculation module, configured to calculate the coordinate positions of the multiple corner points in the coordinate system of the measuring instrument based on the positional relationships of the multiple corner points in the calibration board measured by the measuring instrument; a first pose calculation module, configured to determine the first pose relationship between the sensor to be calibrated on the autonomous vehicle and the calibration board; a second pose calculation module, configured to determine the second pose relationship of the calibration board relative to the vehicle body coordinate system according to the coordinate positions of the multiple corner points in the coordinate system of the measuring instrument and the corresponding coordinate positions of the multiple corner points; and a third pose calculation module, configured to determine the third pose relationship of the sensor to be calibrated in the vehicle body coordinate system according to the first pose relationship and the second pose relationship, and complete the extrinsic parameter calibration of the sensor to be calibrated.

[0023] In a third aspect, an embodiment of the present application further provides an electronic device, including: a processor; and a memory arranged to store computer-executable instructions, where the executable instructions, when executed, cause the processor to execute the above method.

[0024] In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium, where the computer-readable storage medium stores one or more programs, and when the one or more programs are executed by an electronic device including a plurality of application programs, the electronic device is caused to execute the above method.

[0025] The above at least one technical solution adopted in the embodiments of the present application can achieve the following beneficial effects:

[0026] Based on the first pose relationship between the sensor to be calibrated on the autonomous vehicle and the calibration board, as well as the coordinate positions of the multiple corner points in the coordinate system of the measuring instrument and the corresponding coordinate positions of the multiple corner points, determine the second pose relationship of the calibration board relative to the vehicle body coordinate system. Based on the first pose relationship and the second pose relationship, determine the pose relationship of the sensor to be calibrated on the autonomous vehicle in the vehicle body coordinate system. In the external parameter calibration solution, the calculation of the external parameters of the sensor in the vehicle body coordinate system is realized, and more accurate sensor information under the vehicle body is obtained. Description of the Drawings

[0027] The drawings described herein are used to provide a further understanding of the present application, and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation of the present application. In the drawings:

[0028] Figure 1 Schematic diagram of the vehicle position in the multi-sensor external parameter calibration method according to the embodiment of the present application;

[0029] Figure 2 Schematic diagram of the calibration board (initial position) in the multi-sensor external parameter calibration method according to the embodiment of the present application;

[0030] Figure 3 Schematic diagram of the total station instrument (initial position) in the multi-sensor external parameter calibration method according to the embodiment of the present application;

[0031] Figure 4 Schematic flow chart of the multi-sensor external parameter calibration method according to the embodiment of the present application;

[0032] Figure 5 Schematic diagram of the implementation principle of the multi-sensor external parameter calibration method according to the embodiment of the present application;

[0033] Figure 6 Schematic diagram of the structure of the multi-sensor external parameter calibration device according to the embodiment of the present application;

[0034] Figure 7 Schematic diagram of the structure of an electronic device according to the embodiment of the present application. Detailed Embodiments

[0035] To make the objectives, technical solutions and advantages of the present application clearer, the technical solutions of the present application will be clearly and completely described below in conjunction with the specific embodiments of the present application and the corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present application.

[0036] The multi-sensor extrinsic parameter calibration method in the embodiments of this application solves the calculation of the extrinsic parameters of multiple sensors in the vehicle body coordinate system, so that more accurate sensor extrinsic parameter data can be obtained. By applying the four-point method to calculate the position of the calibration board to the calibration process, the accuracy and versatility are improved.

[0037] The following will, with reference to the accompanying drawings, elaborate on the technical solutions provided by the embodiments of this application in detail.

[0038] As Figure 1 shown, according to the vehicle body information, the marking lines of the front axle, rear axle, and central axis are arranged in the calibration site. Among them, point O (in the world coordinate system XYZ) is the intersection point of the front axis and the central axis. Among them, according to the vehicle product specifications, the vehicle body information can be obtained, including the front wheelbase, rear wheelbase, and the positional relationship of the central axis.

[0039] As Figure 2 Place the calibration board within the FOV (field of view) of the vehicle body sensor, and then select four corner points P0, P1, P2, P3 of the calibration board. Since the length and width dimensions of the calibration board are known, the coordinates C0 to C3 of P0, P1, P2, P3 in the calibration board coordinate system can be obtained.

[0040] Among them, the coordinate of C0 is (0, 0, 0), C1 is (w, 0, 0), C2 is (w, h, 0), and C3 is (0, h, 0), where w is the distance between P0 and P1, and h is the distance between P0 and P3.

[0041] As Figure 3 shown, place the plumb point of the measuring instrument (total station) at point O, adjust the horizontal position of the total station, rotate the horizontal scale line to be flush with the front axis, and record the height d of the optical center of the total station from the ground.

[0042] The embodiments of this application provide a multi-sensor extrinsic parameter calibration method. As Figure 4 shown, the schematic flow chart of the multi-sensor extrinsic parameter calibration method in the embodiments of this application is provided. The method at least includes the following steps S410 to step S440:

[0043] Step S410, according to the positional relationship of multiple corner points on the calibration board measured by the measuring instrument, calculate the coordinate positions of the multiple corner points in the coordinate system of the measuring instrument.

[0044] The measuring instrument includes, but is not limited to, a total station. Other measuring instruments that can achieve the same function can also be used.

[0045] According to the positional relationship of multiple corner points on the calibration board measured by the measuring instrument, the corner points usually refer to the corner points at the four corners of the calibration board. Through the method in the related technology, the coordinate positions of the multiple corner points in the coordinate system of the measuring instrument can be calculated.

[0046] For example, taking a total station as the measuring instrument, the positional relationships between P0 to P3 of the calibration plate and the total station are measured to obtain the vertical distances of each point: V0 to V3, horizontal distances: H0 to H3, and right angles (horizontal angles): HAR0 to HAR3. Then, based on V, H, and HAR, the coordinates Q0 to Q3 of P0 to P3 under the total station are calculated. Among them, the coordinate of Q0 is (H0 / tanHAR0, H0, V0 + d), and by analogy, the corner coordinate positions of Q1, Q2, and Q3 are obtained.

[0047] Step S420: Determine the first pose relationship between the sensor to be calibrated on the autonomous vehicle and the calibration plate.

[0048] There are multiple sensors on the autonomous vehicle, and usually, multiple sensors need to be calibrated before they can be used. The sensor to be calibrated includes at least one of the following: lidar, millimeter-wave radar, camera, IMU.

[0049] By collecting the feature points (corner points) on the calibration plate through the sensor to be calibrated, the first pose relationship between the sensor to be calibrated on the autonomous vehicle and the calibration plate is determined. For example, using the sensor to be calibrated on the autonomous vehicle to collect calibration plate data, the pose relationship R1, T1 between the sensor and the calibration plate is calculated. Among them, the sensor and the calibration plate satisfy: Ci = [R1, T1]pi, where pi is the coordinate of the point on the calibration plate obtained from the sensor, and Ci is the coordinate of the point in the calibration plate.

[0050] Step S430: Determine the second pose relationship of the calibration plate relative to the vehicle body coordinate system according to the coordinate positions of the multiple corner points in the coordinate system of the measuring instrument and the corresponding coordinate positions of the multiple corner points.

[0051] Based on Q0 to Q3 of the four points P0 to P3 obtained by calculation in step S410 in the vehicle body coordinate system and C0 to C3 corresponding to the four corner points in the calibration plate coordinate system, the pose R2, T2 of the calibration plate coordinate relative to the vehicle body coordinate system is calculated using 3D-3D pose. Among them, the point Qi in the vehicle body coordinate system and the point Ci in the calibration plate coordinate system satisfy Qi = R2 * Ci + T2.

[0052] The pose relationship of the calibration plate relative to the vehicle body coordinate system is determined through the coordinate positions of the corner points in the coordinate system of the measuring instrument and the corresponding coordinate positions of the multiple corner points in the world coordinate system.

[0053] Step S440: Determine the third pose relationship of the sensor to be calibrated in the vehicle body coordinate system according to the first pose relationship and the second pose relationship, and complete the external parameter calibration of the sensor to be calibrated.

[0054] Based on the pose \(R_1\), \(T_1\) of the sensor in the calibration board coordinate system and the pose \(R_2\), \(T_2\) of the calibration board in the vehicle body coordinate system, calculate the pose \(R_3\), \(T_3\) of the sensor in the vehicle body coordinate system,

[0055] where \(R_3 = R_2\times R_1\), \(T_3 = R_2\times T_1+T_2\),

[0056] After that, complete the calculation of the external parameters of the sensor in the vehicle body coordinate system.

[0057] The third pose relationship is used to characterize the offset of the sensor to be calibrated in the vehicle body coordinate system.

[0058] In an embodiment of the present application, the pose relationship further includes: rotation parameters, translation parameters.

[0059] The process of calculating the external parameters is to solve the relative pose relationship between the sensor to be calibrated and the vehicle body. If the rotation parameters and translation parameters in the pose relationship can be calculated, the sensor on the vehicle can be calibrated according to the pose relationship. The rotation parameters need to adjust the relative angle by rotation correspondingly, and the translation parameters need to adjust the relative distance by translation correspondingly.

[0060] In an embodiment of the present application, the determining the third pose relationship of the sensor to be calibrated in the vehicle body coordinate system according to the first pose relationship and the second pose relationship, and completing the external parameter calibration of the sensor to be calibrated includes: using the product of the rotation parameters in the first pose relationship and the second pose relationship as the rotation parameter for determining the third pose relationship of the sensor to be calibrated in the vehicle body coordinate system; determining the translation parameter of the third pose relationship of the sensor to be calibrated in the vehicle body coordinate system according to the sum of the product of the rotation parameter and the translation parameter in the first pose relationship and the second pose relationship and the translation parameter in the second pose relationship.

[0061] Specifically, when implementing, use the product \(R_1\times R_2\) of the rotation parameters in the first pose relationship \((R_1, T_1)\) and the second pose relationship \((R_2, T_2)\) as the rotation parameter \(R_1\times R_2 = R_3\) for determining the third pose relationship of the sensor to be calibrated in the vehicle body coordinate system. \(R_1\), \(R_2\) represent rotation amounts, and \(T_1\), \(T_2\) represent translation amounts.

[0062] Furthermore, determine the translation parameter \(R_2\times T_1+T_2 = T_3\) of the third pose relationship of the sensor to be calibrated in the vehicle body coordinate system according to the sum of the product of the rotation parameter \(R_1\) and the translation parameter \(T_1\) in the first pose relationship \((R_1, T_1)\) and the second pose relationship \((R_2, T_2)\) and the translation parameter \(T_2\) in the second pose relationship.

[0063] In one embodiment of the present application, determining the first pose relationship between the sensor to be calibrated on the autonomous vehicle and the calibration board includes: collecting the calibration board data by using the sensor to be calibrated on the autonomous vehicle, and calculating the first pose relationship between the sensor to be calibrated and the calibration board. The relationship between the sensor to be calibrated and the calibration board satisfies:

[0064] Ci = [R1, T1]Pi

[0065] where Pi is the coordinate of the point on the calibration board collected according to the sensor to be calibrated, Ci is the coordinate of the midpoint of the calibration board, and [R1, T1] is the first pose relationship; determining the second pose relationship of the calibration board relative to the vehicle body coordinate system according to the coordinate positions of the multiple corner points in the coordinate system of the measuring instrument and the corresponding coordinate positions of the multiple corner points includes: using 3D-3D pose calculation to obtain the second pose relationship of the calibration board relative to the vehicle body coordinate system according to the coordinate positions of the multiple corner points in the coordinate system of the measuring instrument and the coordinate positions of the multiple corner points in the calibration board coordinate system.

[0066] Pi is the coordinate of the point on the calibration board collected according to the sensor to be calibrated, that is, any one of the four corner points P0, P1, P2, and P3 of the calibration board is selected. Since the size (length and width) of the calibration board is known, the coordinates C0 to C3 of P0, P1, P2, and P3 in the calibration board coordinate system can be obtained. Among them, the coordinate of C0 is (0, 0, 0), the coordinate of C1 is (w, 0, 0), the coordinate of C2 is (w, h, 0), and the coordinate of C3 is (0, h, 0), where w is the distance between P0 and P1, and h is the distance between P0 and P3.

[0067] In one embodiment of the present application, the vehicle body information of the autonomous vehicle includes the front wheelbase of the vehicle body, the rear wheelbase of the vehicle body, and the positional relationship of the vehicle body central axis. According to the vehicle body information, the marking lines of the ground front axle, the ground rear axle, and the ground central axis are arranged in the preset calibration site. Before determining the first pose relationship between the sensor to be calibrated on the autonomous vehicle and the calibration board, it further includes: aligning the corresponding positions of the vehicle body of the autonomous vehicle with the projection of the ground preset marking lines, where the front axis of the vehicle body is aligned with the front axis of the ground, and the central axis of the vehicle body is aligned with the central axis of the ground.

[0068] Before determining the first pose relationship, align the corresponding positions of the vehicle body and the projection of the ground marking lines, where the front axis of the vehicle body is aligned with the front axis of the ground, and the central axis of the vehicle body is aligned with the central axis of the ground. Before this, it is also necessary to calculate the coordinate positions of the multiple corner points in the coordinate system of the measuring instrument according to the positional relationship of the multiple corner points of the calibration board measured by the measuring instrument.

[0069] In an embodiment of the present application, calculating the coordinate positions of multiple corner points in the calibration plate in the coordinate system of the measuring instrument based on the positional relationships of the multiple corner points in the calibration plate measured by the measuring instrument includes: placing the measuring instrument at the intersection position of the front axle of the ground and the central axis of the ground, collecting the positional relationships of at least three corner points, and calculating the coordinate positions of the multiple corner points in the coordinate system of the measuring instrument.

[0070] During specific implementation, first place the measuring instrument at the intersection position of the front axle of the ground and the central axis of the ground, as Figure 1 shown in FIGS. 1 and 2. Then, when calculating the coordinate positions of the multiple corner points in the coordinate system of the measuring instrument, it is necessary to collect the positional relationships of at least three corner points. Of course, the calculation results are more accurate with four corner points.

[0071] In an embodiment of the present application, the calibration plate uses a checkerboard or the like as the calibration plate. At the same time, for the calibration field, it is necessary to arrange the marking lines of the front axle, rear axle, and central axis before arranging the calibration field. Among them, point O (in the world coordinate system XYZ) is the intersection of the front axis line and the central axis line. Among them, according to the vehicle product specifications, the vehicle body information can be obtained, including the front wheelbase, rear wheelbase, and the positional relationship of the central axis. Then place the calibration plate within the FOV (field of view) of the vehicle body sensor. In addition, it is also necessary to place the plumb point of the measuring instrument (total station) at point O, adjust the horizontal position of the total station, rotate the horizontal scale line to be flush with the front axis line, and record the height d of the optical center of the total station from the ground.

[0072] As Figure 5 shown, it is a schematic diagram of the implementation principle of the multi-sensor external parameter calibration method in the embodiment of the present application, which specifically includes the following steps:

[0073] S1, obtaining the information of the front axle, rear axle, and wheelbase according to the vehicle body structure.

[0074] According to the vehicle body information, arrange the marking lines of the front axle, rear axle, and central axis on the calibration field. Among them, point O (in the world coordinate system XYZ) is the intersection of the front axis line and the central axis line. Among them, according to the vehicle product specifications, the vehicle body information can be obtained, including the front wheelbase, rear wheelbase, and the positional relationship of the central axis.

[0075] S2, arranging the front, rear, and central axis marking lines on the ground.

[0076] Arrange the front, rear, and central axis marking lines on the ground of the calibration field, as Figure 1 shown, the dashed line is the vehicle stop position, and the solid line is the ground marking line.

[0077] S3, placing the total station at point O and aligning the horizontal line with the front axis line.

[0078] As Figure 3As shown in the figure, place the plumb point of the measuring instrument (total station) at point O, adjust the horizontal position of the total station, rotate the horizontal scale line to be flush with the front axis, and record the height d of the optical center of the total station from the ground.

[0079] S4. Deploy the calibration board according to the characteristics of the sensor.

[0080] S5. Obtain the coordinates of the calibration board P0 - P3 of the vehicle through the total station.

[0081] Place the calibration board within the FOV (field of view) of the vehicle body sensor, and then select four corner points P0, P1, P2, P3 of the calibration board. Since the length and width dimensions of the calibration board are known, the coordinates C0 - C3 of P0, P1, P2, P3 in the calibration board coordinate system can be obtained.

[0082] S6. Drive the vehicle to the corresponding position of the ground marking.

[0083] Align the corresponding position of the vehicle body of the autonomous vehicle with the projection of the ground preset marking line, where the front axis of the vehicle body is aligned with the front axis of the ground, and the central axis of the vehicle body is aligned with the central axis of the ground.

[0084] S7. The vehicle sensor collects calibration board data.

[0085] S8. Calculate the external parameters of the corresponding sensor in the vehicle body coordinate system.

[0086] Determine the first pose relationship between the sensor to be calibrated on the autonomous vehicle and the calibration board;

[0087] According to the coordinate positions of the multiple corner points in the coordinate system of the measuring instrument and the corresponding coordinate positions of the multiple corner points, determine the second pose relationship of the calibration board relative to the vehicle body coordinate system;

[0088] According to the first pose relationship and the second pose relationship, determine the third pose relationship of the sensor to be calibrated in the vehicle body coordinate system, and complete the external parameter calibration of the sensor to be calibrated.

[0089] The pose relationship also includes: rotation parameters, translation parameters.

[0090] The step of determining the third pose relationship of the sensor to be calibrated in the vehicle body coordinate system according to the first pose relationship and the second pose relationship, and completing the external parameter calibration of the sensor to be calibrated includes:

[0091] Use the product of the rotation parameters of the two in the first pose relationship and the second pose relationship as the rotation parameter for determining the third pose relationship of the sensor to be calibrated in the vehicle body coordinate system;

[0092] Determine the translation parameter of the third pose relationship of the sensor to be calibrated in the vehicle body coordinate system according to the sum of the products of the rotation parameter and the translation parameter in the first pose relationship and the second pose relationship and the translation parameter in the second pose relationship.

[0093] The determination of the first pose relationship between the sensor to be calibrated on the autonomous vehicle and the calibration board includes:

[0094] By using the sensor to be calibrated on the autonomous vehicle to collect the calibration board data, calculate the first pose relationship between the sensor to be calibrated and the calibration board. The relationship between the sensor to be calibrated and the calibration board satisfies:

[0095] Ci = [R1, T1]Pi

[0096] Wherein, Pi is the coordinate of the point on the calibration board collected according to the sensor to be calibrated, Ci is the coordinate of the midpoint of the calibration board, and [R1, T1] is the first pose relationship;

[0097] The determination of the second pose relationship of the calibration board relative to the vehicle body coordinate system according to the coordinate positions of the multiple corner points in the coordinate system of the measuring instrument and the corresponding coordinate positions of the multiple corner points includes:

[0098] According to the coordinate positions of the multiple corner points in the coordinate system of the measuring instrument and the coordinate positions of the multiple corner points in the calibration board coordinate system, use 3D-3D pose calculation to obtain the second pose relationship of the calibration board relative to the vehicle body coordinate system.

[0099] The embodiment of the present application also provides a multi-sensor extrinsic parameter calibration device 600, as Figure 6 shown, provides a structural schematic diagram of the multi-sensor extrinsic parameter calibration device in the embodiment of the present application. The multi-sensor extrinsic parameter calibration device 600 at least includes: a corner point calculation module 610, a first pose calculation module 620, a second pose calculation module 630, and a third pose calculation module 640, wherein:

[0100] In an embodiment of the present application, the corner point calculation module 610 is specifically configured to: calculate the coordinate positions of the multiple corner points in the coordinate system of the measuring instrument according to the positional relationship of the multiple corner points in the calibration board measured by the measuring instrument.

[0101] The measuring instrument includes, but is not limited to, a total station. Other measuring instruments that can achieve the same function can also be used.

[0102] According to the positional relationship of multiple corner points in the calibration board measured by the measuring instrument, the corner points usually refer to the corner points at the four corners of the calibration board. Through the methods in the related technologies, the coordinate positions of the multiple corner points in the coordinate system of the measuring instrument can be calculated.

[0103] For example, taking the total station as the measuring instrument, by measuring the positional relationship between P0 to P3 of the calibration board and the total station, the vertical distance of each point: V0 to V3, horizontal distance: H0 to H3, and right angle (horizontal angle): HAR0 to HAR3 are obtained. Then, according to V, H, and HAR, the coordinates Q0 to Q3 of P0 to P3 under the total station are calculated. Among them, the coordinate of Q0 is (H0 / tanHAR0, H0, V0 + d), and by analogy, the corner point coordinate positions of Q1, Q2, and Q3 are obtained.

[0104] In an embodiment of the present application, the first pose calculation module 620 is specifically configured to: determine the first pose relationship between the sensor to be calibrated on the autonomous vehicle and the calibration board.

[0105] There are multiple sensors on the autonomous vehicle, and usually, multiple sensors need to be calibrated before they can be used. The sensor to be calibrated includes at least one of the following: lidar, millimeter-wave radar, camera, IMU.

[0106] By collecting the feature points (corner points) on the calibration board through the sensor to be calibrated, the first pose relationship between the sensor to be calibrated on the autonomous vehicle and the calibration board is determined. For example, using the sensor to be calibrated on the autonomous vehicle to collect calibration board data, and calculating the pose relationship R1, T1 between the sensor and the calibration board. Among them, the sensor and the calibration board satisfy: Ci = [R1, T1]pi, where pi is the coordinate of the point on the calibration board obtained from the sensor, and Ci is the coordinate of the point in the calibration board.

[0107] In an embodiment of the present application, the second pose calculation module 630 is specifically configured to: determine the second pose relationship of the calibration board relative to the vehicle body coordinate system according to the coordinate positions of the multiple corner points in the coordinate system of the measuring instrument and the corresponding coordinate positions of the multiple corner points.

[0108] According to Q0 to Q3 of the four points P0 to P3 in the vehicle body coordinate system calculated in step S410, and C0 to C3 corresponding to the four corner points in the calibration board coordinate system, the pose R2, T2 of the calibration board coordinate relative to the vehicle body coordinate system is calculated using 3D-3D pose. Among them, the point Qi in the vehicle body coordinate system and the point Ci in the calibration board coordinate system satisfy Qi = R2 * Ci + T2, and P0 to P3 are the names of the points.

[0109] Determine the pose relationship of the calibration plate relative to the vehicle body coordinate system based on the coordinate positions of the corner points in the coordinate system of the measuring instrument and the coordinate positions of the corresponding multiple corner points in the world coordinate system.

[0110] In one embodiment of the present application, the third pose calculation module 640 is specifically configured to: determine the third pose relationship of the sensor to be calibrated in the vehicle body coordinate system according to the first pose relationship and the second pose relationship, and complete the external parameter calibration of the sensor to be calibrated.

[0111] According to the pose R1, T1 of the sensor coordinates in the calibration plate coordinate system and the pose R2, T2 of the calibration plate in the vehicle body coordinate system, calculate the pose R3, T3 of the sensor in the vehicle body coordinate system.

[0112] Wherein, R3 = R2 * R1, T3 = R2 * T1 + T2.

[0113] After that, complete the calculation of the external parameters of the sensor in the vehicle body coordinate system.

[0114] The third pose relationship is used to characterize the offset of the sensor to be calibrated in the vehicle body coordinate system.

[0115] It can be understood that the above multi-sensor external parameter calibration device can implement each step of the multi-sensor external parameter calibration method provided in the foregoing embodiments. The relevant explanations of the multi-sensor external parameter calibration method are applicable to the multi-sensor external parameter calibration device and will not be elaborated here.

[0116] Figure 7 It is a schematic structural diagram of an electronic device according to an embodiment of the present application. Please refer to Figure 7 , at the hardware level, the electronic device includes a processor, and optionally also includes an internal bus, a network interface, and a memory. Among them, the memory may include a memory, such as a high-speed random access memory (Random-Access Memory, RAM), and may also include a non-volatile memory, such as at least one disk memory, etc. Of course, the electronic device may also include other hardware required for other services.

[0117] The processor, network interface, and memory can be interconnected through an internal bus, which can be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus, an EISA (Extended Industry Standard Architecture) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For the sake of representation, Figure 7 only a bidirectional arrow is used in

[0118] Memory, used to store programs. Specifically, the program can include program code, and the program code includes computer operation instructions. The memory can include a memory and a non-volatile memory, and provide instructions and data to the processor.

[0119] The processor reads the corresponding computer program from the non-volatile memory into the memory and then runs it, forming a multi-sensor extrinsic calibration device at the logical level. The processor executes the program stored in the memory and is specifically used to perform the following operations:

[0120] According to the positional relationship of multiple corner points in the calibration board measured by the measuring instrument, calculate the coordinate positions of the multiple corner points in the coordinate system of the measuring instrument;

[0121] Determine the first pose relationship between the sensor to be calibrated on the autonomous vehicle and the calibration board;

[0122] According to the coordinate positions of the multiple corner points in the coordinate system of the measuring instrument and the corresponding coordinate positions of the multiple corner points, determine the second pose relationship of the calibration board relative to the vehicle body coordinate system;

[0123] According to the first pose relationship and the second pose relationship, determine the third pose relationship of the sensor to be calibrated in the vehicle body coordinate system, and complete the extrinsic calibration of the sensor to be calibrated.

[0124] The above is as in this application Figure 4The method executed by the multi-sensor extrinsic parameter calibration device disclosed in the illustrated embodiment can be applied to or implemented by a processor. The processor may be an integrated circuit chip with signal processing capabilities. During implementation, each step of the above method can be completed by the integrated logic circuit in the hardware of the processor or by instructions in software form. The above-mentioned processor may be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it may also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. It can implement or execute the various methods, steps, and logic block diagrams disclosed in the embodiments of the present application. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc. The steps of the method disclosed in combination with the embodiments of the present application can be directly embodied as being executed by a hardware decoding processor, or executed by a combination of hardware and software modules in the decoding processor. The software module may be located in a mature storage medium in the art such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable memory, a register, etc. This storage medium is located in the memory, and the processor reads the information in the memory and combines its hardware to complete the steps of the above method.

[0125] The electronic device can also execute Figure 4 the method executed by the multi-sensor extrinsic parameter calibration device in Figure 4 the illustrated embodiment and implement the functions of the multi-sensor extrinsic parameter calibration device in

[0126] Embodiments of the present application also propose a computer-readable storage medium that stores one or more programs. The one or more programs include instructions that, when executed by an electronic device including multiple application programs, can enable the electronic device to execute Figure 4 the method executed by the multi-sensor extrinsic parameter calibration device in the illustrated embodiment, and specifically used to execute:

[0127] According to the positional relationship of multiple corner points in the calibration board measured by the measuring instrument, calculate the coordinate positions of the multiple corner points in the coordinate system of the measuring instrument;

[0128] Determine the first pose relationship between the sensor to be calibrated on the autonomous vehicle and the calibration board;

[0129] Determine a second pose relationship of the calibration board relative to the vehicle body coordinate system according to the coordinate positions of the multiple corner points in the coordinate system of the measuring instrument and the corresponding coordinate positions of the multiple corner points;

[0130] Determine a third pose relationship of the sensor to be calibrated in the vehicle body coordinate system according to the first pose relationship and the second pose relationship, and complete the external parameter calibration of the sensor to be calibrated.

[0131] Those skilled in the art should understand that the embodiments of the present invention can be provided as a method, a system, or a computer program product. Therefore, the present invention can take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present invention can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0132] The present invention is described with reference to the flowcharts and / or block diagrams of methods, apparatuses (systems), and computer program products according to embodiments of the present invention. It should be understood that each flow and / or block in the flowchart and / or block diagram, and the combination of flows and / or blocks in the flowchart and / or block diagram, can be realized by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing devices generate means for realizing the functions specified in one Figure 1 one flow or multiple flows and / or blocks Figure 1 one block or multiple blocks.

[0133] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer-readable memory generate a manufactured article including instruction means, and the instruction means realizes the functions specified in one Figure 1 one flow or multiple flows and / or blocks Figure 1 one block or multiple blocks.

[0134] These computer program instructions can also be loaded onto a computer or other programmable data processing device, so that a series of operation steps are executed on the computer or other programmable device to generate a computer-implemented process, and thus the instructions executed on the computer or other programmable device provide steps for realizing the functions specified in one Figure 1 one flow or multiple flows and / or blocks Figure 1 one block or multiple blocks.

[0135] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0136] The memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0137] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0138] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0139] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0140] The above are only embodiments of the present application and are not intended to limit the present application. For those skilled in the art, various changes and modifications can be made to the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included within the scope of the claims of the present application.

Claims

1. A multi-sensor extrinsic parameter calibration method, wherein, For an autonomous vehicle, the method includes: Calculating the coordinate positions of a plurality of corner points in a calibration board in the coordinate system of a measuring instrument according to the positional relationship of the plurality of corner points measured by the measuring instrument; Determining a first pose relationship between a sensor to be calibrated on the autonomous vehicle and the calibration board; The vehicle body information of the autonomous vehicle includes the front wheelbase of the vehicle body, the rear wheelbase of the vehicle body, and the positional relationship of the central axis of the vehicle body. According to the vehicle body information, marking lines of the ground front axle, the ground rear axle, and the ground central axis are arranged in a preset calibration site. Before determining the first pose relationship between the sensor to be calibrated on the autonomous vehicle and the calibration board, it further includes: Projecting and aligning the corresponding positions of the vehicle body of the autonomous vehicle with the preset ground marking lines, where the front axis of the vehicle body is aligned with the front axis of the ground, and the central axis of the vehicle body is aligned with the central axis of the ground; The calculating the coordinate positions of the plurality of corner points in the calibration board in the coordinate system of the measuring instrument according to the positional relationship of the plurality of corner points measured by the measuring instrument includes: Placing the measuring instrument at the intersection position of the ground front axle and the ground central axis, collecting the positional relationship of at least three corner points, and calculating the coordinate positions of the plurality of corner points in the coordinate system of the measuring instrument; Determining a second pose relationship of the calibration board relative to the vehicle body coordinate system according to the coordinate positions of the plurality of corner points in the coordinate system of the measuring instrument and the coordinate positions corresponding to the plurality of corner points; The determining a second pose relationship of the calibration board relative to the vehicle body coordinate system according to the coordinate positions of the plurality of corner points in the coordinate system of the measuring instrument and the coordinate positions corresponding to the plurality of corner points includes: Using 3D-3D pose calculation to obtain the second pose relationship of the calibration board relative to the vehicle body coordinate system according to the coordinate positions of the plurality of corner points in the coordinate system of the measuring instrument and the coordinate positions of the plurality of corner points in the calibration board coordinate system; Determining a third pose relationship of the sensor to be calibrated in the vehicle body coordinate system according to the first pose relationship and the second pose relationship, and completing the external parameter calibration of the sensor to be calibrated.

2. The method according to claim 1, wherein The pose relationship further includes: rotation parameters, translation parameters.

3. The method according to claim 2, wherein, The determining a third pose relationship of the sensor to be calibrated in the vehicle body coordinate system according to the first pose relationship and the second pose relationship, and completing the external parameter calibration of the sensor to be calibrated includes: Using the product of the rotation parameters of the two in the first pose relationship and the second pose relationship as the rotation parameter for determining the third pose relationship of the sensor to be calibrated in the vehicle body coordinate system; Determining the translation parameter of the third pose relationship of the sensor to be calibrated in the vehicle body coordinate system according to the sum of the product of the rotation parameters and translation parameters of the two in the first pose relationship and the second pose relationship and the translation parameter in the second pose relationship.

4. The method according to claim 1, wherein: The determining a first pose relationship between a sensor to be calibrated on the autonomous vehicle and the calibration board includes: By using the sensors to be calibrated on the autonomous vehicle to collect the calibration plate data, the first pose relationship between the sensor to be calibrated and the calibration plate is calculated, and the relationship between the sensor to be calibrated and the calibration plate satisfies: Ci = [R1, T1]Pi where Pi is the coordinate of the point on the calibration plate collected by the sensor to be calibrated, Ci is the coordinate of the midpoint of the calibration plate, and [R1, T1] is the first pose relationship.

5. The method according to claim 1, wherein The sensor to be calibrated includes at least one of the following: lidar, millimeter-wave radar, camera, IMU.

6. A multi-sensor extrinsic parameter calibration device, wherein, The device includes: A corner point calculation module, configured to calculate the coordinate positions of the multiple corner points in the coordinate system of the measuring instrument according to the positional relationships of the multiple corner points in the calibration plate measured by the measuring instrument; A first pose calculation module, configured to determine the first pose relationship between the sensor to be calibrated on the autonomous vehicle and the calibration plate; The body information of the autonomous vehicle includes the front wheelbase of the vehicle body, the rear wheelbase of the vehicle body, and the positional relationship of the central axis of the vehicle body. According to the body information, the marking lines of the ground front axle, the ground rear axle, and the ground central axis are arranged in a preset calibration site. Before determining the first pose relationship between the sensor to be calibrated on the autonomous vehicle and the calibration plate, it further includes: Aligning the corresponding positions of the vehicle body of the autonomous vehicle with the projection of the ground preset marking lines, where the front axis of the vehicle body is aligned with the front axis of the ground, and the central axis of the vehicle body is aligned with the central axis of the ground; The calculating the coordinate positions of the multiple corner points in the coordinate system of the measuring instrument according to the positional relationships of the multiple corner points in the calibration plate measured by the measuring instrument includes: Placing the measuring instrument at the intersection position of the ground front axle and the ground central axis, collecting the positional relationships of at least three corner points, and calculating the coordinate positions of the multiple corner points in the coordinate system of the measuring instrument; A second pose calculation module, configured to determine the second pose relationship of the calibration plate relative to the vehicle body coordinate system according to the coordinate positions of the multiple corner points in the coordinate system of the measuring instrument and the corresponding coordinate positions of the multiple corner points; The determining the second pose relationship of the calibration plate relative to the vehicle body coordinate system according to the coordinate positions of the multiple corner points in the coordinate system of the measuring instrument and the corresponding coordinate positions of the multiple corner points includes: According to the coordinate positions of the multiple corner points in the coordinate system of the measuring instrument and the coordinate positions of the multiple corner points in the calibration plate coordinate system, using 3D-3D pose calculation to obtain the second pose relationship of the calibration plate relative to the vehicle body coordinate system; A third pose calculation module, configured to determine the third pose relationship of the sensor to be calibrated in the vehicle body coordinate system according to the first pose relationship and the second pose relationship, and complete the external parameter calibration of the sensor to be calibrated.

7. An electronic device, including: A processor; And A memory arranged to store computer-executable instructions, the executable instructions, when executed, cause the processor to execute the method according to any one of claims 1 to 5.

8. A computer-readable storage medium storing one or more programs which, when executed by an electronic device including a plurality of applications, cause the electronic device to perform the method according to any one of claims 1 to 5.

Images