A millimeter-wave calibration method and device

Abstract

The present invention provides a millimeter-wave calibration method and apparatus. The millimeter-wave radar calibration method is as follows: respectively use a lidar and a millimeter-wave radar to detect the key point positions of a plurality of markers with the same height in a to-be-detected area, and obtain the key point positions based on the lidar and the key point positions based on the millimeter-wave radar; match the key point positions based on the lidar and the key point positions based on the millimeter-wave radar, and calculate the 6-degree-of-freedom conversion relationship from the millimeter-wave radar coordinate system to the lidar coordinate system according to the matching result; utilize the conversion relationship between the lidar coordinate system and the vehicle coordinate system, and the 6-degree-of-freedom conversion relationship from the millimeter-wave radar coordinate system to the lidar coordinate system, to calculate the 6-degree-of-freedom conversion relationship from the millimeter-wave radar coordinate system to the vehicle coordinate system. The present invention can achieve a fast and accurate calibration result.

Description

Technical Field

[0001] The present invention relates to the technical field of millimeter-wave radar calibration, and particularly relates to a millimeter-wave radar calibration method and device. Background Art

[0002] The environmental perception sensors involved in autonomous driving include lidar, cameras, millimeter-wave radars, ultrasonic radars, etc.; millimeter-wave radars occupy an unshakable position due to advantages such as low cost, strong environmental adaptability, high reliability, and the ability to calculate the speed, distance, and angle of targets. External parameter calibration of millimeter-wave radars is an essential step. The combined calibration of millimeter-wave radars and lidar is the most commonly used method at present. However, due to the complexity of the external environment, there are deviations in the detection of external targets by lidar and millimeter-wave radars. Millimeter-wave radars have no elevation angle resolution, and in order to simplify the calibration process, it is often assumed that both the elevation angle and the roll angle are 0, which not only further accumulates errors but also poses high requirements for hardware installation. Summary of the Invention

[0003] The object of the present invention is to provide a millimeter-wave radar calibration method and a target detection method and device based on the millimeter-wave radar calibration method for the technical defects existing in the prior art.

[0004] In the first aspect of the present invention, a millimeter-wave radar calibration method is provided, including:

[0005] Detect the key point positions of a plurality of markers with the same height in the area to be measured by lidar and millimeter-wave radar respectively, and obtain the key point positions based on lidar and the key point positions based on millimeter-wave radar;

[0006] Match the key point positions based on lidar and the key point positions based on millimeter-wave radar, and calculate the 6-degree-of-freedom conversion relationship from the millimeter-wave radar coordinate system to the lidar coordinate system according to the matching result;

[0007] Utilize the conversion relationship between the lidar coordinate system and the vehicle coordinate system, and the 6-degree-of-freedom conversion relationship from the millimeter-wave radar coordinate system to the lidar coordinate system, to calculate the 6-degree-of-freedom conversion relationship from the millimeter-wave radar coordinate system to the vehicle coordinate system.

[0008] In the second aspect of the present invention, a target detection method is provided, including:

[0009] Utilize the 6-degree-of-freedom conversion relationship from the millimeter-wave radar coordinate system to the vehicle coordinate system obtained by the millimeter-wave radar calibration method described in the first aspect of the present invention to convert the target position detected by the millimeter-wave radar to the target position in the vehicle coordinate system.

[0010] In the third aspect of the present invention, a target detection device is provided, including:

[0011] A degree of freedom conversion module, used to obtain a 6-degree-of-freedom conversion relationship from a millimeter-wave radar coordinate system to a vehicle coordinate system using the millimeter-wave radar calibration method described in the first aspect of the present invention;

[0012] The coordinate conversion module is used to convert the target position detected by the millimeter wave radar into the target position in the vehicle coordinate system by using the 6-degree-of-freedom conversion relationship from the millimeter wave radar coordinate system to the vehicle coordinate system.

[0013] The millimeter-wave radar calibration method of the present invention adopts a joint calibration method of laser radar and millimeter-wave radar, obtains a 6-degree-of-freedom conversion relationship from a millimeter-wave radar coordinate system to a vehicle coordinate system, and realizes rapid calibration of the millimeter-wave radar.

[0014] The target detection method of the present invention, based on the millimeter wave radar calibration method of the present invention, can quickly and accurately convert the target position detected by the millimeter wave radar to the target position in the vehicle coordinate system, thereby achieving rapid and accurate detection of the target. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 is a flow chart of a millimeter wave radar calibration method according to an embodiment of the present invention;

[0016] Figures 2a - 2c They are respectively a schematic diagram of a laser radar coordinate system, a schematic diagram of a millimeter wave radar coordinate system, and a schematic diagram of the relationship between the millimeter wave radar coordinate system and the laser radar coordinate system;

[0017] Figure 3 A schematic diagram of the layout of the site calibration of an embodiment of the present invention;

[0018] Figure 4 4 is an overall flow chart of the target detection method according to an embodiment of the present invention. DETAILED DESCRIPTION

[0019] The present invention will be further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0020] like Figure 1 As shown, the millimeter wave radar calibration method of the embodiment of the present invention is implemented by the following steps:

[0021] S1. Use laser radar and millimeter-wave radar to detect the key point positions of multiple highly consistent markers 200 in the test area, respectively, to obtain the key point positions based on laser radar and the key point positions based on millimeter-wave radar;

[0022] The marker 200 is placed on a flat and open space 300. Figure 3As shown in the figure, within the effective detection range of the millimeter-wave radar and lidar installed on the host vehicle 100, the marker 200 is placed on an open and flat ground. It can be in front of the host vehicle 100, on one side of the center line 310 of the open and flat ground 300, between the two road edges 320 on both sides. The width W of the ground should meet certain requirements, such as being greater than 10 m. Selecting an open and flat ground 300 to place the marker 200 can ensure that there are no other targets detectable by millimeter-wave radar or lidar near the marker 200.

[0023] As an alternative embodiment, the marker 200 can be a radar reflector or other marker, or a combination of a radar reflector and other markers. The number and placement method of the markers are not limited to Figure 3 the embodiments of.

[0024] As an alternative embodiment, the key point position preferably is the center point, but not limited to taking the center point position as the key point position.

[0025] It should be noted that in the embodiments of the present invention, when using lidar to detect the marker 200, the position of the key point of the marker can be obtained from the clustering result through a clustering method, and any available clustering method can be used as the clustering method.

[0026] S2. Match the key point positions based on lidar and the key point positions based on millimeter-wave radar, and calculate the 6-degree-of-freedom (Six Degrees of Freedom) transformation relationship from the millimeter-wave radar coordinate system to the lidar coordinate system according to the matching result;

[0027] S3. Use the transformation relationship between the lidar coordinate system and the vehicle coordinate system, and the 6-degree-of-freedom transformation relationship from the millimeter-wave radar coordinate system to the lidar coordinate system to calculate the 6-degree-of-freedom transformation relationship from the millimeter-wave radar coordinate system to the vehicle coordinate system.

[0028] Figures 2a - 2c They are respectively the schematic diagram of the lidar coordinate system, the schematic diagram of the millimeter-wave radar coordinate system, and the schematic diagram of the mutual relationship between the millimeter-wave radar coordinate system and the lidar coordinate system.

[0029] In step S3, the 6-degree-of-freedom transformation relationship from the millimeter-wave radar coordinate system to the lidar coordinate system includes the following parameters:

[0030] X-axis translation amount xoffset, Y-axis translation amount yoffset, Z-axis translation amount zoffset, X-axis rotation angle roll, Y-axis rotation angle pitch, Z-axis rotation angle yaw:

[0031] The above parameters have the following relationships:

[0032]

[0033]

[0034]

[0035] The X-axis and Y-axis respectively represent the front-back axis and left-right axis perpendicular to each other on the horizontal plane; combined with Figures 2a - 2c , the X-axis corresponds to Figure 2a , Figure 2c the OX-axis in Figure 2a , Figure 2c ; the Y-axis corresponds to

[0036] The Z-axis represents the up-down axis perpendicular to the said horizontal plane; combined with Figures 2a - 2c , the Z-axis corresponds to Figure 2a , Figure 2c the OZ-axis in

[0037] (lx, ly, lz) are the coordinates of a point in the lidar coordinate system;

[0038] (rx, ry, rz) are the coordinates of a point in the millimeter-wave radar coordinate system;

[0039] R is the rotation matrix, θ and φ respectively represent the Z-axis rotation angle yaw, the Y-axis rotation angle pitch, and the X-axis rotation angle roll;

[0040] T is the translation matrix;

[0041] Among them, the coordinates (rx, ry, rz) of a point in the millimeter-wave radar coordinate system are converted to the coordinates (lx, ly, lz) of a point in the lidar coordinate system by a method of first rotation and then translation.

[0042] Among them, the Z-axis translation amount zoffset is calculated based on the installation heights of the millimeter-wave radar and the lidar. Since the millimeter-wave radar has no height resolution ability, there are relatively high requirements for the Z-axis translation amount zoffset, the X-axis rotation angle roll, and the Y-axis rotation angle pitch during actual installation. Generally, it is required that its height is not less than 400 mm and not more than 800 mm during installation, the angle error is not greater than 1°, and the Y-axis rotation angle pitch approaches 0 degrees (there will be a certain angle error, and the requirement is not greater than 1°). Therefore, after installing the millimeter-wave radar and the lidar on the vehicle, the Z-axis translation amount zoffset and the Y-axis rotation angle pitch of the millimeter-wave radar coordinate system relative to the lidar coordinate system are known. In this way, after calibrating the X-axis translation amount xoffset, the Y-axis translation amount yoffset, the X-axis rotation angle roll, and the Z-axis rotation angle yaw of the millimeter-wave radar coordinate system relative to the lidar coordinate system based on the lidar coordinate system, the calibration of the millimeter-wave radar coordinate system to the vehicle coordinate system can be realized, thereby realizing the calibration and calibration of the millimeter-wave radar coordinate system relative to the vehicle coordinate system.

[0043] As an optional embodiment, the Z-axis rotation angle yaw can be obtained through the following steps:

[0044] Step 1: Fit a straight line to the key point positions based on the millimeter-wave radar on the horizontal plane, and calculate the slope ra of the fitted straight line;

[0045] Step 2: Fit a straight line to the key point positions based on the lidar on the horizontal plane, and calculate the slope la of the fitted straight line;

[0046] Step 3: Calculate the Z-axis rotation angle yaw according to the following formula:

[0047] yaw = arctan(la) - arctan(ra).

[0048] Such as Figure 3As shown in the figure, the height of the five markers 200 arranged in the site is rh, and the positions of marker 1# to marker 5# detected by the millimeter-wave radar are unique, which are r1(x1_r, y1_r, rh), r2(x2_r, y2_r, rh), r3(x3_r, y3_rr, rh), r4(x4_r, y4_rr, rh), r5(x5_r, y5_rr, rh) respectively. The lidar detects that each marker 200 is composed of multiple points. By determining the key points and clustering, the corresponding two markers are found, and the key point positions of each marker are calculated for the positions of the five markers detected by the corresponding millimeter-wave radar. Assuming that after the lidar detects and clusters, the positions of the five markers are l1(x1_l, y1_lr, rh), l2(x2_l, y2_lr, rh), l3(x3_l, y3_l, rh), l4(x4_l, y4_l, rh), l5(x5_l, y5_l, rh) respectively. The data of the five points detected by the millimeter-wave radar and the lidar are linearly fitted on the X and Y planes respectively, and the slope ra corresponding to the millimeter-wave radar data and the slope la corresponding to the lidar data are obtained.

[0049] In the embodiment of the present invention, after obtaining the slope ra corresponding to the millimeter-wave radar data and the slope la corresponding to the lidar data by fitting a straight line using multiple markers, the Z-axis rotation angle yaw is calculated using the above formula, reducing the error of the Z-axis rotation angle yaw.

[0050] In the embodiment of the present invention, the method of linearly fitting the data detected by the millimeter-wave radar and the lidar on the X and Y planes can use the RANSAC method for linear fitting, or any other available fitting method, such as the least squares fitting method.

[0051] As an optional embodiment, the embodiment of the present invention can calculate the X-axis rotation angle roll according to the following formula:

[0052]

[0053]

[0054] i and j respectively represent any two markers;

[0055] xi_l represents the X-axis coordinate value of the key point position of marker i obtained based on the lidar;

[0056] xj_l represents the X-axis coordinate value of the key point position of marker j obtained based on the lidar;

[0057] xi_r represents the X-axis coordinate value of the key point position of marker i obtained based on the millimeter-wave radar;

[0058] xj_r represents the X-axis coordinate value of the key point position of marker j obtained based on the millimeter-wave radar;

[0059] yj_r represents the Y-axis coordinate value of the key point position of marker j obtained based on the millimeter-wave radar;

[0060] yi_r represents the Y-axis coordinate value of the key point position of marker i obtained based on the millimeter-wave radar;

[0061] N is the number of markers;

[0062] is the permutation and combination of any two markers.

[0063] As an optional embodiment, the embodiments of the present invention may calculate the X-axis translation amount xoffset and the Y-axis translation amount yoffset according to the following formula:

[0064]

[0065]

[0066] xoffset i = xi_l - cos(roll) * xi_r + sin(roll) * cos(yaw) * yi_r - sin(roll) * sin(yaw) * rh

[0067] yoffset i = yi_l - sin(roll) * xi_r - cos(roll) * cos(yaw) * yi_r + cos(roll) * sin(yaw) * rh;

[0068] rh represents the height of the marker;

[0069] yi_l represents the Y-axis coordinate value of the key point position of marker i obtained based on the lidar.

[0070] In the embodiments of the present invention, by taking the average of multiple groups of values instead of a single result as the final result, the errors of the X-axis rotation angle roll, the X-axis translation amount xoffset, and the Y-axis translation amount yoffset are reduced. At the same time, the known conditions, such as the height rh of the marker, are fully utilized to increase the accuracy of the calibration result.

[0071] It should be noted that the methods for calculating the X-axis rotation angle roll, the X-axis translation amount xoffset, and the Y-axis translation amount yoffset in the embodiments of the present invention are not limited to the averaging method, and other available methods can also be used to implement them.

[0072] Based on the same inventive concept, in the second aspect of the embodiments of the present invention, a target detection method is provided. The 6-degree-of-freedom conversion relationship from the millimeter-wave radar coordinate system to the vehicle coordinate system obtained by using the millimeter-wave radar calibration method provided in the first aspect of the embodiments of the present invention is utilized to convert the target position detected by the millimeter-wave radar into the target position in the vehicle coordinate system.

[0073] Based on the same inventive concept, in the third aspect of the embodiments of the present invention, a target detection device is further provided, including:

[0074] A degree-of-freedom conversion module, configured to obtain the 6-degree-of-freedom conversion relationship from the millimeter-wave radar coordinate system to the vehicle coordinate system by using the millimeter-wave radar calibration method provided in the first aspect of the embodiments of the present invention;

[0075] A coordinate conversion module, configured to convert the target position detected by the millimeter-wave radar into the target position in the vehicle coordinate system by using the 6-degree-of-freedom conversion relationship from the millimeter-wave radar coordinate system to the vehicle coordinate system.

[0076] In the embodiments of the present invention, since the lidar coordinate system has been previously calibrated to the vehicle coordinate system by using the millimeter-wave radar calibration method in the first aspect of the embodiments of the present invention, after obtaining the 6-degree-of-freedom conversion relationship of the millimeter-wave radar coordinate system relative to the lidar coordinate system, the 6-degree-of-freedom conversion relationship from the millimeter-wave radar coordinate to the vehicle coordinate system is obtained. Thus, through this 6-degree-of-freedom conversion relationship, the position of the target in the millimeter-wave radar coordinate in the vehicle coordinate can be calculated.

[0077] Specifically, as shown in Figure 4 First, a scene is set up. An empty and flat road surface is selected, and then multiple markers are placed on the road surface. As shown in Figure 3 and arranged at the same height. Then, the markers are detected by the lidar on the vehicle itself and subjected to clustering analysis. According to the clustering results, the positions of the key points of the markers are calculated. At the same time, in combination with the positions of the key points of the markers detected by the millimeter-wave radar, the matching of the positions of the key points of the markers is performed. After the matching of the key points is completed, the 6-degree-of-freedom parameters of the millimeter-wave radar are calculated, that is, the X-axis translation amount xoffset, the Y-axis translation amount yoffset, the Z-axis translation amount zoffset, the X-axis rotation angle roll, the Y-axis rotation angle pitch, and the Z-axis rotation angle yaw.

[0078] As described in the first embodiment of the present invention above, the millimeter-wave radar is installed on the host vehicle, and its height is fixed and known, that is, the Z-axis translation amount zoffset is known, and the Y-axis rotation angle pitch approaches 0 degree, which is also known. After obtaining the parameter values of six degrees of freedom in this way, the corresponding rotation matrix R and translation matrix T can be established, so as to establish a conversion relationship between the coordinates of points in the lidar coordinate system and the coordinates of points in the millimeter-wave radar coordinate system, thereby obtaining the six-degree-of-freedom conversion relationship of the millimeter-wave radar coordinate system relative to the lidar coordinate system. Since the lidar coordinate system has been pre-calibrated to the vehicle coordinate system, the millimeter-wave radar coordinate system can be calibrated to the vehicle coordinate system in this way; after the millimeter-wave radar is calibrated, when performing target detection, after detecting the position of the target by the millimeter-wave radar, the six-degree-of-freedom conversion relationship from the millimeter-wave radar coordinate system to the vehicle coordinate system can be used to obtain the target position in the vehicle coordinate system, thus realizing the detection of the target.

[0079] The above description is only a preferred embodiment of the present invention. It should be noted that for those of ordinary skill in the art, without departing from the principle of the present invention, several improvements and refinements can be made, and these improvements and refinements should also be regarded as the protection scope of the present invention.

Claims

1. A millimeter-wave radar calibration method, characterized in that, Including: Detect the key point positions of multiple markers with the same height in the area to be measured using a lidar and a millimeter-wave radar respectively, and obtain the key point positions based on the lidar and the key point positions based on the millimeter-wave radar; Match the key point positions based on the lidar and the key point positions based on the millimeter-wave radar, and calculate the 6-degree-of-freedom transformation relationship from the millimeter-wave radar coordinate system to the lidar coordinate system according to the matching result; Utilize the transformation relationship between the lidar coordinate system and the vehicle coordinate system, and the 6-degree-of-freedom transformation relationship from the millimeter-wave radar coordinate system to the lidar coordinate system, to calculate the 6-degree-of-freedom transformation relationship from the millimeter-wave radar coordinate system to the vehicle coordinate system; The 6-degree-of-freedom transformation relationship from the millimeter-wave radar coordinate system to the lidar coordinate system includes the Z-axis rotation angle yaw; Wherein, perform a linear fitting on the key point positions based on the millimeter-wave radar on the horizontal plane, and calculate the slope ra of the fitted line; perform a linear fitting on the key point positions based on the lidar on the horizontal plane, and calculate the slope la of the fitted line; calculate the Z-axis rotation angle yaw according to the following formula: yaw = arctan(la) - arctan(ra).

2. The millimeter-wave radar calibration method according to claim 1, characterized in that The 6-degree-of-freedom transformation relationship from the millimeter-wave radar coordinate system to the lidar coordinate system further includes the following parameters: the X-axis translation amount xoffset, the Y-axis translation amount yoffset, the Z-axis translation amount zoffset, the X-axis rotation angle roll, and the Y-axis rotation angle pitch, and the relationships of each parameter are as follows: The X-axis and the Y-axis respectively represent the front-back axis and the left-right axis perpendicular to each other on the horizontal plane; The Z-axis represents the up-down axis perpendicular to the horizontal plane; (lx, ly, lz) are the coordinates of a point in the lidar coordinate system; (rx, ry, rz) are the coordinates of a point in the millimeter-wave radar coordinate system; R is the rotation matrix, θ and φ respectively represent the rotation angle yaw about the Z-axis, the rotation angle pitch about the Y-axis, and the rotation angle roll about the X-axis; T is the translation matrix; Wherein, the coordinates (rx, ry, rz) of a point in the millimeter-wave radar coordinate system are transformed into the coordinates (lx, ly, lz) of a point in the lidar coordinate system by a method of first rotation and then translation.

3. The millimeter-wave radar calibration method according to claim 2, wherein Also including: Calculate the Z-axis translation amount zoffset according to the installation height of the millimeter-wave radar and the installation height of the lidar.

4. The millimeter-wave radar calibration method according to claim 2, wherein Also including: Install the millimeter-wave radar and the lidar according to predetermined requirements so that the Y-axis rotation angle pitch approaches 0 degrees.

5. The millimeter-wave radar calibration method according to claim 2, characterized in that, Also including: Calculate the X-axis rotation angle roll according to the following formula: i and j respectively represent any two markers; xi_l represents the X-axis coordinate value of the key point position of marker i obtained based on the lidar; xj_l represents the X-axis coordinate value of the key point position of marker j obtained based on the lidar; xi_r represents the X-axis coordinate value of the key point position of marker i obtained based on the millimeter-wave radar; xj_r represents the X-axis coordinate value of the key point position of marker j obtained based on the millimeter-wave radar; yj_r represents the Y-axis coordinate value of the key point position of marker j obtained based on the millimeter-wave radar; yi_r represents the Y-axis coordinate value of the key point position of marker i obtained based on the millimeter-wave radar; N is the number of markers; is a permutation and combination of any two markers.

6. The millimeter-wave radar calibration method according to claim 2, wherein Also including: Calculate the X-axis translation offset xoffset and Y-axis translation offset yoffset according to the following formula: xoffset i = xi_l - cos(roll) * xi_r + sin(roll) * cos(yaw) * yi_r - sin(roll) * sin(yaw) * rh yoffset i = yi_l - sin(roll) * xi_r - cos(roll) * cos(yaw) * yi_r + cos(roll) * sin(yaw) * rh; rh represents the height of the marker; yi_l represents the Y-axis coordinate value of the key point position of marker i obtained based on the lidar.

7. The millimeter-wave radar calibration method according to claim 1, characterized in that The marker includes a radar reflector.

8. The millimeter-wave radar calibration method according to claim 1, wherein The key point is the center point of the marker.

9. A target detection method, characterized in that, including: Using the 6-degree-of-freedom conversion relationship from the millimeter-wave radar coordinate system to the vehicle coordinate system obtained by the millimeter-wave radar calibration method according to any one of claims 1-8, convert the target position detected by the millimeter-wave radar to the target position in the vehicle coordinate system.

10. A target detection device, characterized in that, including: A degree-of-freedom conversion module for obtaining the 6-degree-of-freedom conversion relationship from the millimeter-wave radar coordinate system to the vehicle coordinate system by using the millimeter-wave radar calibration method according to any one of claims 1-8; A coordinate conversion module for converting the target position detected by the millimeter-wave radar to the target position in the vehicle coordinate system by using the 6-degree-of-freedom conversion relationship from the millimeter-wave radar coordinate system to the vehicle coordinate system.

Images