High-precision millimeter wave radar-camera calibration method and system based on angle reflection

Abstract

The invention discloses a high-precision millimeter-wave radar-camera calibration method and system based on angular reflection, and belongs to the technical field of sensor calibration, and the method comprises the steps: installing a millimeter-wave radar and a camera on a mobile carrier, and collecting the point cloud data and image data of the angular reflection; calculating the distance from the angle inversion to the millimeter wave radar to obtain an angle inversion point cloud coordinate; based on the angle reflection image data, performing grid traversal from an angle reflection center point to obtain pre-selected pixel coordinates of millimeter wave radar reflection points; based on the angular inversion point cloud coordinates and the pixel coordinates, preliminary estimation of millimeter wave radar-camera external parameters is obtained, a nonlinear optimization algorithm is adopted for iterative optimization, final millimeter wave radar-camera external parameters are obtained, and calibration is completed. According to the method, the observation number is effectively increased, the calibration precision is improved, the influence of noise such as background objects and multi-path signals is reduced, and the problem that it is difficult to accurately find out the pixel position of the millimeter wave signal reflection point from the image is solved.

Description

Technical Field

[0001] The present invention belongs to the technical field of sensor calibration, and particularly relates to a high-precision millimeter-wave radar-camera calibration method and system based on a corner reflector. Background Art

[0002] The external parameter calibration of a millimeter-wave radar-camera is a prerequisite for the millimeter-wave radar and the camera to detect targets with high precision in combination. There are already some millimeter-wave radar-camera calibration methods based on target objects, corner reflectors (hereinafter referred to as corner refs). Since millimeter-wave signals have different reflection intensities on metal materials, a corner ref formed by the intersection of three metal surfaces is a commonly used millimeter-wave signal marker. Such methods use the millimeter-wave radar and the camera to simultaneously detect a certain number of statically placed corner refs, obtain the millimeter-wave point cloud and pictures of the markers, and then use pose recovery methods such as PnP to calculate the external parameters of the millimeter-wave radar-camera.

[0003] However, there are still some problems in the existing millimeter-wave radar-camera calibration. First, in practical applications, the number of corner refs is limited, and it is difficult to obtain enough observation data through static observation, so the calibration accuracy is limited by the number of observations. Second, millimeter-wave observations are easily affected by noises such as background objects and multipath signals. In a restricted scenario, such as when an unmanned boat is on the water and it is difficult to find an open background, it is difficult to extract the corner ref after mixing various noises. Finally, the three metal surfaces of the corner ref are relatively large, and the signal is reflected multiple times and returns along the original path, making it difficult to trace the reflection point position and reflection path of the signal. Therefore, it is difficult to accurately find the pixel position of the millimeter-wave signal reflection point from the image.

[0004] Therefore, it is necessary to design a high-precision millimeter-wave radar-camera calibration method and system based on a corner ref to solve the above problems. Summary of the Invention

[0005] The purpose of the present invention is to provide a high-precision millimeter-wave radar-camera calibration method and system based on a corner ref for the problems of limited number of corner refs, sparse millimeter-wave radar point cloud, and difficulty in determining the millimeter-wave signal reflection point of the corner ref. By moving the carrier to increase the corner ref observations, using GNSS to measure the GNSS coordinates of the corner ref and the GNSS position of the carrier, calculating the distance from the corner ref to the millimeter-wave radar at each moment according to the GNSS coordinates of the two, preselecting the corner ref point cloud coordinates accordingly, determining the final pixel coordinates through grid traversal and non-linear optimization algorithms, and obtaining the optimized external parameters of the millimeter-wave radar-camera.

[0006] According to one aspect of the present specification, a high-precision millimeter-wave radar-camera calibration method based on a corner ref is provided, including:

[0007] Install a millimeter-wave radar and a camera on a moving vehicle. During the movement, the millimeter-wave radar collects point cloud data of the corner reflector, and the camera collects image data of the corner reflector;

[0008] Measure the GNSS coordinates of the center point of the corner reflector and the GNSS coordinates of the moving vehicle. Calculate the distance and direction from the corner reflector to the millimeter-wave radar based on the GNSS coordinates of the center point of the corner reflector and the GNSS coordinates of the moving vehicle. Combine the point cloud data of the corner reflector to obtain the corner reflector point cloud coordinates;

[0009] Based on the corner reflector image data, start grid traversal from the center point of the corner reflector to obtain the preselected pixel coordinates of the millimeter-wave radar reflection point;

[0010] Based on the corner reflector point cloud coordinates and the pixel coordinates of the center point of the corner reflector, obtain a preliminary estimate of the external parameters of the millimeter-wave radar-camera. Combine the preselected pixel coordinates of the millimeter-wave radar reflection point and use a nonlinear optimization algorithm for iterative optimization to obtain the final external parameters of the millimeter-wave radar-camera and complete the calibration.

[0011] Furthermore, obtaining the corner reflector point cloud coordinates includes:

[0012] Based on the distance and direction from the corner reflector to the millimeter-wave radar, filter the point cloud data of the corner reflector by setting a distance error threshold and an angle error threshold to obtain the corner reflector point cloud coordinates.

[0013] Furthermore, obtaining the preselected pixel coordinates of the millimeter-wave radar reflection point includes:

[0014] Start grid traversal from the center point of the corner reflector. The pixel points in the corner reflector image data and the corner reflector point cloud data are used as a set of point cloud pixel pairs;

[0015] Establish an optimization function to perform iterative optimization on the set of point cloud pixel pairs;

[0016] Calculate the residuals of the optimization function one by one, and select the point with the smallest residual as the preselected pixel coordinates.

[0017] Furthermore, using a nonlinear optimization algorithm for iterative optimization includes:

[0018] Use the Levenberg-Marquardt method to perform iterative optimization on the preselected pixel coordinates to obtain the final pixel coordinates of the millimeter-wave radar reflection point;

[0019] Based on the final pixel coordinates of the millimeter-wave radar reflection point and the corner reflector point cloud coordinates, obtain the final external parameters of the millimeter-wave radar-camera.

[0020] According to one aspect of this specification, a high-precision millimeter-wave radar-camera calibration system based on a corner reflector is provided, including:

[0021] The data acquisition module is used to install a millimeter-wave radar and a camera on a mobile carrier. During the movement, the millimeter-wave radar acquires the point cloud data of the corner reflector, and the camera acquires the image data of the corner reflector.

[0022] The point cloud coordinate calculation module is used to measure the GNSS coordinates of the center point of the corner reflector and the GNSS coordinates of the mobile carrier, calculate the distance and direction from the corner reflector to the millimeter-wave radar according to the GNSS coordinates of the center point of the corner reflector and the GNSS coordinates of the mobile carrier, and combine the point cloud data of the corner reflector to obtain the corner reflector point cloud coordinates.

[0023] The pixel coordinate extraction module is used to perform grid traversal starting from the center point of the corner reflector based on the corner reflector image data to obtain the preselected pixel coordinates of the millimeter-wave radar reflection point.

[0024] The optimization and calibration module is used to obtain a preliminary estimate of the external parameters of the millimeter-wave radar-camera based on the corner reflector point cloud coordinates and the pixel coordinates of the center point of the corner reflector, and combine the preselected pixel coordinates of the millimeter-wave radar reflection point, and use a non-linear optimization algorithm to perform iterative optimization to obtain the final external parameters of the millimeter-wave radar-camera and complete the calibration.

[0025] According to one aspect of this specification, an electronic device is provided, including a memory and a processor. The memory stores a computer program. It is characterized in that when the processor executes the computer program, the steps of the high-precision millimeter-wave radar-camera calibration method based on the corner reflector are implemented.

[0026] According to one aspect of this specification, a computer-readable storage medium is provided, on which a computer program is stored. It is characterized in that when the computer program is executed by a processor, the steps of the high-precision millimeter-wave radar-camera calibration method based on the corner reflector are implemented.

[0027] According to one aspect of this specification, a computer program product containing instructions is provided. When it runs on a computer, it causes the computer to execute the steps of the high-precision millimeter-wave radar-camera calibration method based on the corner reflector.

[0028] Compared with the prior art, the beneficial effects of the present invention are:

[0029] 1. By installing the millimeter-wave radar and the camera on a mobile carrier, the present invention enables the millimeter-wave radar and the camera to observe the corner reflector at different positions, increasing the number of observations and improving the calibration accuracy.

[0030] 2. By using GNSS to measure the GNSS coordinates of the corner reflector and the GNSS position of the carrier, and calculating the distance from the corner reflector to the millimeter-wave radar at each moment according to the GNSS coordinates of both, and preselecting the corner reflector point cloud coordinates, the present invention reduces the influence of noises such as background objects and multipath signals.

[0031] 3. The present invention preselects the pixel coordinates of the reflection points by traversing each point of the corner reflector image in a grid, and uses a non - linear optimization algorithm to determine the final pixel coordinates, solving the problem of difficultly and accurately finding the pixel positions of the millimeter - wave signal reflection points from the image. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained based on these drawings.

[0033] Figure 1 is the flowchart of the millimeter - wave radar - camera calibration method according to the embodiment of the present invention;

[0034] Figure 2 is the schematic diagram of the extrinsic parameters of the millimeter - wave radar - camera according to the embodiment of the present invention;

[0035] Figure 3 is the schematic diagram of the calibration principle of the extrinsic parameters of the millimeter - wave radar - camera according to the embodiment of the present invention;

[0036] Figure 4 is the schematic diagram of the principle of point cloud and pixel preselection according to the embodiment of the present invention;

[0037] Figure 5 is the schematic diagram of the corner reflector point cloud before and after screening according to the embodiment of the present invention;

[0038] Figure 6 is the schematic diagram of the reprojection result of the corner - reflector millimeter - wave point cloud on the image according to the embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0039] The following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the drawings in the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, rather than all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts belong to the scope of protection of the present invention.

[0040] The present invention provides a high - precision millimeter - wave radar - camera calibration method based on corner reflectors, as Figure 1As shown in the figure, it includes: installing a millimeter-wave radar and a camera on a mobile vehicle. During the movement, the millimeter-wave radar collects point cloud data of the corner reflector, and the camera collects image data of the corner reflector; measuring the GNSS coordinates of the center point of the corner reflector and the GNSS coordinates of the mobile vehicle, calculating the distance from the corner reflector to the millimeter-wave radar, and combining the point cloud data of the corner reflector to obtain the point cloud coordinates of the corner reflector; based on the image data of the corner reflector, starting from the center point of the corner reflector for grid traversal to obtain the preselected pixel coordinates of the reflection point of the millimeter-wave radar; based on the point cloud coordinates of the corner reflector and the pixel coordinates of the center point of the corner reflector, obtaining a preliminary estimate of the external parameters of the millimeter-wave radar-camera, and combining the preselected pixel coordinates of the reflection point of the millimeter-wave radar, using a non-linear optimization algorithm for iterative optimization to obtain the final external parameters of the millimeter-wave radar-camera, and completing the calibration.

[0041] Specifically, let the pose represent that the pose of coordinate system a in coordinate system b is , where represents the rotation of the three coordinate axes, represents the translation of the origin of the coordinate system. Let the external parameters of the millimeter-wave radar-camera be , where M represents the coordinate system of the millimeter-wave radar and C represents the coordinate system of the camera. Calibrating is to determine the values of , as shown in Figure 2 .

[0042] Specifically, as shown in Figure 3 , in the embodiment of the present invention, three corner reflectors are placed statically and evenly distributed in three-dimensional space. To increase the number of observation values, the corner reflector or the vehicle can be moved, and there is no essential difference between the two methods. Since the workload of manually moving the corner reflector is relatively large, the present invention proposes to make the vehicle move during calibration, and the point cloud and pixel sequence of the corner reflector can be obtained. Figure 3 The number of point clouds on the yellow solid line in is the number of observation values, and this number is determined by the millimeter-wave sampling rate and the experimental duration.

[0043] Specifically, assume that the millimeter-wave point cloud coordinates of the corner reflector are , and the pixel coordinates of the corner reflector are . The matrix transformation equation including the external parameters is as follows:

[0044] (1)

[0045] Among them, is the internal parameter of the camera, is the external parameter of the millimeter-wave radar-camera, is the depth when the point cloud is projected onto the camera coordinate system. Expanding Equation (1) gives the following:

[0046] (2)

[0047] Expand Equation (2) further and write it in the form of a system of equations as follows:

[0048] (3)

[0049] Substitute Equation (3) into Equations (1) and (2) to obtain:

[0050] (4)

[0051] According to Equation (4), A is the coefficient matrix and F is the column vector of external parameters. For a set of matching points, there will be two equations. One equation will contain 12 coefficients regarding the external parameters, and one equation corresponds to a row in A. Then, for n sets of matching points, there will be 2n equations and 2n×12 coefficients, which can be expressed as the following equation:

[0052] (5)

[0053] Equation (5) is an overdetermined equation, that is, the number of observations far exceeds the number of unknowns. For example, if the sampling rates of the millimeter-wave radar and the camera are set to 2 Hz, 600 matching points can be obtained by collecting data for 5 minutes. The matrix can be calculated using the least squares method to recover the external parameters .

[0054] Specifically, although the point cloud and the image are obtained, it is difficult to determine the corner reflector point cloud and the pixel observations. Since the millimeter-wave radar signal is easily interfered by noises such as background objects and multipath signals, it is difficult to extract the corner reflector point cloud. On the other hand, the corner reflector is made by the intersection of three planes, and the signal is reflected multiple times and returns along the original path, making it difficult to determine the position of the reflection point on the corner reflector, thus making it difficult to determine the pixel observations of the reflection point in the picture. If randomly selected, it will inevitably introduce large errors, directly affecting the calibration result. Therefore, it is necessary to preselect the corner reflector point cloud and the pixel observations.

[0055] Specifically, the embodiment of the present invention also provides the method steps for preselecting the corner reflector point cloud. First, obtain the absolute position of the corner reflector at different observation positions and the relative position relationship between the millimeter-wave radar through the GNSS device, filter out the interference data in the point cloud data, and preliminarily screen the corner reflector point cloud data corresponding to the image data. Measure the GNSS coordinates of the center point of the corner reflector through the GNSS device to obtain its latitude , longitude and altitude in the global coordinate system. At the same time, use the GNSS module on the millimeter-wave radar carrier to obtain the current position and attitude information, and determine the latitude , longitude and altitude After that, the GNSS coordinates are converted into geocentric earth-fixed coordinates to obtain the absolute positions of the corner reflector and the millimeter-wave radar in the geocentric earth-fixed coordinates. and , omitting the subscript of the absolute position, the geodetic coordinates and the geocentric earth-fixed coordinates The conversion relationship between them is:

[0056] (6)

[0057] Among them, is the latitude, is the longitude, is the elevation, , is the radius of curvature of the meridian. In the radius of curvature of the meridian , is the first eccentricity of the earth ellipsoid, is the semi-major axis of the earth ellipsoid. Substituting the geodetic coordinates of the corner reflector and the geodetic coordinates of the millimeter-wave radar into the above formula respectively, the geocentric earth-fixed coordinates of the corner reflector and the geocentric earth-fixed coordinates of the millimeter-wave radar can be obtained.

[0058] After obtaining the two absolute positions at each moment, calculate the relative position between the millimeter-wave radar and the corner reflector, and the coordinates of the corner reflector relative to the millimeter-wave radar can be obtained as , denoted as . In the point cloud data, the corner reflector point cloud coordinates are . Using the distance and direction information of the millimeter-wave radar and the corner reflector, by setting the distance error threshold and the angle error threshold for screening. Then the conditions that the preselected point cloud needs to meet are:

[0059] (7)

[0060] The distance error threshold and the angle error threshold limit the position of the corner reflector in the point cloud around the position obtained by GNSS. Taking the qualified point cloud as the preselected point can effectively remove the interference of noise. After filtering and preselecting, the millimeter-wave radar point cloud can obtain the uniquely determined point cloud of the corner reflector.

[0061] Specifically, the embodiment of the present invention also provides the method steps for preselecting the pixel coordinates of the reflection point. In the calibration of the millimeter-wave radar and the camera, through the external parameter transform the associated point of the millimeter-wave radar into the camera coordinate system, and then through the internal parameter Transform it further to the image plane. Calculate the associated points of the millimeter-wave radar The position in the image plane obtained through transformation and the preselected pixel points in the image The Euclidean distance, that is, the reprojection error function, and use it as the non-linear optimization function, as follows:

[0062] (8)

[0063] Wherein, is the serial number of the corner reflector, is the number of preselected pixel points matching a single point cloud, is the external parameter .

[0064] Specifically, as Figure 4 shown, the embodiment of the present invention provides the principle of point cloud and pixel preselection. First, starting from the center point of the corner reflector on the image, the corner reflector is spaced at a preset pixel interval , that is, a grid with a fixed side length is set. When selecting, start from the center point of the corner reflector triangle and gradually expand outward according to the grid. These preselected pixel points will be respectively associated with the corresponding point clouds and iteratively optimized as a group of point cloud pixel pairs. Traverse according to this method, calculate the residuals of the optimization function one by one, and select the point with the smallest residual as the preselected pixel point.

[0065] Specifically, if there is no obvious obstacle interference near the corner reflector, the corner reflector point cloud obtained after point cloud preselection is usually a real corner reflector point cloud observation. However, the grid method only roughly preselects the pixel coordinates of the reflection points, and there is still a difference from the real pixel observation values. In order to further improve the estimation accuracy, the accurate pixel coordinates of the reflection points need to be iteratively optimized after point cloud preselection. By taking both the pixel of the reflection point and the millimeter-wave radar-camera external parameter to be estimated as optimization parameters for iterative estimation, while continuously reselecting the pixel coordinates of the reflection points, the final external parameter optimization estimation is completed.

[0066] The embodiment of the present invention also provides the method steps for jointly optimizing and solving the corner reflector pixel coordinates / external parameters. With the preselected pixel coordinates as the center, limit the final pixel coordinates of the reflection points within the pixel area where the corner reflector is located, and construct a constrained Lagrangian function. The pixel coordinates of the reflection point are , and the coordinates of the three vertices of the corner reflector in the image are , and the following constraint conditions are obtained:

[0067] (9)

[0068] The constraint conditions for the same corner reflector are the same, and its optimization function is:

[0069] (10)

[0070] The Lagrangian function with constraints is as follows:

[0071] (11)

[0072] Wherein, is the Lagrange multiplier. The above formula includes the optimization function and three Lagrangian functions. The LM algorithm (Levenberg-Marquardt algorithm) is used for iterative optimization of pixel observation values and external parameters. The LM algorithm combines the advantages of the gradient descent method and the Gauss-Newton method, and can adjust the optimization direction and step size according to the error magnitude. The LM algorithm iteratively adjusts the pixel observation values and the external parameters of the camera, so as to continuously reselect the pixel observation values of the reflection points within the limited range and obtain the optimized millimeter-wave radar-camera external parameters.

[0073] Specifically, in the embodiment of the present invention, in the millimeter-wave point cloud containing more noise in the interference scene, the corner reflector point cloud is pre-screened through the GNSS coordinates. As shown in Figure 5 Figure (a) therein, it shows the original point cloud without screening, from which the corner reflector points cannot be distinguished; as shown in Figure 5 Figure (b) therein, it shows the corner reflector point cloud screened by GNSS assistance, which is marked as a larger square.

[0074] Specifically, the embodiment of the present invention also provides that the calibration results of the millimeter-wave radar-camera external parameters are listed in Table 1. It can be seen from the table that the maximum calibration error of the rotation angle is 0.32°, and the maximum translation error is 0.2 m. The millimeter-wave radar point cloud is reprojected onto the image through the external parameters, and its reprojection error is calculated. The average reprojection error of eight pairs of points is 3.857 pixels.

[0075] Table 1 Millimeter-wave Radar-Camera Calibration Results

[0076]

[0077] As shown in Figure 6 shown, the embodiment of the present invention also provides the reprojection result of the corner reflector millimeter-wave point cloud on the image. Figure 6 The red points in it are the effects of projecting 3D points onto the image. It can be seen that the reprojection results are basically within the corner reflector pixel range.

[0078] The implementation basis of each embodiment of the present invention is achieved through programmed processing by a device with processor functions. Therefore, in engineering practice, the technical solutions and functions of each embodiment of the present invention are encapsulated into various modules. Based on this reality, on the basis of the above embodiments, an embodiment of the present invention provides a high-precision millimeter-wave radar-camera calibration system based on corner reflectors, which is used to execute a high-precision millimeter-wave radar-camera calibration method based on corner reflectors in the above method embodiments.

[0079] The system includes: a data acquisition module, which is used to install a millimeter-wave radar and a camera on a mobile carrier. During the movement, the millimeter-wave radar acquires the point cloud data of the corner reflector, and the camera acquires the image data of the corner reflector; a point cloud coordinate calculation module, which is used to measure and obtain the GNSS coordinates of the center point of the corner reflector and the GNSS coordinates of the mobile carrier, and calculate the distance from the corner reflector to the millimeter-wave radar. Combining the point cloud data of the corner reflector, the point cloud coordinates of the corner reflector are obtained; a pixel coordinate extraction module, which is used to perform grid traversal starting from the center point of the corner reflector based on the corner reflector image data to obtain the preselected pixel coordinates of the millimeter-wave radar reflection point; an optimization and calibration module, which is used to obtain a preliminary estimate of the external parameters of the millimeter-wave radar-camera based on the point cloud coordinates of the corner reflector and the pixel coordinates of the center point of the corner reflector. Combining the preselected pixel coordinates of the millimeter-wave radar reflection point, an iterative optimization is performed using a non-linear optimization algorithm to obtain the final external parameters of the millimeter-wave radar-camera and complete the calibration.

[0080] For the problems of limited number of corner reflectors, sparse millimeter-wave radar point clouds, and difficulty in determining the millimeter-wave signal reflection points of corner reflectors in the high-precision millimeter-wave radar-camera calibration system based on corner reflectors provided by the embodiment of the present invention, by using the above-mentioned several modules, by installing the millimeter-wave radar and the camera on a mobile carrier, the millimeter-wave radar and the camera observe the corner reflector at different positions, increasing the number of observations and improving the calibration accuracy. By using GNSS to measure the GNSS coordinates of the corner reflector and the GNSS position of the carrier, calculating the distance from the corner reflector to the millimeter-wave radar at each moment based on the GNSS coordinates of the two, and preselecting the point cloud coordinates of the corner reflector, the influence of noise such as background objects and multipath signals is reduced. By traversing each point of the corner reflector image in a grid to preselect the pixel coordinates of the reflection point, and using a non-linear optimization algorithm to determine the final pixel coordinates, the problem of difficultly accurately finding the pixel position of the millimeter-wave signal reflection point from the image is solved.

[0081] Based on the same inventive concept as the foregoing embodiments, an embodiment of the present invention also provides an electronic device, including a memory and a processor. The memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions to implement the high-precision millimeter-wave radar-camera calibration method based on corner reflectors as proposed in the above embodiments.

[0082] An embodiment of the present invention further provides a computer-readable storage medium, on which a computer program is stored. When the program is executed by a processor, it is used to reduce the influence of noises such as background objects and multipath signals, increase the number of observations, and improve the calibration accuracy. The storage medium can be any non-volatile storage device such as a hard disk, a solid-state drive, a flash drive, an optical disc, etc., for storing computer program codes and necessary data files. The stored computer program includes: a data acquisition module, a point cloud coordinate calculation module, a pixel coordinate extraction module, and an optimization and calibration module.

[0083] An embodiment of the present invention further provides a computer program product containing instructions, which, when running on a computer, wholly or partially generates the high-precision millimeter-wave radar-camera calibration method based on corner reflection proposed in the above embodiment. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices.

[0084] Finally, it should be pointed out that the above specific embodiments are only relatively representative examples of the present invention. Obviously, the present invention is not limited to the above specific embodiments and there can be many variations. Any simple modification, equivalent change, and modification made to the above specific embodiments based on the technical essence of the present invention shall be considered to fall within the protection scope of the present invention.

Claims

1. A high-precision millimeter-wave radar-camera calibration method based on corner reflectors, characterized in that Including: Mount a millimeter-wave radar and a camera on a mobile carrier. During movement, the millimeter-wave radar collects point cloud data of the corner reflector, and the camera collects image data of the corner reflector; Measure the GNSS coordinates of the center point of the corner reflector and the GNSS coordinates of the mobile carrier. Calculate the distance and direction from the corner reflector to the millimeter-wave radar according to the GNSS coordinates of the center point of the corner reflector and the GNSS coordinates of the mobile carrier, and combine the point cloud data of the corner reflector to obtain the corner reflector point cloud coordinates; Based on the image data of the corner reflector, start grid traversal from the center point of the corner reflector to obtain the preselected pixel coordinates of the millimeter-wave radar reflection point; Based on the corner reflector point cloud coordinates and the preselected pixel coordinates of the millimeter-wave radar reflection point, obtain a preliminary estimate of the external parameters of the millimeter-wave radar-camera. Use a nonlinear optimization algorithm for iterative optimization to obtain the final external parameters of the millimeter-wave radar-camera and complete the calibration.

2. The high-precision millimeter-wave radar-camera calibration method based on corner reflection according to claim 1, characterized in that Obtaining the corner reflector point cloud coordinates includes: Based on the distance and direction from the corner reflector to the millimeter-wave radar, screen the point cloud data of the corner reflector by setting a distance error threshold and an angle error threshold to obtain the corner reflector point cloud coordinates.

3. A high-precision millimeter-wave radar-camera calibration method based on corner reflectors according to claim 1, characterized in that, Obtaining the preselected pixel coordinates of the millimeter-wave radar reflection point includes: Start grid traversal from the center point of the corner reflector. The pixel points in the corner reflector image data and the corner reflector point cloud data are used as a set of point cloud pixel pairs; Establish an optimization function to perform iterative optimization on the set of point cloud pixel pairs; Calculate the residuals of the optimization function one by one, and select the point with the smallest residual as the preselected pixel coordinates.

4. A high-precision millimeter-wave radar-camera calibration method based on corner reflection according to claim 1, characterized in that Using a nonlinear optimization algorithm for iterative optimization includes: Use the Levenberg-Marquardt method to perform iterative optimization on the preselected pixel coordinates to obtain the final pixel coordinates of the millimeter-wave radar reflection point; Based on the final pixel coordinates of the millimeter-wave radar reflection point and the corner reflector point cloud coordinates, obtain the final external parameters of the millimeter-wave radar-camera.

5. A high-precision millimeter-wave radar-camera calibration system based on corner reflectors, characterized in that, Including: A data acquisition module for mounting a millimeter-wave radar and a camera on a mobile carrier. During movement, the millimeter-wave radar collects point cloud data of the corner reflector, and the camera collects image data of the corner reflector; A point cloud coordinate calculation module for measuring the GNSS coordinates of the center point of the corner reflector and the GNSS coordinates of the mobile carrier, calculating the distance and direction from the corner reflector to the millimeter-wave radar according to the GNSS coordinates of the center point of the corner reflector and the GNSS coordinates of the mobile carrier, and combining the point cloud data of the corner reflector to obtain the corner reflector point cloud coordinates; A pixel coordinate extraction module for starting grid traversal from the center point of the corner reflector based on the corner reflector image data to obtain the preselected pixel coordinates of the millimeter-wave radar reflection point; An optimization and calibration module for obtaining a preliminary estimate of the external parameters of the millimeter-wave radar-camera based on the corner reflector point cloud coordinates and the pixel coordinates of the center point of the corner reflector, and combining the preselected pixel coordinates of the millimeter-wave radar reflection point, using a nonlinear optimization algorithm for iterative optimization to obtain the final external parameters of the millimeter-wave radar-camera and complete the calibration.

6. An electronic device, comprising a memory and a processor, the memory storing a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the high-precision millimeter-wave radar-camera calibration method based on a corner reflector according to any one of claims 1 to 4.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the high-precision millimeter-wave radar-camera calibration method based on a corner reflector according to any one of claims 1 to 4.

8. A computer program product comprising instructions, characterized in that, When it runs on a computer, it causes the computer to execute the steps of the high-precision millimeter-wave radar-camera calibration method based on corner reflection according to any one of claims 1 to 4.

Images