Multi-millimeter-wave radar rapid calibration method and device and electronic equipment

Abstract

The invention relates to the technical field of millimeter-wave radar and sensor calibration, in particular to a multi-millimeter-wave radar rapid calibration method and device and electronic equipment, and the method comprises the steps: constructing a first motion model of a millimeter-wave radar according to the Doppler velocity and space coordinates of a point cloud, and extracting a stationary target point cloud from the point cloud data of the multi-millimeter-wave radar, estimating the motion speed of each millimeter-wave radar according to the stationary target point cloud, constructing a second motion model of the millimeter-wave radar based on the external parameters between the radars, estimating the motion speed relationship between the radars together with the motion speed of the millimeter-wave radar, and obtaining the motion parameters of the target platform where the multiple millimeter-wave radars are located; and on the basis of the motion data of the target platform where the multi-millimeter-wave radar is located and the motion speed relationship between the radars, calibrating external parameters between the radars of the multi-millimeter-wave radar. Therefore, the problem that the high-precision requirement of the unmanned system in a dynamic environment is difficult to meet due to response delay when external parameters change in related technologies is solved.

Description

Technical Field

[0001] The present application relates to the technical field of millimeter-wave radar and sensor calibration, and particularly relates to a method, device and electronic device for rapid calibration of multiple millimeter-wave radars. Background Art

[0002] In recent years, mobile robots and autonomous driving technologies have developed rapidly. Due to its robustness in complex environments, millimeter-wave radar has become an important sensor for environmental perception of mobile robots and autonomous driving vehicles. To comprehensively perceive complex scenarios, unmanned systems usually carry multiple millimeter-wave radars for detection. To achieve effective fusion of multi-radar data, it is necessary to accurately obtain the installation position relationship between each radar, that is, the external parameters (extrinsic parameters) of multiple radars.

[0003] Currently, common methods for obtaining extrinsic parameters rely on the cooperation of scene targets and obtain the relative position relationship between radars through offline calibration. However, during actual operation, due to factors such as mechanical vibration, the extrinsic parameters of the radar may drift, thereby affecting the perception accuracy of the system. Therefore, realizing online calibration of multiple radars is crucial for ensuring the long-term reliability of unmanned systems.

[0004] Existing online calibration methods for multiple radars usually rely on observation data within a long time window and estimate extrinsic parameters by solving complex optimization problems. However, such methods have a lag in response when the extrinsic parameters change and are difficult to meet the high-precision extrinsic parameter requirements of unmanned systems in dynamic environments. Summary of the Invention

[0005] The present application provides a method, device and electronic device for rapid calibration of multiple millimeter-wave radars to solve the problem that the response is laggy when the extrinsic parameters change in related technologies and it is difficult to meet the high-precision requirements of unmanned systems in dynamic environments.

[0006] The first aspect embodiment of the present application provides a method for rapid calibration of multiple millimeter-wave radars, including the following steps: constructing a first motion model of the millimeter-wave radar according to the Doppler velocity and spatial coordinates of the point cloud; extracting stationary target point clouds from the point cloud data of multiple millimeter-wave radars, and estimating the motion velocities of each millimeter-wave radar according to the first motion model and the stationary target point clouds extracted by each millimeter-wave radar; constructing a second motion model of the millimeter-wave radar based on the extrinsic parameters between radars, and estimating the motion velocity relationship between radars based on the second motion model and the motion velocities of each millimeter-wave radar; obtaining the motion parameters of the target platform where the multiple millimeter-wave radars are located, and calibrating the extrinsic parameters between the multiple millimeter-wave radars based on the motion data of the target platform where the multiple millimeter-wave radars are located and the motion velocity relationship between radars.

[0007] Optionally, the expression of the first motion model is:

[0008]

[0009] Among them, is the spatial coordinate of all stationary target point clouds under radar i, is the horizontal azimuth angle, is the elevation angle, is the Doppler velocity; is the moving velocity of radar i in its own radar coordinate system, is the horizontal component of the moving velocity of radar i in its own radar coordinate system, is the vertical component of the moving velocity of radar i in its own radar coordinate system; N i,s is the number of stationary target point clouds detected by radar i.

[0010] Optionally, extracting stationary target point clouds from the point cloud data of multiple millimeter-wave radars includes: downsampling the point cloud data of multiple millimeter-wave radars based on the farthest point pre-sampling algorithm; clustering and grouping the point cloud data of multiple millimeter-wave radars based on the density of the point cloud; selecting stationary target point clouds from the clustering groups of each millimeter-wave radar according to the downsampled point cloud.

[0011] Optionally, the farthest point pre-sampling algorithm includes: selecting a random point from the point cloud data as the initial sampling point; for each unselected point, calculating the distances between the unselected point and all points in the selected set, and taking the target distance among all distances as the distance between the unselected point and the selected set; selecting new sampling points from the unselected points according to the distance between the unselected point and the selected set until the sampling stop condition is met.

[0012] Optionally, selecting stationary target point clouds from the clustering groups of each millimeter-wave radar according to the downsampled point cloud includes: estimating the moving velocity of each millimeter-wave radar according to the downsampled point cloud; inferring the Doppler velocity of the point cloud of each millimeter-wave radar according to the moving velocity of each millimeter-wave radar, and determining the stationary target point cloud in the clustering group according to the Doppler velocity; filtering the clustering groups with the number of stationary target point clouds less than the filtering threshold.

[0013] Optionally, estimating the moving velocity of each millimeter-wave radar according to the first motion model and the stationary target point clouds extracted by each millimeter-wave radar includes: obtaining the spatial coordinates and Doppler velocity of the stationary target point clouds extracted by each millimeter-wave radar; fitting the first motion model by using the least squares method based on the spatial coordinates and Doppler velocity of the stationary target point clouds to obtain the moving velocity of each millimeter-wave radar.

[0014] Optionally, the expression of the second motion model is:

[0015]

[0016] Among them, α i is the installation angle of radar i; ωe is the rotational angular velocity of the target platform where the multi-millimeter wave radar is located in the two-dimensional plane; is the motion speed of the reference radar in its own radar coordinate system; (x i , y i ) is the position of radar i in the reference coordinate system.

[0017] Optionally, based on the motion data of the target platform where the multi-millimeter wave radar is located and the motion speed relationship between radars, calibrate the external parameters between radars of the multi-millimeter wave radar, including: identifying the angular velocity in the motion data; if the angular velocity is less than or equal to the angular velocity threshold, estimate the installation angle of the current time according to the motion speed relationship between radars, and use the installation position estimated in the previous time as the installation position of the current time; if the angular velocity is greater than the angular velocity threshold, use the installation angle estimated in the previous time as the installation angle of the current time, and estimate the installation position of the current time according to the motion speed relationship between radars.

[0018] The embodiment of the second aspect of the present application provides a multi-millimeter wave radar rapid calibration device, including: a construction module, configured to construct a first motion model of the millimeter wave radar according to the Doppler velocity and spatial coordinates of the point cloud; an extraction module, configured to extract the static target point cloud from the point cloud data of the multi-millimeter wave radar, and estimate the motion speed of each millimeter wave radar according to the first motion model and the static target point cloud extracted by each millimeter wave radar; an estimation module, configured to construct a second motion model of the millimeter wave radar based on the external parameters between radars, and estimate the motion speed relationship between radars based on the second motion model and the motion speed of each millimeter wave radar; a calibration module, configured to obtain the motion parameters of the target platform where the multi-millimeter wave radar is located, and calibrate the external parameters between radars of the multi-millimeter wave radar based on the motion data of the target platform where the multi-millimeter wave radar is located and the motion speed relationship between radars.

[0019] The embodiment of the third aspect of the present application provides an electronic device, including: a memory, a processor, and a computer program stored on the memory and executable on the processor, and the processor executes the program to implement the multi-millimeter wave radar rapid calibration method as described in the above embodiment.

[0020] Therefore, the present application has the following beneficial effects:

[0021] In the embodiment of the present application, a first motion model of a millimeter-wave radar is constructed through the Doppler velocity and spatial coordinates of a point cloud. The stationary target point cloud is extracted from the point cloud data of multiple millimeter-wave radars. The motion velocity of each millimeter-wave radar is estimated according to the stationary target point cloud. Then, a second motion model of the millimeter-wave radar is constructed based on the external parameters between the radars. Together with the motion velocity of the millimeter-wave radar, the motion velocity relationship between the radars is estimated, and the motion parameters of the target platform where the multiple millimeter-wave radars are located are obtained. Based on the motion data of the target platform where the multiple millimeter-wave radars are located and the motion velocity relationship between the radars, the external parameters between the multiple millimeter-wave radars are calibrated. Without relying on the cooperation of scene targets, fast and accurate online external parameter calibration of multiple millimeter-wave radars is achieved based on the velocity of the radars during the motion process, effectively ensuring the long-term accuracy and reliability of the unmanned system. Thus, the problem that the response is laggy when the external parameters change in the related technology and it is difficult to meet the high-precision requirements of the unmanned system in a dynamic environment is solved.

[0022] Additional aspects and advantages of the present application will be given in part in the following description, become apparent in part from the following description, or be learned through the practice of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] The above and / or additional aspects and advantages of the present application will become apparent and be readily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:

[0024] Figure 1 is a schematic flowchart of a method for rapid calibration of multiple millimeter-wave radars according to an embodiment of the present application;

[0025] Figure 2 is a relationship diagram between the motion velocity and external parameters of a radar provided by an embodiment of the present application;

[0026] Figure 3 is a block diagram example of a device for rapid calibration of multiple millimeter-wave radars according to an embodiment of the present application;

[0027] Figure 4 is a schematic structural diagram of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0028] The embodiments of the present application will be described in detail below. The examples of the embodiments are shown in the accompanying drawings, where the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to explain the present application, but should not be construed as limiting the present application.

[0029] The following describes a method, apparatus, and electronic device for rapid calibration of multi-millimeter wave radars according to embodiments of the present application. In view of the problem in the related art that the response is laggy when the external parameters change, making it difficult to meet the high-precision requirements of unmanned systems in dynamic environments, the present application provides a method for rapid calibration of multi-millimeter wave radars. In this method, a first motion model of the millimeter wave radar is constructed based on the Doppler velocity and spatial coordinates of the point cloud. Static target point clouds are extracted from the point cloud data of the multi-millimeter wave radar, and the motion speeds of each millimeter wave radar are estimated according to the static target point clouds. Then, a second motion model of the millimeter wave radar is constructed based on the external parameters between the radars, and the motion speed relationship between the radars is estimated together with the motion speeds of the millimeter wave radars to obtain the motion parameters of the target platform where the multi-millimeter wave radar is located. Based on the motion data of the target platform where the multi-millimeter wave radar is located and the motion speed relationship between the radars, the external parameters between the radars of the multi-millimeter wave radar are calibrated. Without relying on the cooperation of scene targets, rapid and accurate online external parameter calibration of multiple millimeter wave radars is achieved based on the speed of the radar during the motion process, effectively ensuring the long-term accuracy and reliability of the unmanned system. Thus, the problem in the related art that the response is laggy when the external parameters change and it is difficult to meet the high-precision requirements of unmanned systems in dynamic environments is solved.

[0030] Specifically, Figure 1 FIG. is a schematic flowchart of a method for rapid calibration of multi-millimeter wave radars provided by an embodiment of the present application.

[0031] As Figure 1 shown, the method for rapid calibration of multi-millimeter wave radars includes the following steps:

[0032] In step S101, a first motion model of the millimeter wave radar is constructed according to the Doppler velocity and spatial coordinates of the point cloud.

[0033] Among them, the point cloud refers to a set composed of a large number of discrete points. In the embodiments of the present application, it refers to a set of target reflection points detected by the millimeter wave radar. The set of static target reflection points detected by the i-th millimeter wave radar is represented as where represents the j-th point in respectively represent the distance of this point to the radar origin, the horizontal azimuth angle of this point, the elevation angle of this point, and the Doppler velocity of this point. N i,s is the number of static target point clouds detected by radar i. The static target point clouds will be described in detail below and will not be elaborated here; the Doppler velocity refers to the velocity component of the target relative to the radar measured based on the Doppler effect and is determined by analyzing the change in the reflected wave frequency.

[0034] It can be understood that in the embodiments of the present application, a first motion model of a millimeter-wave radar can be constructed based on the Doppler velocity and spatial coordinates of point clouds. Specifically, the first motion model is as follows:

[0035] In the embodiments of the present application, the expression of the first motion model is:

[0036]

[0037] Wherein, is the spatial coordinate of all stationary target point clouds under radar i, is the horizontal azimuth angle, is the altitude angle, is the Doppler velocity; is the motion velocity of radar i in its own radar coordinate system, is the horizontal component of the motion velocity of radar i in its own radar coordinate system, is the vertical component of the motion velocity of radar i in its own radar coordinate system; N i,s is the number of stationary target point clouds detected by radar i. Among the above parameters, is a known quantity, is an unknown quantity to be estimated. By using the least squares method to fit the above motion equation, the motion velocity of radar i can be estimated.

[0038] In step S102, stationary target point clouds are extracted from the point cloud data of multiple millimeter-wave radars. According to the first motion model and the stationary target point clouds extracted by each millimeter-wave radar, the motion velocities of each millimeter-wave radar are estimated.

[0039] Among them, the stationary target point cloud refers to the set of point cloud data reflected by targets that are absolutely stationary in the environment (such as buildings, etc.). These point clouds do not change their positions over time. The method for extracting stationary target point clouds can be an improved RANSAC (Random Sample Consensus) algorithm. When the number of point clouds of moving targets in the environment is large, the traditional RANSAC algorithm often fails to correctly detect stationary targets. Therefore, in the embodiments of the present application, the RANSAC algorithm based on FPS (Farthest Point Sampling) is used to extract stationary target point clouds.

[0040] It can be understood that when extracting the static target point cloud from the point cloud data of multiple millimeter-wave radars containing moving targets and noise points through the traditional RANSAC algorithm, it may be impossible to correctly detect the static target points due to the large number of point clouds of moving targets in the environment. Therefore, in the embodiments of this application, the RANSAC algorithm based on FPS is used to extract the static target point cloud, obtain the set of point cloud data reflected by the targets that are absolutely static in the environment, and estimate the movement speed of each millimeter-wave radar through the static target point cloud, specifically as follows:

[0041] In the embodiments of this application, extracting the static target point cloud from the point cloud data of multiple millimeter-wave radars includes: downsampling the point cloud data of multiple millimeter-wave radars based on the farthest point sampling algorithm; clustering and grouping the point cloud data of multiple millimeter-wave radars based on the density of the point cloud; and selecting the static target point cloud from the clustering groups of each millimeter-wave radar according to the downsampled point cloud.

[0042] Among them, the farthest point sampling algorithm is a downsampling technique that can select a set of representative points from the original point cloud that can cover the entire point cloud distribution as much as possible. It will be described in detail below and will not be elaborated here; the density-based clustering and grouping is a clustering algorithm that can identify dense regions as clusters and filter low-density regions or isolated points as noise or outliers.

[0043] It can be understood that in the embodiments of this application, the farthest point sampling algorithm is used to downsample the point cloud data from multiple millimeter-wave radars to screen out a set of points with uniform spatial distribution and representativeness, and the density-based clustering algorithm is used to cluster and group the original point cloud data of multiple millimeter-wave radars. Finally, based on the downsampled point cloud, the point cloud that conforms to the characteristics of the static target is further screened out from the clustering groups of each radar to determine the static target point cloud.

[0044] In the embodiments of this application, the farthest point sampling algorithm includes: selecting a random point from the point cloud data as the initial sampling point; for each unselected point, calculating the distances between the unselected point and all points in the selected set, and taking the target distance among all distances as the distance between the unselected point and the selected set; and selecting a new sampling point from the unselected points according to the distance between the unselected point and the selected set until the sampling stop condition is met.

[0045] Among them, the sampling stop condition is specifically set according to the actual situation and will not be specifically limited here. For example, if the sampling stop condition is set to reach the expected number of sampling points, then the sampling will stop when the number of sampling points is sufficient.

[0046] It can be understood that in the embodiments of the present application, the farthest point pre-sampling algorithm is used to downsample the point cloud data from multiple millimeter-wave radars. The specific method is as follows: randomly select a point from the point cloud data as the initial sampling point and add it to the selected set. For each unselected point, calculate its distance from each point in the selected set, and regard the minimum value of these distances as the target distance between this point and the selected set. According to the target distances between each unselected point and the selected set, select the point with the farthest distance as the new sampling point and add it to the selected set. Repeat this process until the sampling stop condition is met. By using the farthest point pre-sampling algorithm to downsample the point cloud data from multiple millimeter-wave radars, it can ensure that the selected points are as evenly distributed in space as possible, thereby effectively reducing the influence of densely distributed dynamic targets and providing a more reliable data basis for subsequent processing.

[0047] In the embodiments of the present application, the stationary target point cloud is selected from the clustering groups of each millimeter-wave radar according to the downsampled point cloud, including: estimating the motion speed of each millimeter-wave radar according to the downsampled point cloud; inferring the Doppler speed of the point cloud of each millimeter-wave radar according to the motion speed of each millimeter-wave radar, and determining the stationary target point cloud in the clustering group according to the Doppler speed; filtering out the clustering groups in which the number of stationary target point clouds is less than the filtering threshold.

[0048] Among them, to estimate the motion speed of each millimeter-wave radar according to the downsampled point cloud, from the point cloud obtained by downsampling the point cloud data, according to the first motion model, the traditional RANSAC algorithm is used to estimate the motion speed of each millimeter-wave radar; to infer the Doppler speed of the point cloud of each millimeter-wave radar according to the motion speed of each millimeter-wave radar, the first motion model is fitted by the least squares method to obtain the estimated radar speed, and then the Doppler speed of each point in the point cloud at this radar speed is inferred according to the first motion model; the filtering threshold is specifically set according to the actual situation and will not be specifically limited here.

[0049] It can be understood that in the embodiments of the present application, first, the information in the downsampled point cloud is used to estimate the motion speed of each millimeter-wave radar. Then, based on the estimated motion speed, the Doppler speed of the point cloud detected by each radar is inferred. The predicted Doppler speeds are compared with the actually observed Doppler speeds. If the difference between the inferred Doppler speed and the Doppler speed actually measured by the radar is less than the set threshold, it is determined that this point belongs to the stationary target point cloud. If the actual Doppler speeds of most points in a certain clustering group match the speeds predicted based on the radar motion speed, it is considered that the clustering group contains stationary target point clouds. Check the number of point clouds identified as stationary target point clouds in each clustering group. If the number is less than the set filtering threshold, remove the entire clustering group to ensure that subsequent analysis focuses on reliable stationary target data.

[0050] In the embodiments of the present application, based on the first motion model and the stationary target point clouds extracted by each millimeter-wave radar, the motion speed of each millimeter-wave radar is estimated, including: obtaining the spatial coordinates and Doppler velocities of the stationary target point clouds extracted by each millimeter-wave radar; based on the spatial coordinates and Doppler velocities of the stationary target point clouds, fitting the first motion model by using the least squares method to obtain the motion speed of each millimeter-wave radar.

[0051] It can be understood that the embodiments of the present application obtain the spatial coordinates of the stationary target point clouds extracted by each millimeter-wave radar, that is, the distance, horizontal azimuth angle, and elevation angle, and their corresponding Doppler velocities. Based on the spatial coordinates and Doppler velocities of these stationary target point clouds, the least squares method is used to fit the first motion model, so as to accurately estimate the motion speed of each millimeter-wave radar itself.

[0052] In step S103, based on the external parameters between radars, a second motion model of the millimeter-wave radar is constructed, and based on the second motion model and the motion speeds of each millimeter-wave radar, the motion speed relationship between radars is estimated.

[0053] Among them, the external parameters between radars refer to the relative position and angular relationship between different millimeter-wave radars.

[0054] It can be understood that the embodiments of the present application can construct a second motion model of the millimeter-wave radar through the external parameters between radars, and use the motion speeds of each millimeter-wave radar obtained in the previous steps, combined with the second motion model, to calculate the motion speed relationship between radars. The second motion model is specifically as follows:

[0055] In the embodiments of the present application, the expression of the second motion model is:

[0056]

[0057] Among them, α i is the installation angle of radar i, which is the angle between the y-axis of the radar coordinate system and the y-axis of the reference coordinate system, increasing counterclockwise; ω e is the rotational angular velocity of the target platform where the multi-millimeter-wave radar is located in the two-dimensional plane; is the motion speed of the reference radar in its own radar coordinate system; (x i , y i ) is the position of radar i in the reference coordinate system. Among them, the embodiments of the present application only consider the two-dimensional motion situation, and ω e can be measured by an IMU (Inertial Measurement Unit), and the angular velocity is greater than 0 when rotating counterclockwise.

[0058] In step S104, obtain the motion parameters of the target platform where the multi-millimeter wave radar is located, and calibrate the external parameters between the radars based on the motion data of the target platform where the multi-millimeter wave radar is located and the motion speed relationship between the radars.

[0059] Among them, according to the second motion model, when the target platform where the multi-millimeter wave radar is located moves with a relatively large angular velocity ω e in motion, the difference in the installation positions of the multi-millimeter wave radars will cause a significant difference in the radar speeds; while when the unmanned platform moves in a straight line, that is, the angular velocity ω e is 0, the difference in the installation positions of the multi-millimeter wave radars will not affect the speed of the radars. Therefore, the speed difference between the multi-millimeter wave radars is completely caused by the installation angle α i Based on this, the embodiment of the present application uses a method for estimating different external parameters in stages to calibrate the external parameters between the multi-millimeter wave radars based on the motion data of the target platform where the multi-millimeter wave radar is located and the motion speed relationship between the radars, specifically as follows:

[0060] In the embodiment of the present application, calibrating the external parameters between the multi-millimeter wave radars based on the motion data of the target platform where the multi-millimeter wave radar is located and the motion speed relationship between the radars includes: identifying the angular velocity in the motion data; if the angular velocity is less than or equal to the angular velocity threshold, estimate the current installation angle according to the motion speed relationship between the radars, and use the previously estimated installation position as the current installation position; if the angular velocity is greater than the angular velocity threshold, use the previously estimated installation angle as the current installation angle, and estimate the current installation position according to the motion speed relationship between the radars.

[0061] Among them, the angular velocity threshold is specifically set according to the actual situation and is not specifically limited here. Usually, it can be set to less than 5°; if the angular velocity is less than or equal to the angular velocity threshold, estimate the current installation angle according to the motion speed relationship between the radars, and estimate the current installation angle by the estimated result of the speed of the reference radar and the estimated result of the speed of radar i: Among them If the angular velocity is greater than the angular velocity threshold, estimate the current installation position according to the motion speed relationship between the radars, and obtain the installation position (x and the estimated result of the speed of radar i , y i , i ) as:

[0062] It can be understood that in the embodiments of the present application, first, by analyzing the motion data of the target platform where the multi-millimeter wave radar is located, the current angular velocity is identified. If the detected angular velocity is less than or equal to the set angular velocity threshold, it means that the platform is roughly in a straight-line motion state. At this time, the current installation angle of each radar is estimated according to the motion speed relationship between the radars, and the installation position determined during the previous calibration is kept unchanged. On the contrary, if the angular velocity is greater than the angular velocity threshold, it indicates that the platform is performing a relatively significant rotational motion. Then, the installation angle calibrated last time is maintained unchanged, and the current installation position is re-estimated according to the motion speed relationship between the radars, realizing the adaptive adjustment of the external parameters between the radars, ensuring accurate environmental perception even in a dynamically changing environment, effectively avoiding complex optimization problems, and at the same time improving the response speed and accuracy to changes in external parameters.

[0063] According to the multi-millimeter wave radar rapid calibration method proposed by the embodiments of the present application, a first motion model of the millimeter wave radar is constructed through the Doppler velocity and spatial coordinates of the point cloud. The static target point cloud is extracted from the point cloud data of the multi-millimeter wave radar, the motion speed of each millimeter wave radar is estimated according to the static target point cloud, and then a second motion model of the millimeter wave radar is constructed based on the external parameters between the radars. Together with the motion speed of the millimeter wave radar, the motion speed relationship between the radars is estimated, the motion parameters of the target platform where the multi-millimeter wave radar is located are obtained, and the external parameters between the multi-millimeter wave radars are calibrated based on the motion data of the target platform where the multi-millimeter wave radar is located and the motion speed relationship between the radars. Without relying on the cooperation of scene targets, rapid and accurate online external parameter calibration of multiple millimeter wave radars is achieved based on the speed of the radar during the motion process, effectively ensuring the long-term accuracy and reliability of the unmanned system.

[0064] The multi-millimeter wave radar rapid calibration method will be further described below through a specific embodiment.

[0065] This embodiment provides an external parameter calibration method for a multi-millimeter wave radar based on motion estimation. The specific steps include:

[0066] Step S1: Estimation of the motion speed of the millimeter wave radar.

[0067] (1) According to the Doppler velocity and spatial coordinates of the point cloud, a first motion model of the millimeter wave radar point cloud is constructed.

[0068] The static target point cloud detected by the i-th millimeter wave radar is expressed as where represents the j-th point in respectively represent the distance of this point to the radar origin, the horizontal azimuth angle of this point, the elevation angle of this point, and the Doppler velocity of this point, and N i,sis the number of stationary target point clouds detected by radar i. The y-axis of the radar points to the due front of the radar, and the x-axis points to the right. Then, the velocity of the radar can be represented by the velocity along the x-axis direction and the velocity along the y-axis direction . In this embodiment, only the movement of the platform in the two-dimensional plane is considered. When the radar moves, the Doppler velocity of the stationary target point cloud is the projection of the radar velocity in the radial direction. Therefore the relationship between the Doppler velocity of

[0069]

[0070] and the radar movement velocity can be expressed as: is a known quantity, that is, the spatial coordinates (horizontal azimuth angle and altitude angle) and Doppler velocity of all stationary target point clouds under radar i; is the quantity to be estimated, that is, the movement velocity of radar i in its own radar coordinate system. Using the least squares (LS) [1] method to fit the above movement equation, the movement velocity of radar i can be estimated.

[0071] (2) Extract the stationary target point clouds in the environment using an improved random sample consensus algorithm.

[0072] Since only the point clouds of stationary targets satisfy the above first movement model, before estimating the radar movement velocity, an improved RANSAC algorithm is used to extract the stationary target point clouds from the radar point clouds containing moving targets and noise points. When the number of point clouds of moving targets in the environment is large, the traditional RANSAC algorithm often fails to correctly detect stationary targets. To address this problem, this embodiment proposes a RANSAC algorithm based on farthest point pre-sampling to reduce the influence of large moving targets. Specifically, the following steps are performed on the point cloud of each radar:

[0073] (2-1) Perform farthest point sampling on the point cloud and roughly estimate the radar velocity based on the downsampled point cloud.

[0074] Since the point clouds of large moving targets usually show a relatively concentrated cluster distribution, while the stationary target point clouds in the environment are relatively dispersed, the FPS algorithm is first used to uniformly downsample the radar point cloud to reduce the proportion of moving target point clouds therein.

[0075] Specifically:

[0076] [1] Select a random point as the initial sampling point;

[0077] [2] For each unselected point, calculate its distance from all points in the selected point set, and use the minimum distance as the distance from this point to the selected set;

[0078] [3]. From all the points that have not been selected currently, find the point with the farthest distance from the selected set, and add it as a new sampling point to the selected set;

[0079] [4]. Repeat steps [2] and [3] until enough points are sampled.

[0080] This sampling method ensures that the sampling points are as evenly distributed as possible in space, thereby reducing the proportion of dense moving target point clouds in the sampling points. The point cloud downsampled from the point cloud of radar i is denoted as According to the first motion model satisfied by the static target point cloud in (1), for Adopt the traditional RANSAC algorithm to estimate the motion speed of radar i. First, randomly sample N points from rs , where N rs is a parameter set manually. Assume that these N rs points are all static target point clouds, and use the least squares method to fit the first motion model in 1.1 to estimate a radar speed. Then, according to the first motion model in (1), infer the Doppler speed of each point in at this radar speed. If the difference between the inferred Doppler speed and the Doppler speed actually measured by the radar is less than the manually set threshold, then this point is also considered a static target point. If the number of points determined to be static target points at the current radar speed is less than rs half of the total number of point clouds, then start from sampling N points and repeat the above steps; if the number of points determined to be static target points reaches

[0081] half of the total number of point clouds, then exit the iteration. After the algorithm iteration ends, based on the spatial coordinates and Doppler speeds of all static target point clouds in found currently, use the least squares method to fit the first motion model in (1) to obtain the motion speed of radar i estimated from the downsampled point cloud

[0082] (2-2). Cluster and group the radar point cloud based on the density of the point cloud.

[0083] Adopt the density-based DBSCAN (Density-Based Spatial Clustering of Applications with Noise, clustering algorithm) to cluster and group the point cloud. The outlier points that are not assigned to a certain cluster group after clustering are identified as noise points and filtered out from the point cloud. The points with the same label after clustering belong to the same cluster group and are identified as coming from the same target. The target cluster groups obtained after clustering the point cloud of radar i are denoted as where C i,j represents the j-th clustering group of the i-th radar, and N i is the total number of target clustering groups under radar i.

[0084] (2-3) Determine whether the clustering group is a stationary target based on the radar speed estimated from the downsampled point cloud

[0085] For all target clustering groups under radar i According to the estimated radar speed judge whether the clustering group is a stationary target point cloud. Specifically, for the j-th clustering group of radar i, where p ij,l is the l-th point in C i,j , and N ij is the number of point clouds in C i,j . According to the first motion model in (1), based on the horizontal azimuth angle θ ij,l of p ij,l and the elevation angle the Doppler speed of this point under the radar speed can be inferred as follows:

[0086]

[0087] If and the actual Doppler speed v ij,l of p ij,l satisfy then p ij,l is considered a stationary target point, where δ v is a manually set threshold. Perform the above discrimination on all points in C i,j . If the number of stationary target points in C i,j is less than then C i,j is considered to come from a moving target or noise, and the point cloud of the entire clustering group is filtered out; if the number of stationary target points in C i,j reaches then C i,j is considered to come from a stationary target, the stationary target points in C i,j are retained, and the remaining points are filtered out. Merge the stationary target points retained by all clustering groups under radar i to obtain the point cloud which is the stationary target point cloud detected by radar i.

[0088] (3) Estimate the motion speed of each radar based on the stationary target point cloud extracted from each millimeter-wave radar.

[0089] Based on the stationary target point cloud of radar i extracted in (2) According to its spatial coordinates and Doppler velocity, the first motion model in (1) is fitted by the least squares method to obtain an estimated value of the motion velocity of radar i. The above steps are performed for each millimeter-wave radar to estimate the motion velocity of each radar in its own radar coordinate system. where 1 ≤ j ≤ N r , N r is the number of millimeter-wave radars on the platform.

[0090] Step S2: Estimate the extrinsic parameters between radars based on the motion velocities of the millimeter-wave radars.

[0091] (1) Construct a second motion model for multiple millimeter-wave radars based on the extrinsic parameters between radars.

[0092] Taking the coordinate system of the first millimeter-wave radar as the reference coordinate system, the motion of the unmanned platform can be represented by (in this embodiment, only two-dimensional motion is considered), where ω e is the rotational angular velocity of the platform in the two-dimensional plane, which can be measured by an Inertial Measurement Unit (IMU), and the angular velocity is greater than 0 when rotating counterclockwise. The extrinsic parameters of the radar to be estimated are represented by (x i , y i , α i ), where (x i , y i ) represents the position of radar i in the reference coordinate system; α i represents the installation angle of radar i, which is the angle between the y-axis of the radar coordinate system and the y-axis of the reference coordinate system, increasing counterclockwise. For radar i (2 ≤ i ≤ N r ), the relationship between the motion velocity of the radar and the extrinsic parameters is as shown in Figure 2 . The motion velocity of this radar in the reference coordinate system is the vector sum of the translational velocity of the reference radar and the linear velocity of this radar rotating around the reference radar. After the velocity of radar i in the reference coordinate rotates by an angle of α i , the motion velocity of this radar in its own coordinate system is obtained Therefore, the relationship between the motion velocities of radar i and the reference radar in their respective coordinate systems satisfies the following relationship, that is, the second motion model:

[0093]

[0094] (2) Adaptively estimate the extrinsic parameters between radars based on the motion of the unmanned platform.

[0095] Considering that the extrinsic parameters of different radars may change independently, in order to avoid coupling interference during multi-radar collaborative calibration and to avoid solving complex optimization problems online, this embodiment adopts a method of estimating one by one, and separately estimates the extrinsic parameter relationship between radar i (2 ≤ i ≤ N r ) and the reference radar, so as to obtain the extrinsic parameters of all radars in a unified reference coordinate system. This method improves the independence and stability of the extrinsic parameter estimation of each radar.

[0096] The extrinsic parameter differences between radars result in different motion speeds of different radars on the same rigid body platform. Therefore, the extrinsic parameter relationship between radars can be estimated through the speed differences between radars. The extrinsic parameters to be estimated include the installation position and installation angle of the radar. However, due to the lack of degrees of freedom, the speed difference between radars at a single moment cannot estimate both the position and angle extrinsic parameters simultaneously. Existing methods estimate by combining the observation data within a relatively long time window, but this will cause a lag in the system's response to changes in extrinsic parameters and requires solving complex optimization problems. For this reason, this embodiment proposes a method for estimating different extrinsic parameters in stages to achieve efficient and real-time extrinsic parameter calibration. According to the motion speed relationship between multiple radars in step S2(1), when the unmanned platform moves with a large angular velocity ω e the difference in installation position will cause a significant difference in radar speeds; when the unmanned platform moves in a straight line (i.e., the angular velocity ω e is 0), the difference in installation position will not affect the speed of the radar, and the speed difference between radars is completely caused by the installation angle α i . Based on this, this embodiment adaptively selects to update the installation position or installation angle of the radar according to the angular velocity motion situation of the unmanned platform.

[0097] When the angular velocity ω e of the unmanned platform is less than the given threshold δ ω (usually set to less than 5°), the installation angle α i of the radar is estimated. At this time, the installation position (x i , y i ) adopts the result of the previous estimate. If it has not been estimated, the rough result measured manually during initialization can be used. According to the speed estimation result of the reference radar and the speed estimation result of radar i in step S1(3), the closed-form solution of α i can be obtained as:

[0098]

[0099] where

[0100] When the angular velocity ω e of the unmanned platform is greater than the given threshold δω When estimating the installation position (x i , y i ) of the radar, the installation angle α i adopts the result of the previous estimation. According to the speed estimation result of the reference radar and the speed estimation result of radar i , the closed-form solution of (x i , y i ) can be obtained as follows:

[0101]

[0102] Next, a multi-millimeter wave radar fast calibration device according to an embodiment of the present application will be described with reference to the accompanying drawings.

[0103] Figure 3 is a block diagram of the multi-millimeter wave radar fast calibration device according to an embodiment of the present application.

[0104] As Figure 3 shown, the multi-millimeter wave radar fast calibration device 10 includes: a construction module 201, an extraction module 202, an estimation module 203, and a calibration module 204.

[0105] Among them, the construction module 201 is used to construct the first motion model of the millimeter wave radar according to the Doppler speed and spatial coordinates of the point cloud; the extraction module 202 is used to extract the stationary target point cloud from the point cloud data of the multi-millimeter wave radar, and estimate the motion speed of each millimeter wave radar according to the first motion model and the stationary target point cloud extracted by each millimeter wave radar; the estimation module 203 is used to construct the second motion model of the millimeter wave radar based on the external parameters between radars, and estimate the motion speed relationship between radars based on the second motion model and the motion speed of each millimeter wave radar; the calibration module 204 is used to obtain the motion parameters of the target platform where the multi-millimeter wave radar is located, and calibrate the external parameters between radars of the multi-millimeter wave radar based on the motion data of the target platform where the multi-millimeter wave radar is located and the motion speed relationship between radars.

[0106] In the embodiment of the present application, the expression of the first motion model is:

[0107]

[0108] Among them, is the spatial coordinate of all stationary target point clouds under radar i, is the horizontal azimuth angle, is the elevation angle, is the Doppler speed; is the motion speed of radar i in its own radar coordinate system, is the horizontal component of this speed, is the vertical component of this speed; Ni,s is the number of stationary target point clouds detected by Radar i.

[0109] In the embodiment of the present application, to extract stationary target point clouds from the point cloud data of the multi-millimeter-wave radar, the extraction module 202 is further configured to: downsample the point cloud data of the multi-millimeter-wave radar based on the farthest point sampling algorithm; cluster and group the point cloud data of the multi-millimeter-wave radar according to the density of the point cloud; select stationary target point clouds from the clustering groups of each millimeter-wave radar according to the downsampled point cloud.

[0110] In the embodiment of the present application, the farthest point sampling algorithm includes: selecting a random point from the point cloud data as the initial sampling point; for each unselected point, calculating the distances between the unselected point and all points in the selected set, and taking the target distance among all the distances as the distance between the unselected point and the selected set; selecting new sampling points from the unselected points according to the distance between the unselected point and the selected set until the sampling stop condition is met.

[0111] In the embodiment of the present application, selecting stationary target point clouds from the clustering groups of each millimeter-wave radar according to the downsampled point cloud includes: estimating the motion speed of each millimeter-wave radar according to the downsampled point cloud; inferring the Doppler speed of each millimeter-wave radar according to the motion speed of each millimeter-wave radar, and determining the stationary target point clouds in the clustering group according to the Doppler speed; filtering out the clustering groups where the number of stationary target point clouds is less than the filtering threshold.

[0112] In the embodiment of the present application, to estimate the motion speed of each millimeter-wave radar according to the first motion model and the stationary target point clouds extracted by each millimeter-wave radar, the extraction module 202 is further configured to: obtain the spatial coordinates and Doppler speed of the stationary target point clouds extracted by each millimeter-wave radar; based on the spatial coordinates and Doppler speed of the stationary target point clouds, fit the first motion model by using the least squares method to obtain the motion speed of each millimeter-wave radar.

[0113] In the embodiment of the present application, the expression of the second motion model is:

[0114]

[0115] where α i is the installation angle of Radar i; ω e is the rotational angular velocity of the target platform where the multi-millimeter-wave radar is located in the two-dimensional plane; is the motion speed of the reference radar in its own radar coordinate system; (x i , y i ) is the position of Radar i in the reference coordinate system.

[0116] In the embodiments of the present application, based on the motion data of the target platform where the multi-millimeter wave radars are located and the motion speed relationship between the radars, the external parameters between the multi-millimeter wave radars are calibrated. The calibration module 204 is further configured to: identify the angular velocity in the motion data; if the angular velocity is less than or equal to the angular velocity threshold, estimate the installation angle of the current time according to the motion speed relationship between the radars, and use the installation position estimated in the previous time as the installation position of the current time; if the angular velocity is greater than the angular velocity threshold, use the installation angle estimated in the previous time as the installation angle of the current time, and estimate the installation position of the current time according to the motion speed relationship between the radars.

[0117] It should be noted that the foregoing explanation of the embodiments of the multi-millimeter wave radar fast calibration method is also applicable to the multi-millimeter wave radar fast calibration device of this embodiment, and will not be elaborated here.

[0118] The multi-millimeter wave radar fast calibration device proposed according to the embodiments of the present application constructs a first motion model of the millimeter wave radar through the Doppler velocity and spatial coordinates of the point cloud, extracts the static target point cloud from the point cloud data of the multi-millimeter wave radar, estimates the motion speed of each millimeter wave radar according to the static target point cloud, and then constructs a second motion model of the millimeter wave radar based on the external parameters between the radars, estimates the motion speed relationship between the radars together with the motion speed of the millimeter wave radar, obtains the motion parameters of the target platform where the multi-millimeter wave radars are located, and calibrates the external parameters between the multi-millimeter wave radars based on the motion data of the target platform where the multi-millimeter wave radars are located and the motion speed relationship between the radars. Without relying on the cooperation of the scene target, it realizes fast and accurate online external parameter calibration of multiple millimeter wave radars based on the speed of the radar during the motion process, effectively ensuring the long-term accuracy and reliability of the unmanned system.

[0119] Figure 4 The following is a schematic structural diagram of an electronic device provided by the embodiments of the present application. The electronic device may include:

[0120] A memory 301, a processor 302, and a computer program stored on the memory 301 and executable on the processor 302.

[0121] When the processor 302 executes the program, it implements the multi-millimeter wave radar fast calibration method provided in the foregoing embodiments.

[0122] Further, the electronic device further includes:

[0123] A communication interface 303 for communication between the memory 301 and the processor 302.

[0124] The memory 301 is used to store a computer program executable on the processor 302.

[0125] The memory 301 may include a high-speed RAM (Random Access Memory) memory and may also include a non-volatile memory, such as at least one disk memory.

[0126] If the memory 301, the processor 302, and the communication interface 303 are implemented independently, the communication interface 303, the memory 301, and the processor 302 can be interconnected via a bus and communicate with each other. The bus can be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component) bus, an EISA (Extended Industry Standard Architecture) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 4 only a thick line is used to represent it in the figure, but it does not mean that there is only one bus or one type of bus.

[0127] Optionally, in a specific implementation, if the memory 301, the processor 302, and the communication interface 303 are integrated on a single chip, the memory 301, the processor 302, and the communication interface 303 can communicate with each other through an internal interface.

[0128] The processor 302 may be a CPU (Central Processing Unit), or an ASIC (Application Specific Integrated Circuit), or one or more integrated circuits configured to implement the embodiments of the present application.

[0129] In the description of this specification, the description with reference to terms such as "one embodiment", "some embodiments", "example", "specific example", or "some examples" means that the specific features, structures, materials, or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representations of the above terms are not necessarily directed to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described can be combined in any one or N embodiments or examples in a suitable manner. In addition, without contradiction, those skilled in the art can combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples.

[0130] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the quantity of the technical features indicated. Thus, features defined with "first" and "second" may explicitly or implicitly include at least one such feature. In the description of this application, the meaning of "N" is at least two, such as two, three, etc., unless otherwise specifically defined.

[0131] Any process or method description shown in a flowchart or described otherwise herein may be understood to represent a module, segment, or portion of code including one or N executable instructions for implementing a customized logical function or process. The scope of the preferred embodiments of this application includes additional implementations, where functions may be executed in a substantially simultaneous manner or in a reverse order according to the functions involved, rather than in the order shown or discussed, which should be understood by those skilled in the technical field to which the embodiments of this application pertain.

[0132] It should be understood that each part of this application can be implemented by hardware, software, firmware, or a combination thereof. In the above embodiments, the steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, any one or a combination of the following well-known technologies in the art can be used: discrete logic circuits with logic gate circuits for implementing logical functions on data signals, application-specific integrated circuits with appropriate combinational logic gate circuits, programmable gate arrays, field-programmable gate arrays, etc.

[0133] Those of ordinary skill in the art of this technology can understand that all or part of the steps carried by the methods for implementing the above embodiments can be completed by instructing relevant hardware through a program. The above program can be stored in a computer-readable storage medium. When the program is executed, it includes one or a combination of the steps of the method embodiments.

[0134] Although the embodiments of this application have been shown and described above, it can be understood that the above embodiments are exemplary and should not be construed as limiting this application. Those of ordinary skill in the art can make changes, modifications, substitutions, and variations to the above embodiments within the scope of this application.

Claims

1. A fast calibration method for multi-millimeter wave radar, characterized in that, It includes the following steps: Construct a first motion model of the millimeter-wave radar based on the Doppler velocity and spatial coordinates of the point cloud; Extract stationary target point clouds from the point cloud data of multiple millimeter-wave radars, and estimate the motion speeds of the millimeter-wave radars based on the first motion model and the stationary target point clouds extracted by each millimeter-wave radar; Construct a second motion model of the millimeter-wave radar based on the extrinsic parameters between radars, and estimate the motion speed relationship between radars based on the second motion model and the motion speeds of the millimeter-wave radars; Obtain the motion parameters of the target platform where the multiple millimeter-wave radars are located, and calibrate the extrinsic parameters between the millimeter-wave radars based on the motion data of the target platform where the multiple millimeter-wave radars are located and the motion speed relationship between radars; 2. The multi-millimeter wave radar rapid calibration method according to claim 1, characterized in that The expression of the first motion model is: Among them, are the spatial coordinates of all stationary target point clouds under radar i, is the horizontal azimuth angle, is the elevation angle, is the Doppler velocity; is the motion velocity of radar i in its own radar coordinate system, is the horizontal component of the motion velocity of radar i in its own radar coordinate system, is the vertical component of the motion velocity of radar i in its own radar coordinate system; N i,s is the number of stationary target point clouds detected by radar i.

3. The multi-millimeter wave radar rapid calibration method according to claim 1, wherein The extracting stationary target point clouds from the point cloud data of multiple millimeter-wave radars includes: Downsample the point cloud data of the multiple millimeter-wave radars based on the farthest point sampling algorithm; Cluster and group the point cloud data of the multiple millimeter-wave radars based on the density of the point cloud; Select stationary target point clouds from the cluster groups of each millimeter-wave radar according to the downsampled point cloud; 4. The multi-millimeter wave radar rapid calibration method according to claim 3, wherein, The farthest point sampling algorithm includes: Select a random point from the point cloud data as the initial sampling point; For each unselected point, calculate the distances between the unselected point and all points in the selected set, and take the target distance among all distances as the distance between the unselected point and the selected set; Select new sampling points from the unselected points according to the distance between the unselected point and the selected set until the sampling stop condition is met; 5. The multi-millimeter wave radar rapid calibration method according to claim 3, characterized in that The selecting stationary target point clouds from the cluster groups of each millimeter-wave radar according to the downsampled point cloud includes: Estimate the motion speeds of the millimeter-wave radars according to the downsampled point cloud; Speculate the Doppler velocities of the point clouds of the millimeter-wave radars according to the motion speeds of the millimeter-wave radars, and determine the stationary target point clouds in the cluster groups according to the Doppler velocities; Filter the cluster groups with the number of the stationary target point clouds less than the filtering threshold; 6. The multi-millimeter wave radar rapid calibration method according to claim 1, characterized in that The estimating the motion speeds of the millimeter-wave radars based on the first motion model and the stationary target point clouds extracted by each millimeter-wave radar includes: Obtain the spatial coordinates and Doppler velocities of the stationary target point clouds extracted by the millimeter-wave radars; Based on the spatial coordinates and Doppler velocities of the stationary target point clouds, fit the first motion model by the least squares method to obtain the motion speeds of the millimeter-wave radars; 7. The rapid calibration method for multi-millimeter wave radar according to claim 1, characterized in that The expression of the second motion model is: where α i is the installation angle of radar i; ω e is the angular velocity of the target platform where the multi-millimeter wave radar is located in the two-dimensional plane; is the motion speed of the reference radar in its own radar coordinate system; (x i , y i ) is the position of radar i in the reference coordinate system.

8. The multi-millimeter wave radar rapid calibration method according to claim 1, wherein The calibrating the extrinsic parameters between the millimeter-wave radars based on the motion data of the target platform where the multiple millimeter-wave radars are located and the motion speed relationship between radars includes: Identify the angular velocity in the motion data; If the angular velocity is less than or equal to the angular velocity threshold, estimate the current installation angle according to the motion speed relationship between radars, and take the previously estimated installation position as the current installation position; If the angular velocity is greater than the angular velocity threshold, take the previously estimated installation angle as the current installation angle, and estimate the current installation position according to the motion speed relationship between radars; 9. A multi-millimeter wave radar rapid calibration device, characterized in that, It includes: A construction module for constructing a first motion model of a millimeter-wave radar according to the Doppler velocity and spatial coordinates of a point cloud; An extraction module for extracting stationary target point clouds from the point cloud data of multiple millimeter-wave radars, and estimating the motion speeds of the millimeter-wave radars according to the first motion model and the stationary target point clouds extracted by the millimeter-wave radars; An estimation module for constructing a second motion model of the millimeter-wave radar based on the external parameters between the radars, and estimating the motion speed relationship between the radars based on the second motion model and the motion speeds of the millimeter-wave radars; A calibration module for obtaining the motion parameters of the target platform where the multiple millimeter-wave radars are located, and calibrating the external parameters between the millimeter-wave radars based on the motion data of the target platform where the multiple millimeter-wave radars are located and the motion speed relationship between the radars; 10. An electronic device, characterized in that, Including: A memory, a processor, and a computer program stored on the memory and executable on the processor, and the processor executes the program to implement the multi-millimeter-wave radar rapid calibration method according to any one of claims 1-8.

Images