Multi-radar calibration method and device
By preprocessing the radar point clouds of multiple radars and calculating the overall target residuals, the problem of multiple radar calibration under no specific target object is solved, and attitude parameter calibration and alignment consistency between multiple radars is achieved.
Patent Information
- Application Number
- CN202411997125.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-12-31
- Publication Date
- 2025-04-15
AI Technical Summary
The prior art is difficult to calibrate attitude parameters of multiple radars simultaneously without a specific target object, resulting in accumulation errors between multiple radars and unable to achieve alignment.
By preprocessing the radar point clouds acquired by multiple radars, the target category point clouds under the coordinate system of the target equipment are determined, and the overall target residuals between multiple radars are calculated based on the target category point clouds. Then, when the external installation parameters of the reference radar are fixed, the target attitude parameters of the calibration radar are calibrated according to the target overall residual.
It is realized that the attitude parameters of multiple radars are calibrated without a specific target object, avoiding accumulation errors and ensuring alignment consistency between multiple radars.
Smart Images
Figure CN119828088A_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to the technical field of radar calibration, and in particular to a multi-radar calibration method and device. Background Art
[0002] Radar calibration refers to a series of calibration processes performed on the radar system to ensure the accuracy and reliability of its measurement data.
[0003] In autonomous vehicles, there are usually multiple radars (such as lidar), and the consistency between multiple radars greatly affects the consistency of vehicle perception and positioning, because the fusion of multiple radar point clouds or targets depends on the consistency between multiple radars. Lidar2lidar calibration is an important method of multi-radar joint calibration in autonomous driving systems, mainly used to calibrate the relative position and attitude between two radars. The purpose of calibration is to unify the coordinate systems of the two radars so that data fusion and processing can be accurately performed in the development of autonomous driving algorithms.
[0004] At present, lidar2lidar calibration is mainly used for the calibration of a single radar, the calibration of multiple radars based on specific targets, and the calibration of two radars without specific targets. However, the calibration of a single radar cannot calibrate multiple radars at the same time. The calibration of multiple radars based on specific targets is constrained by the specific targets. It takes a long time to make a qualified target (such as a calibration plate), and it is not convenient to move and is limited by the site. The calibration of two radars without specific targets only meets the calibration between the two radars. If it is applied to the calibration of multiple radars, due to the error between the two radars, the accumulated error will be generated between the multiple radars, which will make it impossible to align the multiple radars. Summary of the invention
[0005] In order to solve the above technical problem or at least partially solve the above technical problem, the present disclosure provides a multi-radar calibration method and device to achieve simultaneous calibration of multiple radars without specific targets.
[0006] In a first aspect, the present disclosure provides a multi-radar calibration method, comprising:
[0007] Preprocessing radar point clouds acquired by multiple radars to determine target category point clouds of the radar point clouds in a target device coordinate system; wherein the multiple radars are installed on the target device and include a reference radar and at least one radar to be calibrated, and the target category point clouds include non-ground point clouds and / or ground point clouds;
[0008] Determine the target overall residual of the multiple radars according to the target category point cloud; wherein the target overall residual is used to represent a comprehensive measure of the target attitude parameter error between every two of the multiple radars, and the target attitude parameter includes at least one of a yaw angle, a pitch angle, and a roll angle;
[0009] When the external installation parameters of the reference radar and the external installation parameters of the radar to be calibrated except the target attitude parameters remain unchanged, the target attitude parameters of each radar to be calibrated are calibrated according to the target overall residual; wherein the external installation parameters include position parameters and attitude parameters.
[0010] In a second aspect, the present disclosure provides a multi-radar calibration device, comprising:
[0011] An image acquisition module, used to acquire images around the vehicle;
[0012] A parking space line segment determination module, used to determine the parking space line segment according to the vehicle surrounding image;
[0013] A parking space correction information determination module, used to determine parking space correction information according to the parking space type and the parking space line segment; wherein the parking space correction information indicates parking space heading information;
[0014] The multi-radar calibration module is used to correct the parking space to be corrected at the current moment according to the parking space correction information to obtain the first corrected parking space at the current moment.
[0015] In a third aspect, the present disclosure provides an electronic device, including:
[0016] memory and one or more processors;
[0017] Among them, the memory is communicatively connected to the one or more processors, and the memory stores instructions that can be executed by the one or more processors. When the instructions are executed by the one or more processors, the electronic device is used to implement the multi-radar calibration method described in the first aspect of the present disclosure.
[0018] In a fourth aspect, the present disclosure provides a computer-readable storage medium having computer-executable instructions stored thereon. When the computer-executable instructions are executed by a computing device, they can be used to implement the multi-radar calibration method described in the first aspect of the present disclosure.
[0019] Compared with the prior art, the technical solution provided by the embodiments of the present disclosure has the following advantages:
[0020] The technical solution provided by the embodiment of the present disclosure pre-processes the radar point clouds acquired by multiple radars to obtain target category point clouds containing non-ground point clouds and / or ground point clouds in the target device coordinate system, and determines the target overall residuals of all radars based on the target category point clouds, and the target overall residuals characterize the comprehensive measure of the target attitude parameter errors between every two radars in all radars. Therefore, in the case of fixing the external installation parameters of the reference radar and the external installation parameters of the radar to be calibrated except the target attitude parameters, the target attitude parameters of all radars to be calibrated can be calibrated at the same time based on the target overall residuals. Therefore, the technical solution disclosed in the present disclosure can calibrate the attitude parameters of multiple radars at the same time without a specific target object. BRIEF DESCRIPTION OF THE DRAWINGS
[0021] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.
[0022] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.
[0023] Figure 1 A schematic diagram of a flow chart of a multi-radar calibration method provided in an embodiment of the present disclosure;
[0024] Figure 2 A functional module block diagram of a multi-radar calibration system provided by an embodiment of the present disclosure;
[0025] Figure 3 A functional module block diagram of a multi-radar calibration device provided in an embodiment of the present disclosure;
[0026] Figure 4 A schematic diagram of the structure of an electronic device suitable for implementing the embodiment of the present disclosure is provided for an embodiment of the present disclosure. DETAILED DESCRIPTION
[0027] In order to more clearly understand the above-mentioned objectives, features and advantages of the present disclosure, the scheme of the present disclosure will be further described below. It should be noted that the embodiments of the present disclosure and the features in the embodiments can be combined with each other without conflict.
[0028] In the following description, many specific details are set forth to facilitate a full understanding of the present disclosure, but the present disclosure may also be implemented in other ways different from those described herein; it is obvious that the embodiments in the specification are only part of the embodiments of the present disclosure, rather than all of the embodiments.
[0029] The radars involved in the embodiments of the present disclosure include but are not limited to laser radar, microwave radar and ultrasonic radar. Among them, laser radar is a sensor that uses laser for distance measurement and target recognition, and is widely used in fields such as autonomous driving, geographic mapping and environmental monitoring. The main purpose of laser radar calibration is to determine the precise position and direction of the laser radar relative to the coordinate system of other devices such as vehicles or robots, and to calibrate the internal parameters of the laser radar to ensure the accuracy of the measurement data. The parameters that need to be calibrated for laser radar calibration include the external installation parameters (external parameters) of the laser radar, such as position parameters (such as X, Y, Z coordinates) and attitude parameters (such as yaw angle, pitch angle and roll angle), as well as the internal parameters (internal parameters) of the laser radar, such as the pointing angle and beam width of the laser beam. Among them, the external parameter calibration of the laser radar is a key step to ensure that the laser radar data can be accurately reflected in the vehicle coordinate system, which is crucial for perception and positioning in the autonomous driving system. The embodiments of the present disclosure mainly describe the calibration of the attitude parameters in the radar external parameters.
[0030] For the calibration of radar attitude parameters, in the related art, radar calibration based on specific targets requires placing a calibration plate of known geometric shape in the scene, using multiple radars to collect data from the calibration plate at the same time, and then processing these data through algorithms to calculate the attitude relationship between multiple radars. However, this calibration method requires the production of a calibration plate, which is time-consuming and labor-intensive, and the calibration plate is not convenient to move. It is restricted by the site and cannot meet the radar calibration in any scene. In addition, radar calibration without specific targets mainly includes the following steps: ground plane alignment: extracting point clouds from ground point clouds and non-ground point clouds and segmenting them, using ground normals for ground registration, and obtaining roll angle, pitch angle and z-axis translation as initial extrinsic parameters. Registration of non-ground feature points: traversing the yaw angle of the transformed non-ground point cloud, automatically calculating the distance between the two radar closest points, and obtaining the yaw angle with the minimum distance. Subsequently, the calibration accuracy of the yaw angle is improved through the normal iterative nearest point (NICP) and octree-based optimization. However, this calibration method is only applicable to the calibration between two radars. When calibrating more than two radars at the same time, the error between the two radars will cause the accumulation error among multiple radars, making it impossible to align multiple radars.
[0031] In view of the above technical problems, the embodiments of the present disclosure provide a multi-radar calibration method. The method is applicable to the case where two or more radars are calibrated simultaneously without a specific target. Based on the multi-radar calibration method provided by the embodiments of the present disclosure, the attitude parameters of multiple radars can be calibrated according to non-ground point clouds and / or ground point clouds, breaking through the constraint that only two radars can be calibrated without a specific target in the related art.
[0032] Figure 1A schematic diagram of a multi-radar calibration method provided by an embodiment of the present disclosure. Figure 1 As shown, the method comprises the following steps:
[0033] S110 , preprocessing radar point clouds acquired by multiple radars to determine target category point clouds of the radar point clouds in a target device coordinate system.
[0034] Among them, multiple radars are installed on the target device and include a reference radar and at least one radar to be calibrated, and the target category point cloud includes non-ground point cloud and / or ground point cloud. The reference radar can be any one of the multiple radars. Preferably, a radar with accurate internal and external parameters is selected as the reference radar. In practical applications, the radar that has not been replaced is used as the reference radar. The radar point cloud of the reference radar and / or the radar to be calibrated contains a corresponding radar identifier to distinguish the reference radar from the radar to be calibrated, so as to subsequently configure the parameters of the reference radar and the radar to be calibrated. The target device includes but is not limited to devices with multi-radar scanning functions such as autonomous driving vehicles and robots.
[0035] In this embodiment, the radar point cloud of each radar in its own coordinate system is converted to the target device coordinate system to unify the coordinate system so as to calibrate the target attitude parameters of each radar to be calibrated.
[0036] The target category point cloud can be set according to the target attitude parameters to be calibrated as required. For example, when the target attitude parameter is the yaw angle, the target category point cloud is a non-ground point cloud; when the target attitude parameter includes the pitch angle and the roll angle, the target category point cloud is a ground point cloud; when the target attitude parameter includes the yaw angle, the pitch angle and the roll angle, the target category point cloud includes both non-ground point clouds and ground point clouds. Therefore, this embodiment can calibrate only the yaw angle of each radar to be calibrated according to the non-ground point cloud; it can also calibrate only the pitch angle and the roll angle of each radar to be calibrated according to the ground point cloud; it can also calibrate the yaw angle, pitch angle and roll angle of each radar to be calibrated according to the non-ground point cloud and the ground point cloud at the same time. The specific settings can be made according to the actual situation.
[0037] In some embodiments, radar point clouds acquired by multiple radars are preprocessed, including: converting the radar point clouds of the multiple radars from a radar coordinate system to a target device coordinate system; and acquiring non-ground point clouds and / or ground point clouds from the radar point clouds in the target device coordinate system.
[0038] Exemplarily, the target device is an autonomous driving vehicle, and the target device can convert the radar point cloud of each radar into the target device coordinate system according to the initial external parameters of each radar. Among them, the target device coordinate system takes the center of the rear axle of the vehicle as the origin, the z-axis is perpendicular to the ground, the x-axis is parallel to the ground and consistent with the direction of the front of the vehicle, and the y-axis is parallel to the ground and facing the left side of the vehicle body. Extract the Z coordinate of each point in the radar point cloud in the target device coordinate system; filter out points whose Z coordinates are lower than the preset height threshold to obtain a non-ground point cloud. The random consistency check (Ransac) plane fitting method is used to extract the ground point cloud from the radar point cloud in the target device coordinate system. The Ransac plane fitting method can extract all the points on the ground point cloud by setting a relatively loose threshold in the z-axis direction, without requiring the ground to be strictly flat.
[0039] S120: Determine overall target residuals of multiple radars according to the target category point cloud.
[0040] The target overall residual is used to represent a comprehensive measure of the target attitude parameter error between every two radars in multiple radars, and the target attitude parameter includes at least one of the yaw angle, the pitch angle and the roll angle.
[0041] In one embodiment, the target category point cloud includes a non-ground point cloud, and the target attitude parameter includes a yaw angle; based on the target category point cloud, determining the target overall residual of multiple radars, including: determining a first residual between every two radars based on the non-ground point cloud; wherein the first residual is used to represent the error of the yaw angle between the two radars; and summing all the first residuals to obtain the target overall residual.
[0042] Exemplarily, each two radars include a first radar and a second radar. All first points are determined from the non-ground point cloud of the first radar; wherein the first point has a nearest neighbor point in the non-ground point cloud of the second radar; according to the coordinates of the first point and the coordinates of the nearest neighbor point, the angle difference between the first point and the corresponding nearest neighbor point is determined; the squares of the angle differences are summed to obtain a first residual. Then, all first residuals are summed to obtain a target overall residual.
[0043] Specifically, take a point p in the non-ground point cloud of the i-th radar, the coordinates are
[0044] (xp, yp), find the coordinates (xq, yq) of the nearest neighbor point q in the non-ground point cloud of the jth radar. In order to ensure that the nearest neighbor point can be found at different distances, the search radius of the nearest neighbor is set to the arc length corresponding to the central angle of 4 degrees on the circumference, where the circumference is the distance from point p to the origin; calculate the angle difference between the i-th radar point p and the j-th radar point q: The square sum of the angle differences corresponding to point p in all the i-th radars that can find the nearest neighbor point is obtained to obtain the first residual (yaw residual) between the i-th radar and the j-th radar, which is defined as:
[0045]
[0046] The residuals of all radar pairs are summed to obtain the overall residual of the target:
[0047]
[0048] In another embodiment, the target category point cloud includes a ground point cloud, and the target attitude parameters include a pitch angle and a roll angle; based on the target category point cloud, determining the target overall residual of multiple radars, including: determining the second residual between every two radars based on the ground point cloud; wherein the second residual is used to represent the error of the pitch angle and the roll angle between the two radars; and summing all the second residuals to obtain the target overall residual.
[0049] Exemplarily, each two radars include a first radar and a second radar. The ground point cloud of each radar is triangulated to obtain multiple triangular planes corresponding to each ground point cloud; all second points are determined from the ground point cloud of the first radar; wherein the projection of the second point in the Z-axis direction is located in the triangular plane corresponding to the ground point cloud of the second radar; the distance error from the second point to the corresponding triangular plane is determined; the squares of the distance errors are summed to obtain the second residual. Then, all the second residuals are summed to obtain the overall residual of the target.
[0050] Specifically, all ground point clouds are processed by triangulation to obtain a triangular mesh of each ground point cloud, where the triangular mesh includes multiple triangular planes Δ. Take a point p in the ground point cloud of the i-th radar and determine whether the point p falls in a triangular plane Δ in the ground point cloud of the j-th radar in the z-axis direction. If it falls, the distance error is calculated using the distance from point p to the triangular plane Δ, which is defined as E p-＞Δ ; The square sum of the distance errors corresponding to all points p in the ground point cloud of the i-th radar that can be projected onto the triangular plane Δ of the j-th radar is obtained to obtain the second residual (roll-pitch residual) between the i-th radar and the j-th radar, which is defined as:
[0051]
[0052] The residuals of all radar pairs are summed to obtain the overall residual of the target:
[0053]
[0054] S130, calibrating the target attitude parameters of each radar to be calibrated according to the target overall residual when the external installation parameters of the reference radar and the external installation parameters of the radar to be calibrated except the target attitude parameters remain unchanged.
[0055] Among them, the external installation parameters include position parameters and posture parameters.
[0056] The reference radar is a standard radar. When calibrating the target attitude parameters of the radar to be calibrated, all external parameters of the reference radar remain unchanged. At the same time, the external parameters of the radar to be calibrated, except for the target attitude parameters, remain unchanged. The final calibration value of the target attitude parameters is output by optimizing the target attitude parameters.
[0057] In one embodiment, the target attitude parameters include a yaw angle; the target attitude parameters of each radar to be calibrated are calibrated according to the target overall residual, including: determining a first gradient vector according to the target overall residual; wherein the first gradient vector is composed of partial derivatives of the target overall residual with respect to the yaw angle of each radar to be calibrated; according to the first gradient vector and an iterative method of gradient descent, the yaw angle of each radar to be calibrated is iterated; when the yaw angle of each radar to be calibrated meets a first preset iteration termination condition, the yaw angle of each radar to be calibrated obtained in the last iteration is used as a yaw angle calibration value.
[0058] Exemplarily, the yaw angle of each radar to be calibrated satisfies a first preset iteration termination condition, which may include: a change in a yaw angle vector formed by the yaw angles of each radar to be calibrated obtained in two adjacent iterations is less than a first preset angle threshold (eg, 0.1 degrees).
[0059] Specifically, the yaw angle is Rz, and the above target overall residual E can be γ Considered as a function combination of the Rz of the radar to be calibrated Right now n is the number of radars to be calibrated. According to the target overall residual E γ , determine the first gradient vector as:
[0060]
[0061] is the yaw angle vector of each radar to be calibrated obtained in the kth iteration.
[0062] The iteration formula is: where α k is the k-th iteration step size. k+1 =0.75*α k The first preset iteration termination condition is: Therefore, when the change in the yaw angle vector formed by the yaw angles of each radar to be calibrated obtained in two consecutive iterations is less than 0.1 degree, the iteration is stopped, and each yaw angle in the yaw angle vector obtained in the last iteration is used as the yaw angle calibration value of each radar to be calibrated.
[0063] Based on the above implementation, in one example, an optimizer such as ceres or g2o can be used to iteratively optimize the Rz of the radar to be calibrated. For the radar to be calibrated, X, Y, Z, Rx, and Ry are fixed; for the reference radar, X, Y, Z, Rx, Ry, and Rz are fixed. The target overall residual E γ Input to the optimizer, and the optimizer can directly output the Rz of all radars to be calibrated.
[0064] In another embodiment, the pitch angle and the roll angle have a strong correlation, so the pitch angle and the roll angle can be calculated at the same time. The target attitude parameters include the pitch angle and the roll angle; according to the target overall residual, the target attitude parameters of each radar to be calibrated are calibrated, including: according to the target overall residual, a second gradient vector is determined; wherein the second gradient vector is composed of the partial derivative of the target overall residual with respect to the pitch angle of each radar to be calibrated, and the partial derivative of the target overall residual with respect to the roll angle of each radar to be calibrated; according to the second gradient vector and the iterative method of gradient descent, the pitch angle and roll angle of each radar to be calibrated are iterated respectively; when the pitch angle of each radar to be calibrated meets the second preset iteration termination condition, the pitch angle of each radar to be calibrated obtained in the last iteration is used as the pitch angle calibration value; when the roll angle of each radar to be calibrated meets the third preset iteration termination condition, the roll angle of each radar to be calibrated obtained in the last iteration is used as the roll angle calibration value.
[0065] Exemplarily, the pitch angle of each radar to be calibrated satisfies the second preset iteration termination condition, which may include: the change amount of the pitch angle vector formed by the pitch angles of each radar to be calibrated obtained in two adjacent iterations is less than the second preset angle threshold (for example, 0.1 degrees). The roll angle of each radar to be calibrated satisfies the third preset iteration termination condition, which may include: the change amount of the roll angle vector formed by the roll angles of each radar to be calibrated obtained in two adjacent iterations is less than the third preset angle threshold (for example, 0.1 degrees).
[0066] Specifically, the roll angle is Ry, the pitch angle is Rx, and the above target overall residual E can be αβ Treated as a function combination of Ry and Rx of the radar to be calibrated Right now n is the number of radars to be calibrated. According to the target overall residual E αβ , determine the second gradient vector as:
[0067]
[0068] is the roll angle vector of each radar to be calibrated obtained in the kth iteration; is the elevation angle vector of each radar to be calibrated obtained in the kth iteration.
[0069] The iteration formula is: where α k is the k-th iteration step size. k+1 =0.75*α k The second preset iteration termination condition is: The third preset iteration termination condition is: Thus, when the change amount of the roll angle vector composed of the roll angles of each radar to be calibrated obtained by two adjacent iterations is less than 0.1 degrees, the iteration is stopped, and each roll angle in the roll angle vector obtained by the last iteration is used as the roll angle calibration value of each radar to be calibrated. When the change amount of the pitch angle vector composed of the pitch angles of each radar to be calibrated obtained by two adjacent iterations is less than 0.1 degrees, the iteration is stopped, and each pitch angle in the pitch angle vector obtained by the last iteration is used as the pitch angle calibration value of each radar to be calibrated.
[0070] Based on the above implementation, in one example, an optimizer such as ceres or g2o can be used to iteratively optimize the Rx and Ry of the radar to be calibrated. For the radar to be calibrated, X, Y, Z, and Rz are fixed; for the reference radar, X, Y, Z, Rx, Ry, and Rz are fixed. The target overall residual E αβ Input to the optimizer, and the optimizer can directly output the Rx and Ry of all radars to be calibrated.
[0071] The multi-radar calibration method provided in this embodiment pre-processes the radar point clouds acquired by multiple radars to obtain target category point clouds containing non-ground point clouds and / or ground point clouds in the target device coordinate system, and determines the target overall residuals of all radars based on the target category point clouds, and the target overall residuals characterize the comprehensive measure of the target attitude parameter errors between every two radars in all radars. Therefore, in the case of fixing the external installation parameters of the reference radar and the external installation parameters of the radar to be calibrated except the target attitude parameters, the target attitude parameters of all radars to be calibrated can be calibrated simultaneously based on the target overall residuals. Therefore, the technical solution disclosed in the present invention can calibrate the attitude parameters of multiple radars at the same time without a specific target object.
[0072] Based on the above embodiments, in some embodiments, the yaw angle, pitch angle and roll angle may be calibrated simultaneously.
[0073] Specifically, the target attitude parameter includes a first attitude parameter and a second attitude parameter, the first attitude parameter includes a yaw angle, and the second attitude parameter includes a pitch angle and a roll angle; the target overall residual includes a first overall residual and a second overall residual, the first overall residual is used to represent a comprehensive measure of the yaw angle error between every two radars, and the second overall residual is used to represent a comprehensive measure of the pitch angle and roll angle errors between every two radars; according to the target overall residual, the target attitude parameters of each radar to be calibrated are calibrated, including: according to the first overall residual, the yaw angle of each radar to be calibrated is calibrated; according to the second overall residual, the pitch angle and roll angle of each radar to be calibrated are calibrated. Among them, the first overall residual is the above-mentioned target overall residual E γ , the first population residual is the above target population residual E αβ The specific calibration method of the yaw angle, pitch angle and roll angle can refer to the above embodiment and will not be repeated here.
[0074] In some embodiments, the fixed external parameters of the radars to be calibrated can be updated based on the calibrated yaw angle and the calibrated pitch angle and roll angle, so that after the external parameters of the radar to be calibrated are updated, the yaw angle, pitch angle and roll angle are recalibrated. In this way, the yaw angle, pitch angle and roll angle are iteratively optimized until the yaw angle, pitch angle and roll angle are stable. Thus, the accuracy of the attitude parameter calibration of the radar to be calibrated is further improved.
[0075] Exemplarily, based on the calibrated yaw angles of each radar to be calibrated, the yaw angles of the configured radars to be calibrated are updated to iteratively update the pitch angles and roll angles of each radar to be calibrated until the errors of the pitch angles and roll angles between two iterative updates are both less than a first error threshold; and / or,
[0076] Based on the calibrated pitch angle and roll angle of each radar to be calibrated, the pitch angle and roll angle of the configured radar to be calibrated are updated, and the yaw angle of each radar to be calibrated is iteratively updated until the error of the yaw angle between two iterative updates is less than a second error threshold.
[0077] Based on the above embodiments, the present disclosure provides a multi-radar calibration system. Figure 2 As shown, the multi-radar calibration system includes: a preprocessing module, a heading angle optimization module and a roll and pitch angle optimization module.
[0078] Among them, the preprocessing module receives the radar point cloud collected by the reference radar and the radar to be calibrated, and converts each radar point cloud to the vehicle body coordinate system through point cloud coordinate system conversion; the radar point cloud in the vehicle body coordinate system is divided into non-ground point cloud (including reference radar non-ground point cloud and calibration radar non-ground point cloud) and ground point cloud (including reference radar ground point cloud and calibration radar ground point cloud) through point cloud segmentation, and the non-ground point cloud is sent to the heading angle optimization module, and the ground point cloud is sent to the roll and pitch angle optimization module. The heading angle optimization module optimizes the heading angle (i.e., the yaw angle) according to the non-ground point cloud. Specifically, the heading angle optimization module calculates the target overall residual E in the above embodiment based on the non-ground point cloud. γ , and the optimizer is based on the target overall residual E γ The input value of the calibrated radar is used to output the heading angle of each radar to be calibrated, wherein the X, Y, Z, Rx and Ry of the radar to be calibrated and the X, Y, Z, Rx, Ry and Rz of the reference radar are configured in the optimizer. The roll and pitch angle optimization module optimizes the roll angle and pitch angle according to the ground point cloud. Specifically, the roll and pitch angle optimization module calculates the target overall residual E in the above embodiment according to the ground point cloud. αβ , and the optimizer is based on the target overall residual E αβ The input value is used to output the roll angle and pitch angle of each radar to be calibrated, wherein the X, Y, Z and Rz of the radar to be calibrated and the X, Y, Z, Rx, Ry and Rz of the reference radar are configured in the optimizer. In addition, in order to obtain the optimal yaw angle, roll angle and pitch angle, the heading angle optimization module and the roll and pitch angle optimization module update the attitude parameters configured in each other's optimizer according to the attitude parameters obtained by each module, and iterate each other several times, so that the errors of the pitch angle and the roll angle between the two iterative updates are less than the first error threshold, and the error of the yaw angle between the two iterative updates is less than the second error threshold.
[0079] Figure 3 This is a functional module block diagram of a multi-radar calibration device provided in an embodiment of the present disclosure. Figure 3 As shown, the multi-radar calibration device includes a preprocessing module 201, an overall residual determination module 202 and an attitude parameter calibration module 203.
[0080] The preprocessing module 201 is used to preprocess the radar point clouds acquired by the multiple radars to determine the target category point clouds of the radar point clouds in the target device coordinate system; wherein the multiple radars are installed on the target device and include a reference radar and at least one radar to be calibrated, and the target category point clouds include non-ground point clouds and / or ground point clouds;
[0081] The overall residual determination module 202 is used to determine the overall residual of the target of multiple radars according to the target category point cloud; wherein the overall residual of the target is used to represent the comprehensive measure of the target attitude parameter error between each two radars in the multiple radars, and the target attitude parameter includes at least one of the yaw angle, the pitch angle and the roll angle;
[0082] The attitude parameter calibration module 203 is used to calibrate the target attitude parameters of each radar to be calibrated according to the target overall residual when the external installation parameters of the reference radar and the external installation parameters of the radar to be calibrated except the target attitude parameters remain unchanged; wherein the external installation parameters include position parameters and attitude parameters.
[0083] In some embodiments, the preprocessing module 201 is specifically used for:
[0084] Convert radar point clouds of multiple radars from radar coordinate system to target device coordinate system;
[0085] Obtain non-ground point cloud and / or ground point cloud from the radar point cloud in the target device coordinate system.
[0086] In some embodiments, the target category point cloud includes a non-ground point cloud; the pre-processing module 201 is specifically used for:
[0087] Extract the Z coordinate of each point in the radar point cloud in the target device coordinate system;
[0088] Points whose Z coordinates are lower than the preset height threshold are filtered out to obtain a non-ground point cloud.
[0089] In some embodiments, the target category point cloud includes a non-ground point cloud, and the target attitude parameter includes a yaw angle; the overall residual determination module 202 includes:
[0090] A first residual determination unit, used to determine a first residual between every two radars according to the non-ground point cloud; wherein the first residual is used to represent an error in a yaw angle between the two radars;
[0091] The first overall residual determination unit is used to sum all the first residuals to obtain a target overall residual.
[0092] In some embodiments, every two radars include a first radar and a second radar; the first residual determination unit is specifically configured to:
[0093] Determine all first points from the non-ground point cloud of the first radar; wherein the first points have nearest neighboring points in the non-ground point cloud of the second radar;
[0094] Determine the angle difference between the first point and the corresponding nearest neighbor point according to the coordinates of the first point and the coordinates of the nearest neighbor point;
[0095] The squares of the angle differences are summed to obtain the first residual.
[0096] In some embodiments, the target attitude parameter includes a yaw angle; the attitude parameter calibration module 203 is specifically used for:
[0097] Determine a first gradient vector according to the target overall residual; wherein the first gradient vector is composed of partial derivatives of the target overall residual with respect to the yaw angle of each radar to be calibrated;
[0098] Iterating the yaw angle of each radar to be calibrated according to the first gradient vector and the iterative method of gradient descent;
[0099] When the yaw angle of each radar to be calibrated meets the first preset iteration termination condition, the yaw angle of each radar to be calibrated obtained in the last iteration is used as the yaw angle calibration value.
[0100] In some embodiments, the target category point cloud includes a ground point cloud; the preprocessing module 201 is specifically used for:
[0101] The ground point cloud is extracted from the radar point cloud in the target device coordinate system using a random consistency check plane fitting method.
[0102] In some embodiments, the target category point cloud includes a ground point cloud, and the target attitude parameters include a pitch angle and a roll angle; the overall residual determination module 202 includes:
[0103] A second residual determination unit is used to determine a second residual between every two radars according to the ground point cloud; wherein the second residual is used to represent the errors of the pitch angle and the roll angle between the two radars;
[0104] The second overall residual determination unit is used to sum all the second residuals to obtain a target overall residual.
[0105] In some embodiments, every two radars include a first radar and a second radar; and the second residual determination unit is specifically configured to:
[0106] Triangulate the ground point cloud of each radar to obtain multiple triangular planes corresponding to each ground point cloud;
[0107] Determine all second points from the ground point cloud of the first radar, wherein the projection of the second points in the Z-axis direction is located in the triangular plane corresponding to the ground point cloud of the second radar;
[0108] Determine the distance error from the second point to the corresponding triangle plane;
[0109] The squares of the distance errors are summed to obtain the second residual.
[0110] In some embodiments, the target attitude parameters include pitch angle and roll angle; the attitude parameter calibration module 203 is specifically used for:
[0111] Determine a second gradient vector according to the target overall residual; wherein the second gradient vector is composed of a partial derivative of the target overall residual with respect to the pitch angle of each radar to be calibrated, and a partial derivative of the target overall residual with respect to the roll angle of each radar to be calibrated;
[0112] According to the second gradient vector and the iterative method of gradient descent, the pitch angle and the roll angle of each radar to be calibrated are iterated respectively;
[0113] When the elevation angle of each radar to be calibrated meets the second preset iteration termination condition, the elevation angle of each radar to be calibrated obtained in the last iteration is used as the elevation angle calibration value;
[0114] When the roll angle of each radar to be calibrated meets the third preset iteration termination condition, the roll angle of each radar to be calibrated obtained in the last iteration is used as the roll angle calibration value.
[0115] In some embodiments, the target attitude parameter includes a first attitude parameter and a second attitude parameter, the first attitude parameter includes a yaw angle, and the second attitude parameter includes a pitch angle and a roll angle; the target overall residual includes a first overall residual and a second overall residual, the first overall residual is used to represent a comprehensive measure of the yaw angle error between every two radars, and the second overall residual is used to represent a comprehensive measure of the pitch angle and roll angle errors between every two radars;
[0116] The posture parameter calibration module 203 is specifically used for:
[0117] Calibrate the yaw angle of each radar to be calibrated according to the first overall residual;
[0118] The pitch angle and roll angle of each radar to be calibrated are calibrated according to the second overall residual.
[0119] In some embodiments, the apparatus further comprises a posture parameter optimization module for:
[0120] Based on the calibrated yaw angles of the radars to be calibrated, the yaw angles of the configured radars to be calibrated are updated to iteratively update the pitch angles and roll angles of the radars to be calibrated until the errors of the pitch angles and roll angles between two iterative updates are both less than a first error threshold; and / or,
[0121] Based on the calibrated pitch angle and roll angle of each radar to be calibrated, the pitch angle and roll angle of the configured radar to be calibrated are updated, and the yaw angle of each radar to be calibrated is iteratively updated until the error of the yaw angle between two iterative updates is less than a second error threshold.
[0122] The multi-radar calibration device disclosed in the above embodiments can execute the multi-radar calibration method disclosed in the above embodiments, and has the same or corresponding beneficial effects. To avoid repetition, it will not be described here.
[0123] An embodiment of the present disclosure further provides an electronic device, comprising: a memory and one or more processors; wherein the memory is communicatively connected to the one or more processors, and the memory stores instructions that can be executed by the one or more processors. When the instructions are executed by the one or more processors, the electronic device is used to implement the multi-radar calibration method described in any embodiment of the present disclosure.
[0124] Figure 4 Schematic diagram of the structure of an electronic device suitable for implementing the embodiments of the present disclosure. Figure 4 As shown, the electronic device 300 includes a central processing unit (CPU) 301, which can perform various processes in the aforementioned embodiments according to a program stored in a read-only memory (ROM) 302 or a program loaded from a storage part 308 into a random access memory (RAM) 303. In the RAM 303, various programs and data required for the operation of the electronic device 300 are also stored. The CPU 301, the ROM 302, and the RAM 303 are connected to each other via a bus 304. An input / output (I / O) interface 305 is also connected to the bus 304.
[0125] The following components are connected to the I / O interface 305: an input section 306 including a keyboard, a mouse, etc.; an output section 307 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 308 including a hard disk, etc.; and a communication section 309 including a network interface card such as a LAN card, a modem, etc. The communication section 309 performs communication processing via a network such as the Internet. A drive 310 is also connected to the I / O interface 305 as needed. A removable medium 311, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 310 as needed, so that a computer program read therefrom is installed into the storage section 308 as needed.
[0126] In particular, according to an embodiment of the present disclosure, the method described above can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program tangibly contained on a readable medium, and the computer program includes a program code for executing the aforementioned obstacle avoidance method. In such an embodiment, the computer program can be downloaded and installed from a network through the communication part 309, and / or installed from the removable medium 311.
[0127] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architecture, functions and operations of the devices, methods and computer program products according to various embodiments of the present disclosure. In this regard, each box in the road map or block diagram can represent a module, a program segment or a part of a code, and a part of a module, a program segment or a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flow chart, and the combination of boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.
[0128] The units or modules involved in the embodiments described in the present disclosure may be implemented by software or hardware. The units or modules described may also be set in a processor, and the names of these units or modules do not constitute limitations on the units or modules themselves in some cases.
[0129] In addition, the embodiment of the present disclosure further provides a computer-readable storage medium, which may be the computer-readable storage medium included in the device described in the above embodiment; or may be a computer-readable storage medium that exists independently and is not assembled into the device. The computer-readable storage medium stores computer-executable instructions, which, when executed by the computing device, can be used to implement the multi-radar calibration method described in any embodiment of the present disclosure.
[0130] It should be noted that, in this article, relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.
[0131] The above description is only a specific embodiment of the present disclosure, so that those skilled in the art can understand or implement the present disclosure. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present disclosure. Therefore, the present disclosure will not be limited to the embodiments described herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.
Claims
1. A multi-radar calibration method, characterized in that: include: Preprocessing radar point clouds acquired by multiple radars to determine target category point clouds of the radar point clouds in a target device coordinate system; wherein the multiple radars are installed on the target device and include a reference radar and at least one radar to be calibrated, and the target category point clouds include non-ground point clouds and / or ground point clouds; Determine the target overall residual of the multiple radars according to the target category point cloud; wherein the target overall residual is used to represent a comprehensive measure of the target attitude parameter error between every two of the multiple radars, and the target attitude parameter includes at least one of a yaw angle, a pitch angle, and a roll angle; When the external installation parameters of the reference radar and the external installation parameters of the radar to be calibrated except the target attitude parameters remain unchanged, the target attitude parameters of each radar to be calibrated are calibrated according to the target overall residual; wherein the external installation parameters include position parameters and attitude parameters.
2. The method according to claim 1, characterized in that Preprocess the radar point clouds acquired by multiple radars, including: Converting the radar point clouds of the multiple radars from the radar coordinate system to the target device coordinate system; The non-ground point cloud and / or the ground point cloud are acquired from the radar point cloud in the target device coordinate system.
3. The method according to claim 1, characterized in that The target category point cloud includes the non-ground point cloud, and the target attitude parameter includes a yaw angle; and determining the target overall residuals of the multiple radars according to the target category point cloud, including: Determine a first residual between every two of the radars according to the non-ground point cloud; wherein the first residual is used to represent an error in the yaw angle between the two radars; All the first residuals are summed to obtain the target overall residual.
4. The method according to claim 3, characterized in that Each of the two radars includes a first radar and a second radar; Determining a first residual between every two of the radars according to the non-ground point cloud comprises: Determine all first points from the non-ground point cloud of the first radar; wherein the first points have nearest neighboring points in the non-ground point cloud of the second radar; Determine, based on the coordinates of the first point and the coordinates of the nearest neighbor point, an angular difference between the first point and the corresponding nearest neighbor point; The squares of the angle differences are summed to obtain the first residual.
5. The method according to claim 1, characterized in that The target attitude parameter includes a yaw angle; and calibrating the target attitude parameter of each radar to be calibrated according to the target overall residual, including: Determine a first gradient vector according to the target overall residual; wherein the first gradient vector is composed of partial derivatives of the target overall residual with respect to the yaw angle of each of the radars to be calibrated; Iterating the yaw angle of each radar to be calibrated according to the first gradient vector and the iterative method of gradient descent; When the yaw angle of each radar to be calibrated meets the first preset iteration termination condition, the yaw angle of each radar to be calibrated obtained in the last iteration is used as the yaw angle calibration value.
6. The method according to claim 1, characterized in that The target category point cloud includes the ground point cloud, and the target attitude parameters include a pitch angle and a roll angle; Determining the target overall residuals of the multiple radars according to the target category point cloud includes: Determine a second residual between every two of the radars according to the ground point cloud; wherein the second residual is used to represent the errors of the pitch angle and the roll angle between the two radars; All the second residuals are summed to obtain the target overall residual.
7. The method according to claim 6, characterized in that Each of the two radars includes a first radar and a second radar; and determining a second residual between each of the two radars according to the ground point cloud includes: Triangulating the ground point cloud of each radar to obtain a plurality of triangular planes corresponding to each ground point cloud; Determine all second points from the ground point cloud of the first radar; wherein the projection of the second point in the Z-axis direction is located in the triangular plane corresponding to the ground point cloud of the second radar; Determining a distance error from the second point to the corresponding triangle plane; The squares of the distance errors are summed to obtain the second residual.
8. The method according to claim 1, characterized in that: The target attitude parameters include a pitch angle and a roll angle; and the target attitude parameters of each of the radars to be calibrated are calibrated according to the target overall residual, including: Determine a second gradient vector according to the target overall residual; wherein the second gradient vector is composed of a partial derivative of the target overall residual with respect to the pitch angle of each of the radars to be calibrated, and a partial derivative of the target overall residual with respect to the roll angle of each of the radars to be calibrated; Iterating the pitch angle and the roll angle of each of the radars to be calibrated respectively according to the second gradient vector and the iterative method of gradient descent; When the elevation angle of each radar to be calibrated satisfies the second preset iteration termination condition, the elevation angle of each radar to be calibrated obtained in the last iteration is used as the elevation angle calibration value; When the roll angle of each radar to be calibrated meets the third preset iteration termination condition, the roll angle of each radar to be calibrated obtained in the last iteration is used as the roll angle calibration value.
9. The method according to claim 1, characterized in that: The target attitude parameter includes a first attitude parameter and a second attitude parameter, the first attitude parameter includes a yaw angle, and the second attitude parameter includes a pitch angle and a roll angle; the target overall residual includes a first overall residual and a second overall residual, the first overall residual is used to represent a comprehensive measure of the yaw angle error between every two of the radars, and the second overall residual is used to represent a comprehensive measure of the pitch angle and roll angle errors between every two of the radars; Calibrate the target attitude parameters of each of the radars to be calibrated according to the target overall residual, including: Calibrate the yaw angle of each of the radars to be calibrated according to the first overall residual; Calibrate the pitch angle and roll angle of each of the radars to be calibrated according to the second overall residual; The method further comprises: Based on the calibrated yaw angle of each radar to be calibrated, the yaw angle of the configured radar to be calibrated is updated to iteratively update the pitch angle and roll angle of each radar to be calibrated until the errors of the pitch angle and the roll angle between two iterative updates are less than a first error threshold; and / or, Based on the calibrated pitch angle and roll angle of each radar to be calibrated, the pitch angle and roll angle of the radar to be calibrated are updated, and the yaw angle of each radar to be calibrated is iteratively updated until the error of the yaw angle between two iterative updates is less than a second error threshold.
10. A multi-radar calibration device, characterized in that: include: A preprocessing module, configured to preprocess radar point clouds acquired by a plurality of radars to determine target category point clouds of the radar point clouds in a target device coordinate system; wherein the plurality of radars are installed on the target device and include a reference radar and at least one radar to be calibrated, and the target category point clouds include non-ground point clouds and / or ground point clouds; an overall residual determination module, configured to determine the overall target residuals of the plurality of radars according to the target category point cloud; wherein the overall target residual is used to represent a comprehensive measure of the target attitude parameter error between every two of the plurality of radars, the target attitude parameter comprising at least one of a yaw angle, a pitch angle, and a roll angle; The attitude parameter calibration module is used to calibrate the target attitude parameters of each radar to be calibrated according to the target overall residual, while the external installation parameters of the reference radar and the external installation parameters of the radar to be calibrated except the target attitude parameters remain unchanged; wherein the external installation parameters include position parameters and attitude parameters.