Method and system for eliminating angle ambiguity of millimeter wave radar, vehicle and computer program product
The two-dimensional image is converted into a three-dimensional point cloud by the on-board camera, and the Euclidean distance is used to filter out the angle blur information in the millimeter-wave radar point cloud, which solves the problem of high computing costs in the existing technology and achieves low-overhead angle blur elimination. Improved efficiency and accuracy.
Patent Information
- Application Number
- CN202411019679.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-29
- Publication Date
- 2024-11-12
AI Technical Summary
The prior art has high computational cost when eliminating the angular fuzziness of millimeter wave radar, making it difficult to achieve efficient solutions with low computing overhead.
The two-dimensional image is obtained through the on-board camera and converted into a three-dimensional point cloud, and the Euclidean distance between the three-dimensional point cloud and the millimeter-wave radar point cloud filters out the radar point cloud carrying fuzzy information, and uses DBSCAN clustering and other methods to quickly filter out false information. Point cloud.
It realizes the rapid, efficient and precise elimination of millimeter-wave radar angle blur without using lidar, reducing computing overhead.
Smart Images

Figure CN118938157A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of millimeter-wave radar, and in particular to a method for eliminating angle ambiguity of millimeter-wave radar, a system for eliminating angle ambiguity of millimeter-wave radar, a vehicle including the system according to the present application, and a computer program product for at least assisting in implementing the steps of the method according to the present application. Background Art
[0002] Millimeter-wave radar can obtain the target's spatial position parameters such as distance, radial velocity and azimuth by transmitting and receiving millimeter waves. Compared with other microwaves, millimeter waves have high resolution, good directivity, strong anti-interference ability and good detection performance, especially in the field of intelligent driving. Compared with other types of radars, target identification, analysis, perception and information transmission in complex road conditions and bad weather are more accurate. However, according to the angle detection principle of millimeter-wave radar, the maximum angle unambiguous detection capability and angle resolution capability of millimeter-wave radar are limited by the antenna aperture, which leads to the possibility that the point cloud data of millimeter-wave radar may still have angle ambiguity points.
[0003] The current methods for eliminating the angle ambiguity of millimeter-wave radar are limited by theoretical factors and require a large amount of computing power. This makes it a technical problem that needs to be solved at present how to eliminate the angle ambiguity of millimeter-wave radar with low computing cost. Summary of the invention
[0004] The purpose of the present application is to provide a method for eliminating angle ambiguity of millimeter-wave radar, a system for eliminating angle ambiguity of millimeter-wave radar, a vehicle including the system according to the present application, and a computer program product, so as to at least partially solve the problems in the prior art.
[0005] According to a first aspect of the present application, a method for eliminating angle ambiguity of a millimeter wave radar is provided, and the method may include the following steps:
[0006] - converting a two-dimensional image of the target object acquired by the vehicle-mounted camera into a three-dimensional point cloud of the target object; and
[0007] - Based at least on the Euclidean distance between the radar point cloud about the target object acquired by the millimeter wave radar and the three-dimensional point cloud, the radar point cloud carrying angle ambiguity information in the radar point cloud about the target object can be filtered out.
[0008] The core concept of this application is: without using a lidar, a three-dimensional point cloud is obtained by converting the two-dimensional image of a vehicle-mounted camera, and the Euclidean distance between the three-dimensional point cloud and the radar point cloud of the millimeter-wave radar is used as a filtering condition to quickly, efficiently and accurately filter out false radar point clouds carrying angular ambiguity information, thereby eliminating the angular ambiguity of the millimeter-wave radar with low computational overhead.
[0009] According to one embodiment of the present application, DBSCAN clustering can be performed on the radar point cloud and the three-dimensional point cloud based on the Euclidean distance, wherein the Euclidean distance is used as the DBSCAN clustering radius to traverse the radar point cloud and the three-dimensional point cloud carrying angle ambiguity information, thereby screening out radar point clouds and three-dimensional point clouds whose Euclidean distance is greater than a preset distance threshold.
[0010] According to another embodiment of the present application, the two-dimensional image can be converted into a three-dimensional point cloud of the target object in a vehicle coordinate system with the help of intrinsic parameter information, extrinsic parameter information and depth information of the vehicle-mounted camera.
[0011] According to another embodiment of the present application, when the vehicle-mounted camera is configured as a monocular camera, the depth information of the vehicle-mounted camera can be obtained based on the predetermined coordinate information of the calibration object in the vehicle coordinate system, the two-dimensional image coordinate information of the calibration object in the two-dimensional image coordinate system, the intrinsic parameter information and the extrinsic parameter information of the vehicle-mounted camera.
[0012] According to another embodiment of the present application, when the vehicle-mounted camera is configured as a binocular camera, the depth information of the vehicle-mounted camera can be estimated by utilizing the stereoscopic vision principle of the binocular camera.
[0013] According to another embodiment of the present application, radar point clouds carrying angle ambiguity information in radar point clouds within a predetermined central angle range of the field of view and / or a boundary angle range of the field of view of the millimeter-wave radar may be filtered out based at least on the Euclidean distance.
[0014] According to another embodiment of the present application, a two-dimensional image of a target object may be converted into a three-dimensional point cloud of the target object through a BEV perception model, wherein the BEV perception model is trained using a training data set including two-dimensional sample images and their corresponding BEV features.
[0015] According to a second aspect of the present application, a system for eliminating angle ambiguity of a millimeter wave radar is provided, and the system may include the following components:
[0016] - a vehicle-mounted camera configured to obtain a two-dimensional image of a target object;
[0017] - a millimeter wave radar configured to acquire a radar point cloud about the target object; and
[0018] - A control unit configured to perform the method according to the application.
[0019] According to a third aspect of the present application, a vehicle is provided, comprising a system according to the present application.
[0020] According to a fourth aspect of the present application, a computer program product, such as a computer-readable program carrier, is provided, which contains or stores computer program instructions, and when the computer program instructions are executed by a processor, they at least assist in implementing the steps of the method described in the present application. BRIEF DESCRIPTION OF THE DRAWINGS
[0021] The principles, features and advantages of the present application may be better understood by describing the present application in more detail below with reference to the accompanying drawings. The accompanying drawings show:
[0022] Figure 1 A flowchart showing a method for eliminating angle ambiguity of a millimeter-wave radar according to an exemplary embodiment of the present application is shown;
[0023] Figure 2 A schematic diagram showing the field of view of a millimeter wave radar according to an exemplary embodiment of the present application is shown;
[0024] Figure 3 A schematic diagram showing the distribution of a radar point cloud and a three-dimensional point cloud of a target object according to an exemplary embodiment of the present application; and
[0025] Figure 4 A block diagram of a system for eliminating angle ambiguity of a millimeter-wave radar according to an exemplary embodiment of the present application is shown. DETAILED DESCRIPTION
[0026] In order to make the technical problems, technical solutions and beneficial technical effects to be solved by the present application clearer, the present application will be further described in detail below in conjunction with the accompanying drawings and multiple exemplary embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application, rather than to limit the scope of protection of the present application.
[0027] According to the angle detection principle of millimeter-wave radar, the maximum angle unambiguous detection capability of millimeter-wave radar is limited by the following formula 1:
[0028]
[0029] Among them, θ maxrepresents the maximum unambiguous angle that the millimeter-wave radar can detect, d represents the distance between two adjacent receiving antennas, and λ represents the wavelength of the intermediate frequency signal generated after the receiving and transmitting signals are processed by the mixer. Considering that the antenna aperture of the millimeter-wave radar cannot be infinitely small or infinitely large, the distance d between two adjacent receiving antennas is limited, so the maximum unambiguous angle θ that the millimeter-wave radar can detect is max It is also limited. For example, when d=λ / 2, the maximum angular unambiguous range of the millimeter-wave radar is ±30°, while the radar point cloud outside this range may still carry angular ambiguity information.
[0030] In addition, the angular resolution capability of mmWave radar—that is, the minimum angle at which two targets of equal size can be distinguished at the same distance—is limited by the following formula 2:
[0031]
[0032] Among them, θ res represents the angular resolution, d represents the distance between two adjacent receiving antennas, λ represents the wavelength of the intermediate frequency signal generated after the receiving and transmitting signals are processed by the mixer, N represents the number of receiving antennas, and θ represents the arrival angle of the radar echo. Similarly, considering that the distance d between two adjacent receiving antennas is limited, the maximum unambiguous angle θ that the millimeter-wave radar can detect is max It is also limited.
[0033] This application proposes to use the perceived image information of the vehicle-mounted camera to eliminate the above-mentioned angular ambiguity for the millimeter-wave radar. Figure 1 A flowchart of a method for eliminating angle ambiguity of a millimeter wave radar according to an exemplary embodiment of the present application is shown. The following exemplary embodiments describe the method according to the present application in more detail.
[0034] like Figure 1 As shown, the method may include steps S1 and S2. In step S1, a two-dimensional image of a target object acquired by a vehicle-mounted camera 11 is converted into a three-dimensional point cloud of the target object. The vehicle-mounted camera 11 is a monocular camera or a binocular camera installed on the vehicle, and its detection direction is substantially the same as that of the millimeter-wave radar, so it can acquire a two-dimensional image of the target object detected by the millimeter-wave radar.
[0035] Here, the two-dimensional image can be converted into a three-dimensional point cloud of the target object in the vehicle coordinate system by means of the internal reference information, external reference information and depth information of the vehicle camera 11. The conversion process is represented by the following formula 3, for example:
[0036]
[0037] Among them, u and v represent the two-dimensional coordinates of the image pixels of the target object acquired by the vehicle-mounted camera 11 in the two-dimensional image coordinate system; K represents the internal parameter matrix of the vehicle-mounted camera 11, which is composed of fixed parameters set at the factory (including focal length, principal point coordinates, distortion coefficient, etc.); Q represents the external parameter matrix of the vehicle-mounted camera 11, which is a parameter describing the position and posture of the vehicle-mounted camera 11 in the vehicle coordinate system. The external parameter matrix Q is composed of a rotation matrix R and a translation matrix T, which is obtained through the installation parameters of the vehicle-mounted camera 11 on the vehicle, including the horizontal angle α, the pitch angle β, the flip angle γ, the lateral displacement Δx, the longitudinal displacement Δy, and the height displacement Δz; x w ,y w 、z w Represents the three-dimensional position coordinates of the three-dimensional point cloud of the target object in the vehicle coordinate system; c Indicates the depth information of the vehicle-mounted camera 11 , which is the basis for describing the grayscale value or color number of each pixel of a two-dimensional image.
[0038] When the vehicle-mounted camera 11 is configured as a monocular camera, the depth information of the vehicle-mounted camera 11 can be obtained based on the predetermined coordinate information of the calibration object in the vehicle coordinate system, the two-dimensional image coordinate information of the calibration object in the two-dimensional image coordinate system, the internal reference information and the external reference information of the vehicle-mounted camera 11. w ,y w 、z w , when the external parameter matrix Q and the internal parameter matrix K of the vehicle-mounted camera 11 are both known, the depth information z of the vehicle-mounted camera 11 can be obtained according to the above formula 3 c .
[0039] When the vehicle-mounted camera 11 is configured as a binocular camera, since there is a certain distance between the two lenses of the binocular camera, the same calibration object has certain parallax information when imaged in the two lenses. The depth information z of the vehicle-mounted camera 11 can be estimated based on the parallax information using the stereoscopic vision principle of the binocular camera. c .
[0040] Here, the three-dimensional point cloud obtained by conversion is a bird's eye view (BEV) composed of BEV features, which converts the two-dimensional image of the vehicle-mounted camera 11 into a three-dimensional space and observes from an upper perspective, thereby obtaining a comprehensive environmental perception, which helps to simplify the processing of three-dimensional space problems. Optionally, the two-dimensional image of the target object can be converted into a three-dimensional point cloud of the target object through a BEV perception model, wherein the BEV perception model can be trained through a training data set containing a large number of two-dimensional sample images and their corresponding BEV features.
[0041] In step S2, the radar point cloud carrying angle ambiguity information in the radar point cloud about the target object can be filtered out based on at least the Euclidean distance between the radar point cloud about the target object and the three-dimensional point cloud obtained by the millimeter wave radar 12. Without using a laser radar, a three-dimensional point cloud similar to a laser radar point cloud is generated using a two-dimensional image of the vehicle-mounted camera 11. This three-dimensional point cloud does not have Doppler information and reflectivity information. Therefore, the present application uses the Euclidean distance between the radar point cloud and the three-dimensional point cloud as a filtering condition for screening false radar point clouds.
[0042] Figure 2 A schematic diagram of the field of view of a millimeter-wave radar 11 according to an exemplary embodiment of the present application is shown. For example, when the distance d between two adjacent receiving antennas is λ / 2, due to the limitation of the maximum angle-unambiguous detection capability of the millimeter-wave radar 12, the confidence of the radar point cloud within the central angle range C of the field of view of ±30° of the millimeter-wave radar 12 is 100%, and it does not carry angle ambiguity information, so all radar point clouds within the angle range C in the field of view will be retained, and the confidence of the radar point cloud outside the central angle range C of the field of view (for example, the angle range of 30° to 50° and -30° to -50°) is less than 100%, and it carries angle ambiguity information. Here, for a central angle range A of the field of view larger than the central angle range C of the field of view (for example, an angle range of ±50°), the radar point cloud carrying angle ambiguity information in the radar point cloud within the central angle range A of the field of view can be filtered out at least based on the Euclidean distance D between the radar point cloud of the target object and the three-dimensional point cloud.
[0043] In addition, within the detectable field of view of the millimeter wave radar 12 (which is Figure 2 The angle range outside the field of view (represented by the angle range surrounded by solid straight lines in the figure), for example, a field of view boundary angle range B of 10° also carries angle ambiguity information. Here, the radar point cloud carrying angle ambiguity information in the radar point cloud within the field of view boundary angle range B can also be filtered out based on the Euclidean distance D between the radar point cloud and the three-dimensional point cloud of the target object.
[0044] The following combination Figure 3 The distribution diagram of the radar point cloud and the three-dimensional point cloud of the target object according to an exemplary embodiment of the present application is shown to elaborate on the filtering process of the radar point cloud. The millimeter wave radar 12 of the vehicle 1 transmits millimeter waves to the target object in the center of the driving lane, wherein the transmitted millimeter waves can propagate along the dotted arrow pointing straight ahead, and after being reflected by the target object, they return along the original path and are detected by the millimeter wave radar 12, thereby obtaining the radar point cloud of the target object in the rectangular frame (which is marked with a solid circle). At the same time, there is also a three-dimensional point cloud of the target object obtained by converting the perception information of the vehicle-mounted camera 11 in the rectangular frame, which is marked with a hollow circle. The Euclidean distance between the radar point cloud and the three-dimensional point cloud in the rectangular frame is less than or equal to the preset distance threshold, and it can be determined that these radar point clouds and the three-dimensional point clouds are point cloud data about the same target object, so these radar point clouds can be retained.
[0045] In addition, the millimeter wave emitted by the millimeter wave radar 12 may also propagate along the dotted arrow toward the left front, and reach the target after being reflected at the curb, and propagate along the dotted arrow toward the rear after being reflected by the target and detected by the millimeter wave radar 12. The millimeter wave radar 12 calculates the straight-line distance of the target based on the time difference between the moment when the millimeter wave is sent and the moment when the millimeter wave is received, and the reflection of the millimeter wave at the curb cannot be recognized by the millimeter wave radar. Therefore, the radar point cloud determined by the millimeter wave radar 12 based on the millimeter wave of the above propagation path is a false point cloud of the mirror image of the target relative to the curb, that is, the radar point cloud in the circular frame (which is marked with a solid triangle). The Euclidean distance D between the radar point cloud in the circular frame and the three-dimensional point cloud in the matrix frame is greater than the preset distance threshold, so the radar point cloud of the mirror image of the target can be screened out and filtered out.
[0046] Optionally, DBSCAN clustering is performed on the radar point cloud and the three-dimensional point cloud based on the Euclidean distance, and in particular, the Euclidean distance is used as the DBSCAN clustering radius to traverse all radar point clouds carrying angle ambiguity information and converted three-dimensional point clouds within a predetermined angle range (for example, the central angle range A of the field of view and / or the angle range B of the field of view boundary), thereby quickly and accurately screening out radar point clouds and three-dimensional point clouds whose Euclidean distance is greater than a preset distance threshold (for example, 0.5m or 1m), and the screened radar point clouds are filtered out as false radar point clouds.
[0047] According to the present application, without using a lidar, a three-dimensional point cloud is obtained by converting the two-dimensional image of a vehicle-mounted camera, and the Euclidean distance between the three-dimensional point cloud and the radar point cloud of a millimeter-wave radar is used as a filtering condition to quickly, efficiently and accurately filter out false radar point clouds carrying angle ambiguity information, thereby eliminating the angle ambiguity of the millimeter-wave radar in a low computational overhead manner.
[0048] In addition, it should be noted that the step numbers described herein do not necessarily represent a chronological order, but are merely a figure mark. The order can be changed according to specific circumstances as long as the technical purpose of the present application can be achieved.
[0049] Figure 4 A block diagram of a system 1 for eliminating angle ambiguity of a millimeter-wave radar according to an exemplary embodiment of the present application is shown.
[0050] like Figure 4 As shown, the system 1 includes the following components:
[0051] - a vehicle-mounted camera 11, which is configured to obtain a two-dimensional image of a target object, wherein:
[0052] The vehicle-mounted camera 11 is configured as a monocular camera or a binocular camera, for example;
[0053] - a millimeter wave radar 12, which is configured to obtain a radar point cloud about the target object; and
[0054] - A control unit 13 configured to carry out the method according to the present application.
[0055] According to another embodiment of the present application, a vehicle 10 is further provided, wherein the vehicle 10 includes the system 1 .
[0056] If an embodiment includes an “and / or” relationship between a first feature and a second feature, it should be interpreted as follows: according to one embodiment, the embodiment has both the first feature and the second feature, and according to another embodiment, the embodiment has either only the first feature or only the second feature.
[0057] Although specific embodiments have been described above, these embodiments are not intended to limit the scope of the present disclosure, even when only a single embodiment is described with respect to specific features. The feature examples provided in the present disclosure are intended to be illustrative, not restrictive, unless otherwise stated. In specific implementations, multiple features may be combined with each other, where technically feasible, according to actual needs. Various substitutions, changes, and modifications may also be conceived without departing from the spirit and scope of the present application.
Claims
1. A method for eliminating angle ambiguity of millimeter wave radar, the method The following steps are involved: Converting a two-dimensional image of a target object acquired by a vehicle-mounted camera (11) into a three-dimensional point cloud of the target object; and Based at least on the Euclidean distance between the radar point cloud about the target object acquired by the millimeter wave radar (12) and the three-dimensional point cloud, the radar point cloud carrying angle ambiguity information in the radar point cloud about the target object is filtered out.
2. The method according to claim 1, in, DBSCAN clustering is performed on the radar point cloud and the three-dimensional point cloud based on the Euclidean distance, wherein the Euclidean distance is used as a DBSCAN clustering radius to traverse the radar point cloud and the three-dimensional point cloud carrying angle ambiguity information, thereby screening out radar point clouds and three-dimensional point clouds whose Euclidean distances are greater than a preset distance threshold.
3. The method according to any one of the preceding claims, in, The two-dimensional image is converted into a three-dimensional point cloud about the target object in a vehicle coordinate system by means of the internal reference information, external reference information and depth information of the vehicle-mounted camera (11).
4. The method according to any one of the preceding claims, in, When the vehicle-mounted camera (11) is configured as a monocular camera, depth information of the vehicle-mounted camera (11) is obtained based on predetermined coordinate information of a calibration object in a vehicle coordinate system, two-dimensional image coordinate information of the calibration object in a two-dimensional image coordinate system, and internal and external reference information of the vehicle-mounted camera (11).
5. The method according to any one of the preceding claims, in, When the vehicle-mounted camera (11) is configured as a binocular camera, the depth information of the vehicle-mounted camera (11) is estimated using the stereoscopic vision principle of the binocular camera.
6. The method according to any one of the preceding claims, in, Radar point clouds carrying angle ambiguity information in radar point clouds within a predetermined field of view center angle range (A) and / or field of view boundary angle range (B) of the millimeter wave radar (12) are filtered out based at least on the Euclidean distance.
7. A method according to any one of the preceding claims, in, The two-dimensional image of the target object is converted into a three-dimensional point cloud of the target object through a BEV perception model, wherein the BEV perception model is trained by a training data set including two-dimensional sample images and their corresponding BEV features.
8. A system (1) for eliminating angle ambiguity of millimeter wave radar, the system (1) comprising the following components: A vehicle-mounted camera (11), which is configured to obtain a two-dimensional image of a target object; A millimeter wave radar (12) configured to obtain a radar point cloud about the target object; and A control unit (13) configured to execute the method according to any one of claims 1 to 7.
9. A vehicle (10) comprising a system (1) according to claim 8. 10 . A computer program product, such as a computer-readable program carrier, comprising or storing computer program instructions, which, when executed by a processor, at least assist in implementing the steps of the method according to claim 1 .
Citation Information
Cited By
Method for suppressing false echo of laser radar in high-reflection environment
CN120431437A
