3D Data Annotation Method and System for Millimeter-Wave Radar

Abstract

This invention discloses a 3D data annotation method and system for millimeter-wave radar. The method includes: joint calibration of millimeter-wave radar and lidar to achieve spatial alignment and temporal synchronization; FFT processing and coordinate system transformation of the raw millimeter-wave radar data to generate a 3D data tensor; preprocessing of the lidar point cloud and transformation to the millimeter-wave radar coordinate system; obtaining the 3D bounding box of the target object using a lidar 3D point cloud target detection algorithm; matching the millimeter-wave radar points with the 3D bounding box, labeling points within the bounding box as real points and assigning them a category; for unmatched millimeter-wave radar points, matching them with lidar points using a point cloud matching algorithm, and labeling them as real or false points based on a distance threshold. This invention, through multi-sensor fusion and a two-stage matching strategy, achieves automated and high-precision annotation of millimeter-wave radar 3D data, significantly reducing annotation difficulty and improving annotation efficiency and consistency.

Description

Technical Field

[0001] This invention relates to the field of millimeter-wave radar data processing technology, and in particular to a 3D data annotation method and system for millimeter-wave radar. Background Technology

[0002] Millimeter-wave radar has garnered significant attention due to its crucial applications in autonomous driving, intelligent transportation, and industrial automation. It detects information such as the distance, velocity, and angle of target objects by transmitting and receiving millimeter-wave signals. In recent years, applying deep learning to millimeter-wave radar target detection has become a research hotspot. However, with the increasing complexity of application scenarios and the rising accuracy requirements, the need for annotation of millimeter-wave radar data is becoming increasingly urgent, especially the annotation of 3D millimeter-wave radar data, which is crucial for subsequent target recognition, tracking, and decision-making processes.

[0003] Traditional millimeter-wave radar data annotation methods have the following limitations: The annotation process is challenging: Millimeter-wave radar 3D data is not presented intuitively. Its annotation usually requires experienced millimeter-wave radar signal processing engineers to identify targets on spectrum maps such as range-Doppler (RD), range-Angle (RA), and angle-Doppler (AD) before annotation can be performed. This not only requires a high level of expertise from the annotators, but also makes the annotation process complex and difficult to achieve quickly and accurately.

[0004] Insufficient annotation accuracy: Existing methods often rely on simple threshold judgment or manual annotation, which makes it difficult to accurately annotate the three-dimensional spatial position and shape of the target object, and is easily affected by noise and complex environment.

[0005] Inefficient annotation: Manual annotation is not only time-consuming and labor-intensive, but also prone to inconsistencies, making it difficult to meet the needs of large-scale data annotation.

[0006] Lack of automation and intelligence: Existing technologies lack effective automated annotation algorithms, making it impossible to process and annotate dynamically changing 3D data in real time, which limits the application of millimeter-wave radar in scenarios with high real-time requirements. Summary of the Invention

[0007] This invention provides a 3D data annotation method and system for millimeter-wave radar, aiming to solve the problems of low annotation accuracy, poor efficiency, lack of automation and intelligence, and high annotation difficulty in existing millimeter-wave radar 3D data annotation methods. It can achieve high-precision and high-efficiency automated annotation, while adapting to complex environments and dynamic scenes, thereby improving the availability and reliability of millimeter-wave radar data.

[0008] To address the aforementioned technical problems, as one aspect of the present invention, a 3D data annotation method for millimeter-wave radar is provided, comprising the following steps: Joint calibration of millimeter-wave radar and lidar achieves spatial alignment and time synchronization; FFT processing and coordinate system transformation are performed on the raw millimeter-wave radar data to generate a 3D data tensor for the millimeter-wave radar. The lidar point cloud is preprocessed and transformed to the millimeter-wave radar coordinate system; The 3D bounding box of the target object is obtained by using a LiDAR 3D point cloud target detection algorithm; The millimeter-wave radar points are matched with the 3D bounding box, and the points within the bounding box are labeled as real points and assigned a category. For millimeter-wave radar points that do not have matching bounding boxes, a point cloud matching algorithm is used to match them with lidar points, and they are marked as real points or false points according to the matching distance threshold.

[0009] Preferably, the joint calibration includes obtaining the rotation matrix R and translation vector T between the millimeter-wave radar and the lidar; The rotation matrix R and translation vector T are represented as follows: , .

[0010] Preferably, the FFT processing of the raw millimeter-wave radar data includes: range FFT processing, Doppler FFT processing, and angle FFT processing; The coordinate system transformation includes converting the data processed by angle FFT from polar coordinates to Cartesian coordinates, achieved through the following formula: , ,

[0011] in, For distance, The pitch angle, It is the azimuth angle; The millimeter-wave radar 3D data tensor includes: range index (range_index), Doppler index (doppler_index), angle index (angle_index), and the point coordinates of the millimeter-wave radar in the world coordinate system. , , The world coordinate system mentioned above is based on the millimeter-wave radar coordinate system.

[0012] Preferably, the transformation of the lidar point cloud to the millimeter-wave radar coordinate system is achieved by the following formula:

[0013] in, A point in the lidar coordinate system. For points in the transformed millimeter-wave radar coordinate system, R Let be a rotation matrix. T It is a translation vector.

[0014] Preferably, the step of matching millimeter-wave radar points with 3D bounding boxes specifically includes: Calculate the point coordinates of the millimeter-wave radar in the world coordinate system. , , The geometric containment relationship between the point and the 3D bounding box is determined. If a point is inside the box, it is marked as 1 and inherits the category information of the bounding box. The point cloud matching algorithm is the nearest neighbor matching or iterative nearest point algorithm. It calculates the distance between the millimeter-wave radar point and the nearest lidar point. If the distance is less than a set threshold, it is marked as 1 and the category is set to "other". Otherwise, it is marked as 0.

[0015] As another aspect of the present invention, a 3D data annotation system for millimeter-wave radar is also provided, comprising: The sensor calibration module is used to jointly calibrate millimeter-wave radar and lidar to achieve spatial alignment and time synchronization. The millimeter-wave radar processing module is used to perform FFT processing and coordinate system transformation on the raw ADC data to generate millimeter-wave radar 3D data tensors. The lidar processing module is used to preprocess lidar point clouds and convert them to the millimeter-wave radar coordinate system. A 3D target detection module is used to detect the 3D bounding box of a target object based on a LiDAR point cloud. The first matching and annotation module is used to match millimeter-wave radar points with 3D bounding boxes, annotate the points inside the boxes, and assign categories. The second matching and labeling module is used to perform point cloud matching between unmatched millimeter-wave radar points and lidar points, and label them as real points or fake points based on a distance threshold.

[0016] Preferably, the sensor calibration module is used to obtain the rotation matrix R and translation vector T between the millimeter-wave radar and the lidar; The rotation matrix R and translation vector T are represented as follows: , .

[0017] Preferably, the millimeter-wave radar processing module includes: The FFT processing unit is used to perform range FFT processing, Doppler FFT processing, and angle FFT processing on the raw millimeter-wave radar data. The polar coordinate to Cartesian coordinate transformation unit is used to achieve coordinate transformation using the following formula: , ,

[0018] in, For distance, The pitch angle, It is the azimuth angle; The data tensor acquisition unit is used to obtain the 3D data tensor of the millimeter-wave radar, which includes the range index (range_index), the Doppler index (doppler_index), the angle index (angle_index), and the point coordinates of the millimeter-wave radar in the world coordinate system. , , The world coordinate system mentioned above is based on the millimeter-wave radar coordinate system.

[0019] Preferably, the lidar processing module is used to transform the lidar point cloud to the millimeter-wave radar coordinate system using the following formula:

[0020] in, , is a point in the lidar coordinate system. This refers to the point in the transformed millimeter-wave radar coordinate system.

[0021] Preferably, the first matching and labeling module is used to determine the point coordinates of the millimeter-wave radar in the world coordinate system. , , The matching relationship is determined by whether the point is inside the 3D bounding box, and the category of the matching point is set to the category of the 3D bounding box. The second matching and labeling module uses the nearest neighbor matching algorithm to calculate the minimum Euclidean distance between the millimeter-wave radar point and the lidar point. If the distance is less than a preset threshold, it is labeled as 1 and the category is set to "other"; otherwise, it is labeled as 0.

[0022] Implementing the embodiments of the present invention has the following beneficial effects: By fusing millimeter-wave radar and lidar data and applying deep learning algorithms, the embodiments of the present invention can accurately mark the three-dimensional spatial position and shape of the target object, and the marking accuracy is greatly improved compared with traditional methods, reaching centimeter-level accuracy.

[0023] By implementing the embodiments of the present invention, automated annotation is achieved without manual intervention, which greatly improves annotation efficiency and enables rapid processing of large-scale millimeter-wave radar 3D data, meeting the application scenarios with high real-time requirements.

[0024] By implementing the embodiments of the present invention, it can adapt to complex environments and dynamic scenes. Through multi-sensor fusion and adaptive learning mechanisms, it can effectively cope with noise interference and dynamic changes of target objects, and has strong robustness.

[0025] By implementing the embodiments of the present invention, the deep learning-based annotation method has learning and adaptive capabilities, which can continuously optimize the annotation algorithm, improve the annotation quality, and provide higher quality data support for the subsequent processing and application of millimeter-wave radar data.

[0026] Implementing the embodiments of the present invention eliminates the need for experienced millimeter-wave radar signal processing engineers to perform complex manual annotations, lowering the annotation threshold and improving the operability and consistency of annotations. Attached Figure Description

[0027] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, obtaining other drawings based on these drawings without creative effort still falls within the scope of the present invention.

[0028] Figure 1 This is a schematic diagram of the main flow of an embodiment of the 3D data annotation method for millimeter-wave radar of the present invention.

[0029] Figure 2 This is a schematic diagram of an embodiment of the 3D data annotation system for millimeter-wave radar of the present invention.

[0030] Figure 3 for Figure 2 A schematic diagram of the structure of a mid-millimeter wave radar processing module. Detailed Implementation

[0031] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the following illustrative embodiments and descriptions are only used to explain the technical principles and feasible solutions of the present invention, and are not intended to limit the present invention. Moreover, in the absence of conflict, the embodiments and features in the embodiments of the present invention can be combined with each other.

[0032] like Figure 1The diagram illustrates the main flowchart of an embodiment of a 3D data annotation method for millimeter-wave radar provided by the present invention; in this embodiment, the method includes at least the following steps: Step S1, Millimeter-wave radar and lidar calibration steps: Perform joint calibration of millimeter-wave radar and lidar to achieve spatial alignment and time synchronization; Understandably, millimeter-wave radar and lidar are calibrated before data processing to ensure spatial alignment and temporal synchronization. The calibration process includes external parameter calibration and time synchronization. Specifically: External parameter calibration: Using corner reflectors, calibration boards, or other calibration tools, the relative position and attitude relationship between the millimeter-wave radar and the lidar is determined, including the rotation matrix R and the translation vector T. The specific representations of R and T are as follows: ,

[0033] Both can be obtained through corner reflectors or calibration plates.

[0034] Time synchronization: The acquisition time of millimeter-wave radar and lidar is synchronized to ensure the consistency of the data in time, so as to facilitate subsequent data fusion and annotation.

[0035] Step S2, millimeter-wave radar data preprocessing step: Perform FFT processing and coordinate system transformation on the raw millimeter-wave radar data to generate millimeter-wave radar 3D data tensor; Specifically, the raw ADC (analog-to-digital converter) data acquired by millimeter-wave radar is processed through the following steps: Distance FFT (Fast Fourier Transform) processing: Perform distance FFT on the original ADC data to transform the data from the time domain to the frequency domain, thereby obtaining the distance information of the target object.

[0036] Doppler FFT processing: Perform Doppler FFT on the data after distance FFT processing to further extract the velocity information of the target object.

[0037] Angle FFT processing: Perform angle FFT on the data after Doppler FFT to obtain the angle information of the target object.

[0038] Coordinate system transformation: The data processed by angle FFT is transformed from polar coordinates to Cartesian coordinates to obtain the point coordinates in the world coordinate system. , , (By default, the millimeter-wave radar coordinate system is used as the world coordinate system to distinguish it from the Cartesian coordinate system.) This transformation process is achieved through the following formula: , ,

[0039] in, Indicates distance, Indicates pitch angle, Indicates the azimuth angle.

[0040] Data Tensor Construction: The processed data is integrated into a millimeter-wave radar 3D data tensor. This tensor contains the range index (range_index), Doppler index (doppler_index), angle index (angle_index), and the point coordinates of the millimeter-wave radar in the world coordinate system. , , The world coordinate system mentioned above is based on the millimeter-wave radar coordinate system.

[0041] Step S3, LiDAR data preprocessing step: preprocess the LiDAR point cloud and convert it to the millimeter-wave radar coordinate system; Specifically, the preprocessing includes: Noise removal: The raw point cloud data collected by lidar is processed to remove ground reflection points, noise points, and outliers, thereby improving data quality.

[0042] Coordinate system transformation: Based on the rotation matrix R and translation vector T obtained in step S1, the lidar point cloud data is transformed from the lidar coordinate system to the millimeter-wave radar coordinate system (i.e., the world coordinate system). The specific transformation formula is as follows: Assume that the point in the lidar coordinate system is The point of the lidar in the millimeter-wave radar coordinate system is The transformation relationship is then:

[0043] Data transformation: Converting the LiDAR points after coordinate system transformation into points cloud information. , , The format is (,intensity), where intensity is the energy intensity, used for subsequent target detection and data matching.

[0044] Step S4, LiDAR point cloud target detection step: Use the LiDAR 3D point cloud target detection algorithm to obtain the 3D bounding box of the target object; In this step, a 3D point cloud target detection algorithm for LiDAR (an existing open-source deep learning algorithm) is used to detect targets in the LiDAR point cloud, obtaining 3D bounding box data of the target objects, including information such as length (l), width (w), height (h), center point coordinates (x, y, z), heading angle (angle), and class. This target detection algorithm, based on a deep learning model, can accurately identify and locate target objects, providing a reference for the annotation of millimeter-wave radar data.

[0045] Step S5, Millimeter-wave radar data matching with 3D bounding box: Match the millimeter-wave radar points with the 3D bounding box, label the points within the bounding box as real points and assign them a category; Specifically, the preprocessed millimeter-wave radar data is matched with the 3D bounding box obtained from lidar target detection, and the millimeter-wave radar points within the 3D bounding box are detected and labeled.

[0046] By calculating the point coordinates of the millimeter-wave radar in the world coordinate system ( , , The geometric relationship between the LiDAR point cloud and the 3D bounding box is used to determine which millimeter-wave radar points belong to the target objects within the 3D bounding box. The millimeter-wave radar points within the bounding box are marked as 1, and the category is set to the category of the 3D bounding box. Specifically, it refers to the category (class) obtained by target detection of the LiDAR point cloud. The corresponding category information can be obtained from the LiDAR point cloud through the target detection algorithm.

[0047] Step S6, matching the remaining millimeter-wave radar points with lidar points: For millimeter-wave radar points that have not been matched with bounding boxes, a point cloud matching algorithm is used to match them with lidar points, and they are marked as real points or false points according to the matching distance threshold.

[0048] Specifically, for millimeter-wave radar points that are not matched within the 3D bounding box, they are matched with lidar points, and the matched points are labeled.

[0049] Point cloud matching algorithms (such as nearest neighbor matching and iterative nearest point algorithm) are used to find corresponding matching point pairs based on the spatial relationship between millimeter-wave radar points and lidar points, and the distance between the millimeter-wave radar point and the nearest point is calculated. While a millimeter-wave radar point might search for all nearby lidar points during matching, we only need the point closest to the millimeter-wave radar point.

[0050] If the value is less than the set threshold, it will be marked as 1 and the category will be set to "other".

[0051] Marking points greater than a set threshold as 0 will allow for the marking of the remaining millimeter-wave radar points.

[0052] The labels 1 and 0 above represent different classifications. 1 represents a real point and 0 represents a fake point. Labeling them with 0 and 1 is equivalent to classifying the millimeter-wave radar points, thereby achieving data labeling. The labeling results will be used for deep learning algorithm training.

[0053] like Figure 2 The diagram shown illustrates a structural schematic of an embodiment of a 3D data annotation system for millimeter-wave radar provided by the present invention; in conjunction with... Figure 3 As shown, in this embodiment, it is used to achieve the following: Figure 1 The described method, specifically, the 3D data annotation system 1 for the millimeter-wave radar includes: Sensor calibration module 10 is used for joint calibration of millimeter-wave radar and lidar to achieve spatial alignment and time synchronization; The millimeter-wave radar processing module 11 is used to perform FFT processing and coordinate system transformation on the raw ADC data to generate millimeter-wave radar 3D data tensors. The lidar processing module 12 is used to preprocess the lidar point cloud and convert it to the millimeter-wave radar coordinate system. 3D target detection module 13 is used to detect the 3D bounding box of the target object based on the point cloud of the lidar; The first matching and annotation module 14 is used to match millimeter-wave radar points with 3D bounding boxes, annotate the points inside the boxes and assign categories; The second matching and labeling module 15 is used to perform point cloud matching between unmatched millimeter-wave radar points and lidar points, and label them as real points or false points based on a distance threshold.

[0054] In a specific example, the sensor calibration module 10 is used to obtain the rotation matrix R and translation vector T between the millimeter-wave radar and the lidar; The rotation matrix R and translation vector T are represented as follows: ,

[0055] Both can be obtained through corner reflectors or calibration plates.

[0056] In specific examples, such as Figure 3 As shown, the millimeter-wave radar processing module 11 includes: FFT processing unit 110 is used to perform range FFT processing, Doppler FFT processing and angle FFT processing on the raw millimeter-wave radar data. Polar to Cartesian coordinate transformation unit 111 is used to achieve coordinate transformation using the following formula: , ,

[0057] in, For distance, The pitch angle, It is the azimuth angle; Data tensor acquisition unit 112 is used to acquire the millimeter-wave radar 3D data tensor, including the range index (range_index), Doppler index (doppler_index), angle index (angle_index), and the point coordinates of the millimeter-wave radar in the world coordinate system. , , The world coordinate system mentioned above is based on the millimeter-wave radar coordinate system.

[0058] In a specific example, the lidar processing module 12 transforms the lidar point cloud to the millimeter-wave radar coordinate system using the following formula:

[0059] in, , is a point in the lidar coordinate system. This refers to the point in the transformed millimeter-wave radar coordinate system.

[0060] In a specific example, the first matching and labeling module 14 determines the point coordinates of the millimeter-wave radar in the world coordinate system ( , , The matching relationship is determined by whether the point is inside the 3D bounding box, and the category of the matching point is set to the category of the 3D bounding box. The second matching and labeling module 15 uses the nearest neighbor matching algorithm to calculate the minimum Euclidean distance between the millimeter-wave radar point and the lidar point. If the distance is less than a preset threshold, it is labeled as 1 and the category is set to "other"; otherwise, it is labeled as 0.

[0061] For more details, please refer to and combine with the above. Figure 1 The description of that will not be repeated here.

[0062] Implementing the embodiments of the present invention has the following beneficial effects: This invention proposes a 3D data annotation method and system for millimeter-wave radar. By fusing millimeter-wave radar and lidar data and applying deep learning algorithms, the three-dimensional spatial position and shape of the target object can be accurately annotated. The annotation accuracy is significantly improved compared with traditional methods, reaching centimeter-level accuracy.

[0063] By implementing the embodiments of the present invention, automated annotation is achieved without manual intervention, which greatly improves annotation efficiency and enables rapid processing of large-scale millimeter-wave radar 3D data, meeting the application scenarios with high real-time requirements.

[0064] By implementing the embodiments of the present invention, it can adapt to complex environments and dynamic scenes. Through multi-sensor fusion and adaptive learning mechanisms, it can effectively cope with noise interference and dynamic changes of target objects, and has strong robustness.

[0065] By implementing the embodiments of the present invention, the deep learning-based annotation method has learning and adaptive capabilities, which can continuously optimize the annotation algorithm, improve the annotation quality, and provide higher quality data support for the subsequent processing and application of millimeter-wave radar data.

[0066] Implementing the embodiments of the present invention eliminates the need for experienced millimeter-wave radar signal processing engineers to perform complex manual annotations, lowering the annotation threshold and improving the operability and consistency of annotations.

[0067] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0068] The embodiments of the present invention have been described above with reference to the accompanying drawings. However, the present invention is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many variations under the guidance of the present invention without departing from the inventive spirit and scope of protection of the claims. These variations are all within the scope of protection of the present invention.

Claims

1. A 3D data annotation method for millimeter-wave radar, characterized in that, Includes the following steps: Joint calibration of millimeter-wave radar and lidar achieves spatial alignment and time synchronization; FFT processing and coordinate system transformation are performed on the raw millimeter-wave radar data to generate a 3D data tensor for the millimeter-wave radar. The lidar point cloud is preprocessed and transformed to the millimeter-wave radar coordinate system; The 3D bounding box of the target object is obtained by using a LiDAR 3D point cloud target detection algorithm; The millimeter-wave radar points are matched with the 3D bounding box, and the points within the bounding box are labeled as real points and assigned a category. For millimeter-wave radar points that do not have matching bounding boxes, a point cloud matching algorithm is used to match them with lidar points, and they are marked as real points or false points according to the matching distance threshold.

2. The method according to claim 1, characterized in that, The joint calibration includes obtaining the rotation matrix R and translation vector T between the millimeter-wave radar and the lidar; The rotation matrix R and translation vector T are represented as follows: ， 。 3. The method according to claim 2, characterized in that, The FFT processing of the raw millimeter-wave radar data includes: range FFT processing, Doppler FFT processing, and angle FFT processing. The coordinate system transformation includes converting the data processed by angle FFT from polar coordinates to Cartesian coordinates, achieved through the following formula: ， ， in, For distance, The pitch angle, It is the azimuth angle; The millimeter-wave radar 3D data tensor includes: range index (range_index), Doppler index (doppler_index), angle index (angle_index), and the point coordinates of the millimeter-wave radar in the world coordinate system. , , The world coordinate system mentioned above is based on the millimeter-wave radar coordinate system.

4. The method according to claim 3, characterized in that, The conversion of lidar point clouds to millimeter-wave radar coordinates is achieved through the following formula: in, A point in the lidar coordinate system. For points in the transformed millimeter-wave radar coordinate system, R For rotation matrix, T It is a translation vector.

5. The method according to any one of claims 1 to 4, characterized in that, The step of matching millimeter-wave radar points with 3D bounding boxes is as follows: Calculate the point coordinates of the millimeter-wave radar in the world coordinate system. , , The geometric containment relationship between the point and the 3D bounding box is determined. If a point is inside the box, it is marked as 1 and inherits the category information of the bounding box. The point cloud matching algorithm is the nearest neighbor matching or iterative nearest point algorithm. It calculates the distance between the millimeter-wave radar point and the nearest lidar point. If the distance is less than a set threshold, it is marked as 1 and the category is set to "other". Otherwise, it is marked as 0.

6. A 3D data annotation system for millimeter-wave radar, characterized in that, include: The sensor calibration module is used to jointly calibrate millimeter-wave radar and lidar to achieve spatial alignment and time synchronization. The millimeter-wave radar processing module is used to perform FFT processing and coordinate system transformation on the raw ADC data to generate millimeter-wave radar 3D data tensors. The lidar processing module is used to preprocess lidar point clouds and convert them to the millimeter-wave radar coordinate system. A 3D target detection module is used to detect the 3D bounding box of a target object based on a LiDAR point cloud. The first matching and annotation module is used to match millimeter-wave radar points with 3D bounding boxes, annotate the points inside the boxes, and assign categories. The second matching and labeling module is used to perform point cloud matching between unmatched millimeter-wave radar points and lidar points, and label them as real points or fake points based on a distance threshold.

7. The system according to claim 6, characterized in that, The sensor calibration module is used to obtain the rotation matrix R and translation vector T between the millimeter-wave radar and the lidar. The rotation matrix R and translation vector T are represented as follows: ， 。 8. The system according to claim 7, characterized in that, The millimeter-wave radar processing module includes: The FFT processing unit is used to perform range FFT processing, Doppler FFT processing, and angle FFT processing on the raw millimeter-wave radar data. The polar coordinate to Cartesian coordinate transformation unit is used to achieve coordinate transformation using the following formula: ， ， in, For distance, The pitch angle, It is the azimuth angle; The data tensor acquisition unit is used to obtain the 3D data tensor of the millimeter-wave radar, which includes the range index (range_index), the Doppler index (doppler_index), the angle index (angle_index), and the point coordinates of the millimeter-wave radar in the world coordinate system. , , The world coordinate system mentioned above is based on the millimeter-wave radar coordinate system.

9. The system according to claim 8, characterized in that, The lidar processing module is used to transform lidar point clouds to millimeter-wave radar coordinates using the following formula: in, , where is a point in the lidar coordinate system. This refers to the point in the transformed millimeter-wave radar coordinate system.

10. The system according to any one of claims 6 to 9, characterized in that, The first matching and annotation module is used to determine the point coordinates of the millimeter-wave radar in the world coordinate system. , , The matching relationship is determined by whether the point is inside the 3D bounding box, and the category of the matching point is set to the category of the 3D bounding box. The second matching and labeling module uses the nearest neighbor matching algorithm to calculate the minimum Euclidean distance between the millimeter-wave radar point and the lidar point. If the distance is less than a preset threshold, it is labeled as 1 and the category is set to "other"; otherwise, it is labeled as 0.

Images