Automatic labeling, target point classification, training set generation, classification model training, point cloud classification method, integrated circuit and terminal device

Abstract

An automatic labeling method, a target point classification method, a training set generation method, a classification model training method, a point cloud classification method, an integrated circuit and a terminal device. The automatic labeling method is used for labeling target points in multiple frames of point cloud data obtained by a vehicle-mounted radar. The method comprises: dividing the target points into static target points and dynamic candidate points according to vehicle motion state information; dividing the dynamic candidate points into dynamic target points and false target points according to the positional relationship between the dynamic candidate points; and marking the type of the target points, the type including at least one of the following: static target points, dynamic target points and false target points. The labeling scheme based on point cloud data provided by the present disclosure improves the automation degree and labeling accuracy of labeling, greatly reduces the workload of manual labeling, and improves the labeling efficiency.

Description

Technical Field

[0001] This application relates to the field of radar detection technology, and more particularly to an automatic annotation method, a target point classification method, a training set generation method, a classification model training method, a point cloud classification method, an integrated circuit, and a terminal device. Background Technology

[0002] In radar target detection systems, inaccurate target detection results, including false targets, are often caused by factors such as multipath effects, clutter, antenna noise, and receiver noise. The presence of these false targets interferes with the detection of real targets and the effectiveness and accuracy of subsequent applications based on the detection results. Therefore, improving the accuracy of radar target detection and effectively identifying false targets is a core objective for the continuous improvement and optimization of radar detection solutions. Summary of the Invention

[0003] This disclosure provides an automatic annotation method, a target point classification method, a training set generation method, a classification model training method, a point cloud classification method, an integrated circuit, and a terminal device. Based on vehicle motion state information, target points in point cloud data are initially divided into static target points and dynamic candidate points. Then, based on positional relationships, the dynamic candidate points are further distinguished into false target points and (real) dynamic target points before automatic annotation. This improves the automation and accuracy of point cloud data annotation, significantly reduces the workload of manual annotation, and increases annotation efficiency. By fully utilizing point cloud data acquired from radar for ground truth identification, reliance on other ground truth identification systems is greatly reduced.

[0004] This disclosure provides an automatic annotation method for annotating target points in multi-frame point cloud data obtained from vehicle-mounted radar. The method includes: Based on the vehicle motion status information, the target points are divided into static target points and dynamic candidate points; Based on the positional relationship between the dynamic candidate points, the dynamic candidate points are divided into dynamic target points and false target points; The target point is labeled with its type, which includes at least one of the following: static target point, dynamic target point, and false target point; The dynamic target points in a frame include: core target points and edge target points; the core target point is a dynamic candidate point that has at least L other dynamic candidate points within a set distance threshold in the dynamic candidate point range of multiple consecutive frames; the edge target point is a dynamic candidate point that is within a set distance threshold of any core target point in the dynamic candidate point range of multiple consecutive frames, where L is an integer greater than 1; the multiple consecutive frames include the frame.

[0005] This disclosure also provides a target point classification method for classifying target points in multi-frame point cloud data obtained by radar. The method includes: Based on the motion state information of the radar, the target points are divided into static target points and dynamic candidate points; Based on the positional relationship between the dynamic candidate points, false target detection is performed on the dynamic candidate points, and the detected target points are determined as false target points, while other target points are determined as dynamic target points; The radar includes: frequency modulated continuous wave (FMCW) radar, phase modulated continuous wave (PMCW) radar, or pulse compression radar. The dynamic target points in a frame include: core target points and edge target points; the core target point is a dynamic candidate point that exists within a set distance threshold of at least L other dynamic candidate points in the dynamic candidate point range of multiple consecutive frames; the edge target point is a dynamic candidate point that is within a set distance threshold of any core target point in the dynamic candidate point range of multiple consecutive frames, where L is an integer greater than 1; the multiple consecutive frames include the frame.

[0006] This disclosure also provides a training set generation method, including: For the multi-frame point cloud data obtained from the vehicle-mounted radar, the following steps are performed respectively to generate training samples to form the training set: Extract feature data of the target point based on the current frame point cloud data; Get M target points in the current frame point cloud data, and label the type corresponding to the target points according to the automatic labeling method as described in any one of claims 1-7 to obtain the training samples corresponding to the current frame point cloud data, where M is a preset value greater than 0; or, Based on multi-frame point cloud data obtained from radar, the following steps are performed to generate training samples to form the training set: Extract feature data of the target point based on the current frame point cloud data; Get M target points in the current frame point cloud data, determine the type of the target points according to the method described in any one of claims 8-10, and label them to obtain training samples corresponding to the current frame point cloud data, where M is a preset value greater than 0; The radar includes: frequency modulated continuous wave (FMCW) radar, phase modulated continuous wave (PMCW) radar, or pulse compression radar.

[0007] This disclosure also provides a training set generation method, including: The following steps are performed on the multi-frame point cloud data obtained by the vehicle-mounted radar to generate training samples to form the training set: Extract feature data of the target point based on the current frame point cloud data; Get M target points in the current frame point cloud data, and label the type corresponding to the target points to obtain the training samples corresponding to the current frame point cloud data, where M is a preset value greater than 0; The feature data includes: spatial location data and Doppler velocity; or, the feature data includes: spatial location data and Doppler velocity, and further includes: radar signal-to-noise ratio and / or azimuth angle.

[0008] This disclosure also provides a classification model training method, including: Obtain a training set, and use the training samples included in the training set to train a point cloud target point classification model; The training set includes: training samples generated from multiple frames of point cloud data obtained from the vehicle-mounted radar; each training sample includes at least one labeled target point. The type of the target point is labeled according to the automatic labeling method as described in any embodiment of this disclosure; or, the type of the target point is determined and labeled according to the target point classification method as described in any embodiment of this disclosure. The point cloud target point classification model is used to identify the type of target points in point cloud data.

[0009] This disclosure also provides a classification model training method, including: Obtain a training set, and use the training samples included in the training set to train a point cloud target point classification model; The training set includes: training samples generated from multiple frames of point cloud data obtained by vehicle-mounted radar; each training sample includes at least one labeled target point; and the point cloud target point classification model is used to identify the type of target points in the point cloud data. The point cloud target point classification model is a deep learning model, which includes at least one symmetrically arranged feature abstraction module and feature reconstruction module. The feature abstraction module is configured to perform feature extraction after downsampling based on the input data of the current layer. The feature reconstruction module is configured to perform feature extraction after upsampling based on the input data of the current layer. The feature abstraction module is also configured to perform downsampling based on the vertical position, horizontal position, and Doppler velocity of the target point in the point cloud data.

[0010] This disclosure also provides a point cloud classification method, including: Feature data of target points are extracted from the current frame point cloud data obtained by vehicle-mounted radar. The feature data of the target point is input into the trained classification model to classify the target point. The classification model is pre-trained according to the classification model training method described in any embodiment of this disclosure.

[0011] This disclosure also provides an integrated circuit, including a processor configured to implement the automatic annotation method as described in any embodiment of this disclosure; or, to implement the target point classification method as described in any embodiment of this disclosure; or, to implement the training set generation method as described in any embodiment of this disclosure; or, to implement the classification model training method as described in any embodiment of this disclosure; or, to implement the point cloud classification method as described in any embodiment of this disclosure.

[0012] This disclosure also provides a terminal device, including: One or more processors; Storage device for storing one or more programs. When the one or more programs are executed by the one or more processors, the one or more processors implement the point cloud classification method as described in any embodiment of this disclosure.

[0013] Other features and advantages of this application will be set forth in the following description, and will be apparent in part from the description, or may be learned by practicing the application. Other advantages of this application can be realized and obtained by means of the solutions described in the description and the accompanying drawings. Attached Figure Description

[0014] The accompanying drawings are used to provide an understanding of the technical solutions of this application and constitute a part of the specification. They are used together with the embodiments of this application to explain the technical solutions of this application and do not constitute a limitation on the technical solutions of this application.

[0015] Figure 1 A flowchart illustrating an automatic annotation method provided in this application embodiment; Figure 2 A flowchart illustrating another automatic annotation method provided in this application embodiment; Figure 3 A flowchart of a target point classification method provided in an embodiment of this application; Figure 4 A flowchart illustrating a training set generation method provided in an embodiment of this application; Figure 5 A flowchart illustrating another training set generation method provided in this application embodiment; Figure 6 A flowchart illustrating a classification model training method provided in this application embodiment; Figure 7 This is a schematic diagram of the structure of a classification model provided in an embodiment of this application; Figure 8A schematic diagram illustrating another classification model training method provided in an embodiment of this application; Figure 9 A flowchart of a point cloud classification method provided in an embodiment of this application; Figure 10 A flowchart of another point cloud classification method provided in an embodiment of this application. Detailed Implementation

[0016] This application describes several embodiments, but these descriptions are exemplary and not restrictive, and it will be apparent to those skilled in the art that many more embodiments and implementations are possible within the scope of the embodiments described herein. Although many possible combinations of features are shown in the drawings and discussed in the detailed description, many other combinations of the disclosed features are also possible. Unless specifically limited, any feature or element of any embodiment may be used in combination with, or may replace, any feature or element of any other embodiment.

[0017] This application includes and contemplates combinations of features and elements known to those skilled in the art. The embodiments, features, and elements disclosed in this application may also be combined with any conventional features or elements to form a unique inventive scheme as defined by the claims. Any feature or element of any embodiment may also be combined with features or elements from other inventive schemes to form another unique inventive scheme as defined by the claims. Therefore, it should be understood that any feature shown and / or discussed in this application may be implemented individually or in any suitable combination. Therefore, the embodiments are not limited except by the limitations imposed by the appended claims and their equivalents. Furthermore, various modifications and changes may be made within the scope of the appended claims.

[0018] Furthermore, in describing representative embodiments, the specification may have presented methods and / or processes as a specific sequence of steps. However, the method or process should not be limited to the specific order of steps described herein, to the extent that it does not depend on such a specific order. As will be understood by those skilled in the art, other sequences of steps are also possible. Therefore, the specific order of steps set forth in the specification should not be construed as a limitation of the claims. Moreover, the claims concerning the method and / or process should not be limited to the steps performed in the written order, and those skilled in the art will readily understand that these orders can be varied and still remain within the spirit and scope of the embodiments of this application.

[0019] The introduction of AI models into radar target detection and recognition schemes elevates radar systems from mere detection to intelligent perception, significantly enhancing the applicability of radar detection solutions for complex application scenarios. This improves overall detection and recognition accuracy and robustness, laying a data foundation for further applications such as autonomous driving and smart cities. Model training is a crucial step in AI model-related solutions, and a wide range of high-quality training samples significantly impacts the training results. Therefore, accurate annotation of sample data to obtain abundant training samples is a prerequisite for AI model training.

[0020] Some feasible solutions employ manual annotation to label the target point types in point cloud data. Due to the large volume of point cloud data and abundant training samples, this manual annotation workload is enormous, impacting both annotation efficiency and overall training efficiency. Other feasible solutions utilize video or other sensors to determine target point types and annotate training samples, but this requires cross-sensor spatial and temporal synchronization, increasing model training complexity. This disclosure provides an automatic annotation method that automatically labels target point types using radar-obtained point cloud data. This method requires no manual intervention and does not rely on other ground truth systems, significantly improving annotation efficiency.

[0021] This disclosure provides an automatic annotation method, such as... Figure 1 As shown, the method for labeling target points in multi-frame point cloud data obtained by vehicle-mounted radar includes: Step 110: Based on the vehicle motion state information, the target points are divided into static target points and dynamic candidate points; Step 120: Based on the positional relationship between the dynamic candidate points, the dynamic candidate points are divided into dynamic target points and false target points; Step 140: Mark the type of the target point, the type including at least one of the following: static target point, dynamic target point, and false target point; The dynamic target points in a frame include: core target points and edge target points; the core target point is a dynamic candidate point that has at least L other dynamic candidate points within a set distance threshold in the dynamic candidate point range of multiple consecutive frames; the edge target point is a dynamic candidate point that is within a set distance threshold of any core target point in the dynamic candidate point range of multiple consecutive frames, where L is an integer greater than 1; the multiple consecutive frames include the frame.

[0022] Based on the annotation requirements of the training samples, step 140 can annotate one or more types of targets among the identified static target points, dynamic target points, and false target points to generate model training samples. In step 120, using the positional relationships between target points, discrete points (also known as outliers) and non-discrete points (also known as clustered points) can be identified from the dynamic candidate points of the multi-frame point cloud. It can be understood that for real and stable moving targets, the corresponding target points in consecutive frames usually form a set of target points that satisfy the clustering / aggregation conditions, while dynamic candidate points that do not belong to any set are likely not target points generated by real moving targets. Therefore, discrete points identified from the dynamic candidate points that do not belong to any set of target points are considered to be instantaneous and unstable target points, which are likely caused by clutter or noise, and are therefore identified as false target points. Thus, for the dynamic candidate points identified in each frame, they are further divided into dynamic target points and false target points.

[0023] The dynamic target points in a frame include core target points and edge target points. A core target point is a dynamic candidate point within a range of dynamic candidate points across multiple consecutive frames, where at least L other dynamic candidate points exist within a set distance threshold. An edge target point is a dynamic candidate point within the range of dynamic candidate points across multiple consecutive frames, located within a set distance threshold of any core target point. L is an integer greater than 1, also known as the minimum number of points. The distance threshold and the minimum number of points L are set values, determined according to radar detection performance, and are not limited to any specific aspect. Assuming the set distance threshold is ε, for a given dynamic candidate point p, the number of target points L1 with a distance less than or equal to ε from the dynamic candidate points across multiple consecutive frames is obtained. If L1 is greater than or equal to the minimum number of points L, then the dynamic candidate point p is determined to be a core target point. If L1 is less than the minimum number of points L, then it is further determined whether the distance between the dynamic candidate point p and any other determined core target point is less than or equal to ε. If so, then the dynamic candidate point p falls within the range of the set distance threshold ε of other core target points, i.e., the dynamic candidate point p is an edge target point. Therefore, for dynamic candidate points in a frame, the identified core target points and edge target points are determined as dynamic target points, while others are determined as false target points.

[0024] In some exemplary embodiments, step 120 includes: determining clusters of points by employing a spatial clustering algorithm based on the positional relationships between the dynamic candidate points; Based on whether the dynamic candidate point belongs to the cluster of points, its type is determined to be a dynamic target point or a false target point; The cluster of clusters includes one or more.

[0025] For dynamic candidate points in multiple consecutive frames, a spatial clustering algorithm is used to determine one or more clusters. Each cluster corresponds to a set of target points. Dynamic candidate points belonging to a certain cluster are determined as dynamic target points, while dynamic candidate points not belonging to any cluster are discrete points and are identified as false target points. The spatial clustering algorithm includes: DBSCAN (Density-Based Spatial Clustering of Applications with Noise), Euclidean clustering, HDBSCAN (Hierarchical Density-Based Spatial Clustering of Applications with Noise), or OPTICS (Ordering Points To Identify the Clustering Structure), etc.

[0026] In other words, if a dynamic candidate point in the current frame can successfully cluster with other dynamic candidate points in multiple consecutive frames, including the current frame, it is considered a stable, real target point and is identified as a dynamic target point; otherwise, it is considered not a stable, real target point, i.e., a false target point. Therefore, a multi-frame clustering algorithm is used to distinguish real moving targets from unstable noise points among dynamic candidate points.

[0027] The "continuous frames" can be a series of consecutive frames including the current frame's t-th frame and ti-th frame, or a series of consecutive frames including the current frame's t-th frame, ti-th frame, and t+j-th frame; it can also be a series of consecutive frames including the current frame's t-th frame and t+j-th frame, without limitation to any specific aspect. i and j are both integers greater than 0. It can be understood that when performing step 120 based on continuous frame point cloud data including t+j-th frame, the execution is delayed compared to the acquisition time of the t-th frame point cloud data. The number of frames can be flexibly determined as needed and is not limited to a specific number.

[0028] In some exemplary embodiments, the vehicle motion state information is determined based on at least one frame of point cloud data obtained by the vehicle-mounted radar; or, it is determined based on at least one of the following external motion sensing modules of the vehicle-mounted radar: a satellite positioning module, an accelerometer, a gyroscope, an inertial measurement unit, a wheel speed sensor, a steering angle sensor, and a vehicle speed sensor.

[0029] In some exemplary embodiments, the vehicle motion state information is determined according to the following method: Based on at least one frame of point cloud data obtained from the vehicle-mounted radar, radar attitude estimation is performed to obtain radar attitude information; and, based on the radar attitude information, the vehicle motion state information is determined. The radar attitude information includes: radar speed and radar azimuth angle; or, radar speed, radar azimuth angle, and radar pitch angle; the vehicle motion status information includes motion speed.

[0030] In some exemplary embodiments, vehicle motion state information includes: speed, or vehicle speed for short, and heading. In some exemplary embodiments, radar attitude information is used as vehicle motion state information, where radar speed is vehicle speed and radar azimuth is heading.

[0031] In some exemplary embodiments, the radar attitude information is determined according to the following method: Based on the first radar attitude information, select multiple candidate stationary target points from the target points detected in the current frame; Using the first radar attitude information as the initial value for fitting, the radar attitude model is fitted according to the radar detection information of the multiple candidate stationary target points to determine the radar attitude information corresponding to the current frame. The first radar attitude information is obtained based on at least one of the following: The radar attitude information corresponding to historical frames and the radar attitude information collected in real time by the radar system's carrying equipment.

[0032] A radar attitude model describes the relationship between radar attitude and radar detection information of multiple target points. This model includes multiple model parameters. Fitting the radar attitude model determines the values ​​of these model parameters, and all or some of these parameters correspond to the radar attitude information of the current frame. Different models correspond to different model parameters. The process of fitting the model and determining the corresponding radar attitude information is also called radar attitude information estimation. In the embodiments of this application, the first radar attitude information, the radar attitude information corresponding to the current frame, the radar attitude information corresponding to historical frames, and the radar attitude information collected from sensors outside the radar system are all radar attitude information, corresponding to data with the same or different values ​​determined at different times, from different sources, or by different methods.

[0033] In some exemplary embodiments, the model includes multiple model parameters: radar velocity and radar azimuth angle; or, the model includes multiple model parameters: radar velocity, radar azimuth angle, and radar elevation angle. Successful fitting means that the fitting has determined these model parameter values, thereby determining the radar's attitude information. It can be understood that the model parameter values ​​are all fitted values, also called estimated values, corresponding to: radar velocity estimate, radar azimuth angle estimate, and radar elevation angle estimate.

[0034] For example, the radar attitude model is a target point velocity-azimuth cosine function, and the model parameters include: radar velocity estimate. Radar azimuth estimate : (1); in, The radar detection information for the i-th target point includes: elevation angle. Azimuth Doppler velocity .

[0035] It should be noted that each frame of radar signal corresponds to radar detection information for multiple target points. The radar detection information for the i-th target point in the t-th frame includes: elevation angle. Azimuth Doppler velocity , The radar detection information for the i-th target point in the (t-1)-th frame includes: elevation angle. Azimuth Doppler velocity If frame t is the current frame, then frame t-1 is the previous frame. Without distinguishing which frame it is, all frames represent the data of the current frame.

[0036] Optionally, the radar attitude model can also be other forms of models or functions, not limited to the cosine function exemplified in the embodiments of this application. In this embodiment, the target point velocity-azimuth cosine function is used as an example to illustrate aspects related to model fitting; this fitting process is also called cosine fitting. In some exemplary embodiments, one of the following methods is used for radar attitude model fitting: Newton's algorithm, LM algorithm, and Doggleg algorithm. That is, the radar attitude model is a target point velocity-azimuth cosine function, and cosine fitting is performed using Newton's algorithm, LM algorithm, or Doggleg algorithm to determine the model parameters of the target point velocity-azimuth cosine function.

[0037] For example, based on one or more frames of point cloud data, the RANSAC (Random Sample Consensus) algorithm is used to fit the Doppler velocity and azimuth distribution of the point cloud to estimate the radar attitude information. The RANSAC algorithm is then used for in-class point identification, where in-class points are stationary target points and non-in-class points are non-stationary target points. That is, after determining more accurate candidate stationary target points from among the candidate stationary target points, model fitting is performed to obtain the radar attitude information.

[0038] In some exemplary embodiments, radar point cloud data is obtained by detecting radar signals in each frame, including radar detection information corresponding to multiple target points.

[0039] In some exemplary embodiments, the radar detection information of the target point includes one or more of the following: elevation angle, azimuth angle, and Doppler velocity; The plurality of candidate stationary target points are multiple target points among the target points detected in the current frame that satisfy one or more of the following constraints: Constraint 1: The pitch angle of the target point falls within the set pitch angle threshold range; Constraint 2: The Doppler velocity of the target point is less than the difference between the set Doppler velocity threshold and the first velocity; wherein, the first velocity is the ground motion velocity of the target point determined based on the first radar attitude information; Constraint 3: The azimuth angle of the target point falls within the first azimuth angle range; wherein, the first azimuth angle range is the azimuth angle range of a stationary target determined based on the first radar attitude information under the set maximum unambiguous velocity condition.

[0040] Based on radar attitude information, vehicle motion state information can be calculated. In some exemplary embodiments, since single-frame estimation may have significant errors, radar attitude information can be obtained by fitting data from multiple consecutive frames of radar point cloud data and then smoothing it to obtain corrected radar attitude information. For example, median filtering can be used for smoothing to obtain more stable and accurate radar attitude information, and thus the corresponding vehicle motion state information.

[0041] In some exemplary embodiments, vehicle motion state information can also be determined using one or more motion sensing modules external to the onboard radar. To improve the accuracy of the motion state information, the results from multiple modules can be combined to determine the overall speed. For example, both satellite positioning modules and wheel speed sensors can obtain the vehicle's speed, and the average of the two can be taken as the vehicle's speed.

[0042] In some exemplary embodiments, the step of dividing the target point into static target points and dynamic candidate points based on vehicle motion state information includes: Based on the vehicle's motion speed in the vehicle motion state information and the speed of the target point in the point cloud data, and based on the principle of speed consistency, the target point is determined to be a static target point or a dynamic candidate point.

[0043] In some exemplary embodiments, determining whether a target point is a static target point or a dynamic candidate point based on the principle of speed consistency means classifying target points in the point cloud data according to the analytical model of motion compensation, and determining the target points as static target points or dynamic candidate target points; wherein, the analytical model of motion compensation is constructed based on the principle of speed consistency between the vehicle and the target point. It can be understood that speed consistency means that the speed of a static target point conforms to the expected speed caused by vehicle motion, and a target point that does not conform to the principle of speed consistency is not a static target point and is initially determined as a dynamic candidate point.

[0044] In some exemplary embodiments, the vehicle motion state information is the radar attitude information; correspondingly, based on the speed consistency principle, determining the target point as a static target point or a dynamic candidate point includes: Target points that meet the following conditions are identified as dynamic candidate points: (2); in, For maximum unambiguous speed The Leyen velocity of the i-th target point in frame t (the current frame) The azimuth angle of the i-th target point in the t-th frame (current frame) The radar velocity estimate in the radar attitude information of frame t. The estimated radar azimuth angle in the radar attitude information of frame t. k is the maximum fuzziness factor. This is the dynamic candidate speed threshold, which is a preset value.

[0045] It can be understood that target points that meet the above conditions are dynamic candidate points, while target points that do not meet the conditions are static target points. This achieves preliminary identification of the type of target points in point cloud data.

[0046] In some exemplary embodiments, such as Figure 2 As shown, the method further includes: Step 130: Perform mirror multipath filtering on the dynamic target points and update the type of the filtered-out dynamic target points to false target points.

[0047] Vehicle-mounted radar systems are installed on moving cars. The presence of roadside facilities such as guardrails and tunnel walls creates strong reflective surfaces for radar signals. These strong reflective surfaces cause multipath reflections, resulting in ghosting. Therefore, among the identified dynamic target points, there may also be target points corresponding to these ghosting images, which can further improve identification accuracy and thus enhance the accuracy of target point type labeling.

[0048] In some exemplary embodiments, the mirror multipath filtering process performed on the dynamic target point includes: Obtain road boundaries; Using the road boundary as a reflective surface, for the dynamic target point, determine whether there is a mirror source corresponding to the dynamic target point according to the reflection model; if there is, filter out the dynamic target point.

[0049] In some exemplary embodiments, the road boundary is identified based on multiple static target points; that is, the road boundary is identified based on the static target points identified in step 110. In some exemplary embodiments, the location of the road boundary can be identified by performing histogram analysis on the static target points accumulated from multiple frames of point cloud data in the vehicle's lateral direction. The road boundary includes: guardrails, median strips, green belts, walls, etc. The vehicle's lateral direction is also called the y-axis direction, and correspondingly, the vehicle's longitudinal direction, which is the direction of the vehicle's movement, is called the x-axis direction.

[0050] It is understandable that in some exemplary embodiments, the road boundary is used as the reflecting surface. According to the principle of reflection, a radar wave emitted by a real dynamic target (such as a vehicle or pedestrian) is reflected by the road boundary (such as a guardrail, median, or wall) and then returns to the radar receiver. This is misinterpreted by the radar as a false dynamic target existing on the other side of the road boundary. In other words, the radar incorrectly treats the echo received after reflection as a direct transmission from a new target, thus "imagining" a target on the other side of the reflecting surface. This real target is considered the mirror source of this imaginary / false target, which is also called a ghost image. Based on the known reflecting surface and the principle of reflection, it can be determined whether a corresponding mirror source (real target) exists for a certain imaginary / false target. Therefore, this embodiment further identifies dynamic target points and filters out imaginary / false target points that satisfy the electromagnetic wave reflection relationship with real targets (points). The type of these filtered-out dynamic target points is reclassified as false target points. This further improves the accuracy of target point category labeling.

[0051] It should be noted that the road boundary can also be obtained through other methods, such as obtaining the road boundary based on map data, or determining it through image recognition, and is not limited to any particular aspect. In comparison, road boundary identification based on identified static target points can introduce more auxiliary systems, reduce system complexity, and improve execution efficiency.

[0052] In this embodiment of the disclosure, the vehicle-mounted radar or general radar system includes: an FMCW (Frequency Modulated Continuous Wave) radar system, a PMCW (Phase Modulated Continuous Wave) radar system, or a pulse compression radar system. It can be a millimeter-wave radar, microwave radar, or radar operating in multiple frequency bands, and is not limited to any particular aspect.

[0053] This disclosure also provides a target point classification method for classifying target points in multi-frame point cloud data obtained by radar, such as... Figure 3 As shown, the method includes: Step 310: Based on the radar's motion status information, the target points are divided into static target points and dynamic candidate points; Step 320: Based on the positional relationship between the dynamic candidate points, perform false target detection on the dynamic candidate points, and determine the detected target points as false target points, and the other target points as dynamic target points; The dynamic target points in a frame include: core target points and edge target points; the core target point is a dynamic candidate point that has at least L other dynamic candidate points within a set distance threshold in the dynamic candidate point range of multiple consecutive frames; the edge target point is a dynamic candidate point that is within a set distance threshold of any core target point in the dynamic candidate point range of multiple consecutive frames, where L is an integer greater than 1; the multiple consecutive frames include the frame.

[0054] In some exemplary embodiments, the radar includes: frequency modulated continuous wave (FMCW) radar, phase modulated continuous wave (PMCW) radar, or pulse compression radar. Multiple selectable frequency bands are supported, such as millimeter-wave radar, etc.

[0055] In some exemplary embodiments, the motion state information of the radar is the vehicle motion state information in the automatic labeling method. It can be the radar attitude information estimated from point cloud data, or the motion state information determined by the motion sensor module connected to the radar system equipment. The detailed steps will not be described here.

[0056] In some exemplary embodiments, the method for detecting false target points in step 320 is similar to the method for identifying false target points in step 120 of the automatic annotation method. Clustering is performed based on the positional relationship of dynamic candidate points, and the discrete points are the detected false target points. For detailed steps, please refer to the relevant implementation of step 120.

[0057] In some exemplary embodiments, the method further includes: Step 330: Perform mirror multipath filtering on the dynamic target points and determine the filtered-out dynamic target points as false target points.

[0058] Similarly, after step 320, step 330 is performed to further process the initially determined dynamic target points, identify any mirrored ghost target points that may be included, and change their type to false target points.

[0059] As can be seen, after classifying target points based on multi-frame point cloud data, different types of point cloud data can be input, such as outputting static target point clouds, dynamic target point clouds, or spurious target point clouds, which enriches the semantic information of the output point cloud data and meets the different needs of other applications.

[0060] This disclosure also provides a training set generation method, such as... Figure 4 As shown, it includes: Step 410: For the multi-frame point cloud data obtained by the vehicle-mounted radar, perform the following steps respectively to generate training samples to form the training set: Step 4110: Extract feature data of the target point based on the current frame point cloud data; Step 4120: Obtain M target points in the current frame point cloud data, and label the type corresponding to the target points according to the automatic labeling method as described in any embodiment of this disclosure, to obtain the training samples corresponding to the current frame point cloud data, where M is a preset value greater than 0.

[0061] This disclosure also provides a training set generation method, such as... Figure 5 As shown, it includes: Step 510: Based on the multi-frame point cloud data obtained by radar, perform the following steps respectively to generate training samples to constitute the training set: Step 5110: Extract feature data of the target point based on the current frame point cloud data; Step 5120: Obtain M target points in the current frame point cloud data, determine the type of the target points according to the method described in any embodiment of this disclosure, and label them to obtain the training samples corresponding to the current frame point cloud data, where M is a preset value greater than 0.

[0062] It is understandable that feature data of target points are extracted for each frame of point cloud data, M target points are determined and their types are labeled, and training samples corresponding to each frame of point cloud data are generated, which constitute the training set.

[0063] This disclosure also provides a training set generation method, including: The following steps are performed on the multi-frame point cloud data obtained by the vehicle-mounted radar to generate training samples to form the training set: Extract feature data of the target point based on the current frame point cloud data; Get M target points in the current frame point cloud data, and label the type corresponding to the target points to obtain the training samples corresponding to the current frame point cloud data, where M is a preset value greater than 0; The feature data includes: spatial location data and Doppler velocity; or, the feature data includes: spatial location data and Doppler velocity, and further includes: radar signal-to-noise ratio and / or azimuth angle.

[0064] In some exemplary embodiments, the automatic annotation method described in any embodiment of this disclosure is used to annotate the type corresponding to the target point.

[0065] The feature data includes: spatial location data and Doppler velocity; or, the feature data includes: spatial location data and Doppler velocity v, and further includes: radar signal-to-noise ratio p and / or azimuth angle θ. The spatial location data includes: the longitudinal, lateral, and vertical positions of the target point, denoted as (x, y, z). The feature data is extracted from point cloud data; specific feature extraction methods are not discussed in detail here and are not limited to any particular aspect.

[0066] As can be seen, in some exemplary embodiments, the feature data also includes the radar signal-to-noise ratio p and / or azimuth angle θ, which can further enrich the feature elements of the training samples, provide richer classification basis, and improve the training effect of the classification model.

[0067] In some exemplary embodiments, obtaining M target points in the current frame point cloud data includes: The M target points are determined based on the current frame point cloud data by using a set data completion or sampling method.

[0068] Each frame of point cloud data corresponds to a training sample containing M target points, along with M feature data and labeled target point types. M is a preset value, flexibly set according to the amount of point cloud data and application requirements. For example, M=2048, meaning each frame's training sample includes 2048 target points. It can be seen that the number of target points in each frame of point cloud data is not consistent. When the original number of target points is less than M, a pre-defined data padding method is used to determine M target points based on the current frame's point cloud data. For example, if the pre-defined data is 0, then zero-padding is used to pad the current frame's point cloud data with the original target points to obtain M target points. When the original number of target points is greater than M, a sampling method is used to sample (extract) M target points from all target points. The sampling strategy can be random sampling, or conditional sampling based on one or more of the feature data, depending on the model's future potential application scenarios. For example, priority sampling can be based on the signal-to-noise ratio p, or it can be limited to specific aspects.

[0069] The feature data and automatically generated type labels corresponding to M target points accumulated over multiple frames constitute the training samples for the accumulated point cloud data, laying the foundation for the next step of model training.

[0070] This disclosure also provides a classification model training method, which trains a point cloud target point classification model based on the constructed training set.

[0071] This disclosure also provides a classification model training method, such as... Figure 6 As shown, it includes: Step 610: Obtain the training set and use the training samples included in the training set to train the point cloud target point classification model; The training set includes: training samples generated from multiple frames of point cloud data obtained from vehicle-mounted radar; the training samples include at least one labeled target point; the type of the target point is labeled according to the automatic labeling method described in any embodiment of this disclosure; or, the type of the target point is determined and labeled according to the target point classification method described in any embodiment of this disclosure. The point cloud target point classification model is used to identify the type of target points in point cloud data.

[0072] In some exemplary implementations, the point cloud target point classification model is a deep learning model, which includes at least one symmetrically configured feature abstraction module and feature reconstruction module. The feature abstraction module is configured to perform feature extraction after downsampling based on the input data of this layer. The feature reconstruction module is configured to perform feature extraction after upsampling based on the input data of this layer. The feature abstraction module is further configured to perform downsampling on the input data of this layer based on the longitudinal position, lateral position, and Doppler velocity (x, y, v) of the target point in the point cloud data.

[0073] This disclosure also provides a classification model training method, including: Obtain a training set, and use the training samples included in the training set to train a point cloud target point classification model; The training set includes: training samples generated from multiple frames of point cloud data obtained by vehicle-mounted radar; each training sample includes at least one labeled target point; and the point cloud target point classification model is used to identify the type of target points in the point cloud data. The point cloud target point classification model is a deep learning model, which includes at least one symmetrically arranged feature abstraction module and feature reconstruction module. The feature abstraction module is configured to perform feature extraction after downsampling based on the input data of the current layer. The feature reconstruction module is configured to perform feature extraction after upsampling based on the input data of the current layer. The feature abstraction module is also configured to perform downsampling based on the longitudinal position, lateral position, and Doppler velocity (x, y, v) of the target point in the point cloud data.

[0074] Regarding the above-described model training method, in some exemplary embodiments, training a point cloud target point classification model is performed using training samples included in the training set, including: The point cloud target point classification model is trained after performing one or more of the following data augmentation processes on the training samples: feature data jitter, feature data random scaling, feature data rotation along the z-axis, random discarding of some target points, and rearrangement of the order of point cloud target points.

[0075] Based on the characteristics of radar point cloud data, employing one or more of the above data augmentation techniques can effectively enhance the diversity of training data and improve the model's generalization ability.

[0076] Regarding the above-described model training method, in some exemplary embodiments, training a point cloud target point classification model is performed using training samples included in the training set, including: When calculating the loss function during training, label smoothing is performed on the types of target points labeled in the training samples.

[0077] The label smoothing process includes: Label smoothing is performed using the following formula: (3); in, For smoothed labels, To smooth out the previous labels, For smoothing coefficients, K Number of types. Smoothing coefficient. To set a value, for example, K=3 represents three types: static target point, dynamic target point, and false target point.

[0078] Since the types of each target point in the training sample are labeled according to the automatic labeling method or classification method provided in this disclosure embodiment, and do not depend on other truth systems or manual confirmation, when calculating the loss function during training, one-hot hard labels are not used, but the label smoothing formula (3) is applied. This will assign a very small probability to the incorrect category, prevent the model from being too confident in the potential noise in the automatically labeled data, and effectively ensure the reliability of the model training results.

[0079] As can be seen, considering that automated labeling may introduce a small amount of noise (incorrect labels), the implementation scheme of this disclosure incorporates label smoothing and data augmentation techniques during the model training phase. Label smoothing prevents the model from "overfitting" to potentially incorrect labels, while data augmentation expands the diversity of the training data. This makes the finally trained model insensitive to noisy labels, exhibiting stronger generalization ability and robustness.

[0080] In some exemplary embodiments, the feature abstraction module in the deep learning model passes the extracted features to the feature reconstruction module at the same layer through a bridging link.

[0081] In some exemplary embodiments, the deep learning model includes one of the following: PointTransformer model, PointTransformerV2 model, PointTransformerV3 model and PointNet++ model.

[0082] For example, Figure 7 The deep learning model shown—the PointTransformer model—includes a multi-layered symmetrical feature abstraction module 710 and a feature reconstruction module 720. The feature abstraction module performs the following steps: point cloud downsampling, nearest neighbor lookup, and feature extraction transformation. The feature reconstruction module performs the following steps: point cloud feature interpolation (also known as point cloud upsampling) and feature extraction transformation. Point cloud downsampling uses either farthest point sampling or random sampling. Nearest neighbor lookup uses the KNN algorithm or the ball-query algorithm; the number of points required for nearest neighbor lookup is flexibly set as needed. Feature extraction transformation uses a shared multilayer perceptron, convolution, or Transformer approach. Point cloud feature interpolation uses weighted interpolation based on examples. A cross-link is established between the feature abstraction module and the feature reconstruction module at the same layer, enabling the feature reconstruction module to obtain the features extracted during feature abstraction when performing feature extraction transformation.

[0083] Some exemplary PointTransformer models built for millimeter-wave point cloud data include a three-layer symmetrically configured feature abstraction module and feature reconstruction module. Depending on the characteristics of the final model's application scenario, in some exemplary implementations, to improve training efficiency, the bridging link between the feature abstraction module and feature reconstruction module in the configuration layer can be disconnected, provided training requirements are met.

[0084] It should be noted that the feature abstraction module performs downsampling on the input data of this layer based on the longitudinal, lateral, and Doppler velocities (x, y, v) of the target point. Compared to some feasible solutions where the downsampling dimension of the model is spatial coordinates, i.e., the longitudinal, lateral, and vertical positions (x, y, z) of the target point, the point cloud target point classification model provided in this embodiment optimizes the sampling by adjusting the model's sampling dimension to more suitable radar data (x, y, v). This fully utilizes the advantage of radar's accurate velocity measurement and overcomes the limitation of limited accuracy of the vertical position z in traditional point cloud data, thereby improving the model training effect.

[0085] In some exemplary implementations, the classification model training method is as follows: Figure 8 As shown, it includes: Step 810, feature extraction, to obtain the feature data corresponding to a point cloud frame: x, y, z, v, p, θ; Step 820: Automatic annotation, complete the annotation of the target point type corresponding to the point cloud; Step 830: Pad with zeros or sample to obtain M target points and generate corresponding training samples; Step 840: Accumulate training samples from multiple frames to form a training set; Step 850: Perform model training to obtain the trained point cloud target point classification model; Step 820 includes: Step 8210, obtaining vehicle motion state information: performing radar attitude estimation based on point cloud data, and then determining the corresponding vehicle motion state information; Step 8220, Static / Dynamic Recognition: Based on the principle of speed consistency, identify static target points and dynamic candidate points; Step 8230: Identify road boundaries based on static target points; Step 8240: Perform spatial clustering based on dynamic candidate points in multiple consecutive frames to identify false target points and dynamic target points; Step 8250: Based on the identified road boundaries, perform mirror multipath filtering on the dynamic target points identified in step 8240, and change the filtered dynamic target points into false target points; Therefore, target points are labeled according to the determined type. This completes the automatic labeling of raw, unlabeled point cloud data into three types: static target points, dynamic target points, and spurious target points.

[0086] As can be seen from the automatic annotation scheme provided in this disclosure, the core of the scheme relies on radar point cloud data for automated target point identification, avoiding reliance on manual annotation and sensor-based truth systems. This improves annotation efficiency, simplifies the complexity of the annotation scheme, and achieves low-cost, high-efficiency training set construction. For the AI ​​model, richer feature data is added, the data sampling dimension in the model is optimized, and more accurate velocity measurement results from radar point cloud data are used for data downsampling. Label smoothing and data augmentation are introduced during the training phase to compensate for potential inaccuracies in the automatic annotation scheme. This makes the final trained model insensitive to noisy labels and has stronger generalization ability and robustness. After training samples are annotated using this method and the AI ​​model is trained, it is applied to point cloud classification in practical applications. Compared to rule-based target recognition schemes in some feasible solutions, this scheme fully utilizes the intelligent advantages of the AI ​​model, not only solving the difficulty and accuracy problems of classification annotation but also improving classification accuracy, laying the foundation for performance assurance in more downstream intelligent applications.

[0087] This disclosure also provides a point cloud classification method, such as... Figure 9 As shown, it includes: Step 910: Extract feature data of the target point based on the current frame point cloud data obtained by the vehicle-mounted radar; Step 920: Input the feature data of the target point into the trained classification model to classify the target point; The classification model is pre-trained according to the classification model training method described in any embodiment of this disclosure.

[0088] In some exemplary embodiments, such as Figure 10 As shown, the feature data includes: x, y, z, v, p, θ; Step 920 includes determining the M target points based on the current frame point cloud data using a set data completion or sampling method, and inputting the feature data of the M target points into the trained classification model to classify the target points.

[0089] As can be seen, after training is complete, the trained classification model can be used to classify target points based on the original point cloud data. Based on the prior automatic annotation, automatic classification, training set construction, and model training, the trained classification model can classify target points more intelligently and efficiently, meeting the classification needs of more complex scenarios and improving classification accuracy and execution efficiency. Applied to in-vehicle intelligent driving system solutions, this lays a solid foundation for the performance of autonomous driving perception systems.

[0090] This disclosure also provides an integrated circuit including a processor configured to implement the automatic labeling method as described in any embodiment of this disclosure.

[0091] This disclosure also provides an integrated circuit including a processor configured to implement the target point classification method as described in any embodiment of this disclosure.

[0092] This disclosure also provides an integrated circuit including a processor configured to implement the training set generation method as described in any embodiment of this disclosure.

[0093] This disclosure also provides an integrated circuit including a processor configured to implement the classification model training method as described in any embodiment of this disclosure.

[0094] This disclosure also provides an integrated circuit including a processor configured to implement the point cloud classification method as described in any embodiment of this disclosure.

[0095] This disclosure also provides an electromagnetic wave device, including: A carrier; an integrated circuit as described in any embodiment of this disclosure is disposed on the carrier; An antenna is disposed on the carrier, or the antenna and the integrated circuit are integrated into a single device and disposed on the carrier. The integrated circuit is connected to the antenna and is used to transmit the radio frequency transmission signal and / or receive the radio frequency reception signal.

[0096] This disclosure also provides a terminal device, including: One or more processors; Storage device for storing one or more programs. When the one or more programs are executed by the one or more processors, the one or more processors implement the point cloud classification method as described in any embodiment of this disclosure.

[0097] In some exemplary embodiments, the terminal device includes: a millimeter-wave radar device; or, the terminal device includes: a frequency modulated continuous wave (FMCW) radar device, a phase modulated continuous wave (PMCW) radar device, or a pulse compression radar device.

[0098] In some exemplary implementations, the terminal device is installed on a smart car or integrated with a smart driving system, and is not limited to a specific method.

[0099] It will be understood by those skilled in the art that all or some of the steps, systems, or apparatuses disclosed above, and their functional modules / units, can be implemented as software, firmware, hardware, or suitable combinations thereof. In hardware implementations, the division between functional modules / units mentioned above does not necessarily correspond to the division of physical components; for example, a physical component may have multiple functions, or a function or step may be performed collaboratively by several physical components. Some or all components may be implemented as software executed by a processor, such as a digital signal processor or microprocessor, or as hardware, or as an integrated circuit, such as an application-specific integrated circuit (ASIC). Such software may be distributed on a computer-readable medium, which may include computer storage media (or non-transitory media) and communication media (or transient media). As is known to those skilled in the art, the term computer storage media includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information (such as computer-readable instructions, data structures, program modules, or other data). Computer storage media include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technologies, CD-ROM, digital versatile disc (DVD) or other optical disc storage, magnetic cartridges, magnetic tape, disk storage or other magnetic storage devices, or any other medium that can be used to store desired information and can be accessed by a computer. Furthermore, it is well known to those skilled in the art that communication media typically contain computer-readable instructions, data structures, program modules, or other data in modulated data signals such as carrier waves or other transmission mechanisms, and may include any information delivery medium.

Claims

1. An automatic annotation method, characterized in that, The method for labeling target points in multi-frame point cloud data obtained by vehicle-mounted radar includes: Based on the vehicle motion status information, the target points are divided into static target points and dynamic candidate points; Based on the positional relationship between the dynamic candidate points, the dynamic candidate points are divided into dynamic target points and false target points; The target point is labeled with its type, which includes at least one of the following: static target point, dynamic target point, and false target point; The dynamic target points in a frame include: core target points and edge target points; the core target point is a dynamic candidate point that has at least L other dynamic candidate points within a set distance threshold in the dynamic candidate point range of multiple consecutive frames; the edge target point is a dynamic candidate point that is within a set distance threshold of any core target point in the dynamic candidate point range of multiple consecutive frames, where L is an integer greater than 1; the multiple consecutive frames include the frame.

2. The automatic annotation method according to claim 1, characterized in that, The vehicle motion state information is determined based on at least one frame of point cloud data obtained by the vehicle-mounted radar. or, The vehicle motion state information is determined based on at least one of the following external motion sensing modules of the onboard radar: Satellite positioning module, accelerometer, gyroscope, inertial measurement unit, wheel speed sensor, steering angle sensor and vehicle speed sensor.

3. The automatic annotation method according to claim 2, characterized in that, The vehicle motion state information is determined according to the following method: Based on at least one frame of point cloud data obtained from the vehicle-mounted radar, radar attitude estimation is performed to obtain radar attitude information. And, the vehicle motion state information is determined based on the radar attitude information; The radar attitude information includes: radar speed and radar azimuth angle; or, radar speed, radar azimuth angle, and radar pitch angle; the vehicle motion status information includes motion speed.

4. The automatic annotation method according to any one of claims 1-3, characterized in that, The step of dividing the target point into static target points and dynamic candidate points based on vehicle motion state information includes: Based on the vehicle's motion speed in the vehicle motion state information and the speed of the target point in the point cloud data, and based on the principle of speed consistency, the target point is determined to be a static target point or a dynamic candidate point.

5. The automatic annotation method according to any one of claims 1-3, characterized in that, Before labeling the type of the target point, the method further includes: Mirror multipath filtering is performed on the dynamic target points to update the type of the filtered-out dynamic target points to false target points.

6. The automatic annotation method according to claim 5, characterized in that, The mirror multipath filtering process performed on the dynamic target point includes: Identify road boundaries based on multiple static target points; Using the road boundary as a reflective surface, for the dynamic target point, determine whether there is a mirror source corresponding to the dynamic target point according to the reflection model; if there is, filter out the dynamic target point.

7. The automatic annotation method according to any one of claims 1-3, characterized in that, The step of classifying the dynamic candidate points into dynamic target points and false target points based on the positional relationship between the dynamic candidate points includes: Based on the positional relationship between the dynamic candidate points, a spatial clustering algorithm is used to determine the cluster point clusters; Based on whether the dynamic candidate point belongs to the cluster of points, its type is determined to be a dynamic target point or a false target point; The cluster of points includes one or more.

8. A target point classification method, characterized in that, The method for classifying target points in multi-frame point cloud data obtained by radar includes: Based on the motion state information of the radar, the target points are divided into static target points and dynamic candidate points; Based on the positional relationship between the dynamic candidate points, false target detection is performed on the dynamic candidate points, and the detected target points are determined as false target points, while other target points are determined as dynamic target points; The radar includes: frequency modulated continuous wave (FMCW) radar, phase modulated continuous wave (PMCW) radar, or pulse compression radar. The dynamic target points in a frame include: core target points and edge target points; the core target point is a dynamic candidate point that exists within a set distance threshold of at least L other dynamic candidate points in the dynamic candidate point range of multiple consecutive frames; the edge target point is a dynamic candidate point that is within a set distance threshold of any core target point in the dynamic candidate point range of multiple consecutive frames, where L is an integer greater than 1; the multiple consecutive frames include the frame.

9. The target point classification method according to claim 8, characterized in that, The method further includes: Mirror multipath filtering is performed on the dynamic target points to identify the filtered-out dynamic target points as false target points.

10. The target point classification method according to claim 8 or 9, characterized in that, The step of detecting false targets by the dynamic candidate points based on the positional relationship between the dynamic candidate points includes: Based on the positional relationship between the dynamic candidate points, a spatial clustering algorithm is used to determine the cluster point clusters; If the dynamic candidate point does not belong to the cluster of points, the dynamic candidate point is determined to be a false target point; The cluster of points includes one or more.

11. A training set generation method, characterized in that, include: For the multi-frame point cloud data obtained from the vehicle-mounted radar, the following steps are performed respectively to generate training samples to form the training set: Extract feature data of the target point based on the current frame point cloud data; Get M target points in the current frame point cloud data, and label the type corresponding to the target points according to the automatic labeling method as described in any one of claims 1-7 to obtain the training samples corresponding to the current frame point cloud data, where M is a preset value greater than 0; or, Based on multi-frame point cloud data obtained from radar, the following steps are performed to generate training samples to form the training set: Extract feature data of the target point based on the current frame point cloud data; Get M target points in the current frame point cloud data, determine the type of the target points according to the method of any one of claims 8-10, and label them to obtain the training samples corresponding to the current frame point cloud data, where M is a preset value greater than 0; The radar includes: frequency modulated continuous wave (FMCW) radar, phase modulated continuous wave (PMCW) radar, or pulse compression radar.

12. The training set generation method according to claim 11, characterized in that, The feature data includes: spatial location data and Doppler velocity; or, the feature data includes: spatial location data and Doppler velocity, and further includes: radar signal-to-noise ratio and / or azimuth angle.

13. A training set generation method, characterized in that, include: The following steps are performed on the multi-frame point cloud data obtained by the vehicle-mounted radar to generate training samples to form the training set: Extract feature data of the target point based on the current frame point cloud data; Get M target points in the current frame point cloud data, and label the type corresponding to the target points to obtain the training samples corresponding to the current frame point cloud data, where M is a preset value greater than 0; The feature data includes: spatial location data and Doppler velocity; or, the feature data includes: spatial location data and Doppler velocity, and further includes: radar signal-to-noise ratio and / or azimuth angle.

14. The training set generation method according to claim 13, characterized in that, The types corresponding to the labeled target points include: The target point is labeled with the type according to the automatic labeling method as described in any one of claims 1-7.

15. The training set generation method according to claim 13, characterized in that, The step of obtaining M target points from the current frame point cloud data includes: The M target points are determined based on the current frame point cloud data by using a set data completion or sampling method.

16. A classification model training method, characterized in that, include: Obtain a training set, and use the training samples included in the training set to train a point cloud target point classification model; The training set includes: training samples generated from multiple frames of point cloud data obtained from the vehicle-mounted radar; each training sample includes at least one labeled target point. The type of the target point is labeled according to the automatic labeling method as described in any one of claims 1-7; or, the type of the target point is determined and labeled according to the target point classification method as described in any one of claims 8-10. The point cloud target point classification model is used to identify the type of target points in point cloud data.

17. The classification model training method according to claim 16, characterized in that, The point cloud target point classification model is a deep learning model, which includes at least one symmetrically arranged feature abstraction module and feature reconstruction module. The feature abstraction module is configured to perform feature extraction after downsampling based on the input data of this layer. The feature reconstruction module is configured to perform feature extraction after upsampling based on the input data of this layer. The feature abstraction module is further configured to perform downsampling on the input data of this layer based on the longitudinal position, lateral position and Doppler velocity of the target point in the point cloud data.

18. A classification model training method, characterized in that, include: Obtain a training set, and use the training samples included in the training set to train a point cloud target point classification model; The training set includes: training samples generated from multiple frames of point cloud data obtained by vehicle-mounted radar; each training sample includes at least one labeled target point; and the point cloud target point classification model is used to identify the type of target points in the point cloud data. The point cloud target point classification model is a deep learning model, which includes at least one symmetrically arranged feature abstraction module and feature reconstruction module. The feature abstraction module is configured to perform feature extraction after downsampling based on the input data of the current layer. The feature reconstruction module is configured to perform feature extraction after upsampling based on the input data of the current layer. The feature abstraction module is also configured to perform downsampling based on the vertical position, horizontal position, and Doppler velocity of the target point in the point cloud data.

19. The classification model training method according to claim 18, characterized in that, The training set is generated according to the training set generation method as described in any one of claims 11-15.

20. The classification model training method according to claim 18, characterized in that, The step of training a point cloud target point classification model using training samples included in the training set includes: After performing one or more of the following data augmentation processes on the training samples, the point cloud target point classification model is trained: Feature data jitter, random scaling of feature data, rotation of feature data along the z-axis, random discarding of some target points, and rearrangement of the order of point cloud target points.

21. A point cloud classification method, characterized in that, include: Feature data of target points are extracted from the current frame point cloud data obtained by vehicle-mounted radar. The feature data of the target point is input into the trained classification model to classify the target point. The classification model is pre-trained according to the method described in any one of claims 16-20.

22. An integrated circuit, characterized in that, The device includes a processor configured to implement the automatic annotation method as described in any one of claims 1-7; or, to implement the target point classification method as described in any one of claims 8-10; or, to implement the training set generation method as described in any one of claims 11-15; or, to implement the classification model training method as described in any one of claims 16-20; or, to implement the point cloud classification method as described in claim 21.

23. A terminal device, characterized in that, include: One or more processors; Storage device for storing one or more programs. When the one or more programs are executed by the one or more processors, the one or more processors implement the point cloud classification method as described in claim 21.

24. The terminal device according to claim 23, characterized in that, The terminal equipment includes: millimeter-wave radar equipment; or, the terminal equipment includes: frequency modulated continuous wave (FMCW) radar equipment, phase modulated continuous wave (PMCW) radar equipment, or pulse compression radar equipment.

Images