Cross-radar model multi-ray distillation 3D target detection method and system
The multi-ray distillation method across radar models addresses the challenge of high-cost LiDAR sensors by transferring knowledge from high-beam to low-beam models, enhancing feature learning and generalization for efficient 3D object detection in industrial settings.
Patent Information
- Application Number
- CN202411585646.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-11-08
- Publication Date
- 2024-12-06
- Estimated Expiration
- 2044-11-08
AI Technical Summary
Existing 3D object detection methods are not suitable for industrial applications due to high costs of high-resolution LiDAR sensors and varying LiDAR beam counts across different models, leading to insufficient training data and poor performance in real-world environments.
A multi-ray distillation method using a teacher-student knowledge distillation model across radar models, involving preprocessing, feature extraction, alignment, and distillation to transfer knowledge from high-beam LiDAR models to low-beam models, enhancing feature learning and generalization.
The method achieves lightweight model deployment with high accuracy and improved generalization, effectively handling low-density data sets and unseen environments, suitable for industrial applications.
Smart Images

Figure CN119091434A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of target detection technology, and in particular to a multi-ray distillation 3D target detection method and system across radar models. Background Art
[0002] The rise of autonomous driving has attracted more and more attention to 3D object detection. Autonomous driving systems need to perceive the surrounding environment in real time to plan safe driving routes. 3D object detection can predict the position, size and category of key 3D objects near autonomous vehicles, such as motor vehicles, pedestrians, cyclists, etc., providing a basis for the decision-making of autonomous driving systems.
[0003] 3D object detection is a key task in the field of computer vision, which aims to identify and locate objects in three-dimensional space. Different from traditional 2D object detection, 3D object detection not only needs to identify the category and position of objects in images, but also needs to determine the size, direction and posture of objects in three-dimensional space. 3D object detection has important application value in the fields of autonomous driving, robot navigation, augmented reality (AR), virtual reality (VR) and so on.
[0004] 3D object detection has been increasingly used in industry, especially LiDAR-based 3D object detection has attracted widespread attention due to its critical importance in autonomous driving. Although substantial progress has been made, most existing research focuses on a single domain, where both training and test data are obtained from the same LiDAR sensor. In many practical cases, we usually only have access to low-beam point clouds (e.g., 16 beams) because the cost of high-resolution LiDAR (e.g., 64 beams) is prohibitively high; for example, the cost of a 64-beam LiDAR sensor exceeds that of a cleaning robot or an unmanned delivery vehicle. In addition, as product models evolve, robots and vehicles may be equipped with LiDAR sensors with different beam counts, making it impractical to collect and annotate a large dataset for each product type. Unlike research-driven datasets, training data in industrial settings usually have environments similar to those encountered during inference, and the LiDAR beam counts in industrial settings tend to be low.
[0005] Most existing 3D object detection methods are only suitable for academic research, without considering industrial applications. Therefore, although many existing methods have achieved good results on open source datasets, they are not representative and difficult to apply to industrial environments. Summary of the invention
[0006] In order to solve the above-mentioned problems, the present invention provides a multi-ray distillation 3D target detection method and system across radar models.
[0007] In a first aspect, the present invention provides a multi-ray distillation 3D target detection method across radar models, which adopts the following technical solution: A multi-ray distillation 3D target detection method across radar models, comprising: Get radar point cloud information; Preprocess radar point cloud information; Construct a teacher-student knowledge distillation model; The teacher-student knowledge distillation model is used to extract features from radar point cloud information, and the extracted features are distilled to obtain feature maps; Perform feature alignment on the feature maps obtained by distillation to obtain aligned features; The object detection model is used to predict the aligned features and obtain the prediction results.
[0008] Furthermore, the preprocessing of the radar point cloud information includes segmenting the radar point cloud information and assigning wave number labels to the segmented points through a clustering method to obtain downsampled radar point cloud information.
[0009] Furthermore, the teacher-student knowledge distillation model is constructed, including a teacher model based on LiDAR, using the BackBone of the PointPillars network as the teacher model, using the 3D backbone to extract voxel features from the LiDAR points, and then processing the voxel features through the 2D BackBone to obtain BEV features, and at the same time setting an output layer for target detection in the last layer, that is, the Head part, which is used to predict the category and bounding box of the target existing in the BEV feature.
[0010] Furthermore, the teacher-student knowledge distillation model is used to extract features from radar point cloud information, including voxelizing the three-dimensional coordinates of the point cloud in the radar point cloud information to obtain voxel features, and performing feature learning on the voxel features through a PointPillars network to obtain enhanced voxel features.
[0011] Furthermore, the extracted features are subjected to feature distillation to obtain feature maps, including sequentially performing ROD feature distillation and FORD feature distillation on radar point cloud information, wherein the ROD feature distillation includes taking the ROI area predicted by the teacher model as a (0, 1) mask, cropping the BEV feature maps of the teacher and the student, and then using the mean square error MSE to calculate the ROI loss, and finally using a dynamic adaptive weight to balance the ROI loss.
[0012] Furthermore, the FORD feature distillation includes using the foreground area to generate a (0 1) mask to crop the teacher model BEV feature map and the student model BEV feature map respectively, and using the mean square error MSE loss to transfer the geometric information of the teacher model to the student model, and using dynamic adaptive weights to balance the foreground distillation loss.
[0013] Furthermore, the feature map obtained by distillation is feature aligned to obtain aligned features, including calculating the KL divergence of the BEV feature map of the teacher-student model by respectively defining regression loss and cls loss, and finally obtaining the response loss.
[0014] In the second aspect, a multi-ray distillation 3D target detection method system across radar models includes: A data acquisition module is configured to acquire radar point cloud information; A preprocessing module, configured to preprocess radar point cloud information; A model building module, configured to build a teacher-student knowledge distillation model; A feature module is configured to extract features from radar point cloud information using a teacher-student knowledge distillation model, and perform feature distillation on the extracted features to obtain a feature map; An alignment module is configured to perform feature alignment on the feature maps obtained by distillation to obtain aligned features; The prediction module is configured to predict the aligned features using the target detection model to obtain a prediction result.
[0015] In a third aspect, the present invention provides a computer-readable storage medium storing a plurality of instructions, wherein the instructions are suitable for being loaded and executed by a processor of a terminal device, for executing the multi-ray distillation 3D target detection method across radar models.
[0016] In a fourth aspect, the present invention provides a terminal device including a processor and a computer-readable storage medium, wherein the processor is used to implement various instructions; and the computer-readable storage medium is used to store multiple instructions, wherein the instructions are suitable for being loaded by the processor and executing the multi-ray distillation 3D target detection method across radar models.
[0017] In summary, the present invention has the following beneficial technical effects: 1. The method of the present invention adopts the method of knowledge distillation to effectively transfer the knowledge in the large teacher model to the small student model, so as to achieve lightweight deployment of the model without sacrificing too much performance, which is in line with the application of industrial environment. And through knowledge distillation, the student model can effectively learn the knowledge in the teacher model, such as the relative relationship between different categories and the position, size, trajectory, etc. of the vehicle in the detection task. This information helps the student model to better understand the data, thereby achieving model compression while maintaining high accuracy. And the strategy of knowledge distillation can improve the generalization ability of the model, which enables the student model to learn more information during the training process, so as to show better performance when facing unseen data.
[0018] 2. The method of the present invention samples high-density data sets into low-density data sets through a clustering algorithm. Such a sampling strategy can better bridge the problem of insufficient data information from an industrial perspective and enhance the generalization ability of the model.
[0019] 3. When building the deep learning framework of teachers and students, the method of the present invention adopts a combination of GroundTruth and Roi to pass effective foreground knowledge to the student model and improve the detection accuracy of the model. BRIEF DESCRIPTION OF THE DRAWINGS
[0020] Figure 1 is a schematic diagram of a multi-ray distillation 3D target detection method across radar models according to Example 1 of the present invention; Figure 2 This is a comparison diagram of the 3D vehicle detection effect predicted on a laser radar beam of 64 beams according to Example 1 of the present invention; Figure 3 This is a comparison diagram of the 3D vehicle detection effect predicted when the laser radar beam is 32 beams according to Example 1 of the present invention; Figure 4 This is a comparison diagram of the 3D vehicle detection effect predicted on a laser radar beam of 32*beam according to Example 1 of the present invention; Figure 5 This is a comparison chart of the 3D vehicle detection effect predicted on a 16-beam laser radar beam in Example 1 of the present invention. DETAILED DESCRIPTION
[0021] The present invention is further described in detail below in conjunction with the accompanying drawings.
[0022] Example 1 Reference Figure 1 , a multi-ray distillation 3D target detection method across radar models of this embodiment includes: Get radar point cloud information; Preprocess radar point cloud information; Construct a teacher-student knowledge distillation model; The teacher-student knowledge distillation model is used to extract features from radar point cloud information, and the extracted features are distilled to obtain feature maps; Perform feature alignment on the feature maps obtained by distillation to obtain aligned features; The object detection model is used to predict the aligned features and obtain the prediction results.
[0023] Specifically, the following steps are included: S1. Get radar point cloud information This embodiment uses the open source KITTI dataset (the KITTI dataset is currently the world's largest computer vision algorithm evaluation dataset for autonomous driving scenarios, jointly created by the Karlsruhe Institute of Technology in Germany and Toyota Technology Research Institute of America) to train and predict the model. All dataset information can be downloaded from the official website of the KITTI dataset.
[0024] S2. Preprocessing the radar point cloud information includes segmenting the radar point cloud information and assigning wave number labels to the segmented points through a clustering method to obtain downsampled radar point cloud information.
[0025] Specifically, Down-sampling of real LiDAR data from the source domain to align the point cloud density. To achieve this, we first need to segment the point cloud into different beams. While some datasets include beam annotations, many LiDAR points lack these labels. Therefore, we introduce a clustering algorithm to assign a beam label to each point. We denote the number of beams and the average number of points per beam in the source and target domains as , and , We start by converting the Cartesian coordinates (x, y, z) to spherical coordinates: , , where θ and φ are the azimuth and zenith angles. Beam and The intuitive alignment of the beams is incorrect, which will result in different beam densities. Defined as source domain: =[ ]in , and represents the vertical field of view in the source and target domains. and indicates high-density beams and low-density beams. With the beam labels at each point, we can easily The beam is downsampled to In addition, we should also pass the sampling point in each beam to and Alignment. Instead of random sampling, we sort the points according to the azimuth angle φ and subsample them in order.
[0026] S3. Construct a teacher-student knowledge distillation model, including a LiDAR-based teacher model. Use the BackBone of the PointPillars network as the teacher model to extract voxel features from the LiDAR points. Then, process the voxel features through the 2D BackBone to obtain BEV features. At the same time, set an output layer for target detection at the last layer to predict the category and bounding box of the target present in the BEV features.
[0027] Among them, the overall teacher-student knowledge distillation framework is designed. Teacher model: For the teacher model based on LiDAR, we use PointPillars, a commonly used BackBone in 3D detection, as the teacher model to extract voxel features from LiDAR points, and then process the voxel features through 2D BackBone to obtain BEV features that are convenient for downstream tasks. The last layer is usually an output layer for target detection (usually a pre-designed convolution operation), which is responsible for predicting the category and bounding box of the target that may exist in the BEV feature.
[0028] S4. The teacher-student knowledge distillation model is used to extract features from radar point cloud information, including voxelizing the three-dimensional coordinates of the point cloud in the radar point cloud information to obtain voxel features, and performing feature learning on the voxel features through the PointPillars network to obtain enhanced voxel features.
[0029] These include: Coordinate conversion: Convert the three-dimensional coordinates (x, y, z) of the point cloud to indices in the voxel grid. Assuming the size of the voxel is voxel_size, the coordinates of each point can be converted to a voxel index by a simple integer division operation: Voxelization: Aggregate the features of multiple points in each voxel (there may be no points in the voxel, which is called an empty voxel). PointPillars uses a simple aggregation method (average) to merge the points in each voxel into a fixed-length feature vector. Features usually include location information (x, y, z), reflection intensity, etc.
[0030] Feature mapping: All voxel features are organized into a grid. The size of this grid is determined by the number of voxels. Usually, a 2D plane (x, y axis) is used to represent voxel features, and the depth (z axis) is used for feature channels. After feature mapping, voxel features are obtained. , where X, Y, and Z represent the width, length, and height of the voxel feature volume, respectively, and C represents the number of feature channels, and t indicates that the feature is a voxel feature in the teacher network.
[0031] Mesh processing: Next, PointPillars uses a neural network (PointNet) to process these voxel features. The network performs deeper feature learning on each voxel feature and maps it to a higher-dimensional space, essentially enhancing the voxel features to obtain more expressive voxel features. .
[0032] 2D convolution layer: The voxel features obtained after grid processing cannot be subsequently operated by traditional convolution, so we need a 2D BackBone to convert the voxel features into BEV feature maps that are convenient for subsequent processing. , and then a 2D convolutional neural network can be used to process the BEV features. This is because the feature map after voxelization is similar to image data and is very suitable for convolution operations.
[0033] Object detection: After 2D BackBone processing, the last layer is usually an output layer for object detection (category convolution and regression convolution), which is responsible for predicting the category and bounding box of the object that may exist in each voxel.
[0034] To ensure the performance of the teacher model, the teacher model is pre-trained on the input high beam. Student model: For the LiDAR-based student model, it shares the same architecture and neural network depth with the teacher, and the only difference is the density of the input point cloud. We downsample the high beam point cloud into a low beam point cloud through a clustering algorithm, and the student model is trained on the low beam point cloud.
[0035] S5. Perform feature distillation on the extracted features to obtain feature maps, including performing ROD feature distillation and FORD feature distillation on the radar point cloud information in turn, where the ROD feature distillation includes taking the ROI area predicted by the teacher model as a (0, 1) mask, cropping the BEV feature maps of the teacher and the student, and then using the mean square error MSE to calculate the ROI loss, and finally using dynamic adaptive weights to balance the ROI loss.
[0036] in, (1) ROD feature distillation. BEV feature map extracted by the teacher model Contains accurate 3D information derived from the high beam point, which is valuable for 3D object detection in point clouds. Student BEV features extracted from the low beam point , due to insufficient training data and sparse density, it is inaccurate and has poor robustness. Therefore, we use the ROI region predicted by the teacher model as a (0, 1) mask , the BEV feature maps of teachers and students are cropped to make them represent more potential foreground information; then, the mean square error (MSE) is used to calculate and The ROI loss between them makes the two as close as possible; finally, a dynamic adaptive weight is used To balance the ROI extraction loss.
[0037] Where N and M represent the number of samples (batches) and the number of ROIs in the BEV feature. represents the jth ROI of the BEV feature map of the i-th sample, and denote the (0, 1) mask and dynamic adaptive weight of ROI distillation loss, respectively.
[0038] (2) FORD feature distillation. The foreground region is the region that is manually marked when the dataset is collected. It represents the most likely location of the target to be detected and contains extremely reliable information. Therefore, the knowledge of this region needs to be passed to the student model. Similarly, we use the foreground region to generate a (0 1) pair and The feature map in is masked and cropped, and then the mean square error (MSE) loss is used to transfer the most valuable geometric information from the teacher model to the student model, and a dynamic adaptive weight is used to balance the foreground distillation loss. Where N and M represent the sample number (batch) and foreground number in the BEV feature. represents the jth foreground region of the BEV feature map of the i-th sample, and denote the (0, 1) mask and dynamic adaptive weights of foreground distillation loss, respectively.
[0039] (3) Dynamic weight adaptation: During the model training process, the foreground area is a manually marked area with strong credibility. Therefore, in the early stage of model training, our main knowledge comes from the foreground area. However, as the model training rounds deepen, the credibility of the model becomes higher and higher, and the ROI area predicted by the model (which may contain the target area) becomes more and more accurate. Therefore, in the later stage of model training, our main knowledge comes from the ROI area. Therefore, we need a gradually increasing weight , making the model more and more focused on the ROI area; accordingly, since the groundtruth is the manually marked foreground area, it remains unchanged during the network training process. As the number of model training rounds deepens, the model has learned enough knowledge from the foreground area. Therefore, we need a gradually decreasing weight , so that the model increasingly ignores the manually annotated true target areas and avoids overfitting the model.
[0040] in is the dynamic loss weight of the roi distillation loss. is the initial weight, step and e represent the scaling factor, and training time respectively. is the dynamic loss weight of the foreground distillation loss.
[0041] also, Figure 1 In the above example, point cloud is represented by the input radar point cloud, Pretrained weights represents the pre-trained weights of the teacher; ROD represents ROD feature distillation; DSA represents dynamic weight adaptation; FORD represents FORD feature distillation; 64beam represents that the input of the teacher model is point cloud data with a radar point cloud beam of 64; 32beam represents that the input of the student model is point cloud data with a radar point cloud beam of 32. Represents the (0, 1) mask in ROD feature distillation. express The (0, 1) mask in feature distillation. Head represents the convolution operation of the output layer. Class represents the category loss. Box represents the regression loss. Represents the calculation of the category loss between the teacher model and the student model. Represents the calculation of the regression loss between the teacher model and the student model. Figure 1 The T in the upper right corner represents the teacher model. Figure 1 The S in the lower right corner represents the student model.
[0042] S6. Perform feature alignment on the feature maps obtained by distillation to obtain the aligned features, including calculating the KL divergence of the BEV feature map of the teacher-student model by defining the regression loss and the cls loss respectively, and finally obtaining the response loss.
[0043] Among them, include: Response Distillation. The predictions from the teacher model are represented as (x, y, z, h, w, l, θ, c, s), where (x, y, z) represents the center of the 3D bounding box, (h, w, l) specify the dimensions of the bounding box, θ is the rotation angle, c represents the predicted class, and s represents the confidence score. We use these predictions as response guidance for the student model. Compared with hard labels, soft labels provide more information for each training sample. In addition, the teacher model acts as a sample filter in that samples that are challenging to it tend to receive lower confidence scores or are eliminated, while stable samples are assigned higher confidence scores. We calculate the KL divergence of the teacher-student BEV feature maps to make the probability distributions closer. This makes the model more robust than other methods. The regression loss is defined as: where KL represents the KL divergence of the two probability distributions used to balance the predictions, are the soft labels predicted by the teacher model, is the student model prediction, and finally we divide it by the size of the feature map to make the two probability distributions more balanced. Similarly, we use the quality focus loss (QFL) to calculate the class loss to alleviate the class imbalance. The cls loss is defined as: where QFL is the quality focus loss, and are the classification parameters of the soft labels and predictions. S is a scaling factor used to balance the difference between the predicted probabilities and the true labels. Finally, our response loss is defined as and sum.
[0044] S7. Use the object detection model to predict the aligned features and obtain the prediction results.
[0045] Among them, include: End-to-end testing. After the model is trained, we can directly obtain the 3D target detection effect predicted on the point cloud by using any input radar point cloud information through the point cloud processing architecture. The detailed effect is shown in the figure below. Our model is an end-to-end model, and no further processing is required on the model or input to obtain the vehicle perception effect in autonomous driving.
[0046] In order to further verify the feasibility and effectiveness of the proposed method, extensive experimental studies were conducted on a challenging large-scale dataset (KITTI).
[0047] The Python-based deep learning library Keras2.1.5 and pytorch1.10.0 are used to build an effective teacher-student network framework based on knowledge distillation. The large and widely used KITTI test set is used to analyze the road scene perception image predicted by the method of the present invention. Here, the commonly used objective parameters for evaluating 3D object detection methods are used as evaluation indicators, namely , which represents the average precision obtained by directly calculating the intersection over union (IoU) of the detection result and the ground truth (real value) in 3D space. This is an important indicator for measuring the performance of 3D object detection algorithms. The test was conducted under three different levels of occlusion conditions (indicating the degree of occlusion of the vehicle. The higher the occlusion, the more difficult it is to detect and the more challenging it is for the model), namely easy, moderate, and hard, to evaluate the performance of predicting 3D detection in vehicles.
[0048] The method of the present invention is used to predict each radar point cloud information in the KITTI test set, which is a new and widely used test set, and obtain the 3D detection effect corresponding to each road scene image, reflecting the average accuracy of the 3D detection effect of the method of the present invention. (Average Precision) is listed in Table 1. From the data listed in Table 1, it can be seen that the 3D prediction effect of the road scene image obtained by the method of the present invention is good, indicating that it is feasible and effective to use the method of the present invention to obtain the 3D detection prediction effect corresponding to the road scene image.
[0049] Table 1 Evaluation results of the method of the present invention on the test set Figure 2 (a) shows the prediction effect of the first point cloud beam with 64 beams in the same scene; Figure 2 (b) shows the method of the present invention for Figure 2 The point cloud beam shown in (a) is a 64-beam radar point cloud information sampled to obtain 32-beam radar point cloud information for prediction, and the resulting 3D detection effect; Figure 3 (a) shows the prediction effect of the second point cloud of the same scene with a beam size of 64 beams. Figure 3 (b) shows the method of the present invention for Figure 3The point cloud beam shown in (a) is a 64-beam radar point cloud information sampled to obtain 32-beam radar point cloud information for prediction, and the resulting 3D detection effect; Figure 4 (a) shows the prediction effect of the third point cloud of the same scene with a beam size of 64 beams; Figure 4 (b) shows the method of the present invention for Figure 4 The point cloud beam shown in (a) is a 64-beam radar point cloud information sampled to obtain 32-beam radar point cloud information for prediction, and the resulting 3D detection effect; Figure 5 (a) shows the prediction effect when the point cloud beam is 64 beams; Figure 5 (b) shows the method of the present invention for Figure 5 The point cloud beam shown in (a) is a 64-beam radar point cloud information sampled to obtain a 32-beam radar point cloud information for prediction, resulting in a 3D detection effect. Figure 2 (a) and Figure 2 (b) in the figure, compared with Figure 3 (a) and Figure 3 (b) in the figure, compared with Figure 4 (a) and Figure 4 (b) in the figure, compared with Figure 5 (a) and Figure 5 In (b), it can be seen that even when the beam of the point cloud is greatly reduced, the model can still achieve good detection results using the method of the present invention. In practical industrial applications, point cloud beams are often relatively small, so this method is suitable for practical industrial applications to perform 3D scene perception for autonomous driving.
[0050] Example 2 This embodiment provides a multi-ray distillation 3D target detection method system across radar models, including: A data acquisition module is configured to acquire radar point cloud information; A preprocessing module, configured to preprocess radar point cloud information; A model building module, configured to build a teacher-student knowledge distillation model; A feature module is configured to extract features from radar point cloud information using a teacher-student knowledge distillation model, and perform feature distillation on the extracted features to obtain a feature map; An alignment module is configured to perform feature alignment on the feature maps obtained by distillation to obtain aligned features; The prediction module is configured to predict the aligned features using the target detection model to obtain a prediction result.
[0051] A computer-readable storage medium stores a plurality of instructions, wherein the instructions are suitable for being loaded and executed by a processor of a terminal device, for a multi-ray distillation 3D target detection method across radar models.
[0052] A terminal device includes a processor and a computer-readable storage medium, wherein the processor is used to implement various instructions; the computer-readable storage medium is used to store multiple instructions, wherein the instructions are suitable for being loaded by the processor and executing the multi-ray distillation 3D target detection method across radar models.
[0053] The above are all preferred embodiments of the present invention, and are not intended to limit the protection scope of the present invention. Therefore, any equivalent changes made based on the structure, shape, and principle of the present invention should be included in the protection scope of the present invention.
Claims
1. A multi-ray distillation 3D target detection method across radar models, characterized in that include: Get radar point cloud information; Preprocess radar point cloud information; Construct a teacher-student knowledge distillation model; The teacher-student knowledge distillation model is used to extract features from radar point cloud information, and the extracted features are distilled to obtain feature maps; Perform feature alignment on the feature maps obtained by distillation to obtain aligned features; The object detection model is used to predict the aligned features and obtain the prediction results.
2. The multi-ray distillation 3D target detection method across radar models according to claim 1 is characterized in that: The preprocessing of the radar point cloud information includes segmenting the radar point cloud information and assigning wave number labels to the segmented points by a clustering method to obtain downsampled radar point cloud information.
3. The multi-ray distillation 3D target detection method across radar models according to claim 2 is characterized in that: The teacher-student knowledge distillation model is constructed, including a teacher model based on LiDAR, using BackBone of the PointPillars network as the teacher model, extracting voxel features from LiDAR points, and then processing the voxel features through 2D BackBone to obtain BEV features, and at the same time setting an output layer for target detection at the last layer to predict the category and bounding box of the target present in the BEV features.
4. The multi-ray distillation 3D target detection method across radar models according to claim 3 is characterized in that: The teacher-student knowledge distillation model is used to extract features from radar point cloud information, including voxelizing the three-dimensional coordinates of the point cloud in the radar point cloud information to obtain voxel features, and performing feature learning on the voxel features through a PointPillars network to obtain enhanced voxel features.
5. The multi-ray distillation 3D target detection method across radar models according to claim 4 is characterized in that: The extracted features are subjected to feature distillation to obtain feature maps, including sequentially performing ROD feature distillation and FORD feature distillation on radar point cloud information, wherein the ROD feature distillation includes taking the ROI area predicted by the teacher model as a (0, 1) mask, cropping the BEV feature maps of the teacher and the student, then using the mean square error MSE to calculate the ROI loss, and finally using a dynamic adaptive weight to balance the ROD loss.
6. The multi-ray distillation 3D target detection method across radar models according to claim 5, characterized in that: The FORD feature distillation includes using the foreground area to generate a (0 1) mask to crop the teacher model BEV feature map and the student model BEV feature map respectively, and using the mean square error MSE loss to transfer the geometric information of the teacher model to the student model, and using dynamic adaptive weights to balance the foreground distillation loss.
7. The multi-ray distillation 3D target detection method across radar models according to claim 6 is characterized in that: The feature map obtained by distillation is subjected to feature alignment to obtain aligned features, including calculating the KL divergence of the BEV feature map of the teacher-student model by respectively defining regression loss and classification loss, and finally obtaining the response loss.
8. A multi-ray distillation 3D target detection method system across radar models, characterized in that include: A data acquisition module is configured to acquire radar point cloud information; A preprocessing module, configured to preprocess radar point cloud information; A model building module, configured to build a teacher-student knowledge distillation model; A feature module is configured to extract features from radar point cloud information using a teacher-student knowledge distillation model, and perform feature distillation on the extracted features to obtain a feature map; An alignment module is configured to perform feature alignment on the feature maps obtained by distillation to obtain aligned features; The prediction module is configured to predict the aligned features using the target detection model to obtain a prediction result.
9. A computer-readable storage medium storing a plurality of instructions, characterized in that: The instructions are suitable for being loaded by a processor of a terminal device and executing the method according to claim 1 .
10. A terminal device, comprising a processor and a computer-readable storage medium, wherein the processor is used to implement each instruction; and the computer-readable storage medium is used to store multiple instructions, characterized in that: The instructions are suitable for being loaded by a processor and executing the method as claimed in claim 1 .
Citation Information
Patent Citations
Improved 3D target detection system based on PointPill
CN114648698A
Point cloud 3D target detection method based on foreground enhanced knowledge distillation
CN116246096A
Three-dimensional target detection method based on point cloud structure perception
CN116778468A
3D target detection method and system
CN117636295A
LiDAR-Free monocular 3D target detection knowledge distillation method
CN117726906A
Cited By
Scene generalization method and system for millimeter wave radar model, terminal and medium
CN119537960A
Open vocabulary few-sample three-dimensional part segmentation method and system
CN121415067A
