Three-dimensional target detection model generation and detection method, equipment and medium

Abstract

The invention provides a three-dimensional target detection model generation and detection method and device, and a medium, and belongs to the technical field of computer vision and photoelectron, and the method comprises the steps: training a teacher model based on a preset dense depth map, training a student model based on a preset RGB image, and obtaining a three-dimensional target detection model; a preset three-dimensional bounding box is used as a real label to supervise training of the teacher model and the student model; extracting a first feature output by the teacher model intermediate layer and a second feature output by the student model intermediate layer in the training process; inputting the first feature and the second feature into a preset distillation model, and calculating a distillation loss function; and optimizing network parameters of the student model based on the distillation loss function and the standard cross entropy loss, and taking the student model as a final three-dimensional target detection model for outputting a three-dimensional bounding box according to the RGB image. According to the method, the technical effect of improving the three-dimensional target detection capability of the monocular vision system on the premise of not increasing the model calculation complexity is effectively realized.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of computer vision and optoelectronic technology, and in particular to a three-dimensional target detection model generation and detection method, device and medium. BACKGROUND

[0002] With the wide application of photoelectric tracking systems in military reconnaissance, intelligent monitoring, automatic driving and other fields, higher requirements are put forward for target detection accuracy and three-dimensional perception capability. Traditional three-dimensional target detection systems usually rely on multi-sensor fusion (such as laser radar and camera), which can achieve high accuracy, but the system structure is complex, the cost is high, and the deployment is limited in resource-limited scenarios. In comparison, the three-dimensional target detection method based on monocular camera has become an important research direction in light-weight photoelectric systems due to its simple equipment, low cost and easy integration. However, due to the lack of real depth information, the monocular three-dimensional detection task is essentially a severely under-constrained problem, and the detection accuracy and stability are difficult to meet the actual demand.

[0003] To make up for the lack of depth information in monocular images, researchers try to introduce a depth estimation model to generate a depth map to assist detection. However, such methods rely on the generalization ability of the depth estimation model, are easily disturbed by noise, and significantly increase the computational burden during inference, limiting their application in real-time photoelectric tracking systems. In recent years, knowledge distillation has been proposed as an effective means of model optimization, which transfers the knowledge in large models or multi-modal models to light student models, improving model performance without increasing running overhead. In particular, cross-modal knowledge distillation technology can realize information transfer between different modalities (such as radar and image), providing a new idea for introducing additional explicit depth information for monocular three-dimensional detection tasks. Therefore, how to effectively improve the three-dimensional target detection capability of monocular vision systems without increasing the computational complexity of the model, and enhance the target recognition accuracy and robustness of photoelectric systems in complex scenes, is one of the key challenges faced by three-dimensional target detection technology in current photoelectric tracking systems. SUMMARY

[0004] Therefore, it is necessary to provide a three-dimensional target detection model generation and detection method, device and medium to improve the three-dimensional target detection capability of monocular vision systems without increasing the computational complexity of the model.

[0005] To achieve the above technical effects, in a first aspect, the present application provides a three-dimensional target detection model generation method, comprising: training a teacher model based on a preset dense depth map, training a student model based on a preset RGB image, and using a preset three-dimensional bounding box as a real label to supervise the training of the teacher model and the student model, wherein the dense depth map is generated from radar point cloud data; Extract the first feature output from the intermediate layer of the teacher model and the second feature output from the intermediate layer of the student model during the training process; The first and second features are input into a preset distillation model to calculate the distillation loss function; The network parameters of the student model are optimized based on the distillation loss function and the standard cross-entropy loss of the student model. The optimized student model is then used as the final 3D object detection model for outputting 3D bounding boxes based on RGB images.

[0006] In some embodiments of the present invention, the dense depth map is generated from the radar point cloud data based on projection and interpolation.

[0007] In some embodiments of the present invention, the first feature includes: a first backbone layer feature output by the backbone layer in the teacher model and a first response feature inside the detection head; The second feature includes: the second backbone layer feature output by the backbone layer in the student model and the second response feature inside the detection head.

[0008] In some embodiments of the present invention, the distillation model includes: a local distillation module, a global distillation module, and an object-responsive distillation module; The step of inputting the first feature and the second feature into a preset distillation model and calculating the distillation loss function includes: The local distillation module extracts the first attention feature of the first feature and the second attention feature of the second feature, and obtains the local distillation loss function of the local distillation module based on the first attention feature and the second attention feature. The global distillation module extracts the first global context feature of the first feature and the second global context feature of the second feature, and obtains the global distillation loss function of the global distillation module based on the first global context feature and the second global context feature. Based on the object response module, the first response region in the first feature and the second response region in the second feature are extracted, and the object response distillation loss of the object response distillation module is obtained based on the first response region and the second response region.

[0009] In some embodiments of the present invention, the local distillation module includes: mask distillation and local feature distillation; The step of extracting a first attention feature of the first feature and a second attention feature of the second feature based on the local distillation module, and obtaining the local distillation loss function of the local distillation module based on the first attention feature and the second attention feature, includes: Based on the mask distillation, the first channel attention and the first spatial attention of the first keel layer features, and the second channel attention and the second spatial attention of the second keel layer features are extracted respectively. Based on the first spatial attention and the second spatial attention, the spatial attention loss function between the teacher model and the student model is obtained. Based on the first channel attention and the second channel attention, the channel attention loss function between the teacher model and the student model is obtained. The two functions are then integrated to obtain the mask distillation loss function. The first spatial attention and the first channel attention are applied to the first keel layer features to obtain the first enhanced feature, and the second spatial attention and the second channel attention are applied to the second keel layer features to obtain the second enhanced feature; A mask is generated based on a two-dimensional bounding box, wherein the two-dimensional bounding box is obtained by projection of the three-dimensional bounding box; Based on the mask, the first enhanced feature and the second enhanced feature are filtered, and the loss function between the filtered first enhanced feature and the second enhanced feature is calculated as the local feature distillation loss function. The local distillation loss function is obtained by integrating the mask distillation loss function and the local feature distillation loss function.

[0010] In some embodiments of the present invention, the global distillation module includes: a first branch for performing global average pooling, a second branch for performing convolution to generate query tensors and key tensors, and a third branch for performing convolution to generate value tensors; The step of extracting the first global context feature of the first feature and the second global context feature of the second feature based on the global distillation module, and obtaining the global distillation loss function of the global distillation module based on the first global context feature and the second global context feature, includes: Based on the first branch, extract the first channel dimension context information of the first backbone layer features and the second channel dimension context information of the second backbone layer features respectively; Based on the second branch, extract the first query tensor and the first key tensor of the first backbone layer features, as well as the second query tensor and the second key tensor of the second backbone layer features; Based on the third branch, extract the first value tensor of the first backbone layer features and the second value tensor of the second backbone layer features; By fusing the first channel dimensional context information and the first value tensor, the first channel inter-context information is obtained; by fusing the second channel dimensional context information and the second value tensor, the second channel inter-context information is obtained. By fusing the first query tensor, the first key tensor, and the first value tensor, the context information between the first pixels is obtained; by fusing the second query tensor, the second key tensor, and the second value tensor, the context information between the second pixels is obtained. By fusing the first inter-channel context information and the first inter-pixel context information, a first pixel-level spatial context information is obtained; by fusing the second inter-channel context information and the second inter-pixel context information, a second pixel-level spatial context information is obtained. The global distillation loss function is calculated based on the first pixel-level spatial context information and the second pixel-level spatial context information.

[0011] In some embodiments of the present invention, the step of extracting a first response region from the first feature and a second response region from the second feature based on the object response module, and obtaining the object response distillation loss of the object response distillation module based on the first response region and the second response region, includes: The first response region is obtained by parsing the first response feature, wherein the first response region includes: the coordinates of the target center point and the size of the two-dimensional bounding box; Generate a Gaussian mask for the first response region; Obtain the first output response of the detector head in the teacher model and the second output response of the detector head in the student model; Align the first and second output responses based on the Gaussian mask; The object response distillation loss is obtained based on the aligned first and second output responses.

[0012] Secondly, the present invention also provides a three-dimensional target detection method, comprising: Acquire an RGB image including the 3D target to be detected; The RGB image is input into a 3D target detection model to obtain a 3D bounding box labeled with the location information of the 3D target to be detected. The 3D target detection model is obtained according to any one of the above methods for generating a 3D target detection model.

[0013] Thirdly, the present invention also provides an apparatus comprising: Memory, used to store programs; A processor, coupled to the memory, is configured to execute the program stored in the memory to implement the steps of the three-dimensional target detection model generation method or the three-dimensional target detection method described in any one of the above method items.

[0014] Fourthly, the present invention also provides a medium comprising: Used to store computer-readable programs or instructions, which, when executed by a processor, can implement the steps of the three-dimensional target detection model generation method or the three-dimensional target detection method described in any one of the above method items.

[0015] The beneficial effects of this invention are as follows: This invention provides a method for generating a 3D target detection model. Through a knowledge distillation network, data from different modalities are input into a teacher model and a student model for training. During the training process of the teacher model, real labels guide the teacher model to output corresponding prediction results. The knowledge features extracted by the teacher model during training are transferred to the student model through the distillation model, making full use of the explicit depth information in the LiDAR modality. This information is transferred to the student model trained based on monocular images without changing the network structure, thereby more effectively guiding the student model to complete the training. Furthermore, this invention ultimately outputs a lighter student model as the final model, without the need for additional depth estimation modules or multi-sensor input, maintaining the computational efficiency of the original model. Thus, it effectively achieves the technical effect of improving the 3D target detection capability of a monocular vision system without increasing the computational complexity of the model. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1 A flowchart illustrating an embodiment of the three-dimensional target detection model generation method provided by the present invention; Figure 2 A framework diagram of the monocular 3D target detection method based on cross-modal knowledge distillation provided by this invention; Figure 3 for Figure 1 A flowchart illustrating an embodiment of step S103; Figure 4 A framework diagram of the partial distillation module provided by the present invention; Figure 5 for Figure 3 A flowchart illustrating an embodiment of step S301; Figure 6 A global distillation module framework diagram provided by the present invention; Figure 7 for Figure 3 A flowchart illustrating an embodiment of step S302; Figure 8 forFigure 3 A flowchart illustrating an embodiment of step S303; Figure 9 This is a flowchart illustrating an embodiment of the three-dimensional target detection method provided by the present invention; Figure 10 This is a schematic diagram of the structure of an embodiment of the device provided by the present invention. Detailed Implementation

[0018] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0019] In the description of the embodiments of the present invention, unless otherwise stated, "multiple" means two or more. "And / or" describes the relationship between related objects, indicating that there can be three relationships. For example, A and / or B can represent three situations: A exists alone, A and B exist simultaneously, and B exists alone.

[0020] The terms "first," "second," etc., used in the embodiments of this invention are for descriptive purposes only and should not be construed as indicating or implying their relative importance or implicitly specifying the number of technical features indicated. Therefore, a technical feature defined with "first" or "second" may explicitly or implicitly include at least one of that feature.

[0021] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of the invention. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0022] This invention provides a method, device, and medium for generating and detecting three-dimensional target detection models, which are described below.

[0023] like Figure 1 As shown, in a first aspect, the present invention provides a method for generating a three-dimensional target detection model, comprising: S101. The teacher model is trained based on a preset dense depth map, the student model is trained based on a preset RGB image, and the training of the teacher model and student model is supervised by a preset 3D bounding box as the real label.

[0024] The dense depth map is generated from radar point cloud data.

[0025] It should be noted that the dense depth map is generated from radar point cloud data using projection and interpolation methods.

[0026] S102. Extract the first feature output by the intermediate layer of the teacher model and the second feature output by the intermediate layer of the student model during the training process.

[0027] Preferably, the teacher model adopts a single-stage monocular 3D object detection backbone model (such as MonoDLE), and the student model also adopts the same structure, as shown in the reference. Figure 2 Both the teacher model and the student model include a backbone layer and a detection head in their intermediate layers.

[0028] It should be noted that, because the teacher model uses dense depth maps as input and is trained under supervised supervision using real 3D bounding box labels, it can effectively extract high-precision depth representations and 3D spatial structure information. After training, the teacher model is used to guide the training process of the student model. Although the student model has the same structure as the teacher model, its input is a monocular RGB image, resulting in weaker feature extraction capabilities. Therefore, a distillation module is needed to obtain cross-modal knowledge from the teacher network, thereby achieving enhanced depth perception and improved 3D detection accuracy.

[0029] In some embodiments of the present invention, the first feature includes: a first backbone layer feature output by the backbone layer in the teacher model and a first response feature inside the detection head; the second feature includes: a second backbone layer feature output by the backbone layer in the student model and a second response feature inside the detection head.

[0030] S103. Input the first feature and the second feature into the preset distillation model and calculate the distillation loss function.

[0031] Specifically, the distillation model includes: a local distillation module, a global distillation module, and an object-responsive distillation module.

[0032] like Figure 3 In some embodiments of the present invention, step S103 includes: S301. Extract the first attention feature of the first feature and the second attention feature of the second feature based on the local distillation module, and obtain the local distillation loss function of the local distillation module based on the first attention feature and the second attention feature.

[0033] like Figure 4 The local distillation module includes mask distillation and local feature distillation. This module first generates channel attention and spatial attention for teacher network features and student network features respectively, and uses the deviation between them as a loss term to drive the student model to capture salient regions consistent with the teacher model during feature encoding.

[0034] In some embodiments of the present invention, such as Figure 5 Step S301 includes: S501. Based on mask distillation, extract the first channel attention and first spatial attention of the first backbone layer features, and the second channel attention and second spatial attention of the second backbone layer features.

[0035] S502. Obtain the spatial attention loss function between the teacher model and the student model based on the first spatial attention and the second spatial attention, obtain the channel attention loss function between the teacher model and the student model based on the first channel attention and the second channel attention, and integrate them to obtain the mask distillation loss function.

[0036] Preferably, the attention mask distillation loss function Using standards Loss function, for teacher characteristics With student characteristics Channel attention for teacher characteristics was calculated separately. Spatial attention Channel attention and student characteristics and spatial attention Attention mask distillation loss was calculated separately. and , and The sum of the average absolute errors is calculated as follows: (1) S503. Apply the first spatial attention and the first channel attention to the first backbone layer features to obtain the first enhanced features, and apply the second spatial attention and the second channel attention to the second backbone layer features to obtain the second enhanced features.

[0037] S504. Generate a mask based on a two-dimensional bounding box.

[0038] The two-dimensional bounding box is obtained by projecting the three-dimensional bounding box.

[0039] S505. Filter the first enhanced feature and the second enhanced feature based on the mask, and calculate the loss function between the filtered first enhanced feature and the second enhanced feature as the local feature distillation loss function.

[0040] S506. Integrate the mask distillation loss function and the local feature distillation loss function to obtain the local distillation loss function.

[0041] It should be noted that applying channel attention and spatial attention to the original feature map can achieve semantic focus and noise suppression in key regions. However, since the background occupies most of the area, it contains less information. Extracting knowledge from these areas may cause the network to deviate from the correct optimization direction. Therefore, it is necessary to use realistic 2D bounding boxes to limit the area of ​​knowledge distillation and avoid the influence of other irrelevant background noise. The enhanced feature values ​​are then masked to obtain the teacher's local features. With students' local characteristics The deviation between the two is calculated as the loss item. The local feature mask distillation loss function uses the standard... Loss function, local feature mask distillation loss function The calculation process is as follows: (2) In equation (2), The mask is generated from a 2D bounding box. It represents the number of valid feature vectors.

[0042] S302. Extract the first global context feature of the first feature and the second global context feature of the second feature based on the global distillation module, and obtain the global distillation loss function of the global distillation module based on the first global context feature and the second global context feature.

[0043] It should be noted that the spatial relationships between different pixels can provide valuable clues for monocular vision 3D object detection. By extracting and distilling these global features, the global distillation module can help improve detection accuracy. In the local distillation module, hybrid attention is used to enhance local features and transfer explicit features of the foreground object, but it fails to effectively transfer the global relationships between pixels. To extract the relationships between pixels in the feature map and distill them from the teacher model to the student model, a context information extraction module is designed to extract the relationships between pixels in the feature map.

[0044] like Figure 6 As shown, the global distillation module includes: a first branch for performing global average pooling, a second branch for performing convolution to generate query tensors and key tensors, and a third branch for performing convolution to generate value tensors.

[0045] Specifically, the first branch uses global average pooling to extract global information, capturing the up and down information of the channel dimension. The second branch uses... Convolution generates the product of the query and the key, establishing long-distance dependencies between pixels. The third branch uses... Convolution is used to generate a linear transformation of the feature map, which serves as an expression of spatial features.

[0046] In some embodiments of the present invention, such as Figure 7Step S302 includes: S701. Based on the first branch, extract the first channel dimension context information of the first backbone layer features and the second channel dimension context information of the second backbone layer features.

[0047] S702. Extract the first query tensor and the first key tensor of the first backbone layer features based on the second branch, as well as the second query tensor and the second key tensor of the second backbone layer features.

[0048] S703, the first-value tensor of the first backbone layer features and the second-value tensor of the second backbone layer features are extracted based on the third branch.

[0049] S704. Fuse the first channel dimension context information and the first value tensor to obtain the first channel context information. Fuse the second channel dimension context information and the second value tensor to obtain the second channel context information.

[0050] S705. Merge the first query tensor, the first key tensor, and the first value tensor to obtain the context information between the first pixels. Merge the second query tensor, the second key tensor, and the second value tensor to obtain the context information between the second pixels.

[0051] S706. Fuse the context information between the first channels and the context information between the first pixels to obtain the first pixel-level spatial context information. Fuse the context information between the second channels and the context information between the second pixels to obtain the second pixel-level spatial context information.

[0052] Specifically, the final output pixel-level spatial context information is expressed in code as follows: (3)

[0053] In equation (3), , , They represent the first The first layer feature map The input, output, and value vectors of each pixel. Indicates the first The total number of pixels in the layer feature map. This indicates a query for the corresponding convolution weight parameters, using... Convolution implementation; Represents an exponential function. Indicates the pixel Global average pooling is performed along the channel dimension. Global average pooling can guide the feature map to select channels containing important information, thereby enabling the global information extraction module to extract contextual information about the channel dimension. Indicates the pixel At the maximum value in the channel dimension, This indicates a join operation.

[0054] S707. Calculate the global distillation loss function based on the first pixel-level spatial context information and the second pixel-level spatial context information.

[0055] Specifically, the differences in contextual information at different scales between the student and teacher models are used as the global distillation loss term. Global Feature Distillation Loss Function as follows: (4) In equation (4), It is the number of feature layers involved in distillation. It is the first The total number of pixels in the layer feature map. and They represent the first Layer Pixels in the teacher network and student network The context in the text is represented as follows: express Norm squared is used to measure the Euclidean distance between context vectors.

[0056] S303. Extract the first response region from the first feature and the second response region from the second feature based on the object response module, and obtain the object response distillation loss of the object response distillation module based on the first response region and the second response region.

[0057] like Figure 8 In some embodiments of the present invention, step S303 includes: S801. Analyze the first response features to obtain the first response region.

[0058] The first response region includes the coordinates of the target center point and the dimensions of the two-dimensional bounding box.

[0059] S802, Generate the Gaussian mask for the first response region.

[0060] Specifically, the Gaussian mask is generated by using a Gaussian function to create the target mask. : (5) In equation (5), With the target center, It is determined by the size of the target.

[0061] S803. Obtain the first output response of the detector head in the teacher model and the second output response of the detector head in the student model.

[0062] S804, first and second output responses aligned based on Gaussian mask.

[0063] S805, Obtain the object response distillation loss based on the aligned first and second output responses.

[0064] Specifically, the detection head output response is aligned using this mask as a weight, and the distillation loss is as follows: (6) In equation (6), For the first Each detection head outputs.

[0065] S104. Optimize the network parameters of the student model based on the distillation loss function and the standard cross-entropy loss of the student model, and use the optimized student model as the final 3D object detection model for outputting 3D bounding boxes based on RGB images.

[0066] Specifically, the distillation loss term is calculated using the three types of distillation modules mentioned above, and combined with the original detection loss. Construct the total loss function : (7) in, , , The distillation weight is set to a value of [value]. , , The student network parameters are optimized through backpropagation, and the final detection model is obtained through iterative training.

[0067] Compared with existing technologies, this invention provides a method for generating a 3D target detection model. Through a knowledge distillation network, data from different modalities are input into a teacher model and a student model for training. During the training of the teacher model, real labels guide the teacher model to output corresponding prediction results. The knowledge features extracted by the teacher model during training are transferred to the student model through the distillation model, fully utilizing the explicit depth information in the LiDAR modality. This information is transferred to the student model trained based on monocular images without changing the network structure, thus more effectively guiding the student model to complete training. Furthermore, this invention ultimately outputs a lighter student model as the final model, requiring no additional depth estimation module or multi-sensor input, maintaining the computational efficiency of the original model. Therefore, it effectively improves the 3D target detection capability of a monocular vision system without increasing the computational complexity of the model.

[0068] Furthermore, this invention proposes three collaborative modules: local distillation, global distillation, and object response distillation, which guide feature learning at three levels: image region, spatial structure, and target response, respectively. Specifically, the local distillation module focuses on key foreground regions to avoid background interference, the global distillation module models the contextual relationships between pixels, and the object response distillation module enhances the spatial consistency of the detection response, thereby comprehensively improving the model's perception quality.

[0069] like Figure 9 Secondly, the present invention also provides a three-dimensional target detection method, comprising: S901. Acquire an RGB image including the 3D target to be detected.

[0070] S902. Input the RGB image into the 3D target detection model to obtain a 3D bounding box with the location information of the 3D target to be detected.

[0071] The three-dimensional target detection model is obtained according to any one of the three-dimensional target detection model generation methods described above.

[0072] like Figure 10 Thirdly, the present invention also provides an apparatus 100, comprising: Memory 1010 is used to store programs; The processor 1020, coupled to the memory 1010, is used to execute the program stored in the memory 1010 to implement the steps of the three-dimensional target detection model generation method or the three-dimensional target detection method described in any one of the above method items.

[0073] Fourthly, the present invention also provides a medium comprising: Used to store computer-readable programs or instructions, which, when executed by a processor, can implement the steps of the three-dimensional target detection model generation method or the three-dimensional target detection method described in any one of the above method items.

[0074] The foregoing has provided a detailed description of the method, equipment, and medium for generating and detecting a three-dimensional target detection model provided by the present invention. Specific examples have been used to illustrate the principles and implementation methods of the present invention. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of the present invention. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of the present invention. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A method for generating a three-dimensional target detection model, characterized in that, include: The teacher model is trained based on a preset dense depth map, the student model is trained based on a preset RGB image, and the training of the teacher model and student model is supervised by a preset 3D bounding box as the ground truth label. The dense depth map is generated from radar point cloud data. Extract the first feature output from the intermediate layer of the teacher model and the second feature output from the intermediate layer of the student model during the training process; The first and second features are input into a preset distillation model to calculate the distillation loss function; The network parameters of the student model are optimized based on the distillation loss function and the standard cross-entropy loss of the student model. The optimized student model is then used as the final 3D object detection model for outputting 3D bounding boxes based on RGB images.

2. The method for generating a three-dimensional target detection model according to claim 1, characterized in that, The dense depth map is generated from the radar point cloud data using projection and interpolation methods.

3. The method for generating a three-dimensional target detection model according to claim 1, characterized in that, The first feature includes: the first backbone layer feature output by the backbone layer in the teacher model and the first response feature inside the detection head; The second feature includes: the second backbone layer feature output by the backbone layer in the student model and the second response feature inside the detection head.

4. The method for generating a three-dimensional target detection model according to claim 3, characterized in that, The distillation model includes: a local distillation module, a global distillation module, and an object-responsive distillation module; The step of inputting the first feature and the second feature into a preset distillation model and calculating the distillation loss function includes: The local distillation module extracts the first attention feature of the first feature and the second attention feature of the second feature, and obtains the local distillation loss function of the local distillation module based on the first attention feature and the second attention feature. The global distillation module extracts the first global context feature of the first feature and the second global context feature of the second feature, and obtains the global distillation loss function of the global distillation module based on the first global context feature and the second global context feature. Based on the object response module, the first response region in the first feature and the second response region in the second feature are extracted, and the object response distillation loss of the object response distillation module is obtained based on the first response region and the second response region.

5. The method for generating a three-dimensional target detection model according to claim 4, characterized in that, The local distillation module includes: mask distillation and local feature distillation; The step of extracting a first attention feature of the first feature and a second attention feature of the second feature based on the local distillation module, and obtaining the local distillation loss function of the local distillation module based on the first attention feature and the second attention feature, includes: Based on the mask distillation, the first channel attention and the first spatial attention of the first keel layer features, and the second channel attention and the second spatial attention of the second keel layer features are extracted respectively. Based on the first spatial attention and the second spatial attention, the spatial attention loss function between the teacher model and the student model is obtained. Based on the first channel attention and the second channel attention, the channel attention loss function between the teacher model and the student model is obtained. The two functions are then integrated to obtain the mask distillation loss function. The first spatial attention and the first channel attention are applied to the first keel layer features to obtain the first enhanced feature, and the second spatial attention and the second channel attention are applied to the second keel layer features to obtain the second enhanced feature; A mask is generated based on a two-dimensional bounding box, wherein the two-dimensional bounding box is obtained by projection of the three-dimensional bounding box; Based on the mask, the first enhanced feature and the second enhanced feature are filtered, and the loss function between the filtered first enhanced feature and the second enhanced feature is calculated as the local feature distillation loss function. The local distillation loss function is obtained by integrating the mask distillation loss function and the local feature distillation loss function.

6. The method for generating a three-dimensional target detection model according to claim 4, characterized in that, The global distillation module includes: a first branch for performing global average pooling, a second branch for performing convolution to generate query tensors and key tensors, and a third branch for performing convolution to generate value tensors. The step of extracting the first global context feature of the first feature and the second global context feature of the second feature based on the global distillation module, and obtaining the global distillation loss function of the global distillation module based on the first global context feature and the second global context feature, includes: Based on the first branch, extract the first channel dimension context information of the first backbone layer features and the second channel dimension context information of the second backbone layer features respectively; Based on the second branch, extract the first query tensor and the first key tensor of the first backbone layer features, as well as the second query tensor and the second key tensor of the second backbone layer features; Based on the third branch, extract the first value tensor of the first backbone layer features and the second value tensor of the second backbone layer features; By fusing the first channel dimensional context information and the first value tensor, the first channel inter-context information is obtained; by fusing the second channel dimensional context information and the second value tensor, the second channel inter-context information is obtained. By fusing the first query tensor, the first key tensor, and the first value tensor, the context information between the first pixels is obtained; by fusing the second query tensor, the second key tensor, and the second value tensor, the context information between the second pixels is obtained. By fusing the first inter-channel context information and the first inter-pixel context information, a first pixel-level spatial context information is obtained; by fusing the second inter-channel context information and the second inter-pixel context information, a second pixel-level spatial context information is obtained. The global distillation loss function is calculated based on the first pixel-level spatial context information and the second pixel-level spatial context information.

7. The method for generating a three-dimensional target detection model according to claim 4, characterized in that, The step of extracting the first response region from the first feature and the second response region from the second feature based on the object response module, and obtaining the object response distillation loss of the object response distillation module based on the first response region and the second response region, includes: The first response region is obtained by parsing the first response feature, wherein the first response region includes: the coordinates of the target center point and the size of the two-dimensional bounding box; Generate a Gaussian mask for the first response region; Obtain the first output response of the detector head in the teacher model and the second output response of the detector head in the student model; Align the first and second output responses based on the Gaussian mask; The object response distillation loss is obtained based on the aligned first and second output responses.

8. A three-dimensional target detection method, characterized in that, include: Acquire an RGB image including the 3D target to be detected; The RGB image is input into a three-dimensional target detection model to obtain a three-dimensional bounding box labeled with the location information of the three-dimensional target to be detected, wherein the three-dimensional target detection model is obtained by the three-dimensional target detection model generation method according to any one of claims 1-7.

9. A device, characterized in that, include: Memory, used to store programs; A processor, coupled to the memory, is configured to execute the program stored in the memory to implement the steps of the three-dimensional target detection model generation method according to any one of claims 1 to 7 or the three-dimensional target detection method according to claim 8.

10. A medium, characterized in that, include: Used to store computer-readable programs or instructions, which, when executed by a processor, can implement the steps in the three-dimensional target detection model generation method according to any one of claims 1 to 7 or the three-dimensional target detection method according to claim 8.

Images