Cross-modal foreground distillation method for multi-view three-dimensional target detection

Abstract

The application discloses a cross-modal foreground distillation method for multi-view three-dimensional target detection, and belongs to the technical field of automatic driving perception. The method uses a laser radar or a multi-modal three-dimensional target detection model as a teacher model, and uses a multi-view bird's eye view (BEV) perception three-dimensional target detection model as a student model, and the model performance is improved through multi-scale feature distillation; a spatial attention mechanism and a foreground mask module are introduced, the student model can imitate the feature response of the teacher model at different spatial positions under multi-scale, focus on the foreground target, and suppress the background noise; an adaptive module ensures the consistency of the multi-scale features in the feature dimension and the spatial resolution, finally, the difference between the features and the regression output is measured through loss functions such as mean square error and KL divergence, and efficient cross-modal and multi-scale foreground feature distillation is realized. Finally, the multi-view three-dimensional target detection precision and efficiency are improved, and the distillation requirements of multi-scale features are fully considered.

Description

Technical Field

[0001] This invention relates to the field of autonomous driving perception technology in computer vision, and in particular to a cross-modal foreground distillation method for multi-view 3D target detection. Background Technology

[0002] In recent years, multi-view 3D object detection has attracted much attention in the field of autonomous driving perception, with broad application scenarios. Camera images, in particular, possess dense color and texture information. 3D detection models based on multi-view bird's-eye view (BEV) perception offer advantages such as low cost and ease of deployment, making them popular in both industry and academia. Especially in complex urban road environments, rich visual information is more conducive to scene understanding and object recognition, helping autonomous driving systems make accurate decisions.

[0003] Compared to LiDAR-based 3D detection models, multi-view BEV-based 3D target detection models, while achieving good accuracy, struggle to accurately infer depth and other 3D geometric information, resulting in lower performance compared to LiDAR-based models. Furthermore, LiDAR-based 3D target detection is typically highly complex, computationally resource-intensive, and inefficient, hindering industrial deployment. This is primarily because constructing 3D information from 2D images is a significant challenge, requiring complex neural networks to learn and infer these mappings. An intuitive solution is to compress the model, reducing the number of parameters to improve efficiency. However, simple model compression often leads to performance degradation. In this context, knowledge distillation, as an effective model compression and acceleration technique, not only significantly improves the efficiency of smaller models but also maintains or approaches the performance of the original larger models while compressing them, demonstrating immense potential.

[0004] Knowledge distillation enables smaller models (student models) to achieve performance close to that of larger models by having a larger model (teacher model) guide the learning process. This method not only effectively compresses model size but also maintains detection accuracy to some extent. By using a LiDAR model as a teacher model to guide the learning of a visual model, the visual model can acquire the precise 3D perception capabilities of LiDAR while retaining the advantages of low cost and easy deployment of visual sensors. However, LiDAR data and camera image data differ fundamentally in modal characteristics, and efficiently transferring the knowledge from the teacher model to the learning model through cross-modal knowledge distillation presents a significant challenge in practical applications. Summary of the Invention

[0005] To address the problems in existing technologies, this invention provides a cross-modal foreground distillation system and method for multi-view 3D target detection, which solves the shortcomings of existing vision-based 3D target detection models in terms of accuracy, depth information reasoning, and computational resource utilization efficiency.

[0006] This invention proposes a cross-modal foreground distillation method for multi-view 3D object detection. It is applicable to a distillation framework including an adaptive module, a foreground masking module, a spatial attention distillation module, a multi-scale foreground feature distillation module, and a responsive distillation module. The method uses a LiDAR or multi-modal 3D object detection model as the teacher model and a multi-view bird's-eye view BEV-perceptive 3D object detection model as the student model. The adaptive module adjusts the features of the student model to ensure that the output features are consistent with the teacher model in terms of spatial distribution and semantic information scale. The foreground masking module generates foreground masks and foreground scaling weights, identifies target regions in the image, and adjusts the distillation weights for target regions of different sizes. The spatial attention distillation module generates a spatial attention map and performs knowledge distillation of spatial attention features. The multi-scale foreground feature distillation module significantly improves the student model's perception of target regions by aligning features of foreground regions at different scales. The responsive distillation module designs distillation methods with loss functions for classification and regression tasks respectively, calculates the differences between the classification and regression outputs of the teacher and student models, and optimizes the distillation of the student model.

[0007] More specifically, the adaptive module in the distillation framework consists of stacked upsampling layers and convolutional layers. The adjustment of the student model's features includes: first, increasing the feature resolution through upsampling layers; then, adjusting the spatial structure and dimensionality of the features through convolutional layers, so that the output features are consistent with the teacher model's features in terms of spatial distribution and semantic information. The expression processed by the adaptive module is as follows:

[0008]

[0009] in, The features of the input student model, For adaptive modules, This is the feature output after processing by the adaptive module.

[0010] More specifically, in the foreground mask module of the distillation framework, the foreground mask... It is obtained by summing the two-dimensional Gaussian masks generated at the center points of the bounding boxes of all target regions, and the formula is as follows:

[0011]

[0012] in, Standard deviation, Representing the A bounding box, This represents the total number of bounding boxes. These are the coordinates of the center point of the bounding box;

[0013] Foreground scaling weight According to the foreground mask The value is generated, where the foreground mask value is greater than a certain threshold. When the point is considered to belong to the foreground, a scaling weight is calculated based on the bounding box of the foreground mask; when the foreground mask value is less than or equal to a certain threshold... When a point is considered to belong to the background region, a scaling weight is generated based on the number of background points; the specific calculation formula is as follows:

[0014]

[0015] in, For point The corresponding foreground mask value; Indicates the first Foreground area mask; These represent the foreground mask regions. The height and width are determined by finding the foreground mask area. The corresponding bounding box is obtained; It is a set threshold used to distinguish between foreground and background;

[0016] Foreground mask and foreground scaling scale weights As a weighting coefficient in the multi-scale foreground feature distillation task, it enables the adjustment of distillation weights for target regions of different sizes.

[0017] More specifically, in the spatial attention distillation module of the distillation framework, the specific calculation formula for the spatial attention map is as follows:

[0018] ,

[0019] in, To perform average pooling on the features along the channel dimension, For use The function normalizes the average pooling result to obtain the spatial attention map. Features of the teacher model or student model Using the number of channels, height, and width as features, the spatial attention maps of the teacher model and the student model are calculated using the formulas described above. And the attention map of the student model is adjusted through an adaptive layer. Then, use hyperparameters. The spatial attention maps of the two models are fused using the following formula:

[0020] ,

[0021] The fused spatial attention is used for subsequent multi-scale feature distillation;

[0022] For knowledge distillation of spatial attention features, the spatial attention maps of the teacher model and the student model are multiplied by their respective original features to obtain the corresponding spatial attention features. This achieves spatial weighting of the features, highlighting the features of important regions. The formula is as follows:

[0023] ,

[0024] in, These are represented as features for the teacher and student models, respectively. These are spatial attention maps for the teacher and student models, respectively. For element-wise multiplication, The spatial attention features of the teacher and student models are respectively used; then, the following methods are employed. The difference in the spatial attention features between the two loss functions is shown in the following formula:

[0025]

[0026] in, These are the length and width of the spatial attention feature, respectively. It is an adaptive module.

[0027] To be more specific, define The features generated by the encoder at different levels are multi-scale features, represented as follows: and in, To represent the features of the student model, we need to obtain the number of channels, height, and width respectively; These represent the number of channels, height, and width of the teacher model, respectively.

[0028] In the multi-scale foreground feature distillation module of the distillation framework, the mean squared error loss function is used to measure the similarity between the features of the teacher model and the student model. The calculation formula is as follows:

[0029]

[0030] in, These represent the number of channels, height, and width of the feature, respectively. This is a spatial attention map. These are respectively represented as the first and second models of the teacher and student. Layer features, Foreground mask, Scale weights for the foreground mask.

[0031] More specifically, in the distillation method that designs loss functions for the classification and regression tasks in the responsive distillation module, the classification loss mainly measures the error of the teacher model and the student model in the classification task, while the regression loss measures their difference in the target attribute regression task.

[0032] In classification tasks, the maximum value of the heatmap is extracted from the outputs of the teacher and student models. The loss function measures the absolute error between the maximum values ​​of the heatmaps of the teacher and student models. Finally, the absolute error is weighted and standardized with the foreground mask to obtain the final classification loss. The specific formula is as follows:

[0033]

[0034] in, These represent the maximum values ​​of the heatmaps for the teacher and student models, respectively. Foreground mask, The sum of the mask weights, This is the numerical stability term, and its value is... ;

[0035] In regression tasks, regression output is extracted from the output features of the teacher and student models, using... Divergence measures the difference between the teacher model and the student model. The mathematical expression for divergence is:

[0036] ,in It is the output of the teacher model. This is the output of the student model; secondly, calculations are performed for different regression outputs. Divergence includes position, height, size, rotation, and velocity;

[0037] Finally, The final regression loss is obtained by weighting and standardizing the divergence and foreground mask. The specific formula is as follows:

[0038] in, The outputs show the positions of the teacher and student models, respectively. The height outputs for the teacher model and the student model are respectively. Output the dimensions of the teacher model and the student model respectively. These are the rotation outputs for the teacher model and the student model, respectively. The speed outputs for the teacher model and the student model are respectively. The weights are generated using Gaussian masks. The sum of the mask weights, This is the numerical stability term, and its value is... .

[0039] The beneficial effects of this invention are:

[0040] 1. Adaptive Module Gain: The adaptive module adjusts the feature dimension and spatial resolution of the student model to align it with the features of the teacher model, thus solving the modal difference problem between LiDAR and camera image data and ensuring the effectiveness of cross-modal knowledge transfer.

[0041] 2. Foreground Mask Module Gain: By generating a foreground mask and foreground scaling scale weights, this module effectively identifies and isolates foreground targets and background regions, avoiding background interference and ensuring a more balanced contribution from targets of different scales during distillation. This module optimizes the key areas of feature distillation, significantly improving the alignment effect and distillation accuracy of target region features.

[0042] 3. Multi-scale Foreground Feature Distillation Module Gain: This module combines foreground masking and scaling weights, enabling the model to perform weighted alignment of foreground target features at different scales, reducing the impact of background noise. By accurately distilling features at different scales, the model's adaptability to multi-scale targets and its feature learning performance are enhanced.

[0043] 4. Spatial Attention Distillation Module Gain: By calculating the spatial attention map, features of important regions are highlighted with weights. The student model can better learn the spatial structure and dimensional information of the teacher model, improving its attention to the target region and feature extraction ability, effectively improving distillation accuracy.

[0044] 5. Response Distillation Module Gains: By optimizing the classification loss and regression loss, the student model can better learn the classification and regression outputs of the teacher model, thereby significantly improving the performance of the student model in 3D object detection tasks, especially the accuracy of classification and localization. Attached Figure Description

[0045] Figure 1 This is a schematic diagram of a cross-modal foreground distillation method for multi-view 3D target detection, provided as an example of the present invention. Detailed Implementation

[0046] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments, but the scope of protection of the present invention is not limited to the content described.

[0047] This invention proposes a cross-modal foreground distillation method for multi-view 3D object detection, applicable to a newly constructed distillation framework. This framework aims to transfer knowledge of multi-scale foreground region features, spatial attention information, and high-quality outputs (classification scores, regression results) from the teacher model to a compact student model. Specifically, multi-scale foreground feature distillation is performed on the BEV (Browser-Eye View), spatial attention distillation is performed on the spatial attention information, and responsive distillation is performed on the classification scores and regression results. Simultaneously, to alleviate the imbalance in distillation loss caused by background and large foreground objects occupying a large portion of the feature map, a foreground scaling scale weight is introduced to balance objects of different sizes and reduce background interference. Furthermore, since the feature representation and spatial structure of LiDAR data and camera image data differ significantly, direct feature distillation leads to poor results; therefore, an adaptive module is introduced to correct for features of different modalities.

[0048] Based on multiple perspectives While perceptual 3D object detection models offer advantages such as low cost and ease of deployment, pure vision models cannot accurately infer depth and other geometric information, resulting in significantly lower performance compared to LiDAR-based or multimodal 3D object detection models. Knowledge distillation is a general model compression technique that extracts knowledge from a teacher model and transfers this knowledge to smaller student models. During training, a LiDAR-based or multimodal 3D object detection model is used as the teacher model, transferring its rich spatial structure and depth information to the student models (based on multi-viewpoints). The perceptual 3D target detection model can help student models learn cross-modal geometric features, thereby making up for their lack of depth information.

[0049] like Figure 1 As shown, in this embodiment of the invention, a distillation framework is constructed between the teacher model and the student model to... Distillation is performed on features, classification, and regression results. This distillation framework mainly consists of five important parts: an adaptive module, a foreground masking module, a multi-scale foreground feature distillation module, a spatial attention distillation module, and a responsive distillation module. A LiDAR or multimodal 3D object detection model is used as the teacher model, with multi-view bird's-eye views... The perceptual 3D target detection model is used as a student model;

[0050] This distillation framework begins with the output features of the teacher and student models. First, the adaptive module adjusts the feature dimension and spatial resolution of the student model's features to align with those of the teacher model, reducing feature mismatch. Then, the foreground mask module generates a foreground mask and foreground scaling weights. The foreground mask accurately locates the target regions in the feature maps of both the teacher and student models, ensuring that the multi-scale foreground feature distillation module can focus on these regions, reducing interference from the background. The foreground scaling weights balance the influence of different targets in the distillation process by adapting to different target sizes, especially when there are significant differences in the size of some foreground targets, where larger objects dominate the distillation loss calculation. By applying different adaptive scaling weights to targets of different sizes, it ensures that all types of targets contribute equally to the distillation loss, avoiding the background and large targets dominating the distillation loss calculation, and improving the feature alignment and distillation accuracy for targets of different sizes. Next, the spatial attention distillation module generates spatial attention maps for both the teacher and student models and uses them to weight the original features, highlighting important regional features. Finally, the responsive distillation module uses classification loss and regression loss respectively to transfer knowledge based on the classification and regression outputs of the teacher model and the student model, significantly improving the classification and regression performance of the student model in the object detection task.

[0051] This distillation framework introduces an adaptive module composed of stacked upsampling and convolutional layers to adjust the features of the student model. Specifically, the adaptive module takes the features of the student model as input and adjusts its feature dimension and spatial resolution through a multi-layer structure to match the features of the teacher model. This process is represented as:

[0052]

[0053] in, For the input student model features, For adaptive modules, This is the feature output after processing by the adaptive module.

[0054] In this distillation framework, the foreground mask module is used to generate the foreground mask and foreground scaling weight of the feature map, which are used to identify target regions in the image. At the same time, the distillation weights are adjusted for target regions of different sizes.

[0055] Furthermore, generate a foreground mask. The specific solution is as follows: First, for a single target bounding box, generate a two-dimensional Gaussian mask with the target center as the origin based on the true value. Calculate the Gaussian masks of all target bounding boxes and combine them into the final foreground mask. The specific formula is as follows:

[0056]

[0057] in, Standard deviation, Representing the A bounding box, This represents the total number of bounding boxes. These are the coordinates of the center point of the bounding box.

[0058] Furthermore, foreground scaling weights It is based on the foreground mask Foreground region generation. Specifically, when the foreground mask value is greater than a certain threshold... When the point is considered to belong to the foreground, a scaling weight is calculated based on the bounding box of the foreground mask; when the foreground mask value is less than or equal to a certain threshold... When a point is considered to belong to the background region, a scaling weight is generated based on the number of background points. The specific formula is as follows:

[0059]

[0060] in, For point The corresponding foreground mask value; Indicates the first Foreground area mask; These represent the foreground mask regions. The height and width are determined by finding the foreground mask area. The corresponding bounding box is obtained; It is a set threshold used to distinguish between foreground and background; foreground mask. and foreground scaling scale weights As weighting coefficients in the multi-scale foreground feature distillation task.

[0061] The spatial attention distillation module in this distillation framework consists of two important parts: computation of spatial attention maps and spatial attention distillation.

[0062] First, spatial attention maps of the teacher and student models are calculated separately. This involves calculating the mean, transforming the dimensions, and normalizing the features to obtain the attention mask. The calculation formulas are as follows:

[0063] ,

[0064] in, To perform average pooling on the features along the channel dimension, For use The function normalizes the average pooling result to obtain the spatial attention map. Features of the teacher model or student model The features are the number of channels, height, and width. The spatial attention maps of the teacher and student models are calculated using the formulas described above. And the attention map of the student model is adjusted to [the desired value] through an adaptive layer. .

[0065] Then use hyperparameters Spatial attention maps of the two models and The fusion process yields the final fused spatial attention map A. The calculation formulas are as follows:

[0066] ,

[0067] The fused spatial attention map is used for subsequent multi-scale feature distillation.

[0068] For spatial attention distillation, the spatial attention maps of the teacher model and the student model are multiplied by their respective original features to obtain the corresponding spatial attention features. This spatial weighting of features highlights the features of important regions, as shown in the following formula:

[0069] ,

[0070] in, These represent the teacher model features and the student model features after being corrected by the adaptive module, respectively. These are spatial attention maps for the teacher and student models, respectively. For element-wise multiplication, These are the spatial attention features of the teacher and student models, respectively. Then, using... Loss function calculation spatial attention distillation loss The difference in spatial attention features between the two is measured by the following formula:

[0071]

[0072] in, These are the length and width of the spatial attention feature, respectively. It is an adaptive module.

[0073] The multi-scale foreground feature distillation module in this distillation framework defines... The features generated by the encoder at different levels are multi-scale features, represented as follows: and .in, To represent the features of the student model, we need to obtain the number of channels, height, and width respectively; These represent the number of channels, height, and width of the teacher model, respectively. An adaptive module is used. The multi-scale features of the student model are adjusted in terms of feature dimension and spatial resolution to align with the multi-scale features of the teacher model. Then, a foreground mask is generated using the foreground mask module. and foreground scaling scale weights This is used for feature weighting. Finally, based on the multi-scale foreground feature distillation module, the mean squared error loss function is used to measure the difference between the output features of the teacher model and the student model, as shown in the following formula:

[0074]

[0075]

[0076] in, These represent the number of channels, height, and width of the feature, respectively. This is a spatial attention map. These are respectively represented as the first and second models of the teacher and student. Layer features, Foreground mask, Scale weights for the foreground mask.

[0077] The responsive distillation module of this distillation framework optimizes the student model through distillation by calculating the differences between the classification and regression outputs of the teacher and student models. This distillation method designs separate loss functions for classification and regression tasks. The classification loss primarily measures the errors of the teacher and student models in the classification task, while the regression loss measures their differences in the target attribute regression task.

[0078] In classification tasks, the maximum value of the heatmap is extracted from the outputs of the teacher and student models. The loss function measures the absolute error between the teacher model and the student model. Finally, the classification loss is obtained by weighting and standardizing the loss values ​​using foreground masks. The specific formula is as follows:

[0079]

[0080] in, These represent the maximum values ​​of the heatmaps for the teacher and student models, respectively. Foreground mask, The sum of the mask weights, This is the numerical stability term, and its value is... .

[0081] In regression tasks, regression outputs are extracted from the outputs of the teacher and student models, and KL divergence is used to measure the difference between the teacher and student models. The mathematical expression for divergence is: ,in It is the output of the teacher model. This is the output of the student model. The KL divergence is calculated for different regression outputs, including position, height, size, rotation, and velocity. Finally, it is weighted with a Gaussian mask and standardized to obtain the final regression loss. The formula is as follows:

[0082] in, The outputs show the positions of the teacher and student models, respectively. The height outputs for the teacher model and the student model are respectively. Output the dimensions of the teacher model and the student model respectively. These are the rotation outputs for the teacher model and the student model, respectively. The speed outputs for the teacher model and the student model are respectively. The weights are generated using Gaussian masks. The sum of the mask weights, This is the numerical stability term, and its value is... .

[0083] Although embodiments of the present invention have been described above in conjunction with the accompanying drawings, the present invention is not limited to the specific embodiments and application fields described above. The specific embodiments described above are merely illustrative and instructive, and not restrictive. Those skilled in the art can make many other forms based on the guidance of this specification and without departing from the scope of protection of the claims of the present invention, and all of these are within the scope of protection of the present invention.

Claims

1.A cross-modal foreground distillation method for multi-view three-dimensional target detection, characterized in that, the cross-modal foreground distillation method is applied to a distillation framework comprising an adaptive module, a foreground mask module, a spatial attention distillation module, a multi-scale foreground feature distillation module, and a responsive distillation module; a laser radar or a multi-modal three-dimensional target detection model is used as a teacher model, and a multi-view bird's eye view (BEV) perception three-dimensional target detection model is used as a student model; the adaptive module is used to adjust the features of the student model so that the output features are consistent with the feature scale of the teacher model in terms of spatial distribution and semantic information; the foreground mask module is used to generate a foreground mask and a foreground scaling scale weight, and to identify the target region in the image, while adjusting the distillation weight of different size target regions; the spatial attention distillation module is used to generate a spatial attention map and perform knowledge distillation of spatial attention features; the multi-scale foreground feature distillation module is used to significantly improve the perception ability of the student model to the target region by aligning the features of different scales of the foreground region; the responsive distillation module is used to design a distillation method for loss functions through classification and regression tasks, to calculate the difference between the classification and regression outputs of the teacher model and the student model, and to realize the distillation optimization of the student model. 2.The cross-modal foreground distillation method of claim 1, characterized in that, the adaptive module in the distillation framework is stacked by an up-sampling layer and a convolution layer, and the expression processed by the adaptive module is as follows: ； wherein, is a feature of an input student model, is an adaptation module, is a feature output after processing by the adaptation module. 3.The cross-modal foreground distillation method of claim 1, characterized in that, In the foreground mask module of the distillation framework, the foreground mask is obtained by summing the two-dimensional Gaussian masks generated for all target region bounding box center points, and the formula is as follows: ； wherein, is the standard deviation, represents the first bounding box, is the total number of bounding boxes, is the center point coordinate of the bounding box; foreground scaling scale weight according to the foreground mask values, specifically as follows: ； wherein, is a point corresponding to the foreground mask value; represents the th foreground region mask; respectively represent the height and width of the foreground mask region obtained by looking up the corresponding bounding box of the foreground mask region is a set threshold value for distinguishing foreground and background;​ foreground mask and foreground scale weight as a weighting coefficient in the multi-scale foreground feature distillation task to adjust the distillation weight of different size target regions. 4.The cross-modal foreground distillation method of claim 1, characterized in that, in the spatial attention distillation module of the distillation framework, for the spatial attention map, the specific calculation formula is as follows: ， ； wherein, is an average pooling operation on the channel dimension of the feature, is a normalization processing on the average pooling result using function to obtain a spatial attention map, is a feature of the teacher model or the student model, is the number of channels, height and width of the feature, and the spatial attention maps of the teacher model and the student model are calculated using the above formula , and the attention map of the student model is adjusted through the adaptive layer , then the spatial attention maps of the two models are fused using the hyperparameter , and the specific formula is as follows: ， ； wherein the fused spatial attention is used for subsequent multi-scale feature distillation; for the knowledge distillation of spatial attention features, the spatial attention map of the teacher model and the student model is multiplied by the respective original features to obtain the corresponding spatial attention features, realizing the spatial weighting of the features and highlighting the features of important regions, and the specific formula is as follows: ， ； in, These are represented as features for the teacher and student models, respectively. These are spatial attention maps for the teacher and student models, respectively. For element-wise multiplication, The spatial attention features of the teacher and student models are respectively used; then, the following methods are employed. The difference in the spatial attention features between the two loss functions is shown in the following formula: ； wherein, respectively the length and the width of the spatial attention feature, is an adaptive module. 5.The cross-modal foreground distillation method of claim 1, characterized in that, Definitions The features generated by the encoder at different levels are multi-scale features, denoted as: and wherein, are respectively the number of channels, height, and width of the student model feature; are respectively the number of channels, height, and width of the teacher model; in the multi-scale foreground feature distillation module of the distillation framework, a mean square error loss function is used to measure the similarity between the features of the teacher model and the student model, and the calculation formula is as follows: ； wherein, are the number of channels, height, width of the feature, respectively, is the spatial attention map, are the first and second layer features, respectively, are the first and second layer features, respectively, is the foreground mask, is the foreground mask scaling scale weight. 6.The cross-modal foreground distillation method of claim 1, characterized in that, in the distillation method for designing loss functions for classification and regression tasks in the responsive distillation module, the classification loss mainly measures the error of the teacher model and the student model in the classification task, while the regression loss measures the difference between them in the target attribute regression task. In the classification task, the maximum value of the heat map is extracted from the output of the teacher model and the student model, and the The loss function measures the absolute error between the maximum value of the heat map of the teacher model and the student model, and finally, the absolute error is weighted and standardized with the foreground mask to obtain the final classification loss The specific formula is as follows: ； wherein, respectively represent the maximum values of the heat maps of the teacher and student models, is a foreground mask, is the sum of the mask weights, is a numerical stability term with a value of ; For regression tasks, the regression outputs are extracted from the output features of the teacher model and the student model, using The divergence measures the difference between the teacher model and the student model, The mathematical expression of the divergence is: where is the output of the teacher model, is the output of the student model; second, the divergence is computed for different regression outputs separately including position, height, size, rotation, and velocity; Finally, the KL divergence is weighted with the foreground mask and normalized to get the final regression loss The specific formula is as follows: ； wherein, are position outputs of the teacher and student models, respectively, are height outputs of the teacher and student models, respectively, are size outputs of the teacher and student models, respectively, are rotation outputs of the teacher and student models, respectively, are velocity outputs of the teacher and student models, respectively, is a weight generated by a Gaussian mask, is a sum of mask weights, is a numerical stability term with a value of .

Images