A target detection knowledge distillation method based on multi-scale channel-space decoupling

Abstract

The application relates to a target detection knowledge distillation method based on multi-scale channel-space decoupling, which comprises the following steps: multi-scale feature decoupling: introducing 1x1, 2x2 and 4x4 grid division in the output layer of a feature pyramid network, independently calculating channel attention weights of each region, and eliminating the interference of background noise on small target features; dynamic knowledge weighting: dynamically adjusting loss weights according to the prediction consistency of teacher-student models, applying a reinforcement coefficient to a prediction conflict region, and improving the learning efficiency of local discriminative features; double weight fusion: combining channel activation intensity and region complementarity evaluation to generate a composite weight, and preferentially retaining key detail features in high activation channels. The application solves the problem of low cross-scale knowledge transfer efficiency caused by the coupling of channel and spatial features in the prior art by constructing a teacher-student collaborative distillation framework, and improves the detection precision and generalization ability of a lightweight student model.

Description

Technical Field

[0001] This invention relates to the field of target detection technology in computer vision, specifically a target detection knowledge distillation method based on multi-scale channel-spatial decoupling. Background Technology

[0002] Object detection, as one of the core tasks in the field of computer vision, mainly realizes the category recognition and spatial localization of specific objects in images. Its technology has been deeply applied in diverse scenarios such as autonomous driving systems (such as pedestrian / vehicle recognition), industrial quality inspection (equipment fault detection), public security (dangerous goods screening at stations) and disaster early warning.

[0003] Object detection relies on scene information acquired by optical sensors, but the imaging performance of optical cameras is significantly affected by scene complexity. For example, problems such as excessively small target size, large variations in target scale, and background noise interference can all reduce detection accuracy. Furthermore, there is a severe class imbalance problem between foreground and background targets (the number of background targets is significantly greater than that of foreground targets), and the feature space is mixed with a large amount of noise information, making object detection a challenging task.

[0004] Driven by the trends of edge intelligence and the autonomy of terminal devices, object detection algorithms are gradually migrating from cloud servers to edge devices (such as autonomous vehicle computing units and drone flight control systems) to meet the demands for real-time performance and offline availability. This technological evolution, through the synergistic optimization of lightweight models and hardware acceleration, achieves efficient inference in resource-constrained environments, supporting the decision-making capabilities of intelligent terminal devices. Edge devices lack the sufficient computing power of the cloud, thus object detection in edge computing scenarios faces a dual challenge: on the one hand, limited by the hardware resources of edge devices, the actual deployed models must meet lightweight requirements; on the other hand, detection scenarios with complex background interference and a high proportion of small targets (such as drone remote sensing and industrial defect detection) significantly reduce the model's feature discrimination ability. Although object detection performance can be improved by optimizing the network structure, such methods may increase model complexity, leading to decreased computational efficiency and making it difficult to meet the real-time requirements of edge devices. Therefore, finding a balance between model lightweighting and detection accuracy has become a key issue in the field of object detection. To facilitate the deployment of neural network models on edge devices, researchers have proposed lightweight techniques such as model acceleration and compression. These techniques can further reduce the number of model parameters and complexity while maintaining model accuracy, thus simplifying deployment. Lightweight model techniques include pruning, quantization, and knowledge distillation.

[0005] As a core method of lightweight technology, knowledge distillation achieves knowledge transfer from a high-capacity teacher model to a compact student model by constructing a collaborative training framework for teacher and student models. Its technical path mainly includes two categories: the first is distillation based on probability distribution: the student model learns implicit dark knowledge such as inter-class similarity by fitting the soft targets output by the teacher model, i.e., the category probability distribution modulated by temperature parameters; the second is distillation based on feature transfer: by constraining the similarity between the activation values ​​of the intermediate feature maps of the student model and the corresponding layers of the teacher model, cross-model transfer of deep semantic features is achieved.

[0006] Knowledge transfer mechanisms enable the distilled student model to maintain performance close to that of the teacher model while reducing the number of parameters. Currently, knowledge distillation has become the preferred solution for computationally limited scenarios such as edge computing and mobile intelligence, demonstrating significant advantages in both model compression and accuracy improvement.

[0007] To address the challenges faced by object detection algorithms when deployed on edge devices, this invention proposes a knowledge distillation method for object detection based on multi-scale channel-space decoupling. By enabling a lightweight student model to learn the knowledge representations of a complex teacher model, the method more closely approximates the real-world data distribution, thereby achieving a balance between accuracy and efficiency on edge devices. Summary of the Invention

[0008] To address existing problems and shortcomings in current technologies, this invention proposes a knowledge distillation method for target detection based on multi-scale channel-spatial decoupling. This method helps student models improve their local discriminative power and cross-scale feature learning capabilities, thereby significantly enhancing the detection accuracy and robustness of student models and ultimately improving the performance of target detection models on edge devices.

[0009] The method of this invention first extracts feature maps from the output layers of the feature pyramid network for both the teacher and student models, and aligns the feature sizes. Channel normalization is then performed on the feature maps of both the teacher and student models, and adaptive pooling is executed at multiple levels to extract multi-scale features. Attention weights are generated based on the channel activation values ​​of the teacher model and used to weight the pooled features of the student model. Distillation loss is calculated at each scale, and weighting factors are introduced for complementary regions to enhance differential guidance. Finally, the distillation losses from each level are weighted and fused to obtain the final total distillation loss, which is used to optimize the network parameters of the student model.

[0010] In the above method, feature maps are extracted from the feature pyramid network: at the output layer of the feature pyramid network, feature maps of multiple levels are extracted from the teacher model T and the student model S.

[0011] T∈R L×C×H×W

[0012] S∈R L×C×H×W

[0013] Where L represents the number of layers in the feature pyramid network, C represents the number of channels, and H and W represent the height and width of the feature map, respectively; then, the spatial size and number of channels of the student model feature map are adjusted by bilinear interpolation or convolution to match those of the teacher model feature map. Figure 1 To:

[0014]

[0015] In the above method, multi-scale features are extracted: channel normalization is performed on the feature maps of the teacher model and the student model respectively to reduce the interference of activation bias.

[0016] T′=φ(T)

[0017]

[0018] Where φ() represents the normalization function, T′∈R L×C×H×W , S′∈R L×C×H×W Then, adaptive average pooling is performed at multiple scales to extract region feature representations at different spatial granularities:

[0019] T (s) =AvgPool(T′,s×s)

[0020] S (s) =AvgPool(S′,s×s)

[0021] Where AvgPool represents adaptive average pooling, s refers to different scales, s∈{1,2,4}, and T (s) ∈R L×C×H×W S (s) ∈R L×C×H×W .

[0022] In the above method, the pooling features of the weighted student model are as follows: the teacher network outperforms the student network in performance, and its channels with higher feature activation values ​​usually reflect features that are more discriminative for the detection task; the channel attention weights of the teacher network are calculated through global average pooling; firstly, global average pooling is performed on the pooling features of the teacher model at each scale to calculate the activation value of each channel:

[0023]

[0024] The activation values ​​are normalized using the Softmax function and then scaled to obtain the channel attention weights:

[0025] w (s) =softmax(g (s) )·γ+1

[0026] Where γ is the scaling factor; this weight is applied to the pooled features of the student model at the same scale to obtain the weighted student feature map, which is used for subsequent loss calculation:

[0027]

[0028] In the above method, the distillation loss is calculated at each scale: at each scale, the root mean squared error loss between the weighted student feature map and the teacher feature map is calculated.

[0029]

[0030] Using the global scale (s=1) loss as the base loss, the losses at other scales (s=2,4) are multiplied by a weighting coefficient β=2 to construct complementary region losses; the base loss and complementary loss are summed to obtain the total distillation loss for each layer.

[0031]

[0032] In the above method, the overall distillation loss is calculated as follows: the multi-scale distillation process is performed on the feature maps of all levels to obtain the distillation loss of each level; the losses of all levels are then weighted and summed to obtain the overall distillation loss, which is used to optimize the parameters of the student model.

[0033]

[0034] Among them, L origin This refers to the loss of the original model, where λ represents the distillation loss weight. Represents a pyramid hierarchy. This represents the total number of floors.

[0035] Compared with the prior art, the method described in this invention has the following obvious and prominent substantive features and significant technological advancements:

[0036] 1) A channel-space decoupling distillation mechanism is proposed to effectively separate channel semantic information from spatial localization features, solve the problem of mixed background noise in the high-activation channels of the teacher model, and enhance the student model's ability to perceive key regions.

[0037] 2) Introducing a multi-scale fine-grained supervision strategy, feature distillation is performed at multiple scales (1×1, 2×2, 4×4), which improves the balance between feature learning for small and large targets, helps student models to mine and inherit fine-grained knowledge, and effectively enhances the detection performance of student models in complex multi-scale scenes.

[0038] 3) Design a differential guidance weighting mechanism to explicitly weight distill complementary regions, highlight the guiding role of key regions, and further improve the discriminativeness and efficiency of knowledge transfer. Attached Figure Description

[0039] Figure 1 A flowchart illustrating the specific training process of the method of this invention;

[0040] Figure 2 The present invention discloses a framework diagram of a target detection knowledge distillation method based on multi-scale channel-spatial decoupling. Detailed Implementation

[0041] The following combination Figure 1 and Figure 2 The specific embodiments of the present invention will be described in detail below. It should be noted that the specific embodiments described herein are for illustrative and explanatory purposes only and are not intended to limit the present invention.

[0042] This invention presents a knowledge distillation method for target detection based on multi-scale channel-space decoupling, using the Python and PyTorch deep learning frameworks. In this embodiment, a Yolov8-small network is selected as the teacher model, and a Yolov8-nanos network is selected as the student model.

[0043] See Figure 1 This invention discloses a target detection knowledge distillation method based on multi-scale channel-spatial decoupling, the specific operation steps of which are as follows:

[0044] 1) Prepare the training dataset: Convert the image dataset used for training to a resolution of 640×640, convert the annotation format to a format supported by YOLOv8, and divide it into training and test sets.

[0045] 2) Training the teacher model: Construct an improved deep neural network model and train the teacher model T using a YOLOv8-small network. Determine parameters such as the optimizer, learning rate, and maximum number of iterations, and begin training the teacher network. After each forward propagation, calculate the loss between the training result and the ground truth, and then update the network parameters using the backpropagation algorithm. Repeat the above steps until the pre-set maximum number of iterations is reached to obtain the teacher model T.

[0046] 3) Training the student model: Use the minimum parameter network yolov8-nanos as the student model S, and the teacher model T trained in the previous step. Before starting the student model training, input the teacher model T, such as... Figure 2As shown, during the training process, a target detection knowledge distillation method based on multi-scale channel-space decoupling of this invention is used to distill the features of the teacher model into the student model. The process is as follows: In the output layer of the feature pyramid network, feature maps of multiple levels are extracted from the teacher model T and the student model S:

[0047] T∈R L×C×H×W

[0048] S∈R L×C×H×W

[0049] Where L represents the number of layers in the feature pyramid network, C represents the number of channels, and H and W represent the height and width of the feature map, respectively; then, the spatial size and number of channels of the student model feature map are adjusted by bilinear interpolation or convolution to match those of the teacher model feature map. Figure 1 To:

[0050]

[0051] Channel normalization was performed on the feature maps of both the teacher and student models to reduce the interference of activation bias.

[0052] T′=φ(T)

[0053]

[0054] Where φ() represents the normalization function, T′∈R L×C×H×W , S′∈R L×C×H×W Then, adaptive average pooling is performed at multiple scales to extract region feature representations at different spatial granularities:

[0055] T (s) =AvgPool(T′,s×s)

[0056] S (s) =AvgPool(S′,s×s)

[0057] Where AvgPool represents adaptive average pooling, s refers to different scales, s∈{1,2,4}, and T (s) ∈R L×C×H×W S (s) ∈R L×C×H×W Considering that the teacher network outperforms the student network in performance, channels with higher feature activation values ​​typically reflect features that are more discriminative for the detection task. The channel attention weights of the teacher network are calculated using global average pooling. First, global average pooling is performed on the pooled features of the teacher model at each scale to calculate the activation value of each channel:

[0058]

[0059] The activation values ​​are normalized using the Softmax function and then scaled to obtain the channel attention weights:

[0060] w (s) =softmax(g (s) )·γ+1

[0061] Where γ is the scaling factor; this weight is applied to the pooled features of the student model at the same scale to obtain the weighted student feature map, which is used for subsequent loss calculation:

[0062]

[0063] Calculate the root mean squared error loss between the weighted student and teacher feature maps at each scale:

[0064]

[0065] Using the global-scale loss as the base loss, the losses at other scales are multiplied by a weighting coefficient β = 2 to construct complementary region losses; the base loss and complementary loss are then summed to obtain the total distillation loss for each layer.

[0066]

[0067] The multi-scale distillation process described above is performed on the feature maps of all levels to obtain the distillation loss for each level. The losses of all levels are then weighted and summed to obtain the overall distillation loss, which is used to optimize the parameters of the student model.

[0068]

[0069] Among them, L origin This refers to the loss of the original model, where λ represents the distillation loss weight. Represents a pyramid hierarchy. This represents the total number of floors.

[0070] After calculating the distillation loss, the network parameters are updated using the backpropagation algorithm. The above steps are repeated until the preset maximum number of iterations is reached to complete the training of the student model S.

[0071] 6) Save the network: Solidify the set of network weights with the highest evaluation index during the training process and save it as the final student network model.

[0072] The parameter comparison between the Teacher model and the Student model is shown in Table 1:

[0073]

[0074] Table 1 compares the number of parameters in the teacher and student models. The evaluation results of the teacher model T and the student model S are shown in Table 2.

[0075]

[0076] Table 2 shows the evaluation results of the teacher model and the student model. As can be seen from Table 2 compared with Table 1, the student model trained using the target detection knowledge distillation method based on multi-scale channel-space decoupling of the present invention has higher accuracy than the YOLOv8-Nanos model trained alone. The improvement in accuracy is very significant for the minimum scale YOLOv8-N network, which quantitatively demonstrates the effectiveness of the present invention.

[0077] The preferred embodiments of the present invention have been described in detail above. It should be understood that those skilled in the art can make numerous modifications and variations based on the concept of the present invention without creative effort. Therefore, all technical solutions that can be obtained by those skilled in the art based on the concept of the present invention through logical analysis, reasoning, or limited experimentation on the basis of existing technology should be within the scope of protection defined by the claims.

Claims

1. A target detection knowledge distillation method based on multi-scale channel-spatial decoupling, characterized in that, Includes the following steps: S1. In the output layer of the feature pyramid network, feature maps are extracted from the teacher model and the student model respectively, and the feature sizes are aligned. S2. Channel normalization is performed on the feature maps of the teacher model and the student model respectively, and adaptive pooling is performed at multiple levels to extract multi-scale features; S3. Generate attention weights based on the channel activation values ​​of the teacher model, and use them to weight the pooling features of the student model; S4. Calculate the distillation loss at each of the aforementioned scales, and introduce a weighting factor for the complementary region; S5. Weighted fusion of the distillation losses at each stage to obtain the final total distillation loss.

2. The target detection knowledge distillation method based on multi-scale channel-spatial decoupling according to claim 1, characterized in that, In step S1, feature maps of multiple levels are extracted from the teacher model T and student model S at the output layer of the feature pyramid network: T∈R L×C×H×W S∈R L×C×H×W Where L represents the number of layers in the feature pyramid network, C represents the number of channels, and H and W represent the height and width of the feature map, respectively; then, the spatial dimensions and number of channels of the student model feature map are adjusted by bilinear interpolation or convolution to make it consistent with the teacher model feature map:

3. The target detection knowledge distillation method based on multi-scale channel-spatial decoupling according to claim 1, characterized in that, In step S2, channel normalization is performed on the feature maps of the teacher model and the student model respectively: T′=φ(T) Where φ() represents the normalization function, T′∈R L×C×H×W , S′∈R L×C×H×W Then, adaptive average pooling is performed at multiple scales to extract region feature representations at different spatial granularities: T (s) =AvgPool(T′,s×s) S (s) =AvgPool(S′,s×s) Where AvgPool represents adaptive average pooling, s refers to different scales, s∈{1,2,4}, and T (s) ∈R L×C×H×W S (s) ∈R L×C×H×W .

4. The target detection knowledge distillation method based on multi-scale channel-spatial decoupling according to claim 1, characterized in that, In step S3, global average pooling is first performed on the pooled features of the teacher model at each scale to calculate the activation value of each channel: The activation values ​​are normalized using the Softmax function and then scaled to obtain the channel attention weights: w (s) =softmax(g (s) )·γ+1 Where γ is the scaling factor; this weight is applied to the pooled features of the student model at the same scale to obtain the weighted student feature map, which is used for subsequent loss calculation:

5. The target detection knowledge distillation method based on multi-scale channel-spatial decoupling according to claim 1, characterized in that: In step S4, the mean squared error loss between the weighted student feature map and the teacher feature map is calculated at each scale: Using the global-scale loss as the base loss, the losses at other scales are multiplied by a weighting coefficient β = 2 to construct complementary regional losses; The total distillation loss for each layer is obtained by summing the basic loss and the complementary loss:

6. The target detection knowledge distillation method based on multi-scale channel-spatial decoupling according to claim 5, characterized in that: In step S5, the above multi-scale distillation process is performed on the feature maps of all levels to obtain the distillation loss of each level. The total distillation loss is obtained by weighted summation of the losses from all layers: Among them, L origin The loss refers to the original model's loss, where λ represents the distillation loss weight, l represents the pyramid level, and N... l This represents the total number of floors.

7. The target detection knowledge distillation method based on multi-scale channel-spatial decoupling according to claim 1, characterized in that: Step S5 involves fixing the set of network weights with the highest evaluation metrics during training and saving them as the final student network model. and This represents the student model trained by YOLOv8-X distillation. This indicates that the student models trained by yolov8-s distillation are evaluated and compared, and the best one is selected as the final deployable model.

Images