A lightweight target detection method based on model compression and knowledge distillation

Abstract

The application discloses a lightweight target detection method based on model compression and knowledge distillation, first reconstructing a YOLO model backbone network based on a deep separable convolution and an inverted residual structure of a MobileNet architecture, fusing a C2fCIB module, an SE attention module and a Hard-Swish activation function, and constructing a compact student model; further building a teacher-student heterogeneous knowledge distillation framework, aligning the channel numbers through an adaptive layer, adopting a channel knowledge distillation loss and a Logits mean square error loss, and migrating deep features and output response knowledge of a teacher model to a student model; finally, dynamically adjusting a distillation weight through a cosine annealing strategy, and obtaining a lightweight target detection model with high precision and low complexity through end-to-end training. The application effectively realizes lightweight deployment of a target detection model, greatly reduces model parameter quantity and computational complexity, maintains high detection precision, and is suitable for resource-limited edge device scenes.

Description

Technical Field

[0001] This invention belongs to the field of computer vision and deep learning technology, specifically relating to a lightweight object detection method based on model compression and knowledge distillation. Background Technology

[0002] Object detection, as one of the core tasks in computer vision, plays a crucial role in fields such as autonomous driving, intelligent video surveillance, industrial defect detection, and aerospace. With the rapid development of deep learning technology, object detection algorithms based on Convolutional Neural Networks (CNNs) have gradually replaced traditional manual feature extraction methods. Among them, single-stage detection algorithms, represented by the YOLO (You Only Look Once) series, as described in article 1 "Redmon J, Divvala S, Girshick R, et al. You only look once: Unified, real-time object detection[C] / / Proceedings of the IEEE conference on computer vision and pattern recognition. 2016: 779-788.", achieve fast end-to-end detection by transforming the object detection task into a regression problem, achieving a good balance between detection speed and accuracy.

[0003] However, with the improvement of algorithm performance, the number of network layers in the YOLO series models has been continuously increasing, leading to a significant increase in the number of model parameters and computational complexity. For example, in pursuit of higher detection accuracy, modern YOLO models are often equipped with complex backbone and neck networks. As described in article 2, "Bochkovskiy A, Wang CY, Liao HYM. Yolov4: Optimal speed and accuracy of object detection[J]. arXiv preprintarXiv:2004.10934, 2020," although optimizing the network structure improves accuracy, it results in a huge weight file size generated by the model, placing extremely high demands on computing resources and storage space. In practical applications, especially in resource-constrained environments such as mobile terminals, embedded devices, or edge computing nodes, deploying these large models faces severe challenges.

[0004] To address these issues, lightweight model techniques have emerged. Current mainstream lightweight methods primarily include model pruning, quantization, and designing lightweight network architectures. Among these, the MobileNet series of networks has garnered significant attention due to its proposed Depthwise Separable Convolution (DSConv) structure. Article 3, “Howard AG, Zhu M, Chen B, et al. Mobilenets: Efficient convolutional neural networks for mobile vision applications[J]. arXiv preprint arXiv:1704.04861, 2017,” proposes the MobileNet structure, which significantly reduces the number of model parameters and computational cost by decomposing standard convolution into two steps: depthwise convolution and pointwise convolution. While replacing the YOLO backbone network with MobileNet can significantly compress the model size, the inherent weakness of lightweight networks in feature extraction capabilities often leads to a noticeable decrease in detection accuracy.

[0005] Knowledge distillation (KD), as an emerging model compression paradigm, provides an effective way to compensate for the accuracy loss of lightweight models. This method was first proposed in paper 4, "Hinton G, Vinyals O, Dean J. Distilling the knowledge in a neural network[J]. arXiv preprint arXiv:1503.02531, 2015." It constructs a "teacher-student" learning framework, allowing a lightweight student model to mimic the behavior of a high-performance teacher model. Existing knowledge distillation methods are mainly divided into three categories: response-based, feature-based, and relation-based. For example, article 5, “Li Q, Jin S, Yan J. Mimicking very efficient network for object detection[C] / / Proceedings of the IEEE conference on computer vision and pattern recognition, 2017: 6356-6364,” proposes a feature-mimicking-based distillation method, and article 6, “Yang Z, Li Z, Shao M, et al. Masked generative distillation[C] / / Europeanconference on computer vision. Cham: Springer Nature Switzerland, 2022: 53-69,” proposes the Masked Generative Distillation (MGD) method. However, in the distillation process for object detection models like YOLO, traditional feature-based distillation methods often struggle to handle the significant differences in channel count and feature semantics between teacher and student models.Article 7, "Shu C, Liu Y, Gao J, et al. Channel-wise knowledge distillation for dense prediction[C] / / Proceedings of the IEEE / CVF international conference on computer vision. 2021: 5311-5320," proposes Channel-wise Knowledge Dissillation (CWD) to focus on the probability distribution along the channel dimension. However, when combining lightweight network architectures (such as MobileNet) with the YOLO detection head for end-to-end optimization, the issues of feature alignment and multidimensional loss balance still need to be further addressed.

[0006] Existing object detection algorithms still face the dilemma of "difficulty in balancing high accuracy and lightweight design" when deployed at edge computing. Therefore, there is an urgent need for a design method that can deeply integrate the advantages of lightweight network architecture and restore or even improve model accuracy through improved knowledge distillation strategies, so as to achieve efficient and high-accuracy object detection on resource-constrained devices. Summary of the Invention

[0007] The purpose of this invention is to provide a lightweight target detection method based on model compression and knowledge distillation, in order to solve the problems of existing YOLO target detection models having a large number of parameters, high computational complexity, and difficulty in efficient deployment on resource-constrained devices.

[0008] To achieve the above objectives, the present invention employs the following technical solution: A lightweight object detection method based on model compression and knowledge distillation includes: The image to be detected is acquired and input into a lightweight object detection model to obtain the object detection result; wherein, the construction method of the lightweight object detection model includes: The backbone network of the YOLO model is reconstructed by introducing depthwise separable convolutions and inverted residual structures from MobileNet. Depthwise separable convolutions replace standard convolutions, decomposing the standard convolution operation into depthwise convolutions and pointwise convolutions. An inverted residual structure is introduced into the backbone network to replace the original residual structure. An SE attention module is embedded inside the Bottleneck module of the backbone network, and a Hard-Swish activation function replaces some of the original ReLU activation functions in the backbone network. A C2fCIB module is fused into the neck network; the C2fCIB module is formed by replacing the original Bottleneck module in the C2f structure of the neck network with a CIB module, ultimately constructing the student model. A dual knowledge distillation strategy based on features and responses was adopted. This strategy constructs a teacher-student distillation framework. During the distillation process, a feature-based channel knowledge distillation loss is used to guide the student model to imitate the intermediate layer feature representation of the teacher model by minimizing the KL divergence between the channel probability maps of the teacher and student models. At the same time, a response-based Logits distillation loss is combined, and the final prediction output of the teacher model is used as a soft label to guide the training of the student model. Combined with the detection loss of the student model, a total loss function is constructed, and the student model is trained end-to-end iteratively to obtain a lightweight object detection model.

[0009] Furthermore, a CBHw module is constructed in the backbone network, consisting of a concatenated convolutional layer Conv, a batch normalization layer BatchNorm, and a Hard-Swish activation function, which serves as a feature extraction unit in the backbone network. The ReLU activation function in the Bottleneck module is replaced with a Hard-Swish activation function, and an SE attention module is incorporated to obtain an improved Bottleneck module.

[0010] Furthermore, an adaptive layer is designed to map the number of feature channels in the student model to be consistent with the number of feature channels in the teacher model using convolution operations; For the intermediate feature layers of the teacher and student models, each channel of the output intermediate feature map is subjected to Softmax normalization to obtain a probability distribution containing spatial location information; then, KL divergence is used to measure the difference between the teacher and student models in the channel probability distribution, and the CWD distillation loss is calculated.

[0011] Furthermore, temperature parameters are introduced. The Softmax function is used to extract the intermediate layer feature map. The feature values ​​of each channel are converted into a probability distribution; for each channel in the intermediate layer feature map... Its spatial location The probability value is: ; in, Indicates the first The first channel in the The probability values ​​of each spatial location after Softmax normalization; Represents the first in the feature graph The first channel, the first The activation value of each spatial location; Indicates the feature map height. Indicates the width of the feature map; KL divergence is used to measure the difference in channel probability distribution between the teacher model and the student model, and feature-based CWD distillation loss is calculated. ,Right now: ; in, Indicates the number of channels. This represents a function that measures the difference in channel probability distributions. The middle layer feature map of the teacher model represents the first... The first channel Location The normalized probability value; The middle layer feature map of the student model represents the first... The first channel Location The normalized probability value.

[0012] Furthermore, the final predicted output of the teacher model is introduced as additional supervision information, and the Logits output of the teacher model is used as a soft label; the mean squared error loss function is used to calculate the difference between the student model output and the teacher model Logits output, and the response-based Logits distillation loss is obtained.

[0013] Furthermore, the response-based Logits distillation loss is represented as follows: ; in, Indicates batch size; Indicates the sample index within the batch; The teacher model represents the first The Logits output for each sample; Represents the student model for the first The Logits output for each sample; This represents the square of the L2 norm.

[0014] Furthermore, define the total loss function. Including the original detection loss Feature-based CWD distillation loss and response-based Logits distillation loss ;in, The original detection loss for the student model consists of classification loss, localization loss, and confidence loss. Introducing a cosine annealing strategy to dynamically adjust the weighting coefficients of distillation losses The total loss function is expressed as follows: ; in, This represents the balance coefficient for Logits distillation loss, used to adjust the ratio of Logits distillation loss to CWD distillation loss.

[0015] Furthermore, weighting coefficients The formula for how this changes during the training process is: ; in, This indicates the total number of batches used in model training; This indicates the batch index of the current training session.

[0016] Furthermore, based on the total loss function, a stochastic gradient descent optimizer is used to perform end-to-end iterative training on the student model. During training, the parameters of the teacher model remain frozen and act solely as a knowledge provider, transferring deep features and output response knowledge to the student model through CWD distillation loss and Logits distillation loss. The parameters of the student model are adjusted according to the total loss function. By continuously updating through backpropagation and guided by the teacher model, the feature extraction capability and detection accuracy are gradually improved until the model converges, resulting in the final lightweight object detection model.

[0017] A terminal device includes a processor, a memory, and a computer program stored in the memory; when the processor executes the computer program, it implements the lightweight target detection method based on model compression and knowledge distillation.

[0018] A computer-readable storage medium storing a computer program; when executed by a processor, the computer program implements the lightweight target detection method based on model compression and knowledge distillation.

[0019] Compared with the prior art, the present invention has the following technical features: First, it significantly reduces resource consumption and improves the model's deployment capability on edge devices. This invention reconstructs the backbone network by introducing a lightweight MobileNet architecture and integrates the efficient C2fCIB module, greatly optimizing the model's computational efficiency and storage requirements. Experimental data shows that the optimized DS-YOLO-M model has only 2.2M parameters, reduces computation to 6.5 GFLOPs, and the final weight file size is only 4.3MB. Compared to the original YOLOv8n model, this model reduces the number of parameters by 31.2%, computation by 27.0%, and the weight file size by 34.2%. This extremely lightweight design allows the model to be easily deployed on embedded devices or mobile terminals with limited computing resources, greatly expanding the application scenarios of object detection technology.

[0020] Secondly, it maintains high detection accuracy, effectively solving the problem of accuracy degradation in lightweight models. This invention employs a dual strategy combining channel knowledge distillation (CWD) and Logits response distillation to effectively transfer the deep knowledge of the high-performance teacher model YOLO11m to the student model. Experimental results show that the knowledge-distilled DS-YOLO-M (distilled) model performs exceptionally well on the data-augmented NWPU VHR-10 dataset, achieving an mAP@0.5 score of 99.4% and an F1 score of 0.98. This performance not only compensates for the accuracy loss caused by network structure simplification but also outperforms many similar lightweight models, achieving high-precision target recognition with low computational cost.

[0021] Third, it enhances feature extraction capabilities, improving the model's representation of complex image details. Visual analysis and comparison of feature maps reveal that the model distilled using the method of this invention has a significant advantage in feature extraction compared to the undistilled model. This model can more clearly and completely preserve key features in the image, such as the edge contours and texture details of targets, effectively solving the problems of blurred feature maps and difficulty in identifying key information in lightweight models. This enhanced feature representation capability allows the model to maintain high robustness and accuracy even when facing complex backgrounds or fine-grained target detection tasks. Attached Figure Description

[0022] Figure 1 This is a diagram illustrating the framework of model compression technology. Figure 2 A flowchart for knowledge distillation; Figure 3 This is a schematic diagram of the principle of depthwise separable convolution. Figure 4 A comparison diagram of residual structure and inverse residual structure; Figure 5 Here is a diagram of the SE module structure; Figure 6 Here is a structural diagram of the CBHw module; Figure 7 A structural diagram of the Bottleneck module integrated into SE; Figure 8 Here is a structural diagram of the C2fCIB module; Figure 9 This is a diagram of the DS-YOLO-M model architecture. Figure 10 Detailed flowchart for knowledge distillation; Figure 11 This is a diagram of a feature-based knowledge distillation architecture. Figure 12 A diagram of a response-based knowledge distillation architecture; Figure 13A comparison chart of mAP for model parameters; Figure 14 A comparison chart of model size versus computational complexity; Figure 15 A comparison chart of the detection accuracy of the model for each category; Figure 16 Comparison curves for performance metrics (precision, recall, mAP@0.5, mAP@0.5-0.95); Figure 17 The confusion matrix diagram for the DS-YOLO-M (distilled) model; Figure 18 A visualization of the feature map of the DS-YOLO-M model; Figure 19 A visualization of the feature map of the DS-YOLO-M (distilled) model. Detailed Implementation

[0023] This invention proposes a lightweight object detection method based on model compression and knowledge distillation. This method addresses the problems faced by YOLO series models when deployed on edge devices, such as large parameter count, high computational resource consumption, and significant accuracy degradation caused by traditional lightweight methods. By reconstructing the network structure and introducing an efficient distillation strategy, it significantly reduces model resource consumption while maintaining a high level of detection accuracy. This invention mainly includes three core stages: constructing a lightweight student model DS-YOLO-M, building a teacher-student heterogeneous distillation framework, and performing end-to-end training under multidimensional loss constraints.

[0024] This invention reconstructs the backbone network of the YOLOv8n model by introducing depthwise separable convolutions and inverted residual structures from MobileNet. Depthwise separable convolutions replace standard convolutions, decomposing the standard convolution operation into depthwise convolutions (DWConv) and pointwise convolutions (PWConv), thus significantly reducing the number of parameters and computational cost. Simultaneously, inverted residual structures are introduced into the backbone network to replace the original residual structure, improving feature representation by first increasing the dimensionality to extract features and then reducing the dimensionality. Channel compression and excitation (SE) attention are embedded within the Bottleneck module of the backbone network. The model incorporates modules to enhance its sensitivity to important channel features and replaces some of the original ReLU activation functions in the backbone network with Hard-Swish activation functions, improving computational speed while maintaining accuracy. Furthermore, the neck network integrates the C2fCIB module, which replaces the original Bottleneck module with a CompactInvertedBlock (CIB). This further compresses computational costs and enhances feature fusion capabilities through a combination of depthwise convolution and pointwise convolution. Ultimately, this results in DS-YOLO-M (Depthwise Separable-YOLO-MobileNet, a YOLO model based on depthwise separable convolution and the MobileNet architecture) as a lightweight student model. The model compression technology framework diagram is shown below. Figure 1 As shown.

[0025] Secondly, this invention adopts a dual knowledge distillation strategy based on both features and responses. This strategy constructs a teacher-student distillation framework, selecting the high-performance YOLO11m as the teacher model. The knowledge distillation flowchart is as follows: Figure 2As shown, during the distillation process, a feature-based channel-wise knowledge distillation (CWD) loss is employed. By minimizing the KL divergence between the channel probability maps of the teacher and student models, the student model is guided to mimic the intermediate layer feature representation of the teacher model. Simultaneously, a response-based Logits distillation loss is combined, using the final prediction output of the teacher model as a soft label to guide the training of the student model. Combined with the detection loss of the student model, a total loss function is constructed, and the student model undergoes end-to-end iterative training. This results in a lightweight object detection model, DS-YOLO-M (distilled), which combines high detection accuracy with low computational complexity, for high-precision and efficient object detection of input images. To address the issue of inconsistent channel numbers in the intermediate feature layers of the teacher and student models, an adaptive layer is designed for channel alignment. The method of this invention specifically includes the following steps: Step 1: Construct a lightweight object detection model DS-YOLO-M based on the MobileNet architecture.

[0026] This step involves deeply reconstructing the YOLO model (network structure including the backbone, neck, and head) by introducing key components of the MobileNet architecture (depthseparable convolution and inverse residual structure), thus building a compact student model, DS-YOLO-M, reducing the model's space and time complexity. The model compression technology framework is as follows: Figure 1 As shown.

[0027] Step 1.1 introduces depthwise separable convolution and inverse residual structures.

[0028] In the design of the backbone network, depthwise separable convolution is used to completely replace the standard convolution operation; depthwise separable convolution decomposes the standard convolution into two independent processes: depthwise convolution and pointwise convolution, such as... Figure 3 As shown.

[0029] Assume the size of the input feature map is The number of channels in the output feature map is The kernel size is ;in, This indicates the height of the input feature map (i.e., the number of pixels or feature points in the vertical direction); Indicates the width of the input feature map; This indicates the number of channels in the input feature map (i.e., the number of pixels in the horizontal direction); This indicates the size of the convolution kernel.

[0030] The computational cost of standard convolution is: ; in, This represents the number of floating-point operations in a standard convolution.

[0031] The computational cost of depthwise separable convolution consists of two parts: depthwise convolution and pointwise convolution. ; in, This represents the number of floating-point operations performed in a depthwise separable convolution.

[0032] Compared to standard convolution, the computational cost is reduced by a certain percentage. for: ; in, Indicates the percentage reduction in computational load. When At that time, the amount of computation is reduced to approximately half of the original amount. arrive This significantly reduces computational costs.

[0033] Based on this, inverted residuals are introduced to replace the original residual structure in the backbone network, such as... Figure 4 As shown. The inverse residual structure differs from the traditional residual structure's "dimensionality reduction-convolution-dimensionality increase" approach; it employs a "dimensionality increase-convolution-dimensionality reduction" strategy. First, it uses... Convolution performs dimensionality expansion, typically with a scaling factor of 6, allowing features to undergo non-linear transformations in a higher-dimensional space, thus preventing information loss. Then, in the higher-dimensional space... Feature extraction is performed using depthwise convolution; finally, features are extracted using... Convolution reduces the number of channels to the number of output channels. This spindle-shaped structure, which is small at both ends and large in the middle, can effectively reduce information loss during transmission and improve feature representation capabilities.

[0034] Step 1.2: Embed the SE attention module with the Hard-Swish activation function.

[0035] To compensate for the shortcomings of lightweight networks in feature extraction, an SE attention module is embedded inside the Bottleneck module in the backbone network, and a Hard-Swish activation function is introduced to construct an improved Bottleneck module. The structure of the SE attention module is as follows: Figure 5 As shown, its calculation process includes two stages: squeezing and excitation. (1) Compression stage: Global Average Pooling (GAP) is used to compress the dimension of the data. Input feature map Compress to Features . No. Output of each channel The calculation formula is: ; in, Indicates the first The global feature value output by each channel after global average pooling is a scalar. This represents the global average pooling function; Represents the input feature map The first in One channel; Indicates the first The coordinates in each channel are The specific pixel value; Indicates the height of the feature map; Indicates the width of the feature map; This indicates the number of channels in the feature map.

[0036] (2) Excitation Phase: To capture the nonlinear dependencies between channels, two fully connected layers (FC) are used; the first fully connected layer reduces the number of channels to After ReLU activation, the second fully connected layer restores the channel to its original state. Finally, the channel weights are generated using the Sigmoid function. : ; in, This represents the generated channel weight vector; This represents the excitation function, used to capture the nonlinear relationship between channels; The feature vector and weight vector represent the output of the compression stage. , This represents the weight matrix of the first fully connected layer. This represents the weight matrix of the second fully connected layer. The scaling parameter SERatio is set to 0.25 in this embodiment; This represents the Sigmoid activation function; Represents the ReLU activation function; It is the space of real numbers.

[0037] (3) Recalibration: Finally, the calculated channel weights are recalibrated using the Scale operation. Compared with the original input feature map Multiply each channel sequentially to obtain the final output. : ; in, Indicates the recalibrated first Each channel feature map, together with all channel feature maps, constitutes... ; This represents the recalibration function for channel-wise multiplication; Represents the input feature map The One channel; Indicates the first The weight scalar corresponding to each channel.

[0038] Meanwhile, to reduce computational overhead and maintain nonlinear characteristics, the Hard-Swish activation function is introduced to replace the ReLU activation function in the original Bottleneck module, thus constructing the CBHw module; Hard-Swish utilizes The piecewise linear property of the Swish function is used to approximate it, and its mathematical expression is: ; in, This represents the output of the Hard-Swish activation function; This represents the input feature value.

[0039] The definition of is: ; in, This represents the output of the ReLU6 activation function, which restricts the input values ​​to the range of 0 to 6; Indicates taking The larger of 0 and 0 is selected; min indicates the smaller of the two. This activation function maintains precision while avoiding the loss of numerical precision during quantization, and its computation speed on embedded devices is much faster than Sigmoid and its derivative.

[0040] Combining the above components, the CBHw (Convolution-BatchNorm-HardSwish) module is constructed; for example... Figure 6 As shown, the CBHw module consists of a concatenated convolutional layer Conv, a batch normalization layer BatchNorm, and a Hard-Swish activation function; the improved Bottleneck module is as follows: Figure 7As shown, the Bottleneck module, which incorporates the SE attention module and replaces the ReLU activation function with the Hard-Swish activation function, is applied to the backbone network.

[0041] Among them, the CBHw module is used to replace the original Conv-BN-ReLU module (i.e., the module consisting of convolutional layers, batch normalization layers and ReLU activation functions) in each feature extraction stage of the backbone network; the improved Bottleneck module is used to replace the original Bottleneck module.

[0042] Step 1.3: Construct the Neck network by integrating the C2fCIB module.

[0043] In the neck network, a C2fCIB (C2f-Compact Inverted Block) module is introduced, such as... Figure 8 As shown, the C2fCIB module is a lightweight improvement on the C2f structure of YOLOv8, replacing the original Bottleneck module with a compact inverted bottleneck block (CIB). The CIB module uses a combination of depthwise convolution and pointwise convolution, which can further reduce computational cost and number of parameters while maintaining high performance.

[0044] Using the improved backbone network, neck network, and the original detection head of the YOLOv8n model, the final student model DS-YOLO-M architecture is constructed as follows: Figure 9 As shown.

[0045] Experiments show that the student model DS-YOLO-M has approximately 2.2M parameters and a computational cost of approximately 6.5 GFLOPs. Compared to the original YOLOv8n model (3.2M parameters, 8.9 GFLOPs), the student model DS-YOLO-M reduces the number of parameters by 31.2%, the computational cost by 27.0%, and the weight file size by 34.2%.

[0046] Step 2: Construct a knowledge distillation training framework for a heterogeneous teacher-student network.

[0047] This step constructs as follows: Figure 10 The teacher-student knowledge distillation process shown is used to recover the accuracy loss caused by lightweight design.

[0048] Teacher model selection: YOLO11m, with 20.1M parameters, strong feature extraction capabilities, and structural similarity to the student model, was selected as the teacher model.

[0049] Student model selection: Select the lightweight student model DS-YOLO-M constructed in step 1.

[0050] Adaptive layer design: To address the inconsistency in the number of channels in the intermediate feature layers (such as Layer 15, 18, 21, etc.) of the backbone and neck regions of the teacher and student models, an adaptive layer is designed; utilizing... Convolution operation Number of feature channels in the student model Mapped to the number of feature channels in the teacher model Consistency, that is This ensures dimensionality matching for subsequent distillation loss calculations. This indicates the number of channels after mapping.

[0051] Step 3: Calculate the feature-based channel knowledge distillation loss function.

[0052] This step employs a feature-based distillation strategy, specifically using the Channel-wise Knowledge Distillation (CWD) method, such as... Figure 11 As shown, CWD focuses on probability distribution matching along the channel dimension, rather than simple spatial pixel matching, making it more suitable for scenarios where the foreground and background are imbalanced in object detection tasks.

[0053] Feature-based knowledge distillation follows a general loss function expression as follows: ; in, This represents the feature-based knowledge distillation loss; The intermediate layer feature map representing the teacher model; The intermediate layer feature map represents the student model; Indicates the input image; and These represent the transformation functions used for channel alignment in the teacher and student models, respectively. This represents a metric function that measures the similarity of feature maps.

[0054] like Figure 11 As shown, CWD focuses on probability distribution matching along the channel dimension, rather than simple spatial pixel matching, making it more suitable for foreground-background imbalance scenarios in object detection tasks. The intermediate feature maps (feature maps output by the intermediate feature layers) of the teacher and student models are represented as follows: and .in Indicates the number of channels. Indicates the feature map height. Indicates the width of the feature map.

[0055] First, let's introduce the temperature parameter. Using the Softmax function to... The feature values ​​of each channel are converted into a probability distribution; for each channel in the feature map... Its spatial location ( The probability value on ) is: ; in, Indicates the first The first channel in the The probability values ​​of each spatial location after Softmax normalization; Represents the first in the feature graph The first channel, the first The activation value of each spatial location; Used to control the smoothness of the probability distribution; Represented by natural constant It is an exponential function with base 0. The Softmax operation eliminates the difference in feature value magnitudes between the teacher and student models, focusing instead on the relative relationships of feature distributions.

[0056] Then, KL divergence is used to measure the difference in channel probability distribution between the teacher model and the student model, and the feature-based CWD distillation loss is calculated. ,Right now: ; in, This represents a function that measures the difference in channel probability distributions. The middle layer feature map of the teacher model represents the first... The first channel Location The normalized probability value; The middle layer feature map of the student model represents the first... The first channel Location The normalized probability value. This loss function allows the student model to mimic the distribution of the most salient regional features in each channel (usually corresponding to the target foreground) of the teacher model, while ignoring the interference of background noise.

[0057] Step 4: Calculate the response-based Logits distillation loss function.

[0058] This step employs a response-based distillation strategy. The final prediction output (Logits) of the teacher model's detection head is incorporated. As additional supervisory information, the output of the teacher model is directly used as "soft targets," such as... Figure 12 As shown.

[0059] Let the Logits output of the teacher model be... The Logits output of the student model is The difference between the two is calculated using the mean squared error loss function, resulting in the response-based Logits distillation loss. : ; in, Indicates batch size; Indicates the sample index within the batch; The teacher model represents the first The Logits output for each sample; Represents the student model for the first The Logits output for each sample; This represents the square of the L2 norm.

[0060] Logits distillation can pass the teacher model’s predicted probability distribution for categories (including probability information for incorrect categories, i.e., “dark knowledge”) to the student model.

[0061] Step 5: Construct the total loss function and perform end-to-end training.

[0062] To balance the learning and knowledge transfer processes of the detection task, this step defines a total loss function, introduces a cosine annealing strategy to dynamically adjust the distillation loss weights, and performs end-to-end training on the student model.

[0063] Define the total loss function Including the original detection loss The feature-based CWD distillation loss obtained in step 3 and the response-based Logits distillation loss obtained in step 4 ;in, The original detection loss for the student model DS-YOLO-M consists of classification loss, localization loss, and confidence loss.

[0064] Introducing a cosine annealing strategy to dynamically adjust the weighting coefficients of distillation losses In the early stages of training, the distillation loss is given a larger weight to accelerate knowledge transfer, and the weight is gradually reduced as training progresses; the total loss function expression is: ; in, This represents the balance coefficient for Logits distillation loss, used to adjust the ratio of Logits distillation loss to CWD distillation loss.

[0065] Dynamic weights The formula for how this changes during the training process is: ; in, This indicates the total number of batches used in model training; This represents the batch index of the current training session. This weight adjustment strategy ensures that the model performs optimally in the early stages of training (…). When it is smaller), The model is relatively large, focusing more on the transfer of teacher knowledge and quickly establishing feature extraction capabilities; in the later stages of training ( near hour), As the size of the model gradually decreases, it focuses more on optimizing its own detection task, avoiding the negative impact of overfitting or erroneous information from the teacher model on the final convergence.

[0066] Based on this total loss function, the student model DS-YOLO-M is trained end-to-end iteratively using a stochastic gradient descent optimizer. During training, the parameters of the teacher model YOLO11m remain frozen and act only as a knowledge provider. The loss is distilled using CWD. and Logits distillation loss Deep features and output response knowledge are transferred to the student model; the parameters of the student model DS-YOLO-M are based on the total loss function. By continuously updating through backpropagation and guided by the teacher model, the feature extraction capability and detection accuracy are gradually improved until the model converges, resulting in the final lightweight object detection model DS-YOLO-M (distilled).

[0067] The technical effects of the present invention will be further explained below with reference to simulation experiments.

[0068] 1. Simulation conditions and content.

[0069] Simulation environment: Operating system: Windows 11; CPU: AMD Ryzen 5 5600G with Radeon Graphics @ 3.90 GHz; RAM: 32.0 GB; GPU: GeForce RTX 3090 with 24 GB VRAM; Deep learning framework: PyTorch 2.1; CUDA 11.8 and cuDNN are used to accelerate the GPU.

[0070] Dataset: The dataset used is the data-enhanced NWPU VHR-10 dataset, which contains 8,000 high-resolution remote sensing images covering 10 categories of geospatial objects, including aircraft, ships, storage tanks, baseball fields, tennis courts, basketball courts, athletic fields, ports, bridges, and vehicles.

[0071] Training parameters: Batch size is set to 16, Epochs are set to 50, initial learning rate is set to 0.01, momentum is set to 0.937, weight decay is set to 0.0005, and SGD is used as the optimizer.

[0072] 2. Analysis of simulation results.

[0073] (1) Analysis of the effect of model lightweighting.

[0074] like Figure 13 The scatter plot shown visually illustrates the comparison between the number of parameters and detection accuracy (mAP@0.5) of different models. The student model DS-YOLO-M constructed in this invention has only 2.2M parameters, which is significantly reduced compared to the non-lightweighted DS-YOLO model (3.0M parameters), while maintaining a high detection accuracy (94.3%).

[0075] Figure 14 The comparison between model weight file size and computational cost is shown. The DS-YOLO-M (distilled) model has a weight file size of only 4.3MB and a computation cost of 6.5 GFLOPs. Compared to the original YOLOv8n model (weights 6.53MB, computation cost 8.9 GFLOPs), the storage footprint of this invention is reduced by 34.2%, and the computation cost is reduced by 27.0%, significantly reducing resource consumption and making it very suitable for deployment on resource-constrained edge devices (such as embedded boards for drones).

[0076] (2) Analysis of detection accuracy and distillation effect.

[0077] Figure 15 This demonstrates the improvement in accuracy achieved by the distillation strategy. Using the DS-YOLO-M model alone, the mAP@0.5 is 94.3%. After introducing CWD and Logits double distillation, the DS-YOLO-M (distilled) model significantly improves its mAP@0.5 to 99.4% on the NWPU VHR-10 dataset, exhibiting excellent detection accuracy across all categories (e.g., 99.5% for aircraft, 99.1% for ships, and 99.5% for storage tanks). like Figure 16 The performance metric curves shown compare the convergence of each model during training. The DS-YOLO-M (distilled) model outperforms YOLOv8n, YOLO11n, and the undistilled DS-YOLO-M model in all metrics, including precision, recall, mAP@0.5, and mAP@0.5-0.95. In particular, after distillation, the mAP@0.5 value improved by 5.1% compared to before distillation, demonstrating the effectiveness of the knowledge distillation strategy.

[0078] like Figure 17 As shown, the confusion matrix of the DS-YOLO-M (distilled) model shows that the model's detection accuracy for each category approaches 1.00, with an extremely low false detection rate and a background false detection rate of almost 0.

[0079] (3) Feature extraction capability analysis.

[0080] The distillation effect was verified using feature map visualization technology. (Comparison) Figure 18 (DS-YOLO-M model feature map) and Figure 19 Visualization results of (DS-YOLO-M (distilled) model feature map). Figure 18 The feature map is relatively blurry, texture details are unclear, and key features are difficult to identify; while Figure 19 In this study, the model feature map after knowledge distillation clearly displays key features in the image, such as the edges and textures of the target, while retaining sufficient detail. This demonstrates that the distillation strategy of this invention effectively enhances the feature extraction capability of the lightweight model, enabling it to learn deep abstract semantic information from the teacher model.

[0081] Conclusion: The lightweight target detection method based on model compression and knowledge distillation proposed in this invention successfully solves the resource bottleneck problem of traditional YOLO models when deployed on edge devices by constructing a DS-YOLO-M model and combining it with a multi-dimensional knowledge distillation strategy. Simulation results show that this method achieves extremely high detection accuracy (99.4% mAP@0.5) while significantly reducing the number of model parameters (2.2M) and computational cost (6.5GFLOPs). It achieves a significantly better balance between detection speed, accuracy, and resource consumption than existing technologies, providing efficient algorithmic support for the construction of embedded AI inference platforms. The above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A lightweight target detection method based on model compression and knowledge distillation, characterized in that, include: The image to be detected is acquired and input into a lightweight object detection model to obtain the object detection result; wherein, the construction method of the lightweight object detection model includes: The backbone network of the YOLO model is reconstructed by introducing depthwise separable convolutions and inverted residual structures from MobileNet. Depthwise separable convolutions replace standard convolutions, decomposing the standard convolution operation into depthwise convolutions and pointwise convolutions. An inverted residual structure is introduced into the backbone network to replace the original residual structure. An SE attention module is embedded inside the Bottleneck module of the backbone network, and a Hard-Swish activation function replaces some of the original ReLU activation functions in the backbone network. A C2fCIB module is fused into the neck network; the C2fCIB module is formed by replacing the original Bottleneck module in the C2f structure of the neck network with a CIB module, ultimately constructing the student model. A dual knowledge distillation strategy based on features and responses was adopted. This strategy constructs a teacher-student distillation framework. During the distillation process, a feature-based channel knowledge distillation loss is used to guide the student model to imitate the intermediate layer feature representation of the teacher model by minimizing the KL divergence between the channel probability maps of the teacher and student models. At the same time, a response-based Logits distillation loss is combined, and the final prediction output of the teacher model is used as a soft label to guide the training of the student model. Combined with the detection loss of the student model, a total loss function is constructed, and the student model is trained end-to-end iteratively to obtain a lightweight object detection model.

2. The lightweight target detection method based on model compression and knowledge distillation according to claim 1, characterized in that, A CBHw module is constructed in the backbone network, which consists of a concatenated convolutional layer Conv, a batch normalization layer BatchNorm, and a Hard-Swish activation function, and is used as a feature extraction unit in the backbone network. The improved Bottleneck module is obtained by replacing the ReLU activation function in the Bottleneck module with the Hard-Swish activation function and incorporating it into the SE attention module.

3. The lightweight target detection method based on model compression and knowledge distillation according to claim 1, characterized in that, Design an adaptive layer that uses convolution operations to map the number of feature channels in the student model to be consistent with the number of feature channels in the teacher model. For the intermediate feature layers of the teacher and student models, each channel of the output intermediate feature map is subjected to Softmax normalization to obtain a probability distribution containing spatial location information; then, KL divergence is used to measure the difference between the teacher and student models in the channel probability distribution, and the CWD distillation loss is calculated.

4. The lightweight target detection method based on model compression and knowledge distillation according to claim 1, characterized in that, Introducing temperature parameters Using the Softmax function to extract the intermediate layer feature map... The feature values ​​of each channel are converted into a probability distribution; for each channel in the intermediate layer feature map... Its spatial location The probability value is: ； in, Indicates the first The first channel in the The probability values ​​of each spatial location after Softmax normalization; Represents the first in the feature graph The first channel, the first The activation value of each spatial location; Indicates the feature map height. Indicates the width of the feature map; KL divergence is used to measure the difference in channel probability distribution between the teacher model and the student model, and feature-based CWD distillation loss is calculated. ,Right now: ； in, Indicates the number of channels. This represents a function that measures the difference in channel probability distributions. The middle layer feature map of the teacher model represents the first... The first channel Location The normalized probability value; The middle layer feature map of the student model represents the first... The first channel Location The normalized probability value.

5. The lightweight target detection method based on model compression and knowledge distillation according to claim 1, characterized in that, The final prediction output of the teacher model is introduced as additional supervision information, and the Logits output of the teacher model is used as a soft label. The difference between the student model output and the teacher model Logits output is calculated using the mean squared error loss function to obtain the response-based Logits distillation loss.

6. The lightweight target detection method based on model compression and knowledge distillation according to claim 1, characterized in that, The response-based Logits distillation loss is represented as follows: ； in, Indicates batch size; Indicates the sample index within the batch; The teacher model represents the first The Logits output for each sample; Represents the student model for the first The Logits output for each sample; This represents the square of the L2 norm.

7. The lightweight target detection method based on model compression and knowledge distillation according to claim 1, characterized in that, Define the total loss function Including the original detection loss Feature-based CWD distillation loss and response-based Logits distillation loss ;in, The original detection loss for the student model consists of classification loss, localization loss, and confidence loss. Introducing a cosine annealing strategy to dynamically adjust the weighting coefficients of distillation losses The total loss function is expressed as follows: ； in, This represents the balance coefficient for Logits distillation loss, used to adjust the ratio of Logits distillation loss to CWD distillation loss.

8. The lightweight target detection method based on model compression and knowledge distillation according to claim 1, characterized in that, Weighting coefficient The formula for how this changes during the training process is: ； in, This indicates the total number of batches used in model training; This indicates the batch index of the current training session.

9. A terminal device, comprising a processor, a memory, and a computer program stored in the memory; characterized in that, When the processor executes the computer program, it implements the lightweight target detection method based on model compression and knowledge distillation as described in any one of claims 1-8.

10. A computer-readable storage medium storing a computer program; characterized in that, When the computer program is executed by a processor, it implements the lightweight target detection method based on model compression and knowledge distillation as described in any one of claims 1-8.

Images