Target detection model training method based on dynamic pruning and double-teacher knowledge distillation

Abstract

The application provides a target detection model training method based on dynamic pruning and double-teacher knowledge distillation, comprising: obtaining a sample set; based on a target detection model, a lightweight student model is constructed by performing dynamic pruning processing based on channel contribution degree evaluation on the target detection model; two pre-trained target detection models with complementary structures are selected as a first teacher model and a second teacher model respectively; a training framework is constructed with the student model, the first teacher model and the second teacher model, and the student model is trained by using the sample set under the training framework to obtain a final target detection model with lightweight characteristics. By fusing dynamic pruning and double-teacher knowledge distillation, the detection accuracy is effectively guaranteed and improved while the model is lightweight, especially the recognition ability of small targets is enhanced; not only the parameter quantity and the calculation cost can be greatly reduced, but also the accuracy can approach or even surpass the original model, and the practicability of the model in a resource-limited environment is significantly improved.

Description

Technical Field

[0001] This invention relates to the fields of computer vision and deep learning technology, and in particular to a method for training a target detection model based on dynamic pruning and dual-teacher knowledge distillation. Background Technology

[0002] Object detection, a core task of computer vision, is widely used in fields such as autonomous driving, video surveillance, and medical image analysis. Single-stage detectors, represented by the YOLO series, are favored for their high detection speed; however, in pursuit of high accuracy, their model depth and parameter count continuously increase, leading to a sharp rise in computational and storage costs, making them difficult to deploy on resource-constrained devices such as drones and mobile devices. Therefore, model compression technology has become a research hotspot. Current lightweighting mainly relies on model pruning and knowledge distillation. Pruning techniques reduce model size and computation by removing redundant parameters from the network; however, traditional static pruning methods often rely on manually designed criteria, which can easily cause irreversible accuracy loss during compression, especially severely damaging the ability to recognize fine features such as small targets. Knowledge distillation restores accuracy by having a lightweight "student" model mimic the knowledge of a high-performance "teacher" model; however, a single teacher model often only provides a limited knowledge perspective and cannot comprehensively guide student models to learn the advantageous features of different architectures or task preferences, thus limiting further performance improvements of student models. Summary of the Invention

[0003] In view of the above-mentioned deficiencies of the prior art, the present invention provides a target detection model training method based on dynamic pruning and dual-teacher knowledge distillation to solve the technical problems of serious accuracy loss and insufficient small target detection capability of existing lightweight model techniques.

[0004] To achieve the above and other related objectives, this invention provides a method for training a target detection model based on dynamic pruning and dual-teacher knowledge distillation, comprising: acquiring a sample set; constructing a lightweight student model by performing dynamic pruning on the target detection model based on channel contribution evaluation; selecting two structurally complementary pre-trained target detection models as the first teacher model and the second teacher model, respectively; constructing a training framework using the student model, the first teacher model, and the second teacher model, and training the student model using the sample set within the training framework to obtain a final target detection model with lightweight characteristics; wherein, the total training loss function is composed of the detection loss based on sample labeling and the distillation loss based on the difference between the outputs of the student model and the dual-teacher model.

[0005] In one embodiment of the present invention, a lightweight student model is constructed based on an object detection model by performing dynamic pruning based on channel contribution evaluation. The process includes: inserting a trainable mask layer after one or more convolutional layers of the object detection model; obtaining the contribution of each feature channel based on the gradient of the total loss function with respect to the mask layer parameters during training; and dynamically suppressing feature channels evaluated as redundant and retaining key feature channels according to the contribution, thereby constructing the student model.

[0006] In one embodiment of the present invention, the feature channels evaluated as redundant are dynamically suppressed according to the contribution degree, including: applying a distribution polarization constraint to force the contribution degree of each feature channel to differentiate towards the high and low ends of the value; generating a dynamic pruning ratio that increases nonlinearly with the training iteration rounds based on an S-shaped curve; determining the redundant channels to be suppressed in each round of training according to the polarized contribution degree ranking and the dynamic pruning ratio, and suppressing the determined redundant channels.

[0007] In one embodiment of the present invention, dynamically suppressing feature channels evaluated as redundant includes: for feature channels determined to be redundant, applying a coefficient that decays with training rounds to perform soft suppression; when the weight gradient of the suppressed channel exceeds a preset threshold, amplifying its gradient to restore its update.

[0008] In one embodiment of the present invention, the student model is a YOLO series object detection model after dynamic pruning; the first teacher model is a YOLOv11 model; and the second teacher model is a YOLO-X model.

[0009] In one embodiment of the present invention, the formula for calculating the total loss function is as follows: , In the formula, L total L det L distill Let be the total loss function, the detection loss, and the distillation loss, respectively, and λ be the coefficient that balances the contributions of the detection loss and the distillation loss.

[0010] In one embodiment of the present invention, the formula for calculating the distillation loss is: , In the formula, L logit The output layer distillation loss is calculated based on the difference in probability distribution between the student model and the first and second teacher models in the predicted output layer; L feat The feature layer distillation loss is calculated based on the difference between the student model and the first and second teacher models in the multi-scale feature maps output by the feature pyramid network; L locThe distillation loss at the localization level is calculated based on the intersection-union ratio (IUU) between the bounding boxes predicted by the student model and the dual-teacher model; λ1 and λ2 are weighting coefficients used to balance the contribution of distillation loss at each level.

[0011] In one embodiment of the present invention, the formula for calculating the distillation loss of the output layer is: , In the formula, p s p t1 p t2 These are the probability distributions of the student model, the first teacher model, and the second teacher model in the prediction output layer, respectively; D(||) represents the divergence function used to measure the difference between the two probability distributions; α and β are weighting coefficients that satisfy α+β=1 and are dynamically adjusted with each training round during model training. The formula for calculating the distillation loss of the feature layer is as follows: , In the formula, These are the output features of the student model, the first teacher model, and the second teacher model at the l-th layer of the feature pyramid network, respectively. The formula for calculating the distillation loss of the positioning layer is: , In the formula, b s b is the bounding box predicted by the student model. t This is a weighted average of the bounding boxes predicted by the dual-classroom model.

[0012] In one embodiment of the present invention, the training adopts an alternating separation optimization strategy, specifically: in a preset portion of the training rounds, the mask parameters in the dynamic pruning process are fixed, and only the network weights of the student model are updated; in the remaining training rounds, the mask parameters are updated, and the network weights of the student model are updated or fixed simultaneously.

[0013] In one embodiment of the present invention, during the training process, the weights of the student model are updated using an exponential moving average, and after training is completed, the resulting exponential moving average weights are used as the weights of the final target detection model.

[0014] The beneficial effects of this invention are as follows: This invention proposes a target detection model training method based on dynamic pruning and dual-teacher knowledge distillation. By integrating dynamic pruning and dual-teacher knowledge distillation, it effectively ensures and improves detection accuracy while making the model lightweight, especially enhancing the ability to identify small targets. First, a dynamic pruning mechanism based on channel contribution evaluation is introduced, which can adaptively identify and remove redundant feature channels in the network, generating a highly compressed student model. Second, a knowledge distillation framework composed of two complementary teacher models is constructed, enabling the student model to simultaneously learn global and detailed features from teachers with different architectures, achieving more comprehensive and robust knowledge transfer. This method, through joint optimization of the pruning and distillation processes, enables the lightweight model to not only significantly reduce the number of parameters and computational overhead, but also approach or even surpass the original model in accuracy, significantly improving the practicality and deployment value of the model in resource-constrained environments. Attached Figure Description

[0015] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below. The accompanying drawings are incorporated in and constitute a part of this specification, illustrating embodiments consistent with this application, and are used together with the description to explain the principles of this application. Obviously, the drawings described below are merely some embodiments of the present invention, and those skilled in the art can obtain other drawings based on these drawings without creative effort.

[0016] Figure 1 A flowchart of a target detection model training method based on dynamic pruning and dual-teacher knowledge distillation provided in an embodiment of the present invention; Figure 2 A detailed flowchart of step S102 provided in an embodiment of the present invention; Figure 3 This is a flowchart illustrating a specific implementation of step S203 according to an embodiment of the present invention; Figure 4 This is a flowchart of the soft suppression and gradient wake-up steps in dynamic suppression provided in an embodiment of the present invention; Figure 5 This is a schematic diagram of the structure of an adaptive dynamic pruning module based on sensitivity feedback provided in an embodiment of the present invention; Figure 6 This is a schematic diagram of the structure of a dual-teacher knowledge distillation module provided in an embodiment of the present invention. Detailed Implementation

[0017] The following specific embodiments illustrate the implementation of the present invention. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. It should be noted that, unless otherwise specified, the following embodiments and features can be combined with each other. In addition to the specific methods, equipment, and materials used in the embodiments, based on the knowledge of the prior art and the description of the present invention by those skilled in the art, any prior art methods, equipment, and materials similar to or equivalent to the methods, equipment, and materials in the embodiments of the present invention can be used to implement the present invention.

[0018] It should be understood that the terminology used in the embodiments of this invention is for describing specific implementations and not for limiting the scope of protection of this invention. Unless otherwise defined, all technical and scientific terms used in this invention have the same meaning as commonly understood by one of ordinary skill in the art.

[0019] In the following description, numerous details are explored to provide a more thorough explanation of embodiments of the invention. However, it will be apparent to those skilled in the art that embodiments of the invention may be practiced without these specific details. In some embodiments, well-known structures and devices are shown in block diagram form rather than in detail to avoid obscuring embodiments of the invention.

[0020] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functions, and operations that may be implemented in the methods and computer program products according to various embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0021] Please see Figure 1 , Figure 1 An embodiment of the present invention provides a target detection model training method based on dynamic pruning and dual-teacher knowledge distillation, including steps S101 to S104.

[0022] Step S101: Obtain the sample set. The sample set is the foundation for model training and typically contains a large number of images with precise annotations (such as category labels and bounding boxes). This step involves standardizing the original images through preprocessing, such as scaling to a uniform size and normalizing pixel values, and pre-setting initial anchor boxes that match the data distribution, providing standardized, high-quality input for subsequent model training. This step ensures the validity and consistency of the training data, which is a prerequisite for the model to converge stably and learn effective features, and is crucial to the generalization performance of the final model.

[0023] Step S102: Based on the object detection model, a lightweight student model is constructed by performing dynamic pruning based on channel contribution evaluation. This step is the core of model lightweighting. By embedding a trainable mask layer after the convolutional layer of the object detection model (such as YOLO), and dynamically evaluating the importance of each feature channel based on the feedback of training gradients, intelligent identification and soft suppression of redundant channels are achieved. Compared with static pruning, this method can adaptively adjust the pruning strategy according to the training state, significantly reducing the model's parameters and computational load while ensuring the model's expressive power, laying a structural foundation for subsequent distillation recovery and accuracy improvement.

[0024] Please see Figure 2 In a specific embodiment of the present invention, step S102 includes steps S201 to S203.

[0025] Step S201: Insert a trainable mask layer after one or more convolutional layers of the object detection model. The object detection model in this step can be, for example, a YOLO series object detection model. Correspondingly, a trainable mask layer needs to be inserted after each convolutional layer of the backbone network and the feature pyramid network, and the mask parameters are initialized to an all-one matrix.

[0026] After inserting the mask layer, during forward propagation, the convolutional output is weighted and modulated using channel-by-channel mask parameters to achieve dynamically controllable feature channel selection. The forward propagation formula for the reconstructed network is: O = m⊙(W*I+b); In the formula, m represents the trainable mask parameter, W is the convolutional layer weight, b is the offset parameter, I and O represent the input and output feature maps, respectively, and ⊙ represents element-wise multiplication.

[0027] Step S202: Based on the gradient of the total loss function with respect to the mask layer parameters during training, the contribution of each feature channel is obtained. In this step, a channel importance evaluation mechanism based on gradient backpropagation is established, and the channel contribution Ω is defined. k The global loss function L totalThe statistical mean of the gradient magnitudes of the mask parameter m is used to quantify the contribution of the k-th feature channel to model convergence. , In the formula, L total The total loss function for model training includes detection loss and distillation loss; N is the number of samples in a batch, m k This is the mask parameter corresponding to the k-th channel. This index Ω k The larger the value, the more critical the information carried by the corresponding channel, and the more it should be retained in the screening process.

[0028] Step S203: Based on the contribution, dynamically suppress the feature channels that are evaluated as redundant and retain the key feature channels to construct the student model.

[0029] Please see Figure 3 In a specific embodiment of the present invention, feature channels evaluated as redundant are dynamically suppressed according to their contribution, including steps S301 to S303.

[0030] S301. By imposing a distribution polarization constraint, the contribution of each feature channel is forced to differentiate towards both high and low values. To address the problem of overly smooth channel contribution distribution leading to blurred screening boundaries, a distribution polarization loss function L is constructed. pol This loss function aims to force the channel contribution values ​​to diverge towards the "extremely important" and "extremely unimportant" extremes, automatically increasing the feature distance. The formula for the distribution polarization loss function is as follows: , In the formula, The L1 norm of the contribution vector is represented. λ1 and λ2 represent the average contribution of all channels in the current layer. Note that these are the weighting coefficients used in this formula, and they differ from the weighting coefficients mentioned below.

[0031] Step S302: Based on the S-curve, generate a dynamic pruning ratio that increases non-linearly with the number of training iterations. To ensure that information in redundant channels is smoothly concentrated in important channels during the pruning process, this invention abandons the traditional linear pruning strategy and constructs a non-linear progressive suppression scheduling based on the S-curve. The dynamic pruning ratio R(t) that changes with the iteration number t is defined, and its calculation formula is as follows: , , In the formula, a is the initial attenuation coefficient, b controls the steepness of the S-shaped curve, t is the current iteration number, σ is the target pruning rate, and R(t) determines the proportion of low-contribution channels to be pruned in the current round. As the training progresses, a(t) gradually approaches 0, while R(t) gradually approaches the target pruning rate σ. Introducing the S-shaped attenuation function not only retains the potential feature expression ability and alleviates the problem of model accuracy degradation but also enables smooth control and avoids model performance oscillations caused by sudden changes in the pruning rate.

[0032] Step S303: Determine the redundant channels to be suppressed in each round of training according to the sorted contribution degrees after polarization and the dynamic pruning ratio, and suppress the determined redundant channels.

[0033] Please refer to Figure 4 , in a specific embodiment of the present invention, dynamically suppressing the feature channels evaluated as redundant includes Step S401 and Step S402.

[0034] Step S401: For the feature channels determined to be redundant, perform soft suppression by applying a coefficient that decays with the training rounds. This step can be expressed by the formula: , In the formula, Ω th is the current truncation threshold, which is calculated based on the sorted contribution degrees and the dynamic pruning ratio R(t). In the traditional pruning process, in the otherwise case, it is directly set to 0 instead of a(t); in the present invention, for redundant channels, it is not directly set to 0 but assigned a coefficient a(t) that decays with time. Applying weight decay operation to redundant channels instead of directly setting to zero retains the potential feature expression ability and avoids irreversible information loss caused by direct setting to zero. At the same time, for redundant channels, their weights are actually multiplicatively suppressed by a(t), achieving a smooth transition of the feature extraction ability and retaining the search space of the model in the early stage of training.

[0035] Step S402: When the weight gradient of the suppressed channel exceeds the preset threshold, amplify its gradient to make it resume updating. To prevent mispruning, this step also introduces a gradient monitoring mechanism. When the contribution degree Ω i <0.5, that is, when the filter is regarded as being in the pruning state, if the gradient of its weight is still large, it means that this feature tries to learn. If |g i |<T (T is the threshold, for example, T = 1×e -6 ), then amplify its gradient through the following formula to make it "resurrect" and participate in the update: , In the formula, is the amplified gradient, The sigmoid normalization result of the sensitivity score is used in this step to correct sensitivity-based misjudgments using gradient information, forcing them to re-participate in training and preventing excessive pruning from causing a decrease in model accuracy.

[0036] Figure 5 The diagram illustrates the structure of the adaptive dynamic pruning module based on sensitivity feedback, which corresponds to the entire process of step S102 described above.

[0037] Step S103: Select two pre-trained target detection models with complementary structures, and use them as the first teacher model and the second teacher model, respectively.

[0038] Please see Figure 6 In a specific embodiment of the present invention, the first teacher model is selected as the YOLOv11 model and the second teacher model is the YOLO-X model. YOLOv11 adopts the classic PAFPN three-level structure, which has excellent accuracy-speed balance in conventional tasks. YOLO-X uses the BiFPN four-level structure and introduces the P6 layer to expand the receptive field, which performs well in small target detection. The two models have very different architectures and complement each other to achieve both speed and accuracy.

[0039] Step S104: Construct a training framework using the student model, the first teacher model, and the second teacher model. Within this framework, train the student model using the sample set to obtain a final object detection model that is lightweight. Figure 6 The DTDP-Net (Dual-Teacher Dynamic Pruning Network) in this context comprises a detection loss based on sample annotations and a distillation loss based on the difference between the student model and the dual-teacher models' outputs. This step constructs a collaboratively optimized training framework by associating a lightweight student model with two pre-trained, structurally complementary teacher models (such as YOLOv11 and YOLO-X). This allows the student model to receive supervision from real-labeled data and multi-layered knowledge guidance from both teachers during training. The framework is driven by a joint loss function that integrates task-driven detection loss and imitation-driven distillation loss. This ensures that while compressing the model structure, it fully absorbs and integrates the strengths of the teachers, ultimately training a lightweight yet highly accurate object detection model.

[0040] In a specific embodiment of the present invention, the formula for calculating the total loss function is as follows: , In the formula, L total L det L distillHere, λ represents the total loss function, detection loss, and distillation loss, respectively. λ is the coefficient that balances the contributions of detection loss and distillation loss. In the early stages of training, λ can take a smaller value, such as [0.1, 0.5]. In the later stages of training, this coefficient can be gradually increased to [2, 3].

[0041] In a specific embodiment of the present invention, the formula for calculating distillation loss is as follows: , In the formula, L logit The output layer distillation loss is calculated based on the difference in probability distribution between the student model, the first teacher model, and the second teacher model in the predicted output layer; L feat The feature layer distillation loss is calculated based on the differences in multi-scale feature maps output by the student model, the first teacher model, and the second teacher model in the feature pyramid network; L loc The localization layer distillation loss is calculated based on the intersection-union ratio (IU) between the bounding boxes predicted by the student model and the dual-teacher model. λ1 and λ2 are weighting coefficients used to balance the contribution of distillation loss at each level. Their initial values ​​are both set to 0.5, and can be adjusted in the range of [0.2, 5.0] according to the performance of the validation set. They can be appropriately increased to address deficiencies in detailed features or localization error.

[0042] The distillation loss consists of three parts: an output layer, a feature layer, and a localization layer, achieving multi-level and comprehensive knowledge transfer. The output layer loss allows the student model to imitate the teacher's final judgment; the feature layer loss forces the student model to approximate the teacher's intermediate feature representations, learning its powerful feature extraction capabilities; and the localization layer loss focuses on improving the regression accuracy of the bounding boxes. The weighting coefficients λ1 and λ2 can be flexibly adjusted to adjust the focus on different levels of knowledge, allowing the student model to learn with emphasis based on its own weaknesses, thus achieving a balanced and significant improvement in overall accuracy, feature representation, and localization accuracy.

[0043] In a specific embodiment of the present invention, the formula for calculating the output layer distillation loss is as follows: , In the formula, p s p t1 p t2Let represent the probability distributions of the student model, the first teacher model, and the second teacher model in the predicted output layer, respectively; D(|| ) represents the divergence function used to measure the difference between the two probability distributions, such as KL divergence; α and β are weighting coefficients that satisfy α+β=1 and are dynamically adjusted with each training epoch during model training. The adjustment strategy could be, for example, decreasing α=0.6-0.01×epoch every 10 epochs. The output layer distillation loss employs a dynamically weighted dual-teacher fusion strategy. Two adjustable weighting coefficients, α and β, control the degree to which the student model relies on the output knowledge of the two teachers during the learning process. In the early stages of training, more emphasis can be placed on the teacher providing globally stable supervision, while in later stages, the learning of knowledge from the teacher specializing in details (such as small objectives) is gradually enhanced. This dynamic adjustment mechanism makes the student model's learning process more adaptive, enabling a smoother and more efficient fusion of the complementary advantages of the two teachers, avoiding overfitting to the knowledge of a single teacher.

[0044] In a specific embodiment of the present invention, the formula for calculating the feature layer distillation loss is as follows: , In the formula, These represent the output features of the student model, the first teacher model, and the second teacher model at layer l of the Feature Pyramid Network (FPN). The feature layer distillation loss is achieved by calculating the L2 distance between the output feature maps of the student and dual-teacher models at each layer of the FPN. This method forces the student model to simultaneously approximate the feature representations of both teachers at multiple scales, thereby learning the teacher model's powerful multi-scale feature fusion and representation capabilities. This is crucial for object detection tasks, especially for targets with large scale variations, effectively improving the model's robustness across different scales.

[0045] In a specific embodiment of the present invention, the formula for calculating the distillation loss of the positioning layer is as follows: , In the formula, b s b is the bounding box predicted by the student model. t This is a weighted average of the bounding boxes predicted by the dual-teacher model. The localization layer distillation loss, guided by the intersection-over-union (IoU) ratio, aligns the bounding boxes predicted by the student model with the weighted average of the predictions from the dual-teacher model. IoU, as a direct indicator of box overlap, more accurately guides improvements in localization accuracy. This loss function encourages the student model to not only learn classification knowledge but also refine its localization capabilities, particularly improving the localization of small or overlapping targets, making it a key factor in enhancing the overall performance of the detection model.

[0046] In a specific embodiment of the present invention, the training adopts an alternating separation optimization strategy, specifically: in a preset portion of the training rounds, the mask parameters in the dynamic pruning process are fixed, and only the network weights of the student model are updated; in the remaining training rounds, the mask parameters are updated, and the network weights of the student model are updated or fixed at the same time.

[0047] For example, in the first 10 training rounds, only the network weights are updated, and the mask parameter is fixed at 1 to ensure initial convergence of the model. From the 11th round onwards, mask parameter updates are introduced. From the 11th round onwards, network weights are updated in odd-numbered rounds, with a learning rate lr = 1 × e^(-1 / 2). -3 Using the total loss function L total Calculate gradient g i And update W; update the mask m in even-numbered rounds, with the learning rate lr = 5 × e -4 And add distributed polarization loss L pol That is, the optimization objective includes minimizing L pol This approach aims to polarize the mask distribution. The alternating separation optimization strategy separates the updates of network weights and pruning mask parameters, performing these updates alternately in different rounds. This strategy effectively avoids potential conflicts and oscillations that may occur when simultaneously optimizing two mutually influential but objectively different parameter sets. By first allowing the model to initially converge under a fixed structure, and then gradually introducing and optimizing the pruning mask, the stability of the pruning decisions and the overall training balance of the model are ensured, which is a key guarantee for the successful implementation of dynamic pruning.

[0048] In a specific embodiment of the present invention, during training, the weights of the student model are updated using an exponential moving average, and after training is completed, the resulting exponential moving average weights are used as the weights of the final object detection model. In this embodiment, an exponential moving average (EMA) is introduced to construct a stability smoothing mechanism. By maintaining the moving average of the weights, a certain proportion of historical weight information is retained while considering the new weights obtained in the current iteration, thereby smoothing the weight updates after pruning and suppressing training oscillations. The calculation formula is as follows: θ' ema =0.9θ ema +0.1θ new , In the formula, θ ema This represents historical weight information, with an initial value of 1, θ new θ' represents the new weights obtained during the current training. emaThis represents the updated weight information. During updates, it's not necessary to update the EMA parameters with every iteration; for example, you can choose to update them every 1000 iterations. An Exponential Moving Average (EMA) mechanism is introduced during training to update a historical moving average by smoothing the weights of the student model. This mechanism effectively suppresses drastic weight fluctuations caused by dynamic pruning and alternating optimization during training, smoothing the optimization trajectory. Finally, using EMA weights instead of the instantaneous weights from the last round as model parameters significantly improves model stability and generalization performance, reduces the risk of overfitting, and results in a more robust and reliable final object detection model.

[0049] It should be noted that the steps of the various methods described above are only for clarity. In practice, they can be combined into one step or some steps can be split into multiple steps. As long as they contain the same logical relationship, they are all within the scope of protection of this application. Adding insignificant modifications or introducing insignificant designs to the algorithm or process, but without changing the core design of the algorithm and process, are also within the scope of protection of this patent.

[0050] The beneficial effects of this invention lie in its innovative deep integration of dynamic pruning and dual-teacher knowledge distillation, proposing a complete lightweight, high-precision target detection model training scheme. This method achieves intelligent compression of model parameters through gradient-feedback-based adaptive dynamic pruning, overcoming the problem of significant accuracy loss in traditional pruning. Simultaneously, by constructing a structurally complementary dual-teacher distillation framework, multi-level and comprehensive knowledge transfer is achieved, enabling the student model to fully absorb the advantages of different teachers, significantly improving the model's overall detection capability, especially in small target detection and localization accuracy. Furthermore, strategies such as alternating separation training and EMA weight smoothing ensure the stability and efficiency of the entire training process. This invention effectively resolves the contradiction between lightweight model design and accuracy preservation, providing a practical technical path for deploying high-performance target detection models on resource-constrained edge devices.

[0051] The above embodiments are merely illustrative of the principles and effects of the present invention and are not intended to limit the invention. Any person skilled in the art can modify or alter the above embodiments without departing from the spirit and scope of the present invention. Therefore, all equivalent modifications or alterations made by those skilled in the art without departing from the spirit and technical concept disclosed in the present invention should still be covered by the claims of the present invention.

Claims

1. A training method for a target detection model based on dynamic pruning and dual-teacher knowledge distillation, characterized in that, include: Obtain the sample set; A lightweight student model is constructed by performing dynamic pruning based on channel contribution evaluation on the target detection model. Two pre-trained target detection models with complementary structures were selected as the first teacher model and the second teacher model, respectively. A training framework is constructed using the student model, the first teacher model, and the second teacher model. Under the training framework, the student model is trained using the sample set to obtain a final target detection model that is lightweight. The total training loss function consists of the detection loss based on sample labeling and the distillation loss based on the difference between the outputs of the student model and the dual-teacher model.

2. The target detection model training method based on dynamic pruning and dual-teacher knowledge distillation according to claim 1, characterized in that, Based on the object detection model, a lightweight student model is constructed by performing dynamic pruning based on channel contribution evaluation, including: A trainable mask layer is inserted after one or more convolutional layers of the object detection model; The contribution of each feature channel is obtained based on the gradient of the total loss function with respect to the mask layer parameters during training. Based on the contribution level, feature channels evaluated as redundant are dynamically suppressed, while key feature channels are retained to construct the student model.

3. The target detection model training method based on dynamic pruning and dual-teacher knowledge distillation according to claim 2, characterized in that, Based on the contribution level, feature channels evaluated as redundant are dynamically suppressed, including: By imposing distributed polarization constraints, the contribution of each feature channel is forced to differentiate towards both high and low values; Based on the S-curve, a dynamic pruning ratio is generated that increases non-linearly with the number of training iterations. Based on the polarized contribution ranking and the dynamic pruning ratio, the redundant channels that need to be suppressed in each round of training are determined, and the determined redundant channels are suppressed.

4. The target detection model training method based on dynamic pruning and dual-teacher knowledge distillation according to claim 2, characterized in that, Dynamically suppress feature channels that are evaluated as redundant, including: For feature channels identified as redundant, soft suppression is applied by applying a coefficient that decays with each training epoch. When the weight gradient of a suppressed channel exceeds a preset threshold, its gradient is amplified to enable it to resume updating.

5. The target detection model training method based on dynamic pruning and dual-teacher knowledge distillation according to claim 1, characterized in that, The student model is a YOLO series object detection model that has undergone dynamic pruning. The first teacher model is a YOLOv11 model; The second teacher model is the YOLO-X model.

6. The target detection model training method based on dynamic pruning and dual-teacher knowledge distillation according to claim 1, characterized in that, The formula for calculating the total loss function is as follows: ， In the formula, L total L det L distill Let be the total loss function, the detection loss, and the distillation loss, respectively, and λ be the coefficient that balances the contributions of the detection loss and the distillation loss.

7. The target detection model training method based on dynamic pruning and dual-teacher knowledge distillation according to claim 6, characterized in that, The formula for calculating the distillation loss is as follows: ， In the formula, L logit The output layer distillation loss is calculated based on the difference in probability distribution between the student model and the first and second teacher models in the predicted output layer; L feat The feature layer distillation loss is calculated based on the difference between the student model and the first and second teacher models in the multi-scale feature maps output by the feature pyramid network; L loc The distillation loss at the localization level is calculated based on the intersection-union ratio (IUU) between the bounding boxes predicted by the student model and the dual-teacher model; λ1 and λ2 are weighting coefficients used to balance the contribution of distillation loss at each level.

8. The target detection model training method based on dynamic pruning and dual-teacher knowledge distillation according to claim 7, characterized in that, The formula for calculating the distillation loss of the output layer is as follows: ， In the formula, p s p t1 p t2 These are the probability distributions of the student model, the first teacher model, and the second teacher model in the prediction output layer, respectively; D(||) represents the divergence function used to measure the difference between the two probability distributions; α and β are weighting coefficients that satisfy α+β=1 and are dynamically adjusted with each training round during model training. The formula for calculating the distillation loss of the feature layer is as follows: ， In the formula, These are the output features of the student model, the first teacher model, and the second teacher model at the l-th layer of the feature pyramid network, respectively. The formula for calculating the distillation loss of the positioning layer is: ， In the formula, b s b is the bounding box predicted by the student model. t This is a weighted average of the bounding boxes predicted by the dual-classroom model.

9. The target detection model training method based on dynamic pruning and dual-teacher knowledge distillation according to claim 1, characterized in that, The training employs an alternating separation optimization strategy, specifically: In a preset portion of the training rounds, the mask parameters in the dynamic pruning process are fixed, and only the network weights of the student model are updated. In the remaining training rounds, the mask parameters are updated, and the network weights of the student model are updated or fixed simultaneously.

10. The target detection model training method based on dynamic pruning and dual-teacher knowledge distillation according to claim 1, characterized in that, During training, the weights of the student model are updated using an exponential moving average, and after training is completed, the resulting exponential moving average weights are used as the weights of the final object detection model.

Images