A knowledge distillation method based on decoupled features and adversarial features

Abstract

The present invention provides a knowledge distillation method based on decoupling features and adversarial features, which belongs to the field of target detection technology and comprises the following steps: firstly, constructing a teacher network and a student network, and then training the teacher network to meet the set requirements; then using the trained teacher network in combination with a decoupling feature module and an adversarial feature module to make the student network perform first-stage learning, so that the backbone network of the student network meets the set requirements; then using the teacher network and a candidate frame feature learning module to perform second-stage learning on the region proposal network of the student network; repeatedly iterating the two-stage learning, so that the student network's detection capability of the target meets the set requirements; the method provided by the present invention enables the student network to achieve high-speed detection of defects while also ensuring high-precision detection.

Description

Technical Field

[0001] The present invention belongs to the field of target detection technology, and specifically relates to a knowledge distillation method based on decoupling features and adversarial features. Background Art

[0002] In the field of power line inspection, drone inspections can reduce the workload of maintenance personnel climbing poles for inspections and enable rapid and accurate defect detection. Drone inspections have become a standard inspection method for power transmission lines. Traditional manual defect analysis of transmission line inspection images captured by drones is difficult to adapt to the demands of power grid development and institutional reforms. Exploring a transmission line drone inspection image processing method that improves defect detection efficiency and accuracy is essential for addressing the challenges of intelligent operation and inspection. Introducing deep learning technology into transmission line aerial image processing and fully leveraging prior knowledge of key transmission line components from big data has significant practical value, ensuring the safe and stable operation of the power grid and improving the efficiency of transmission line inspections. However, in real-world environments, the resolution of power line inspection images captured by drones is high. Traditional deep learning object detection networks require a large number of weight parameters to ensure detection accuracy, resulting in a large model footprint and placing higher demands on existing hardware. Researching efficient network models can further streamline networks, removing redundant structures and parameters, while also improving network performance and accelerating the industrialization of deep networks. In order to reduce the amount of network computation and improve the efficiency of target detection, it is necessary to design a detection network with fewer network parameters for intelligent defect identification in drone inspection without sacrificing network performance.

[0003] In deep learning-based object detection, model compression is often used to improve model speed. Knowledge distillation is an excellent model compression technique that can compress large networks into smaller ones. During the training phase, a high-precision large network is trained. During the deployment phase, the smaller network distilled from the large network is deployed. This reduces computational overhead while achieving network accuracy comparable to that of the large network.

[0004] A Chinese invention patent, publication number CN112200062B, discloses a neural network-based object detection method, apparatus, machine-readable medium, and device. The invention proposes a neural network-based object detection method, comprising: constructing a teacher network; training the teacher network using a sample image set; constructing a student network, wherein the number of parameters of the student network is smaller than that of the teacher network; training the student network using a sample image set while extracting knowledge acquired from the teacher network training and transferring it to the student network through knowledge distillation; and performing object detection on an input image using the trained student network. Although this invention simplifies the neural network for object detection through knowledge distillation, its knowledge distillation only distills the target region. This results in the simplified student network's inability to distinguish between target and background features. When some of the target and background features are close in feature information, effective distinction cannot be achieved. Furthermore, the method does not adequately grasp the global features of the image to be detected, potentially missing useful information during the detection process. All of these factors reduce the accuracy of object detection. Summary of the Invention

[0005] The purpose of the present invention is to solve the problems mentioned in the background technology and provide a knowledge distillation method based on decoupling features and adversarial features. The model simultaneously and with emphasis uses the information of the target area and background area in the image to help the model distillation learn more useful information. At the same time, the adversarial network is used to make the distribution of the feature map values ​​output by the simple network learning the complex network as close to the same as possible, so as to achieve the effect of improving the defect detection speed while ensuring the detection accuracy.

[0006] In order to achieve the above technical objectives, the technical solution adopted by the present invention is:

[0007] A knowledge distillation method based on decoupled features and adversarial features includes the following steps:

[0008] S1. Build a teacher network and a student network; use sample images to train the teacher network until it meets the set requirements;

[0009] S2. Input the sample image into the teacher network trained in step S1 and the student network to obtain the first feature map output by the backbone network of the two networks;

[0010] S3. Using the first feature map obtained in step S2 as the input of a decoupling feature module, and performing gradient backpropagation to update the student network parameters based on the results of the decoupling feature module; the decoupling feature module is used to allow the student network to simultaneously learn the target feature information and background feature information of the teacher network;

[0011] S4, using the first feature map obtained in step S2 as the input of the adversarial feature module, training the adversarial feature module, and performing gradient backpropagation to update the student network parameters based on the results of the adversarial feature module; the adversarial feature module is used to allow the student network to learn the global distribution characteristics of the feature map output by the teacher network;

[0012] S5. Input the sample image into the teacher network trained in step S1 and the student network trained in step S4, input the first feature maps output by the backbone networks of the two networks into their respective region proposal networks, and obtain the second feature map of the candidate box containing classification and regression; use the above second feature map as the input of the candidate box feature learning module, perform gradient backpropagation based on the results of the candidate box feature learning module and update the parameters of the student network; the candidate box feature learning module is used to allow the student network to learn the feature information of the teacher network candidate box;

[0013] S6: The detection head network performs further classification and regression based on the information captured by the candidate box in the second feature map to obtain the final detection result; and performs gradient backpropagation based on the result to update the parameters of the student network;

[0014] S7. Repeat steps S2-S6 until the student network meets the set requirements.

[0015] Preferably, the teacher network and the student network both use the target detection network Faster RCNN, wherein the teacher network uses Faster RCNN with a backbone network of ResNet101, and the student network uses Faster RCNN with a backbone network of ResNet18.

[0016] Preferably, before the sample image is input into the teacher network and the student network, the short side is scaled to a set length, and the long side is scaled accordingly according to the aspect ratio of the original image; the gradient backpropagation adopts the stochastic gradient descent method.

[0017] Preferably, the teacher network and the student network have the same downsampling multiple for the input sample image; the last layer of the student network is set to a 1*1 convolution layer to adjust the number of channels of its output feature map to be equal to the number of channels of the teacher network output feature map.

[0018] Preferably, the sample image is manually annotated with several annotation boxes, and a binary mask M is set for the first feature map output by the teacher network and the student network according to the annotation box, and the value of M is 1 or 0. When the value of M is 1, it means that the area of ​​the first feature map contains the target feature information; when the value of M is 0, it means that the area of ​​the first feature map contains the background feature information; the output result of the decoupling feature module is the decoupling feature loss function L DFIt is used to represent the deviation between the target feature information and the background feature information in the first feature map output by the student network and the teacher network, L DF The calculation formula is as follows:

[0019]

[0020] Among them, F t is the image feature of the area corresponding to the first feature map output by the teacher network, F s The image features of the area corresponding to the first feature map output by the student network; and are the adjustment coefficients of target feature information and background feature information respectively; N obj and N bg are the number of regions containing target feature information and background feature information in the first feature map, respectively; A(·) represents the 1*1 convolution operation used to adjust the number of channels of the student network output feature map; H, W, and C are the height, width, and number of channels of the first feature map, respectively.

[0021] Preferably, the adversarial feature module includes a discriminator model, the output of which is the probability P of the input feature map being identified as the output of the student network. s and the probability P of identifying it as the output of the teacher network t ; The loss function L of the discriminator model dis The calculation formula is as follows:

[0022]

[0023] Among them, Y s Y is the label of the first feature map output by the student network, the label feature values ​​are all 1, H' and W' are the height and width of the label, and H'=H, W'=W; t The label of the first feature map output by the teacher network has the same size and dimension as the first feature map output by the teacher network, and all feature values ​​are 0; the discriminator model is based on its loss function L dis The gradient of the result is returned to improve the discriminator model's ability to identify the first feature maps output by the teacher network and the student network.

[0024] Preferably, the adversarial feature module calculates the backbone network generation loss function L of the student network after each training of the discriminator module. adv , L adv The calculation formula is as follows:

[0025]

[0026] The above formula is the output result of the adversarial feature module.

[0027] As a preference, the distillation loss function L of the backbone network of the student network is backbone By decoupling feature loss function L DF And generate loss function L adv Composition, L backbone The calculation formula is as follows:

[0028] L backbone =λ1L DF +λ2L adv

[0029] Among them, the hyperparameter λ1 is set to 1 and λ2 is set to 0.5.

[0030] Preferably, the candidate box feature learning module converts the candidate box corresponding area in the input second feature map into a third feature map of size α*α by downsampling, and then calculates the distillation loss function L of the candidate box feature of the student network according to the following formula RoI :

[0031]

[0032] The above formula is the output result of the candidate box feature learning module; where N′ represents the total number of candidate boxes contained in the second feature image output by the region proposal network of the teacher network or the student network, and the total number of candidate boxes of the two is equal; C′ represents the third feature map F corresponding to the teacher network t The total number of channels of ′; F s ′ is F s F is the abbreviation of ′(n,c,w,h), which represents the image features of the corresponding area of ​​each third feature map of the student network, where n represents the corresponding serial number of the candidate box, c, w and h represent the channel number, width and height of the third feature map respectively; t ′ is F t ′(n,c,w,h) is the abbreviation of , which represents the image features of the area corresponding to each third feature map of the teacher network; σ(.) indicates that the candidate box feature learning module adjusts the number of channels of the third feature map corresponding to the student network and the number of channels of the third feature map corresponding to the teacher network to the same 1*1 convolution operation.

[0033] Preferably, step S7 is based on the following loss function L mimic Determine whether the student network meets the standards:

[0034] L mimic =λ0L GT +L backbone +λ3L RoI

[0035] Among them, λ0 and λ3 are hyperparameters; L GTis the loss function in the Faster RCNN network, and the specific expression is as follows

[0036] L GT =L rpn +L rcnn

[0037] Among them L rpn is the sum of the losses obtained by the classification and regression branches of the region proposal network, L rcnn is the sum of the losses obtained by the classification and regression branches of the detection head network;

[0038] During the training process of steps S2-S6, the distillation loss function L of the backbone network in the student network is optimized alternately. backbone And the distillation loss function L of the candidate box feature RoI , and the loss function L in the Faster RCNN network GT ; and perform repeated iterative training, when the above loss function L mimic The value is the smallest, and the student network meets the standard.

[0039] The beneficial effects of the present invention are:

[0040] 1. The decoupling feature model improves the student network's ability to recognize target feature information and background feature information, and the adversarial feature model improves the student network's ability to recognize global feature information. This allows the trained student network to retain the detection performance of the teacher network as much as possible while being simplified. It can not only effectively distinguish target feature information and background feature information, but also extract as much global feature information as possible. All of these ensure the detection accuracy of the student network, so that the student network can achieve both rapid detection and high-precision detection results.

[0041] 2. The candidate box feature learning module trains the region proposal network of the student network, making the student network's classification and regression results of the candidate boxes closer to those of the teacher network. That is, the student network can accurately classify and regress the candidate boxes while ensuring high-precision identification of the various features of the target to be detected, thereby enabling the student network to detect the target quickly and with high precision. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 It is the overall flow chart of the present invention;

[0043] Figure 2 This is the learning framework diagram for the first stage. DETAILED DESCRIPTION

[0044] The embodiments of the present invention are described in further detail below with reference to the accompanying drawings.

[0045] It should be noted that the terms such as "upper", "lower", "left", "right", "front", "back", etc. cited in the invention are only for the convenience of description and are not used to limit the scope of implementation of the present invention. Changes or adjustments to their relative relationships should be regarded as the scope of implementation of the present invention without substantially changing the technical content.

[0046] like Figure 1 、 2 As shown, the present invention provides a knowledge distillation method based on decoupling features and adversarial features, which uses knowledge distillation technology to compress complex deep learning models and improve the performance of lightweight networks by capturing the knowledge of complex networks. The complex network containing more knowledge is called the teacher network, and the lightweight network that needs to be learned is called the student network. The main method is to first use the teacher network to train a high-precision large model containing more knowledge, and then use the student network to simultaneously learn the knowledge in the large model and the real labels of the data. In this way, the knowledge of the teacher network can be transferred to the student network, thereby obtaining a model that has both the speed of a small model and the accuracy of a large model.

[0047] The present invention specifically comprises the following steps:

[0048] S1. Construct a teacher network and a student network. Both the teacher network and the student network use the target detection network Faster RCNN. Specifically, the residual network ResNet101 has a deeper depth than ResNet18 and can extract richer features. Therefore, the teacher network uses Faster RCNN with the backbone network ResNet101, and the student network uses Faster RCNN with the backbone network ResNet18. In the residual network, the first four layers serve as feature extraction modules, and the fifth layer connects two fully connected layers for classification and regression.

[0049] The teacher network is trained using sample images until it can accurately detect target information. Note that before the sample image is input into the network (including the teacher network and the student network in subsequent steps), the short side is scaled to 600 and the long side is scaled accordingly according to the aspect ratio of the original image. The network is optimized using stochastic gradient descent (SGD), with the momentum parameter set to 0.9, the weight decay parameter set to 0.0005, the number of samples used in one training session set to 4, and the learning rate set to 0.001.

[0050] S2. Input the sample image into the teacher network trained in step S1 and the student network to obtain the first feature map output by the backbone network of the two networks for use in steps S3 and S4; the teacher network and the student network have the same downsampling multiple for the input sample image; the last layer of the student network is set to a 1*1 convolutional layer to adjust the number of channels of its output feature map to be equal to the number of channels of the teacher network output feature map.

[0051] S3. Using the first feature map obtained in step S2 as the input of the decoupling feature module, and performing gradient backpropagation to update the student network parameters based on the results of the decoupling feature module; the decoupling feature module is used to allow the student network to simultaneously learn the target feature information and background feature information of the teacher network, that is, to improve the student network's ability to recognize the target feature information and background feature information;

[0052] The sample image is manually annotated with a number of annotation boxes, which indicate whether the box contains current feature information or background feature information; a binary mask M is set for the first feature map output by the teacher network and the student network according to the annotation box, and the value of M is 1 or 0. When the value of M is 1, it means that the area of ​​the first feature map contains the target feature information; when the value of M is 0, it means that the area of ​​the first feature map contains the background feature information; the output result of the decoupling feature module is the decoupling feature loss function L DF It is used to represent the deviation between the target feature information and the background feature information in the first feature map output by the student network and the teacher network, L DF The calculation formula is as follows:

[0053]

[0054] Among them, F t is the image feature of the area corresponding to the first feature map output by the teacher network, F s The image features of the area corresponding to the first feature map output by the student network; and are the adjustment coefficients of target feature information and background feature information respectively; N obj and N bg are the number of regions containing target feature information and background feature information in the first feature map, respectively; A(·) represents the 1*1 convolution operation used to adjust the number of channels of the student network output feature map; H, W, and C are the height, width, and number of channels of the first feature map, respectively.

[0055] S4. Use the first feature map obtained in step S2 as the input of the adversarial feature module to train the adversarial feature module, wherein the adversarial feature module includes a discriminator model, and training the adversarial feature module is to train the discriminator model; in terms of structural design, the network of the discriminator model adopts three convolutional layers with a step size of 2 for downsampling, and the sizes of the convolution kernels are 3*3*1024, 3*3*512, and 3*3*1, respectively. A Leaky ReLU activation function with a decay rate of 0.2 is set after the first two convolutional layers. In order to make the final output a probability value from 0 to 1, a sigmoid activation function is set after the last convolutional layer.

[0056] The output of the discriminator model is the probability P that the input feature map (here, the first feature map) is identified as the feature map output by the student network. s and the probability P of the feature map output by the teacher network t ; The loss function L of the discriminator model dis The calculation formula is as follows:

[0057]

[0058] Among them, Y s Y is the label of the first feature map output by the student network. The label feature values ​​are all 1, H' and W' are the height and width of the label, and H'=H, W'=W, that is, the label of the first feature map of the student network has the same size as the first feature map; t The label of the first feature map output by the teacher network has the same size and dimension as the first feature map output by the teacher network, and the label feature values ​​of the first feature map of the teacher network are all 0; the discriminator model is based on its loss function L dis The gradient of the result is returned to improve the discriminator model's ability to identify the first feature maps output by the teacher network and the student network.

[0059] At the same time, after each iteration of training the identification model, the backbone network of the student network is calculated to generate the loss function L adv , L adv The calculation formula is as follows:

[0060]

[0061] According to the calculation results of the above formula (that is, the output results of the adversarial feature module), gradient backpropagation is performed to update the student network parameters, so that the feature map output by the student network is increasingly difficult to distinguish from the feature map output by the teacher network, thereby achieving the purpose of simulating the global distribution characteristics of the teacher network feature map.

[0062] It should be noted that steps S3 and S4 are not performed in any particular order and can be performed in reverse order or simultaneously.

[0063] Steps S2-S4 are the first stage of learning for the student network, which mainly trains the backbone network of the student network to transfer the knowledge of the backbone network of the teacher network as much as possible;

[0064] The distillation loss function L of the backbone network of the student network backbone By decoupling feature loss function L DF And generate loss function L adv Composition, L backbone The calculation formula is as follows:

[0065] L backbone =λ1L DF +λ2L adv

[0066] Among them, the hyperparameter λ1 is set to 1 and λ2 is set to 0.5.

[0067] S5. This step is the second stage of learning for the student network. In addition to learning the features of the teacher backbone network, the student network also needs to learn the feature information of the teacher network candidate box, which is called the candidate box feature learning module, to strengthen the student network's learning of the target feature information. The sample image is input into the student network trained in step S4, and the first feature map F is output through its backbone network. s , and then the first feature map F s The input is input into the subsequent region proposal network (RPN) of the student network. The region proposal network preliminarily obtains the second feature map of the candidate box containing classification and regression based on the extracted features; the sample image is input into the teacher network trained in step S1, and the second feature map corresponding to the teacher network is obtained through the same process as the student network; the second feature maps corresponding to the student network and the teacher network are then input into the candidate box feature learning module to obtain the corresponding third feature map. The results obtained by the candidate box feature learning module are gradient-backed to update the parameters of the student network, so that the student network learns the candidate box feature information of the teacher network, and improves the accuracy of candidate box classification and regression output by the region proposal network of the student network;

[0068] The process of obtaining the third feature map by the candidate box feature learning module and its corresponding output results are as follows:

[0069] The candidate box feature learning module downsamples the corresponding area of ​​the candidate box in the second feature map input by the two networks (student network and teacher network) to obtain a third feature map of size α*α, and then calculates the distillation loss function L of the candidate box feature of the student network according to the following formula RoI (That is, the output of the candidate box feature learning module):

[0070]

[0071] Among them, N′ represents the total number of candidate boxes of the first output image of the teacher network or the student network, and the total number of candidate boxes of the two is equal; C′ represents the feature map F t The total number of channels of ′; F s ′ is F s F is the abbreviation of ′(n,c,w,h), which represents the image features of the corresponding area of ​​each second feature map of the student network, where n represents the corresponding serial number of the candidate box, c, w and h represent the channel number, width and height of the second feature map respectively; t ′ is F t ′(n,c,w,h) is the abbreviation of , which represents the image features of the area corresponding to each second feature map of the teacher network; σ(.) represents the 1*1 convolution operation that adjusts the number of channels of the second feature map output by the student network and the number of channels of the second feature map output by the teacher network to be consistent.

[0072] Based on the calculation results of the above formula, the gradient is back-propagated to update the parameters of the regional proposal network of the student network.

[0073] S6. The detection head network performs further classification and regression based on the information intercepted by the candidate box in the second feature map to obtain the final detection result; and performs gradient backpropagation based on the result to update the parameters of the detection head network of the student network;

[0074] S7. Repeat steps S2-S6 until the student network meets the set requirements:

[0075] This step is based on the following loss function L mimic Determine whether the student network meets the standards:

[0076] L mimic =λ0L GT +L backbone +λ3L RoI

[0077] Among them, λ0 and λ3 are hyperparameters, which are set to 1 and 2.5 respectively; L GT is the loss function in the Faster RCNN network, and the specific expression is as follows

[0078] L GT =L rpn +L rcnn

[0079] Among them L rpn is the sum of the losses obtained by the classification and regression branches of the region proposal network, L rcnn is the sum of the losses obtained by the classification and regression branches of the detection head network;

[0080] Repeating the training of steps S2-S6 is actually to alternately optimize the distillation loss function L of the backbone network in the student network. backbone And the distillation loss function L of the candidate box feature RoI , and the loss function L in the Faster RCNN network GT ; When the above loss function L mimic When the value is minimum, the training of the student network reaches the standard, and step S7 can be ended.

[0081] After the above seven steps of training and learning, the student network can complete the training.

[0082] To demonstrate the feasibility and effectiveness of the knowledge distillation method proposed in this paper, based on decoupled and adversarial features, we used real-world images of transmission line defects captured by drones for training and testing. A total of 29,124 images were collected, covering 61 categories. The training and test sets were divided into a 7:3 ratio, and the mAP value and model size were used as the final evaluation criteria for the model. The experimental results are shown in the following table:

[0083] Table 1 Experimental results based on the transmission line drone inspection defect dataset

[0084]

[0085] The experimental results show that without knowledge distillation (i.e., types one and three), it is difficult to strike a balance between model accuracy and speed. When using the deeper network ResNet101 as the backbone network of Faster RCNN, the average accuracy of the model is 53.1%, the number of floating-point operations is 7.6G, and the inference speed is 13.2FPS; when using the shallower ResNet18 as the backbone network of Faster RCNN, the average accuracy of the model is 30.3%, a decrease of about 23%, but the number of floating-point operations of the model is 1.8G, and the inference speed is 76.3FPS, which is nearly 5 times faster than the inference speed of the large model.

[0086] When using the knowledge distillation model proposed in the present invention (i.e., Type 2), the average accuracy of the student network model increased from 30.3% to 48.3% while maintaining the performance of floating-point operations and inference speed, an increase of 18.4%, close to the average accuracy of the teacher network, achieving the expected effect, and proving the effectiveness of the method used in the present invention.

[0087] The above are merely preferred embodiments of the present invention. The scope of protection of the present invention is not limited to the above embodiments. All technical solutions based on the principles of the present invention are within the scope of protection of the present invention. It should be noted that for those skilled in the art, various improvements and modifications that do not depart from the principles of the present invention should be considered within the scope of protection of the present invention.

Claims

1. A knowledge distillation method based on decoupling features and adversarial features, characterized in that: The following steps are involved: S1. Build a teacher network and a student network; use sample images to train the teacher network until it meets the set requirements; S2. Input the sample image into the teacher network trained in step S1 and the student network to obtain the first feature map output by the backbone network of the two networks; S3. Using the first feature map obtained in step S2 as the input of a decoupling feature module, and performing gradient backpropagation to update the student network parameters based on the results of the decoupling feature module; the decoupling feature module is used to allow the student network to simultaneously learn the target feature information and background feature information of the teacher network; S4, using the first feature map obtained in step S2 as the input of the adversarial feature module, training the adversarial feature module, and performing gradient backpropagation to update the student network parameters based on the results of the adversarial feature module; the adversarial feature module is used to allow the student network to learn the global distribution characteristics of the feature map output by the teacher network; S5. Input the sample image into the teacher network trained in step S1 and the student network trained in step S4, input the first feature maps output by the backbone networks of the two networks into their respective region proposal networks, and obtain the second feature map of the candidate box containing classification and regression; use the above second feature map as the input of the candidate box feature learning module, perform gradient backpropagation based on the results of the candidate box feature learning module and update the parameters of the student network; the candidate box feature learning module is used to allow the student network to learn the feature information of the teacher network candidate box; S6: The detection head network performs further classification and regression based on the information captured by the candidate box in the second feature map to obtain the final detection result; and performs gradient backpropagation based on the result to update the parameters of the student network; S7. Repeat steps S2-S6 until the student network meets the set requirements.

2. The knowledge distillation method based on decoupling features and adversarial features according to claim 1, characterized in that: The teacher network and the student network both use the target detection network Faster RCNN, where the teacher network uses Faster RCNN with a backbone network of ResNet101, and the student network uses Faster RCNN with a backbone network of ResNet18.

3. The knowledge distillation method based on decoupling features and adversarial features according to claim 1, characterized in that: Before inputting the teacher network and the student network, the short side of the sample image is scaled to a set length, and the long side is scaled accordingly according to the aspect ratio of the original image; the gradient backpropagation adopts the stochastic gradient descent method.

4. The knowledge distillation method based on decoupling features and adversarial features according to claim 1, characterized in that: The teacher network and the student network have the same downsampling multiples for the input sample image; the last layer of the student network is set to a 1*1 convolution layer to adjust the number of channels of its output feature map to be equal to the number of channels of the teacher network output feature map.

5. The knowledge distillation method based on decoupling features and adversarial features according to any one of claims 1 to 4, characterized in that: The sample image is manually annotated with several annotation boxes, and a binary mask M is set for the first feature map output by the teacher network and the student network according to the annotation box. The value of M is 1 or 0. When the value of M is 1, it means that the area of ​​the first feature map contains the target feature information; when the value of M is 0, it means that the area of ​​the first feature map contains the background feature information. The output result of the decoupling feature module is the decoupling feature loss function L DF It is used to represent the deviation between the target feature information and the background feature information in the first feature map output by the student network and the teacher network, L DF The calculation formula is as follows: Among them, F t is the image feature of the area corresponding to the first feature map output by the teacher network, F s The image features of the area corresponding to the first feature map output by the student network; and are the adjustment coefficients of target feature information and background feature information respectively; N obj and N bg are the number of regions containing target feature information and background feature information in the first feature map, respectively; A(·) represents the 1*1 convolution operation used to adjust the number of channels of the student network output feature map; H, W, and C are the height, width, and number of channels of the first feature map, respectively.

6. The knowledge distillation method based on decoupling features and adversarial features according to claim 1, characterized in that: The adversarial feature module includes a discriminator model, the output of which is the probability P of the input feature map being identified as the output of the student network. s and the probability P of identifying it as the output of the teacher network t ; The loss function L of the discriminator model dis The calculation formula is as follows: Among them, Y s Y is the label of the first feature map output by the student network, the label feature values ​​are all 1, H' and W' are the height and width of the label, and H'=H, W'=W; t The label of the first feature map output by the teacher network has the same size and dimension as the first feature map output by the teacher network, and all feature values ​​are 0; the discriminator model is based on its loss function L dis The gradient of the result is returned to improve the discriminator model's ability to identify the first feature maps output by the teacher network and the student network.

7. The knowledge distillation method based on decoupling features and adversarial features according to claim 6, characterized in that: The adversarial feature module calculates the backbone network generation loss function L of the student network after each training of the discriminator module. adv , L adv The calculation formula is as follows: The above formula is the output result of the adversarial feature module.

8. The knowledge distillation method based on decoupling features and adversarial features according to claim 7, characterized in that: The distillation loss function L of the backbone network of the student network backbone By decoupling feature loss function L DF And generate loss function L adv Composition, L backbone The calculation formula is as follows: L backbone =λ1L DF +λ2L adv Among them, the hyperparameter λ1 is set to 1 and λ22 is set to 0.

5.

9. The knowledge distillation method based on decoupling features and adversarial features according to claim 8, characterized in that: The candidate box feature learning module converts the candidate box corresponding area in the input second feature map into a third feature map of size α*α by downsampling, and then calculates the distillation loss function L of the candidate box feature of the student network according to the following formula RoI : The above formula is the output result of the candidate box feature learning module; where N′ represents the total number of candidate boxes contained in the second feature image output by the region proposal network of the teacher network or the student network, and the total number of candidate boxes of the two is equal; C′ represents the third feature map F corresponding to the teacher network t The total number of channels of ′; F s ′ is F s F is the abbreviation of ′(n,c,w,h), which represents the image features of the corresponding area of ​​each third feature map of the student network, where n represents the corresponding serial number of the candidate box, c, w and h represent the channel number, width and height of the third feature map respectively; t ′ is F t ′(n,c,w,h) is the abbreviation of , which represents the image features of the area corresponding to each third feature map of the teacher network; σ(.) indicates that the candidate box feature learning module adjusts the number of channels of the third feature map corresponding to the student network and the number of channels of the third feature map corresponding to the teacher network to the same 1*1 convolution operation.

10. The knowledge distillation method based on decoupling features and adversarial features according to claim 9, characterized in that: The step S7 is based on the following loss function L mimic Determine whether the student network meets the standards: L mimic =λ0L GT +L backbone +λ3L RoI Among them, λ0λ3 are hyper parameters; L GT is the loss function in the Faster RCNN network, specifically expressed as, The formula is as follows L GT =L rpn +L rcnn Among them L rpn is the sum of the losses obtained by the classification and regression branches of the region proposal network, L rcnn is the sum of the losses obtained by the classification and regression branches of the detection head network; During the training process of steps S2-S6, the distillation loss function L of the backbone network in the student network is optimized alternately. backbone And the distillation loss function L of the candidate box feature RoI , and the loss function L in the Faster RCNN network GT ; and perform repeated iterative training, when the above loss function L mimic The value is the smallest, and the student network meets the standard.

Images