Aerial image target detection method based on cross-head collaborative distillation and feature standardization

Abstract

The invention discloses an aerial image target detection method based on cross-head collaborative distillation and feature standardization, and the method is characterized in that the method comprises the steps: training a teacher network, and obtaining a teacher network with higher accuracy; randomly initializing a student network, and obtaining multi-scale features of the teacher and student networks through a feature pyramid structure after inputting an image; a feature sharing standardization layer is used to obtain the standardized features of the teacher and student networks, and feature correlation loss is solved; the student network head features are input into a teacher network head in a cross-head mode through a shared standardization layer, and prediction consistency loss is obtained; obtaining supervised loss of the student network through the label; and updating the weight value of the student network according to the back propagation gradient information. According to the method, the distillation effect of the target detection model can be improved through a cross-head standardization method, so that the target detection student network with less parameter quantity can obtain the precision similar to that of the teacher model with larger parameter quantity.

Description

Technical Field

[0001] The present invention relates to the field of image processing, and in particular to an aerial image target detection method based on cross-head collaborative distillation and feature standardization. Background Art

[0002] Object detection technology has been widely used in many fields, such as traffic flow monitoring and ship monitoring. Generally speaking, deep learning models with more parameters can achieve higher accuracy, but such models often have high requirements for computing resources. On some end-side devices with limited computing power, too many model parameters may lead to slower inference speed or even failure to deploy. Therefore, how to reduce the computational complexity of the model while ensuring model accuracy has become an important research direction.

[0003] As a model lightweight technology, knowledge distillation can significantly reduce the number of model parameters without significantly reducing the model accuracy. Knowledge distillation transfers the knowledge learned by the large teacher network to the smaller student network. Compared with the traditional supervised learning method of directly training a small model, it can make better use of the knowledge learned by the teacher model, thereby achieving higher accuracy with the same data and model size.

[0004] There are two main existing knowledge distillation methods: the first is predictive imitation, that is, after the teacher network training is completed, the student network is trained by imitating the output of the teacher network. The disadvantage of this method is that for target detection tasks, the distillation efficiency of the positioning information output by the teacher network is low, and there may be conflicts between the pseudo labels and the real labels of the teacher network, which in turn affects the optimization effect of the model. The second is feature imitation, that is, the student network is trained by imitating the feature representation of the intermediate layer of the teacher network. The disadvantage of this method is that it cannot fully utilize the task-specific relevant information for effective distillation, so the performance in target detection tasks is relatively poor.

[0005] The common defect of these two methods is that during the distillation process, the output features between the teacher network and the student network have large differences in numerical range and scale, resulting in unsatisfactory distillation results. Due to this inconsistent numerical range, the model cannot effectively transmit key information, the distillation effect is often poor, and it is difficult to fully improve the performance of the student network.

[0006] In summary, existing knowledge distillation methods face great challenges in solving the inconsistency between pseudo labels and true labels, as well as the problem of inconsistent output feature scales. New technical means are urgently needed to improve existing methods and enhance the distillation effect, especially in target detection tasks. Summary of the invention

[0007] The present invention relates to an aerial image target detection method based on cross-head collaborative distillation and feature standardization, aiming to improve the distillation effect of the target detection model through feature sharing standardization and cross-head input technology. The method of the present invention can effectively improve the detection accuracy of the student network while reducing the computational complexity and model parameters.

[0008] The present invention provides an aerial image target detection method based on cross-head collaborative distillation and feature standardization, comprising the following steps:

[0009] 1. Train a teacher network to obtain a teacher network with higher accuracy;

[0010] 2. Randomly initialize a student network, input the image and extract multi-scale features of the teacher network and the student network through the feature pyramid structure;

[0011] 3. Use the feature sharing normalization layer to normalize the multi-scale features of the teacher network and the student network, and calculate the feature correlation loss;

[0012] 4. Input the head features of the student network into the cross-head of the teacher network through the shared normalization layer for processing, and calculate the prediction consistency loss;

[0013] 5. Obtain supervised loss of student network through labels;

[0014] 6. Update the weight value of the student network based on the back-propagated gradient information.

[0015] The present invention provides an aerial image target detection method based on cross-head collaborative distillation and feature normalization. In the step 1, widely used target detection datasets DOTAv1 and COCO are used as training data to train a teacher network to improve the target detection accuracy of the model.

[0016] The present invention provides an aerial image target detection method based on cross-head collaborative distillation and feature normalization. In step 2, a FPN (Feature Pyramid Network) structure is used to extract multi-scale features so as to better capture target information of different scales and enhance feature alignment between teacher and student networks.

[0017] The present invention provides an aerial image target detection method based on cross-head collaborative distillation and feature standardization. In the step 3, a feature sharing standardization layer is introduced to standardize the multi-scale features of the teacher network and the student network to reduce the feature difference between the two, improve the feature sharing consistency, and thus improve the distillation effect.

[0018] The present invention provides an aerial image target detection method based on cross-head collaborative distillation and feature standardization. In step 4, a head standardization layer is introduced to standardize the head features of the student network, and the head features are input into the head of the teacher network across heads. By calculating the prediction consistency loss, the conflict between the pseudo labels and the real labels of the teacher network is alleviated.

[0019] The present invention provides an aerial image target detection method based on cross-head collaborative distillation and feature normalization. In step 5, the supervised loss obtained by the label further guides the training of the student network to improve its accuracy in the target detection task.

[0020] The present invention provides an aerial image target detection method based on cross-head collaborative distillation and feature normalization. In step 6, only the gradient of the student network is updated according to the loss function to ensure that the student network can better learn the knowledge of the teacher network and improve the target detection accuracy.

[0021] Compared with the prior art, the beneficial effects of the present invention are as follows: compared with the traditional knowledge distillation method, the present invention significantly improves the accuracy and distillation effect of the student network through the feature sharing normalization layer and the cross-head normalization technology, especially performs well in complex target detection tasks; extracts multi-scale features and performs feature normalization to make the feature alignment between the teacher network and the student network more accurate, thereby improving the accuracy of target detection; by introducing the head normalization layer, the conflict between the pseudo-labels and the real labels of the teacher network is resolved, further improving the distillation effect; while ensuring the accuracy of the student network, the computational complexity is reduced, adapting to the deployment requirements of terminal devices with lower computing resources. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Various other advantages and benefits will become apparent to those of ordinary skill in the art by reading the detailed description of the preferred embodiments below. The accompanying drawings are only for the purpose of illustrating the preferred embodiments and are not to be considered as limiting the present invention. Moreover, the same reference symbols are used throughout the accompanying drawings to represent the same components. In the accompanying drawings:

[0023] Figure 1 It is a flow chart of an aerial image target detection method based on cross-head collaborative distillation and feature standardization of the present invention;

[0024] Figure 2 It is a model structure diagram of an aerial image target detection method based on cross-head collaborative distillation and feature standardization of the present invention;

[0025] Figure 3 It is a schematic diagram of the performance of different distillation methods on the target detection dataset. DETAILED DESCRIPTION

[0026] The exemplary embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. Although the exemplary embodiments of the present invention are shown in the accompanying drawings, it should be understood that the present invention can be implemented in various forms and should not be limited by the embodiments described herein. On the contrary, these embodiments are provided in order to enable a more thorough understanding of the present invention and to fully convey the scope of the present invention to those skilled in the art.

[0027] Figure 1 and Figure 2 They are respectively a flow chart of an aerial image target detection method based on cross-head collaborative distillation and feature standardization of the present invention and a model structure diagram of an aerial image target detection method based on cross-head collaborative distillation and feature standardization of the present invention, comprising the following steps:

[0028] 1. Train the teacher network on the object detection dataset using supervised loss The formula is shown in (1):

[0029] (1)

[0030] in, Represents the head prediction output of the teacher model, and GT represents the true label value. Use the commonly used detection loss, the binary cross entropy loss for classification, and the GIoU loss for box regression. Suppose the input labeled image data is (X, Y), where X and Y represent the input image data and label value respectively. The prediction value calculation formula of the teacher network is shown in (2):

[0031] (2)

[0032] The teacher network is trained through multiple rounds on a given dataset to obtain a trained teacher head network. and teacher feature pyramid network . Prepare for the subsequent distillation study.

[0033] 2. Multi-scale features are obtained through the randomly initialized student network and the trained teacher network. The calculation formulas are shown in (3)-(4):

[0034] (3)

[0035] (4)

[0036] 3. Next, the multi-scale features of the teacher and the student are normalized using a shared feature normalization layer. The calculation formulas are shown in (5)-(6):

[0037] (5) (6)

[0038] in and They represent batch normalization and convolutional block attention, respectively. Their function is to make the teacher and student networks focus on the same important features and standardize the scale of output feature numbers to improve the efficiency of distillation learning.

[0039] After standardization, the KL divergence is used to calculate the consistency loss of the standardized teacher and student features. The calculation formula is shown in (7):

[0040] (7)

[0041] in, Represent the feature data of students and teachers in the i-th feature channel respectively.

[0042] 4. Perform cross-head normalization and distillation. For a head network with n convolutional layers, the head features , multiple consecutive convolutional layers are used Indicates that the i-th feature of the student network The teacher network is also standardized using a shared standardization layer. The formula for standardizing features is shown in (8):

[0043] (8)

[0044] in and They represent batch normalization method and SE channel attention module respectively. Their function is to make the teacher and the student focus on the same important channels and make the feature data size of each layer consistent, thus improving the efficiency of distillation.

[0045] The standardized student head features are input into the i+1th convolution of the teacher network across heads. In the example, the last convolutional layer of the teacher network is normalized to adapt to cross-head predictions. The formula for calculating cross-head distillation loss is shown in (9):

[0046] (9)

[0047] in, represents the position of all anchor boxes, It is an indicator to measure the consistency of the prediction results. For classification tasks, KL divergence is used, and for box regression tasks, L1 loss is used.

[0048] 5. Calculate the final loss, there will be label loss , feature consistency loss and cross-head normalized consistency loss Comprehensively obtain the final loss function , The calculation formula is shown in (10):

[0049] (10)

[0050] 6. Finally, only the parameters of the student network are updated according to the gradient of back propagation, and the distilled student model is obtained through multiple rounds of training. This model improves the efficiency of distillation by standardizing multi-scale features and head features. The cross-head distillation method solves the problem of conflict between teacher pseudo labels and real labels, thereby improving the accuracy of the student model in the detection task.

[0051] Embodiment 1:

[0052] The experimental results in this embodiment are obtained by using the widely used public datasets COCO and DOTA as experimental datasets for training and testing. The experimental results show that the technical effects of the present invention are as follows:

[0053] Figure 3 The table shows the effect of using the same ResNet101 as the teacher network and ResNet50 as the student network on different datasets using different distillation methods. The final student network accuracy obtained by distillation is published in the table, where: KD represents the naive pseudo-label knowledge distillation method, PKD represents the result obtained by using the PKD feature distillation method, and CrossKD represents the distillation result obtained by using the CrossKD feature imitation method. CNKD represents the method proposed in this paper.

[0054] from Figure 3 It can be seen that the proposed method is 1.8% higher than the KD method on the COCO dataset and 0.6% higher than the feature imitation method PKD. It is 0.31% higher than the CrossKD prediction imitation method on the DOTAv1 dataset. The reason is that the proposed method can make the attention channels of the features closer and the numerical values ​​consistent, thereby improving the efficiency of distillation.

[0055] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the technical principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A method for aerial image object detection based on cross-head collaborative distillation and feature normalization, characterized in that: The following steps are involved: Train a teacher network to obtain a teacher network with higher accuracy; randomly initialize a student network, input the image and extract the multi-scale features of the teacher network and the student network through the feature pyramid structure; use the feature sharing normalization layer to standardize the features of the teacher network and the student network, and calculate the feature correlation loss; input the head features of the student network into the cross-head of the teacher network through the shared normalization layer for processing, thereby calculating the prediction consistency loss; obtain the supervised loss of the student network through the label; update the weight value of the student network according to the back-propagated gradient information.

2. The aerial image target detection method based on cross-head collaborative distillation and feature normalization according to claim 1, characterized in that: The feature pyramid structure is used to extract multi-scale features between the teacher network and the student network, so that the feature alignment between the teacher network and the student network is more accurate.

3. The aerial image target detection method based on cross-head collaborative distillation and feature normalization according to claim 1, characterized in that: The features of the teacher network and the student network are both standardized by a shared normalization layer to reduce the feature differences between the teacher and student networks, thereby improving the consistency of feature sharing and enhancing the distillation effect.

4. The aerial image target detection method based on cross-head collaborative distillation and feature normalization according to claim 1, characterized in that: The head features of the student network are input into the head of the teacher network through a shared normalization layer across heads to alleviate the inconsistency between the pseudo labels of the teacher network and the true labels.

5. The aerial image target detection method based on cross-head collaborative distillation and feature normalization according to claim 1, characterized in that: Compared with traditional knowledge distillation methods, the present invention effectively improves the accuracy of the student network in target detection tasks through a cross-head normalization method, alleviates the inconsistency between the pseudo labels and the true labels of the teacher network, and further enhances the feature consistency by using a feature sharing normalization layer.

Images