Unmanned aerial vehicle image target identification method and device based on compensation feature adversarial distillation

Abstract

The invention provides an unmanned aerial vehicle image target identification method and device based on compensation feature adversarial distillation, and relates to the technical field of unmanned aerial vehicle image target detection. The method comprises the following steps: training a teacher network model by using a natural sample data set to obtain natural sample features and output layer logic; training a student network model by using the adversarial sample data set to obtain adversarial sample features; performing compensation fusion on the natural sample features and the adversarial sample features; constructing a gradient consistency dynamic adjustment mechanism to distribute distillation weights for characteristic distillation and logic distillation; designing a distillation loss function; and performing feature distillation and logic distillation on the student network model according to the compensation features, the output layer logic, the gradient consistency dynamic adjustment mechanism and the loss function to obtain a constructed student network model. According to the invention, the problems of insufficient natural feature and confrontation feature learning and low unmanned aerial vehicle image recognition precision of a model in an existing defense method based on confrontation training and multi-teacher knowledge distillation can be solved.

Description

Technical Field

[0001] The present invention relates to the technical field of drone image target detection, and in particular to a drone image target recognition method and device based on compensating feature adversarial distillation. Background Art

[0002] UAV remote sensing images have become an important data source in the field of low-altitude remote sensing research due to their high efficiency and low cost. Many researchers have applied deep networks to target detection tasks, but deep neural networks have been proven to be fragile. Deep neural networks are very vulnerable to adversarial samples. These adversarial samples are created by attackers deliberately adding perturbations that are difficult for humans to detect to clean images. These carefully crafted samples cause deep neural networks to make incorrect predictions, as shown in Reference 1 (SZEGEDY C, ZAREMBA W, SUTSKEVER I, et al. Intriguing properties of neural networks[Z].arXiv:1312.6199, 2014).

[0003] To improve the recognition stability of models against adversarial examples, researchers have proposed numerous defense methods. Reference 2 (Kurakin A, Goodfellow IJ, Bengio S. Adversarial examples in the physical world[M]. Artificial intelligence safety and security. Chapman and Hall / CRC, 2018: 99-112.) incorporates adversarial examples into training to optimize model parameters and enhance model robustness. Reference 3 (ZiB, Zhao S, Ma X, et al. Revisiting adversarial robustness distillation: Robust soft labels make student better[C] / / Proceedings of the IEEE / CVFInternational Conference on Computer Vision. 2021: 16443-16452.) leverages the idea of knowledge distillation to extract robustness from a large adversarially trained model. Both references 2 and 3 only improve the recognition stability against adversarial examples; the trained models' recognition accuracy for natural samples decreases.

[0004] To balance accuracy against both natural and adversarial examples, researchers have proposed numerous solutions, the most effective of which are adversarial training and robust distillation. Reference 4 (Zhang H, Yu Y, Jiao J, et al. Theoretically principled trade-off between robustness and accuracy [C] / / International conference on machine learning. PMLR, 2019: 7472-7482) introduces a trade-off between robustness to adversarial examples and accuracy against natural examples, thereby improving the model's performance in both areas. Reference 5 (Zhao S, Yu J, Sun Z, et al. Enhanced accuracy and robustness via multi-teacher adversarial distillation [C] / / European Conference on Computer Vision. Cham: Springer Nature Switzerland, 2022: 585-602.) proposes the Multi-Teacher Adversarial Robustness Distillation (MTARD) method, which uses an adversarial teacher and a clean teacher to guide the learning of a small student model, balancing the influence of the adversarial and clean teacher models. Reference 4's method adds both natural and adversarial examples to the dataset during training, enabling the model to detect both natural and adversarial examples. However, during training, the model fails to fully learn from both natural and adversarial examples, resulting in low recognition accuracy. Reference 5 has the student model learn the features of the natural and adversarial teachers separately, which may lead to confusion in the fusion of the natural and adversarial teacher features. Furthermore, both References 4 and 5 are designed for small images, which can lead to recognition difficulties with larger image data such as drone images. Summary of the Invention

[0005] To address the technical issues of insufficient natural and adversarial feature learning and low drone image recognition accuracy in existing defense methods based on adversarial training and multi-teacher knowledge distillation, the present invention provides a method and device for drone image target recognition based on compensatory feature adversarial distillation. The technical solution is as follows:

[0006] On the one hand, a method for drone image target recognition based on compensatory feature adversarial distillation is provided. The method is implemented by a drone image target recognition device and includes:

[0007] S1. Obtain a natural sample dataset, and generate an adversarial sample dataset based on the natural sample dataset and the adversarial sample generation method.

[0008] S2. Use the natural sample dataset to train the teacher network model to obtain the natural sample features and output layer logic of the teacher network model.

[0009] S3. Use the adversarial sample dataset to train the student network model and obtain the adversarial sample features of the student network model.

[0010] S4. Compensate and fuse the natural sample features and the adversarial sample features to obtain compensated features.

[0011] S5. Construct a dynamic adjustment mechanism for gradient consistency and assign distillation weights to feature distillation and logic distillation in the distillation process based on the dynamic adjustment mechanism for gradient consistency.

[0012] S6. Design the loss function for the distillation process.

[0013] S7. Perform feature distillation and logic distillation on the student network model based on the compensation features, the output layer logic of the teacher network model, the gradient consistency dynamic adjustment mechanism, and the loss function to obtain a constructed student network model; obtain the drone image to be identified, input it into the constructed student network model, and obtain the target recognition result.

[0014] Optionally, the compensatory fusion of the natural sample features and the adversarial sample features in S4 to obtain the compensated features includes:

[0015] Subtract the natural sample features from the adversarial sample features to obtain the difference features.

[0016] The difference features are amplified, and the amplified features are normalized to obtain normalized features.

[0017] The normalized features are multiplied element-wise with the natural sample features to obtain the fused features.

[0018] The fusion features are fused with the adversarial sample features to obtain the compensation features.

[0019] Optionally, a gradient consistency dynamic adjustment mechanism is constructed in S5, and distillation weights are assigned to feature distillation and logic distillation in the distillation process according to the gradient consistency dynamic adjustment mechanism, including:

[0020] Obtain the gradient direction of the current moment and the gradient direction of the previous moment of feature distillation and logical distillation, calculate the cosine similarity of the gradient direction of the current moment and the gradient direction of the previous moment of feature distillation and logical distillation respectively, and calculate the weight of the feature distillation and the weight of the logical distillation in the distillation process according to the cosine similarity of the feature distillation and the cosine similarity of the logical distillation.

[0021] Optionally, the weight calculation method for feature distillation is as follows (1):

[0022] (1)

[0023] Where, represents the weight of feature distillation, Indicates the iterations, Indicates the cosine similarity between the gradient direction of the current moment and the gradient direction of the previous moment of feature distillation, Indicates the cosine similarity between the gradient direction at the current moment and the gradient direction at the previous moment of logical distillation.

[0024] Optionally, the loss function of the distillation process in S6 includes: adversarial sample loss, feature distillation loss, and logistic distillation loss;

[0025] Among them, the adversarial sample loss is shown in the following formula (2):

[0026] (2)

[0027] Where, represents the adversarial sample loss, represents the cross entropy loss function, Indicates that the student model is in adversarial samples The predicted output on represents adversarial samples, Indicates the actual label.

[0028] The feature distillation loss is shown in the following formula (3):

[0029] (3)

[0030] Where, represents the feature distillation loss, Represents the selected feature layer index set, Indicates that the student model is The feature representation extracted by the layer, Indicates that the teacher model The feature representation extracted by the layer, represents a natural sample, Represents the square of the Euclidean distance.

[0031] Logical distillation loss is shown in Equation (4):

[0032] (4)

[0033] Where, represents the logistic distillation loss, represents the Kullback-Leibler divergence, represents the output of the teacher model, represents the distillation temperature coefficient, represents the output of the student model.

[0034] On the other hand, a device for identifying targets in drone images based on compensatory feature adversarial distillation is provided. The device is applied to a method for identifying targets in drone images based on compensatory feature adversarial distillation. The device includes:

[0035] The data acquisition module is used to obtain a natural sample data set and generate an adversarial sample data set based on the natural sample data set and the adversarial sample generation method.

[0036] The teacher network model training module is used to train the teacher network model using a natural sample data set to obtain the natural sample features and output layer logic of the teacher network model.

[0037] The student network model training module is used to train the student network model using the adversarial sample dataset to obtain the adversarial sample features of the student network model.

[0038] The feature compensation fusion module is used to compensate and fuse the natural sample features and the adversarial sample features to obtain compensated features.

[0039] A construction module is used to construct a gradient consistency dynamic adjustment mechanism, and assign distillation weights to feature distillation and logic distillation in the distillation process based on the gradient consistency dynamic adjustment mechanism.

[0040] Design module for designing the loss function of the distillation process.

[0041] The output module is used to perform feature distillation and logic distillation on the student network model based on the compensation features, the output layer logic of the teacher network model, the gradient consistency dynamic adjustment mechanism and the loss function to obtain a constructed student network model; obtain the drone image to be identified, input it into the constructed student network model, and obtain the target recognition result.

[0042] Optionally, the feature compensation fusion module is further used to:

[0043] Subtract the natural sample features from the adversarial sample features to obtain the difference features.

[0044] The difference features are amplified, and the amplified features are normalized to obtain normalized features.

[0045] The normalized features are multiplied element-wise with the natural sample features to obtain the fused features.

[0046] The fusion features are fused with the adversarial sample features to obtain the compensation features.

[0047] Optionally, the building block is further configured to:

[0048] Obtain the gradient direction of the current moment and the gradient direction of the previous moment of feature distillation and logical distillation, calculate the cosine similarity of the gradient direction of the current moment and the gradient direction of the previous moment of feature distillation and logical distillation respectively, and calculate the weight of the feature distillation and the weight of the logical distillation in the distillation process according to the cosine similarity of the feature distillation and the cosine similarity of the logical distillation.

[0049] Optionally, the weight calculation method for feature distillation is as follows (1):

[0050] (1)

[0051] Where, represents the weight of feature distillation, Indicates the iterations, Indicates the cosine similarity between the gradient direction of the current moment and the gradient direction of the previous moment of feature distillation, Indicates the cosine similarity between the gradient direction at the current moment and the gradient direction at the previous moment of logical distillation.

[0052] Optionally, the loss function of the distillation process includes: adversarial sample loss, feature distillation loss, and logistic distillation loss;

[0053] Among them, the adversarial sample loss is shown in the following formula (2):

[0054] (2)

[0055] Where, represents the adversarial sample loss, represents the cross entropy loss function, Indicates that the student model is in adversarial samples The predicted output on represents adversarial samples, Indicates the actual label.

[0056] The feature distillation loss is shown in the following formula (3):

[0057] (3)

[0058] Where, represents the feature distillation loss, Represents the selected feature layer index set, Indicates that the student model is The feature representation extracted by the layer, Indicates that the teacher model The feature representation extracted by the layer, represents a natural sample, Represents the square of the Euclidean distance.

[0059] Logical distillation loss is shown in Equation (4):

[0060] (4)

[0061] Where, represents the logistic distillation loss, represents the Kullback-Leibler divergence, represents the output of the teacher model, represents the distillation temperature coefficient, represents the output of the student model.

[0062] On the other hand, a drone image target recognition device is provided, which includes: a processor; a memory, wherein the memory stores computer-readable instructions, and when the computer-readable instructions are executed by the processor, any one of the above-mentioned drone image target recognition methods based on compensatory feature adversarial distillation is implemented.

[0063] On the other hand, a computer-readable storage medium is provided, wherein the storage medium stores at least one instruction, and the at least one instruction is loaded and executed by a processor to implement any one of the above-mentioned methods for drone image target recognition based on compensatory feature adversarial distillation.

[0064] The beneficial effects brought about by the technical solution provided by the embodiment of the present invention include at least:

[0065] In this paper, a method using compensatory feature adversarial distillation can address the problem of insufficient defense performance against drone reconnaissance images in existing image adversarial defense technologies, thereby improving the recognition accuracy of drone reconnaissance images. A distillation model based on logic distillation and feature distillation is constructed. Feature distillation transfers the feature representations of the intermediate layers of the teacher model, allowing the student model to mimic the internal feature space of the teacher model. Logic distillation guides the student model by transferring the logic of the teacher model's output layer, using the teacher model's predicted probability distribution of input data as a supervisory signal. A feature compensation module is established to compensate and fuse the natural sample features of the teacher network with the adversarial sample features of the student network, allowing the student network to learn not only the features of adversarial samples but also those of natural samples during training. A dynamic gradient consistency adjustment mechanism is designed to dynamically adjust the weights of logic distillation and feature distillation during the distillation process by calculating the consistency of the gradient at the current moment with the gradient at the previous moment. A distillation loss function is designed to calculate the difference between the model output and the actual output, and the magnitude of this loss function is used as the basis for iterative model optimization. The teacher model is trained using natural samples. Based on these natural samples, adversarial samples are generated using PGD, FGSM, and BIM adversarial sample generation methods as a dataset for training the student model. The student model is then used to detect natural samples and adversarial samples. BRIEF DESCRIPTION OF THE DRAWINGS

[0066] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0067] Figure 1 This is a flow chart of a method for drone image target recognition based on compensatory feature adversarial distillation provided by an embodiment of the present invention;

[0068] Figure 2 2 is a structural diagram of a compensatory feature adversarial distillation model provided by an embodiment of the present invention;

[0069] Figure 3 is a structural diagram of a characteristic compensation module provided by an embodiment of the present invention;

[0070] Figure 4 This is a diagram of a feature difference amplification module provided by an embodiment of the present invention;

[0071] Figure 5 This is a block diagram of a drone image target recognition device based on compensatory feature adversarial distillation provided by an embodiment of the present invention;

[0072] Figure 6 The present invention provides a schematic diagram of the structure of a drone image target recognition device. DETAILED DESCRIPTION

[0073] The technical solution of the present invention is described below in conjunction with the accompanying drawings.

[0074] In the embodiments of the present invention, words such as "exemplarily" and "for example" are used to indicate examples, illustrations, or explanations. Any embodiment or design described as an "exemplary" in the present invention should not be interpreted as being preferred or advantageous over other embodiments or designs. Rather, the use of the word "exemplary" is intended to present concepts in a concrete manner. Furthermore, in the embodiments of the present invention, "and / or" can mean both or either of the two.

[0075] In the embodiments of the present invention, the terms "image" and "picture" may sometimes be used interchangeably. It should be noted that, when the distinction is not emphasized, the meanings they convey are the same. The terms "of," "corresponding," and "corresponding" may sometimes be used interchangeably. It should be noted that, when the distinction is not emphasized, the meanings they convey are the same.

[0076] In the embodiments of the present invention, sometimes a subscript such as W1 may be written as a non-subscript such as W1. When the difference is not emphasized, the meanings to be expressed are the same.

[0077] In order to make the technical problems, technical solutions and advantages to be solved by the present invention clearer, a detailed description will be given below with reference to the accompanying drawings and specific embodiments.

[0078] The embodiment of the present invention provides a method for drone image target recognition based on compensatory feature adversarial distillation. The method can be implemented by a drone image target recognition device, which can be a terminal or a server. Figure 1 The flowchart of the method for target recognition in drone images based on compensatory feature adversarial distillation is shown. The processing flow of this method may include the following steps:

[0079] S1. Obtain a natural sample dataset, and generate an adversarial sample dataset based on the natural sample dataset and the adversarial sample generation method.

[0080] In one feasible implementation, the acquired natural sample dataset can be drone images, and adversarial samples are generated based on the natural samples using the Projected Gradient Descent (PGD), Fast Gradient Sign Method (FGSM), and Basic Iterative Method (BIM) adversarial sample generation methods.

[0081] S2. Use the natural sample dataset to train the teacher network model to obtain the natural sample features and output layer logic of the teacher network model.

[0082] In a feasible implementation, Figure 2 As shown, the compensatory feature adversarial distillation model constructed by the present invention includes: a teacher network model, a student network model, a feature distillation branch, and a logic distillation branch.

[0083] S3. Use the adversarial sample dataset to train the student network model and obtain the adversarial sample features of the student network model.

[0084] S4. Compensate and fuse the natural sample features and the adversarial sample features to obtain compensated features.

[0085] Optionally, the above step S4 may include:

[0086] Subtract the natural sample features from the adversarial sample features to obtain the difference features.

[0087] The difference features are amplified, and the amplified features are normalized to obtain normalized features.

[0088] The normalized features are multiplied element-wise with the natural sample features to obtain the fused features.

[0089] The fusion features are fused with the adversarial sample features to obtain the compensation features.

[0090] In a feasible implementation, Figure 3 As shown in the figure, a feature compensation fusion module is constructed, and the natural sample features of the teacher network and the adversarial sample features of the student network are input. The difference features are calculated by difference, and the difference features of the natural samples and the adversarial samples are amplified and normalized and multiplied element-by-element with the natural features. After being fused with the adversarial sample features, the compensation features are obtained to guide the student model to learn.

[0091] Specifically, the feature compensation fusion module includes the difference between natural samples and adversarial samples. Calculation:

[0092] (1)

[0093] Where, represents the feature extraction of natural samples by the teacher model, represents the feature extraction of the student model for adversarial samples, It means taking the absolute value of each element.

[0094] like Figure 4 As shown, the feature differences are amplified and then normalized:

[0095] (2)

[0096] Where, represents the normalized features, represents normalization, mapping the convolution output to [0,1], Represents a convolution operation.

[0097] The natural features are multiplied element by element and fused with the natural sample features:

[0098] (3)

[0099] Where, represents the fusion feature, Indicates the scaling factor, which controls the strength of the difference compensation.

[0100] Then fused with the adversarial sample features, we can compensate for the natural sample features. :

[0101] (4)

[0102] Where, represents the final fusion feature, Indicates the fusion of two feature maps in the feature dimension.

[0103] S5. Construct a dynamic adjustment mechanism for gradient consistency and assign distillation weights to feature distillation and logic distillation in the distillation process based on the dynamic adjustment mechanism for gradient consistency.

[0104] Optionally, the above step S5 may include:

[0105] Obtain the gradient direction of the current moment and the gradient direction of the previous moment of feature distillation and logical distillation, calculate the cosine similarity of the gradient direction of the current moment and the gradient direction of the previous moment of feature distillation and logical distillation respectively, and calculate the weight of the feature distillation and the weight of the logical distillation in the distillation process according to the cosine similarity of the feature distillation and the cosine similarity of the logical distillation.

[0106] In a feasible implementation, a dynamic adjustment mechanism for gradient consistency is constructed to balance the direction of feature distillation and logical distillation. By calculating the gradient consistency between the current moment and the previous round of training, it is determined whether the training gradient descent is stable, thereby assigning appropriate distillation weights to feature distillation and logical distillation.

[0107] Define its cosine similarity with the previous step as:

[0108] (5)

[0109] (6)

[0110] Where, and Respectively The gradient vectors of the student model in feature distillation and logistic distillation in the iteration are different from the traditional fixed or preset weight adjustment method. The learning stability of each distillation branch is estimated by analyzing the consistency of the gradient change direction. The weight of feature distillation is defined as:

[0111] (7)

[0112] Where, represents the weight of feature distillation, Indicates the iterations, Indicates the cosine similarity between the gradient direction of the current moment and the gradient direction of the previous moment of feature distillation, Indicates the cosine similarity between the gradient direction at the current moment and the gradient direction at the previous moment of logical distillation.

[0113] when The closer it is to 1, the more consistent the direction of the current round of training gradient is with the previous round, indicating that stable optimization is being carried out in this direction.

[0114] S6. Design the loss function for the distillation process.

[0115] Optionally, the loss function of the distillation process in S6 includes: adversarial sample loss, feature distillation loss, and logical distillation loss.

[0116] Among them, the adversarial sample loss is shown in the following formula (8):

[0117] (8)

[0118] Where, Adversarial sample loss is the difference between the metric model prediction result and the true label, which aims to make the model better fit the adversarial sample. It is the most basic robustness training goal. Cross entropy loss is used to train the adversarial sample to correctly classify. Represents the cross entropy loss function, which is used to measure the difference between the predicted distribution and the true label. Indicates that the student model is in adversarial samples The predicted output on represents adversarial samples, Indicates the actual label.

[0119] The feature distillation loss is shown in the following formula (9):

[0120] (9)

[0121] Where, Represents feature distillation loss. Feature distillation emphasizes that the student can also align the teacher's key representation under adversarial input. Represents the selected feature layer index set, Indicates that the student model is The feature representation extracted by the layer, Indicates that the teacher model The feature representation extracted by the layer, represents a natural sample, It represents the square of the Euclidean distance, also known as the L2 norm, which is used to measure the difference between two feature vectors.

[0122] Logical distillation loss is shown in Equation (10):

[0123] (10)

[0124] Where, represents the logical distillation loss, indicating that the teacher provides soft labels to guide students to learn output, represents the Kullback-Leibler divergence, which measures the difference between two distributions. represents the output of the teacher model, represents the distillation temperature coefficient, represents the output of the student model.

[0125] The loss function of the distillation process is shown in the following formula (11):

[0126] (11)

[0127] Where, represents the total loss function.

[0128] S7. Perform feature distillation and logic distillation on the student network model based on the compensation features, the output layer logic of the teacher network model, the gradient consistency dynamic adjustment mechanism, and the loss function to obtain a constructed student network model; obtain the drone image to be identified, input it into the constructed student network model, and obtain the target recognition result.

[0129] In an embodiment of the present invention, a method using compensatory feature adversarial distillation can address the inadequate performance of existing image adversarial defense technologies against drone reconnaissance images, thereby improving the recognition accuracy of drone reconnaissance images. A distillation model based on logic distillation and feature distillation is constructed. Feature distillation transfers the feature representations of the intermediate layers of a teacher model, allowing the student model to mimic the teacher model's internal feature space. Logic distillation guides the student model by transferring the logic of the teacher model's output layer, using the teacher model's predicted probability distribution of input data as a supervisory signal. A feature compensation module is established to compensate and fuse the natural sample features of the teacher network with the adversarial sample features of the student network, enabling the student network to learn not only the features of adversarial samples but also those of natural samples during training. A dynamic gradient consistency adjustment mechanism is designed to dynamically adjust the weights of logic distillation and feature distillation during the distillation process by calculating the consistency of the gradient at the current moment with the gradient at the previous moment. A distillation loss function is designed to calculate the difference between the model output and the actual output, and the magnitude of this loss function is used as the basis for iterative model optimization. The teacher model is trained using natural samples. Adversarial samples are generated using PGD, FGSM, and BIM adversarial sample generation methods based on these natural samples as a dataset for training the student model. The student model is then used to detect natural samples and adversarial samples.

[0130] Figure 5 This is a block diagram of a device for identifying targets in drone images based on compensatory feature adversarial distillation according to an exemplary embodiment. The device is used in a method for identifying targets in drone images based on compensatory feature adversarial distillation. Figure 5 The device includes a data acquisition module 310, a teacher network model training module 320, a student network model training module 330, a feature compensation fusion module 340, a construction module 350, a design module 360 and an output module 370.

[0131] The data acquisition module 310 is used to acquire a natural sample dataset and generate an adversarial sample dataset based on the natural sample dataset and the adversarial sample generation method.

[0132] The teacher network model training module 320 is used to train the teacher network model using a natural sample data set to obtain the natural sample features and output layer logic of the teacher network model.

[0133] The student network model training module 330 is used to train the student network model using the adversarial sample dataset to obtain the adversarial sample features of the student network model.

[0134] The feature compensation fusion module 340 is used to compensate and fuse the natural sample features and the adversarial sample features to obtain compensated features.

[0135] The construction module 350 is used to construct a gradient consistency dynamic adjustment mechanism, and allocate distillation weights to feature distillation and logic distillation in the distillation process according to the gradient consistency dynamic adjustment mechanism.

[0136] The design module 360 is used to design a loss function for the distillation process.

[0137] The output module 370 is used to perform feature distillation and logic distillation on the student network model based on the compensation features, the output layer logic of the teacher network model, the gradient consistency dynamic adjustment mechanism and the loss function to obtain a constructed student network model; obtain the drone image to be identified, input it into the constructed student network model, and obtain the target recognition result.

[0138] In an embodiment of the present invention, a method using compensatory feature adversarial distillation can address the inadequate performance of existing image adversarial defense technologies against drone reconnaissance images, thereby improving the recognition accuracy of drone reconnaissance images. A distillation model based on logic distillation and feature distillation is constructed. Feature distillation transfers the feature representations of the intermediate layers of a teacher model, allowing the student model to mimic the teacher model's internal feature space. Logic distillation guides the student model by transferring the logic of the teacher model's output layer, using the teacher model's predicted probability distribution of input data as a supervisory signal. A feature compensation module is established to compensate and fuse the natural sample features of the teacher network with the adversarial sample features of the student network, enabling the student network to learn not only the features of adversarial samples but also those of natural samples during training. A dynamic gradient consistency adjustment mechanism is designed to dynamically adjust the weights of logic distillation and feature distillation during the distillation process by calculating the consistency of the gradient at the current moment with the gradient at the previous moment. A distillation loss function is designed to calculate the difference between the model output and the actual output, and the magnitude of this loss function is used as the basis for iterative model optimization. The teacher model is trained using natural samples. Adversarial samples are generated using PGD, FGSM, and BIM adversarial sample generation methods based on these natural samples as a dataset for training the student model. The student model is then used to detect natural samples and adversarial samples.

[0139] Figure 6 FIG. 1 is a schematic diagram of a structure of a drone image target recognition device provided by an embodiment of the present invention. Figure 6 As shown, the drone image target recognition device may include the above Figure 5 The drone image target recognition device 410 shown is based on the compensation feature adversarial distillation. Optionally, the drone image target recognition device 410 may include a first processor 2001.

[0140] Optionally, the drone image target recognition device 410 may also include a memory 2002 and a transceiver 2003 .

[0141] The first processor 2001, the memory 2002 and the transceiver 2003 may be connected via a communication bus.

[0142] The following combination Figure 6 The components of the drone image target recognition device 410 are described in detail:

[0143] The first processor 2001 is the control center of the drone image target recognition device 410 and can be a single processor or a collective term for multiple processing elements. For example, the first processor 2001 can be one or more central processing units (CPUs), an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement embodiments of the present invention, such as one or more microprocessors (digital signal processors, DSPs) or one or more field programmable gate arrays (FPGAs).

[0144] Optionally, the first processor 2001 can perform various functions of the drone image target recognition device 410 by running or executing software programs stored in the memory 2002 and calling data stored in the memory 2002.

[0145] In a specific implementation, as an embodiment, the first processor 2001 may include one or more CPUs, such as Figure 6 CPU0 and CPU1 are shown in FIG.

[0146] In a specific implementation, as an embodiment, the drone image target recognition device 410 may also include multiple processors, such as Figure 6 1 and 2. The first processor 2001 and the second processor 2004 are shown in FIG. Each of these processors can be a single-core processor (single-CPU) or a multi-core processor (multi-CPU). A processor herein can refer to one or more devices, circuits, and / or processing cores for processing data (e.g., computer program instructions).

[0147] The memory 2002 is used to store the software program for executing the solution of the present invention, and is controlled by the first processor 2001 for execution. The specific implementation method can refer to the above method embodiment and will not be repeated here.

[0148] Alternatively, the memory 2002 may be a read-only memory (ROM) or other type of static storage device capable of storing static information and instructions, a random access memory (RAM) or other type of dynamic storage device capable of storing information and instructions, an electrically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disc storage, an optical disc storage (including a compact disc, laser disc, optical disc, digital versatile disc, Blu-ray disc, etc.), a magnetic disk storage medium or other magnetic storage device, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and capable of being accessed by a computer, but not limited thereto. The memory 2002 may be integrated with the first processor 2001 or may exist independently and be accessed through the interface circuit ( Figure 6 (not shown) is coupled to the first processor 2001, which is not specifically limited in this embodiment of the present invention.

[0149] The transceiver 2003 is used to communicate with a network device or a terminal device.

[0150] Optionally, the transceiver 2003 may include a receiver and a transmitter ( Figure 6 The receiver is used to implement a receiving function, and the transmitter is used to implement a sending function.

[0151] Optionally, the transceiver 2003 may be integrated with the first processor 2001 or may exist independently and communicate with the first processor 2001 through the interface circuit of the drone image target recognition device 410 ( Figure 6 (not shown) is coupled to the first processor 2001, which is not specifically limited in this embodiment of the present invention.

[0152] It should be noted that Figure 6 The structure of the drone image target recognition device 410 shown in the figure does not constitute a limitation on the router. The actual knowledge structure recognition device may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.

[0153] In addition, the technical effects of the drone image target recognition device 410 can refer to the technical effects of the drone image target recognition method based on compensatory feature adversarial distillation described in the above method embodiment, and will not be repeated here.

[0154] It should be understood that the first processor 2001 in the embodiment of the present invention may be a central processing unit (CPU), or may be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor, or the processor may be any conventional processor, etc.

[0155] It should also be understood that the memory in the embodiments of the present invention may be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. The non-volatile memory may be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory may be random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of random access memory (RAM) are available, such as static RAM (SRAM), dynamic random access memory (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), and direct rambus RAM (DR RAM).

[0156] The above embodiments can be implemented in whole or in part via software, hardware (e.g., circuits), firmware, or any other combination thereof. When implemented using software, the above embodiments can be implemented in whole or in part in the form of a computer program product. The computer program product comprises one or more computer instructions or computer programs. When loaded or executed on a computer, the processes or functions described in accordance with the embodiments of the present invention are fully or partially performed. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired means (e.g., infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium accessible by a computer or a data storage device such as a server or data center that contains a collection of one or more available media. The available medium can be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., DVDs), or semiconductor media. The semiconductor media can be a solid-state drive.

[0157] It should be understood that the term "and / or" as used herein simply describes a relationship between associated objects, indicating that three possible relationships exist. For example, "A and / or B" can represent: A alone, A and B together, or B alone. A and B can be singular or plural. Furthermore, the character " / " as used herein generally indicates an "or" relationship between the associated objects, but it may also indicate an "and / or" relationship. For specific understanding, please refer to the context.

[0158] In this disclosure, "at least one" means one or more, and "plurality" means two or more. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, "at least one of a, b, or c" can mean: a, b, c, ab, ac, bc, or abc, where a, b, and c can be single or plural.

[0159] It should be understood that in various embodiments of the present invention, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0160] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present invention.

[0161] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described equipment, devices and units can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0162] In the several embodiments provided by the present invention, it should be understood that the disclosed devices, apparatuses and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of the units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interface, indirect coupling or communication connection of the device or unit, which can be electrical, mechanical or other forms.

[0163] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0164] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0165] If the functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the portion that contributes to the prior art, or the portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in various embodiments of the present invention. The aforementioned storage media include various media that can store program code, such as USB flash drives, mobile hard drives, read-only memories (ROM), random access memories (RAM), magnetic disks, or optical disks.

[0166] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any modifications or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in the present invention should be included in the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be based on the scope of protection of the claims.

Claims

1. A method for target recognition in drone images based on compensatory feature adversarial distillation, characterized in that: The method comprises: S1. Obtain a natural sample dataset, and generate an adversarial sample dataset based on the natural sample dataset and an adversarial sample generation method; S2. Use the natural sample data set to train the teacher network model to obtain the natural sample features and output layer logic of the teacher network model; S3. Use the adversarial sample dataset to train the student network model to obtain adversarial sample features of the student network model; S4. Perform compensation fusion on the natural sample features and the adversarial sample features to obtain compensated features; S5. Construct a dynamic adjustment mechanism for gradient consistency, and assign distillation weights to feature distillation and logic distillation in the distillation process according to the dynamic adjustment mechanism for gradient consistency. S6. Design the loss function of the distillation process; S7. Perform feature distillation and logic distillation on the student network model according to the compensation features, the output layer logic of the teacher network model, the gradient consistency dynamic adjustment mechanism, and the loss function to obtain a constructed student network model; obtain the drone image to be identified, input it into the constructed student network model, and obtain the target recognition result.

2. The method for target recognition in drone images based on compensatory feature adversarial distillation according to claim 1 is characterized in that: The step of performing compensatory fusion on the natural sample features and the adversarial sample features to obtain compensated features in S4 includes: Subtracting the natural sample feature from the adversarial sample feature to obtain a difference feature; amplifying the difference feature, and normalizing the amplified feature to obtain a normalized feature; Multiplying the normalized features by the natural sample features element by element to obtain fused features; The fusion feature is fused with the adversarial sample feature to obtain a compensation feature.

3. The method for drone image target recognition based on compensatory feature adversarial distillation according to claim 1 is characterized in that: The step S5 constructs a dynamic adjustment mechanism for gradient consistency, and allocates distillation weights for feature distillation and logic distillation in the distillation process according to the dynamic adjustment mechanism for gradient consistency, including: Obtain the gradient direction of the current moment and the gradient direction of the previous moment of feature distillation and logical distillation, calculate the cosine similarity of the gradient direction of the current moment and the gradient direction of the previous moment of feature distillation and logical distillation respectively, and calculate the weight of the feature distillation and the weight of the logical distillation in the distillation process according to the cosine similarity of the feature distillation and the cosine similarity of the logical distillation.

4. The method for target recognition in drone images based on compensatory feature adversarial distillation according to claim 3 is characterized in that: The weight calculation method of the feature distillation is as follows (1): （1） Where, represents the weight of feature distillation, Indicates the iterations, Indicates the cosine similarity between the gradient direction of the current moment and the gradient direction of the previous moment of feature distillation, Indicates the cosine similarity between the gradient direction at the current moment and the gradient direction at the previous moment of logical distillation.

5. The method for target recognition in drone images based on compensatory feature adversarial distillation according to claim 1, characterized in that: The loss function of the distillation process in S6 includes: adversarial sample loss, feature distillation loss and logical distillation loss; The adversarial sample loss is as shown in the following formula (2): （2） Where, represents the adversarial sample loss, represents the cross entropy loss function, Indicates that the student model is in adversarial samples The predicted output on represents adversarial samples, represents the actual label; The characteristic distillation loss is shown in the following formula (3): （3） Where, represents the feature distillation loss, Represents the selected feature layer index set, Indicates that the student model is The feature representation extracted by the layer, Indicates that the teacher model The feature representation extracted by the layer, represents a natural sample, represents the square of the Euclidean distance; The logical distillation loss is shown in the following formula (4): （4） Where, represents the logistic distillation loss, represents the Kullback-Leibler divergence, represents the output of the teacher model, represents the distillation temperature coefficient, represents the output of the student model.

6. A device for identifying targets in drone images based on compensatory feature adversarial distillation, wherein the device is used to implement the method for identifying targets in drone images based on compensatory feature adversarial distillation according to any one of claims 1 to 5, characterized in that: The device comprises: A data acquisition module, configured to acquire a natural sample dataset and generate an adversarial sample dataset based on the natural sample dataset and an adversarial sample generation method; A teacher network model training module is used to train the teacher network model using the natural sample data set to obtain the natural sample features and output layer logic of the teacher network model; A student network model training module is used to train the student network model using the adversarial sample dataset to obtain adversarial sample features of the student network model; A feature compensation fusion module is used to compensate and fuse the natural sample features and the adversarial sample features to obtain compensated features; A construction module is used to construct a gradient consistency dynamic adjustment mechanism, and assign distillation weights to feature distillation and logic distillation in the distillation process according to the gradient consistency dynamic adjustment mechanism; Design module, used to design the loss function of the distillation process; The output module is used to perform feature distillation and logic distillation on the student network model according to the compensation features, the output layer logic of the teacher network model, the gradient consistency dynamic adjustment mechanism and the loss function to obtain a constructed student network model; obtain the drone image to be identified, input it into the constructed student network model, and obtain the target recognition result.

7. The device for drone image target recognition based on compensatory feature adversarial distillation according to claim 6 is characterized in that: The compensating and fusing the natural sample features and the adversarial sample features to obtain compensated features includes: Subtracting the natural sample feature from the adversarial sample feature to obtain a difference feature; amplifying the difference feature, and normalizing the amplified feature to obtain a normalized feature; Multiplying the normalized features by the natural sample features element by element to obtain fused features; The fusion feature is fused with the adversarial sample feature to obtain a compensation feature.

8. The device for drone image target recognition based on compensatory feature adversarial distillation according to claim 6 is characterized in that: The step of constructing a dynamic adjustment mechanism for gradient consistency and allocating distillation weights for feature distillation and logic distillation in a distillation process according to the dynamic adjustment mechanism for gradient consistency includes: Obtain the gradient direction of the current moment and the gradient direction of the previous moment of feature distillation and logical distillation, calculate the cosine similarity of the gradient direction of the current moment and the gradient direction of the previous moment of feature distillation and logical distillation respectively, and calculate the weight of the feature distillation and the weight of the logical distillation in the distillation process according to the cosine similarity of the feature distillation and the cosine similarity of the logical distillation.

9. A drone image target recognition device, characterized in that: The drone image target recognition device includes: processor; A memory having computer-readable instructions stored thereon, wherein when the computer-readable instructions are executed by the processor, the method according to any one of claims 1 to 5 is implemented.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores program code, which can be called by a processor to execute the method according to any one of claims 1 to 5.

Images