Adversarial training method and device based on image reconstruction and feature distance optimization

Abstract

The invention relates to an adversarial training method and device based on image reconstruction and feature distance optimization, and the method comprises the steps: obtaining a reconstruction sample set through an SMUNet reconstruction model, and carrying out the basic adversarial training of a to-be-adversarial training model through all samples till convergence, and obtaining a basic adversarial training model; then obtaining features of each category based on the basic adversarial training model, and then constructing a plurality of under-fitting subsets based on feature distances among different categories to train the basic adversarial training model to obtain a plurality of enhanced adversarial training models; and then performing fine tuning training on each enhanced adversarial training model by using all samples to obtain a plurality of fine-tuned adversarial training models, finally performing resistance testing on the plurality of fine-tuned adversarial training models, and selecting the fine-tuned adversarial training model with the best robustness as a target adversarial training model. According to the method, the neural network model for the remote sensing scene classification task can be effectively trained, and the robustness and generalization ability of the model in the face of adversarial attacks are improved.

Description

Technical Field

[0001] The present invention relates to the field of deep learning technology, and in particular to an adversarial training method and device based on image reconstruction and feature distance optimization. Background Art

[0002] Deep neural networks have demonstrated remarkable performance in a variety of applications, including image processing, speech recognition, and natural language processing. However, deep neural networks are very vulnerable to adversarial examples, which means that introducing almost imperceptible perturbations in the input data can cause the model to output incorrect results with high confidence. This vulnerability not only exposes the security risks of deep learning models, but also severely limits their application in security-sensitive fields.

[0003] Adversarial training is currently one of the most effective defenses against adversarial attacks. Adversarial training is a method that intentionally incorporates adversarial samples into the training process to enhance the resistance of deep neural networks to malicious inputs. This method simulates potential attack scenarios by adding small but carefully designed perturbations to the training set, thereby helping the network learn more robust feature representations. This training strategy not only improves the model's ability to defend against unknown attacks, but also enhances the overall security of the system, making it more reliable and trustworthy in practical applications.

[0004] However, for the neural network model used for remote sensing scene classification tasks that is still vulnerable to adversarial samples, if adversarial training is directly applied to the adversarial defense of remote sensing scene classification without fully considering the characteristics of remote sensing scene classification data such as significant scale changes, high similarity between classes, and large differences within classes, the adversarial training effect will be poor. The neural network model used for remote sensing scene classification tasks after the adversarial training is still insufficiently resistant to adversarial samples and has poor robustness. Summary of the invention

[0005] Based on this, it is necessary to provide an adversarial training method and device based on image reconstruction and feature distance optimization to address the above-mentioned technical problems, which can effectively train the neural network model used for remote sensing scene classification tasks and improve the robustness and generalization ability of the model when facing adversarial attacks.

[0006] In a first aspect, the present invention provides an adversarial training method based on image reconstruction and feature distance optimization, comprising the following steps:

[0007] The clean sample set and the adversarial sample set are respectively input into the trained SMUNet reconstruction model to obtain the reconstructed clean sample set and the reconstructed adversarial sample set, wherein the SMUNet reconstruction model is constructed based on the Swin-Transformer subunit and the multi-scale convolution subunit, and the clean sample set contains sample images of multiple different categories;

[0008] Use the clean sample set, adversarial sample set, reconstructed clean sample set and reconstructed adversarial sample set to perform basic adversarial training on the adversarial training model until convergence to obtain the basic adversarial training model;

[0009] Obtain the features of each category based on the basic adversarial training model;

[0010] Calculate the feature distance between any two different categories, and calculate the underfitting threshold based on the mean and standard deviation of all feature distances. The two categories whose feature distance is less than the underfitting threshold are regarded as underfitting pairs.

[0011] Arrange all underfitting pairs from large to small according to the degree of underfitting, and take the clean samples, adversarial samples, reconstructed clean samples and reconstructed adversarial samples corresponding to the first 1, 2, ..., n underfitting pairs in turn to form n underfitting subsets respectively;

[0012] Use n underfitting subsets to train the basic adversarial training model until convergence, and obtain n enhanced adversarial training models. Then use all clean samples, adversarial samples, reconstructed clean samples and reconstructed adversarial samples to fine-tune each enhanced adversarial training model to obtain n fine-tuned adversarial training models.

[0013] Perform resistance tests on n fine-tuned adversarial training models, and select the best robust fine-tuned adversarial training model as the target adversarial training model.

[0014] In one of the embodiments, the adversarial sample set is generated from a clean sample set based on an adversarial attack algorithm for a model to be trained against;

[0015] The categories of samples in the clean sample set, adversarial sample set, reconstructed clean sample set and reconstructed adversarial sample set correspond one to one.

[0016] In one embodiment, the SMUNet reconstruction model includes, from the image input end to the output end, a first 3×3 convolution module, a first SM-convolution module, a second SM-convolution module, a third SM-convolution module, two SM modules, a first SM-deconvolution module, a second SM-deconvolution module, a third SM-deconvolution module, and a second 3×3 convolution module;

[0017] The first SM-convolution module and the first SM-deconvolution module are jump-connected, the second SM-convolution module and the second SM-deconvolution module are jump-connected, and the third SM-convolution module and the third SM-deconvolution module are jump-connected;

[0018] The first SM-convolution module, the second SM-convolution module and the third SM-convolution module are each composed of two SM modules and one 2×2 convolution module, and the first SM-deconvolution module, the second SM-deconvolution module and the third SM-deconvolution module are each composed of two SM modules and one 2×2 deconvolution module;

[0019] The SM module consists of two sub-modules connected in series. Each sub-module includes a 1×1 convolution unit, a parallel unit and a 1×1 convolution unit from the input to the output. The parallel unit is composed of a parallel Swin-Transformer sub-unit and a multi-scale convolution sub-unit.

[0020] In one embodiment, the training steps of the SMUNet reconstruction model include:

[0021] Input the adversarial sample set into the SMUNet reconstruction model to obtain the reconstructed adversarial sample set;

[0022] Use the pre-trained feature extractor to extract all clean sample features in the clean sample set and all reconstructed adversarial sample features in the reconstructed adversarial sample set;

[0023] Based on all clean samples, all reconstructed adversarial samples, all clean sample features and all reconstructed adversarial sample features, the total loss of the SMUNet reconstruction model is calculated and the SMUNet reconstruction model parameters are updated through back propagation of the total loss until the SMUNet reconstruction model converges to obtain a trained SMUNet reconstruction model.

[0024] In one embodiment, obtaining features of each category based on the basic adversarial training model includes the following steps:

[0025] Input clean samples, adversarial samples, reconstructed clean samples, and reconstructed adversarial samples belonging to the same category into the basic adversarial training model in sequence;

[0026] After each input sample, the output feature map after pooling of the last convolutional layer of the basic adversarial training model is converted into a vector;

[0027] The average of the vectors corresponding to the four samples of the same category is taken as the feature of the current category.

[0028] In one embodiment, the total loss of the SMUNet reconstruction model is calculated as follows:

[0029]

[0030]

[0031]

[0032] In the formula, ||·||1 represents the 1-norm, It is visual loss, is the feature loss, is the total loss of the SMUNet reconstruction model, x represents a clean sample, and x′ rec represents the reconstructed adversarial sample, φ(x) represents the clean sample feature, φ(x′ rec ) represents the reconstructed adversarial sample features.

[0033] In one embodiment, the loss function of the basic adversarial training is

[0034]

[0035] In the formula, is the cross entropy loss, x is a clean sample, x′ is an adversarial sample, and x rec is the reconstructed clean sample, x′ rec is the reconstructed adversarial sample, θ represents the parameters of the adversarial training model, and y represents the label of the scene category.

[0036] In one embodiment, the calculation formula for calculating the underfitting threshold based on the mean and standard deviation of all feature distances is:

[0037] l=L mean -2L std

[0038] In the formula, l is the underfitting threshold, L mean is the mean of all feature distances, L std is the standard deviation of all feature distances.

[0039] In a second aspect, the present invention also provides an adversarial training device based on image reconstruction and feature distance optimization, comprising a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the steps of the adversarial training method based on image reconstruction and feature distance optimization described in the claim are implemented.

[0040] The beneficial effects of the present invention are:

[0041] (1) The present invention adopts the SMUNet reconstruction model to reconstruct clean samples and adversarial samples. The SMUNet reconstruction model is built based on the Swin-Transformer subunit and the multi-scale convolution subunit. The Swin-Transformer subunit and the multi-scale convolution subunit can jointly extract global and local features, thereby enhancing the understanding of scenes and details, so as to better eliminate the disturbances in adversarial samples.

[0042] (2) In the present invention, the underfitting subset is constructed based on the underfitting pair, and the underfitting pair is two categories whose feature distance is less than the underfitting threshold, which is used to optimize the inter-class feature distance. The underfitting subset is used to train the basic adversarial training model, which can enhance the learning of the basic adversarial training model on the underfitting content, so that the intra-class feature distribution is more concentrated and the inter-class feature distribution is more dispersed.

[0043] (3) The present invention combines the SMUNet reconstruction model and the feature distance optimization strategy to effectively train the adversarial training model for remote sensing scene classification tasks, which can improve the robustness and generalization ability of the model when facing adversarial attacks. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 This is one of the flowcharts of the adversarial training method based on image reconstruction and feature distance optimization provided by an embodiment of the present invention;

[0045] Figure 2 It is a structural schematic diagram of the SMUNet reconstruction model provided by an embodiment of the present invention;

[0046] Figure 3 It is one of the flow charts of the training steps of the SMUNet reconstruction model provided by an embodiment of the present invention;

[0047] Figure 4 This is one of the flowcharts of obtaining features of each category based on the basic adversarial training model provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0048] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0049] In one embodiment, Figure 1 As shown, Figure 1 This is one of the flow charts of the adversarial training method based on image reconstruction and feature distance optimization provided by an embodiment of the present invention. The method is applied to a computer device, comprising the following steps:

[0050] S101. Input the clean sample set and the adversarial sample set into the trained SMUNet reconstruction model respectively to obtain a reconstructed clean sample set and a reconstructed adversarial sample set, wherein the SMUNet reconstruction model is constructed based on a Swin-Transformer subunit and a multi-scale convolution subunit, and the clean sample set contains sample images of multiple different categories.

[0051] In this embodiment, the clean sample set includes at least three different categories of sample images. Different categories represent sample images of different scenes, such as airport scenes or port scenes.

[0052] Specifically, the adversarial sample set is generated from the clean sample set based on the adversarial attack algorithm for the training model to be adversarial; the categories of samples in the clean sample set, the adversarial sample set, the reconstructed clean sample set and the reconstructed adversarial sample set correspond one to one.

[0053] One-to-one correspondence means that the number of samples in the clean sample set, adversarial sample set, reconstructed clean sample set and reconstructed adversarial sample set is equal and the number of scene categories is also equal.

[0054] like Figure 2 As shown, Figure 2 It is a structural diagram of the SMUNet reconstruction model provided by an embodiment of the present invention. The SMUNet reconstruction model includes a first 3×3 convolution module, a first SM-convolution module, a second SM-convolution module, a third SM-convolution module, two SM modules, a first SM-deconvolution module, a second SM-deconvolution module, a third SM-deconvolution module and a second 3×3 convolution module in sequence from the image input end to the output end. The first SM-convolution module and the first SM-deconvolution module are jump-connected, the second SM-convolution module and the second SM-deconvolution module are jump-connected, and the third SM-convolution module and the third SM-deconvolution module are jump-connected.

[0055] Among them, the first 3×3 convolution module, the first SM-convolution module, the second SM-convolution module, and the third SM-convolution module on the left constitute an encoder, and the first SM-deconvolution module, the second SM-deconvolution module, the third SM-deconvolution module and the second 3×3 convolution module on the right constitute a decoder. The jump connection between the encoder and the decoder can help the model better restore detail information.

[0056] The first SM-convolution module, the second SM-convolution module and the third SM-convolution module are each composed of two SM modules and one 2×2 convolution module, and the first SM-deconvolution module, the second SM-deconvolution module and the third SM-deconvolution module are each composed of two SM modules and one 2×2 deconvolution module;

[0057] The SM module consists of two sub-modules connected in series. Each sub-module includes a 1×1 convolution unit, a parallel unit and a 1×1 convolution unit from the input to the output. The parallel unit is composed of a parallel Swin-Transformer sub-unit and a multi-scale convolution sub-unit.

[0058] The Swin-Transformer subunit and the multi-scale convolution subunit can jointly extract global and local features to enhance the understanding of scenes and details to better eliminate perturbations in adversarial samples.

[0059] Specifically, the multi-scale convolution subunit in this embodiment includes convolutions of four scales, the convolution kernel sizes are 1×1, 3×3, 5×5 and 7×7, and the padding of the four scales are 0, 1, 2 and 3, respectively.

[0060] S102, using the clean sample set, the adversarial sample set, the reconstructed clean sample set and the reconstructed adversarial sample set to perform basic adversarial training on the adversarial training model until convergence to obtain a basic adversarial training model. The basic adversarial training model converges when the loss function value of the basic adversarial training is stable at a low level. For example, the basic adversarial training model converges when the loss function value of the basic adversarial training is less than 0.01.

[0061] Specifically, the loss function of basic adversarial training is

[0062]

[0063] In the formula, is the cross entropy loss, x is a clean sample, x′ is an adversarial sample, and x rec is the reconstructed clean sample, x′ rec is the reconstructed adversarial sample, θ represents the parameters of the adversarial training model, and y represents the label of the scene category.

[0064] S103. Obtain features of each category based on the basic adversarial training model.

[0065] S104, calculating the feature distance between any two different categories, and calculating the under-fitting threshold according to the mean and standard deviation of all feature distances, and taking two categories whose feature distance is less than the under-fitting threshold as under-fitting pairs.

[0066] For example, there are three different categories of sample images in the clean sample set, namely category a, b and c. Calculating the feature distance between any two different categories is to calculate the feature distance between category pairs ab, bc and ac.

[0067] Specifically, the formula for calculating the underfitting threshold based on the mean and standard deviation of all feature distances is:

[0068] l=L mean -2L std

[0069] In the formula, l is the underfitting threshold, L mean is the mean of all feature distances, L std is the standard deviation of all feature distances.

[0070] S105. Arrange all underfitting pairs from large to small according to the degree of underfitting, and take the clean samples, adversarial samples, reconstructed clean samples and reconstructed adversarial samples of the categories corresponding to the first 1, 2, ..., n underfitting pairs in turn to form n underfitting subsets respectively.

[0071] Exemplarily, the under-fitting pairs are sorted as ab, bc and cd, then the first under-fitting subset takes the under-fitting pair ab, that is, all clean samples, adversarial samples, reconstructed clean samples and reconstructed adversarial samples corresponding to categories a and b, the second under-fitting subset takes the under-fitting pairs ab and bc, that is, all clean samples, adversarial samples, reconstructed clean samples and reconstructed adversarial samples corresponding to categories a, b and c, and the third under-fitting subset takes all clean samples, adversarial samples, reconstructed clean samples and reconstructed adversarial samples corresponding to categories a, b, c and d.

[0072] S106. Use n underfitting subsets to train the basic adversarial training model until convergence, and obtain n enhanced adversarial training models. Then use all clean samples, adversarial samples, reconstructed clean samples and reconstructed adversarial samples to fine-tune each enhanced adversarial training model to obtain n fine-tuned adversarial training models.

[0073] Among them, the underfitting pair is two categories whose feature distance is less than the underfitting threshold, which is used to optimize the inter-class feature distance. Using the underfitting subset to train the basic adversarial training model can enhance the learning of the basic adversarial training model on the underfitting content, so that the intra-class feature distribution is more concentrated and the inter-class feature distribution is more dispersed.

[0074] S107. Perform a resistance test on the n fine-tuned adversarial training models, and select the fine-tuned adversarial training model with the best robustness as the target adversarial training model.

[0075] The adversarial training method based on image reconstruction and feature distance optimization in this embodiment, combined with the SMUNet reconstruction model and the feature distance optimization strategy, can effectively train the adversarial training model for remote sensing scene classification tasks, which can improve the robustness and generalization ability of the model when facing adversarial attacks.

[0076] In one embodiment, Figure 3 As shown, Figure 3 It is one of the flow charts of the training steps of the SMUNet reconstruction model provided by an embodiment of the present invention. The training steps of the SMUNet reconstruction model include:

[0077] S301. Input the adversarial sample set into the SMUNet reconstruction model to obtain a reconstructed adversarial sample set.

[0078] S302: Use a pre-trained feature extractor to extract all clean sample features in the clean sample set and all reconstructed adversarial sample features in the reconstructed adversarial sample set.

[0079] S303, based on all clean samples, all reconstructed adversarial samples, all clean sample features and all reconstructed adversarial sample features, calculate the total loss of the SMUNet reconstruction model and update the SMUNet reconstruction model parameters through back propagation of the total loss until the SMUNet reconstruction model converges to obtain a trained SMUNet reconstruction model.

[0080] Specifically, the total loss of the SMUNet reconstruction model is calculated as follows:

[0081]

[0082]

[0083]

[0084] In the formula, ||·||1 represents the 1-norm, It is visual loss, is the feature loss, is the total loss of the SMUNet reconstruction model, x represents a clean sample, and x′ rec represents the reconstructed adversarial sample, φ(x) represents the clean sample feature, φ(x′ rec ) represents the reconstructed adversarial sample features.

[0085] Specifically, when the total loss of the SMUNet reconstruction model gradually decreases with the increase of the number of iterations and finally stabilizes at a lower level, the model is considered to have converged. For example, when the total loss is less than 0.01, the model converges.

[0086] In one embodiment, if Figure 4 As shown, Figure 4 This is one of the flow charts of obtaining the features of each category based on the basic adversarial training model provided by an embodiment of the present invention. Obtaining the features of each category based on the basic adversarial training model includes the following steps:

[0087] S401. Input clean samples, adversarial samples, reconstructed clean samples and reconstructed adversarial samples belonging to the same category into the basic adversarial training model in sequence.

[0088] S402, converting the output feature map after pooling of the last convolutional layer of the basic adversarial training model after each input sample into a vector.

[0089] S403: Take the average value of the vectors corresponding to the four samples of the same category as the feature of the current category.

[0090] In a specific embodiment, the adversarial training method based on image reconstruction and feature distance optimization of the present invention is described by applying the present invention to a ResNet152 model for AID data classification.

[0091] Among them, the AID clean sample set D consists of k category data (D1, D2, ..., D k ),k=30.

[0092] The specific steps include:

[0093] Step 1: Generate an adversarial sample set D′=(D1′, D2′, ..., D k ′), where clean samples correspond to adversarial samples one by one, and the dimension is

[0094] Step 2: Input the adversarial sample x′ in the adversarial sample set D′ into the SMUNet reconstruction model to obtain the reconstructed sample x′ of the adversarial sample rec , and compare the clean sample x with the reconstructed sample x′ of the adversarial sample rec Input feature extractor φ to obtain the respective features φ(x) and φ(x′ rec ), the feature dimension is Clean sample x and adversarial reconstruction sample x′ rec The 1-norm of the difference is used as the visual loss The feature φ(x) of the clean sample and the feature φ(x′) of the adversarial reconstruction sample rec ) The 1 norm of the difference is used as the feature loss The total loss is The gradient is calculated by back-propagation, and the parameters of the SMUNet reconstruction model are updated until the SMUNet reconstruction model converges.

[0095] Step 3: Input the clean sample set D and the adversarial sample set D′ into the SMUNet reconstruction model to obtain the reconstructed clean sample set D rec =(D rec_1 ,D rec_2 ,...,D rec_k ) and reconstruct the adversarial sample set D′ rec =(D′ rec_1 ,D′ rec_2 ,...,D′ rec_k ), using D, D′, D rec and D′ rec The original model M is trained with basic adversarial training until convergence, that is, the loss function value is stabilized at a low level, and the basic adversarial training model M is obtained. adv .

[0096] Step 4: Obtain the features of each category based on the basic adversarial training model, where the calculation process of the i-th category sample is to convert the i-th category sample (D i ,D′ i ,D rec_i ,D′ rec_i ) The four types of data are sequentially input into the basic adversarial training model M adv , take the basic adversarial training model M adv The output feature map after the last convolutional layer pooling has a dimension of Convert it into a 2048-dimensional vector and take the average of the features of all samples in this class as the feature F of category i i , represented as a 2048-dimensional vector.

[0097] Step 5: Calculate feature F i With F j The distance d between (i<j) i-j , the Euclidean distance metric is used. The smaller the feature distance, the greater the degree of underfitting. The underfitting threshold l is determined by the mean and standard deviation of the feature distance. The feature distance d i-j The n category combinations that are less than the underfitting threshold l are recorded as underfitting pairs (ij). The underfitting pairs are arranged from the largest to the smallest underfitting degree, and the data of the categories corresponding to the first 1, 2, ..., n underfitting pairs are taken in turn to form n underfitting subsets S1, S2, ..., S n .

[0098] Step 6: Use n underfitting subsets S1, S2, ..., S n To M adv Train until convergence, that is, the loss function value tends to be stable, and obtain M i adv (i=1,2,...,n), and then use all data D, D′, D rec and D′ rec To M i adv (i=1,2,...,n) to obtain the fine-tuned adversarial training model M1,M2,...,M n .

[0099] Step 7: For M1,M2,...,M n Perform a resistance test and select the best fine-tuned adversarial training model as the target adversarial training model M best .

[0100] Based on the same invention, the present invention also provides an adversarial training device based on image reconstruction and feature distance optimization, including a memory and a processor, the memory stores a computer program, and when the processor executes the computer program, it implements the steps of the adversarial training method based on image reconstruction and feature distance optimization in any one of the above embodiments of the claim.

[0101] The device can combine the SMUNet reconstruction model and the feature distance optimization strategy to effectively train the adversarial training model for remote sensing scene classification tasks, which can improve the robustness and generalization ability of the model when facing adversarial attacks.

[0102] The above-mentioned embodiments only express several implementation methods of the present invention, and the description thereof is relatively specific and detailed, but it cannot be understood as limiting the scope of the present invention. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present invention, and these all belong to the protection scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the attached claims.

Claims

1. An adversarial training method based on image reconstruction and feature distance optimization, characterized in that: The following steps are involved: Inputting the clean sample set and the adversarial sample set into the trained SMUNet reconstruction model respectively to obtain a reconstructed clean sample set and a reconstructed adversarial sample set, wherein the SMUNet reconstruction model is constructed based on a Swin-Transformer subunit and a multi-scale convolution subunit, and the clean sample set contains sample images of multiple different categories; Use the clean sample set, adversarial sample set, reconstructed clean sample set and reconstructed adversarial sample set to perform basic adversarial training on the adversarial training model until convergence to obtain the basic adversarial training model; Obtain the features of each category based on the basic adversarial training model; Calculate the feature distance between any two different categories, and calculate the underfitting threshold based on the mean and standard deviation of all feature distances. The two categories whose feature distance is less than the underfitting threshold are regarded as underfitting pairs. Arrange all underfitting pairs from large to small according to the degree of underfitting, and take the clean samples, adversarial samples, reconstructed clean samples and reconstructed adversarial samples corresponding to the first 1, 2, ..., n underfitting pairs in turn to form n underfitting subsets respectively; Use n underfitting subsets to train the basic adversarial training model until convergence, and obtain n enhanced adversarial training models. Then use all clean samples, adversarial samples, reconstructed clean samples and reconstructed adversarial samples to fine-tune each enhanced adversarial training model to obtain n fine-tuned adversarial training models. Perform resistance tests on n fine-tuned adversarial training models, and select the best robust fine-tuned adversarial training model as the target adversarial training model.

2. The adversarial training method based on image reconstruction and feature distance optimization according to claim 1, characterized in that: The adversarial sample set is generated from a clean sample set based on an adversarial attack algorithm for a model to be trained against; The categories of samples in the clean sample set, adversarial sample set, reconstructed clean sample set and reconstructed adversarial sample set correspond one to one.

3. The adversarial training method based on image reconstruction and feature distance optimization according to claim 1, characterized in that: The SMUNet reconstruction model includes, from the image input end to the output end, a first 3×3 convolution module, a first SM-convolution module, a second SM-convolution module, a third SM-convolution module, two SM modules, a first SM-deconvolution module, a second SM-deconvolution module, a third SM-deconvolution module and a second 3×3 convolution module; The first SM-convolution module and the first SM-deconvolution module are jump-connected, the second SM-convolution module and the second SM-deconvolution module are jump-connected, and the third SM-convolution module and the third SM-deconvolution module are jump-connected; The first SM-convolution module, the second SM-convolution module and the third SM-convolution module are each composed of two SM modules and one 2×2 convolution module, and the first SM-deconvolution module, the second SM-deconvolution module and the third SM-deconvolution module are each composed of two SM modules and one 2×2 deconvolution module; The SM module is composed of two sub-modules connected in series, each of which includes a 1×1 convolution unit, a parallel unit and a 1×1 convolution unit from the input end to the output end, and the parallel unit is composed of a parallel Swin-Transformer sub-unit and a multi-scale convolution sub-unit.

4. The adversarial training method based on image reconstruction and feature distance optimization according to claim 3, characterized in that: The training steps of the SMUNet reconstruction model include: Inputting the adversarial sample set into the SMUNet reconstruction model to obtain a reconstructed adversarial sample set; Use the pre-trained feature extractor to extract all clean sample features in the clean sample set and all reconstructed adversarial sample features in the reconstructed adversarial sample set; Based on all clean samples, all reconstructed adversarial samples, all clean sample features and all reconstructed adversarial sample features, the total loss of the SMUNet reconstruction model is calculated and the SMUNet reconstruction model parameters are updated through back propagation of the total loss until the SMUNet reconstruction model converges to obtain a trained SMUNet reconstruction model.

5. The adversarial training method based on image reconstruction and feature distance optimization according to claim 1, characterized in that: The method of obtaining the features of each category based on the basic adversarial training model includes the following steps: Input clean samples, adversarial samples, reconstructed clean samples, and reconstructed adversarial samples belonging to the same category into the basic adversarial training model in sequence; After each input sample, the output feature map after pooling of the last convolutional layer of the basic adversarial training model is converted into a vector; The average of the vectors corresponding to the four samples of the same category is taken as the feature of the current category.

6. The adversarial training method based on image reconstruction and feature distance optimization according to claim 4, characterized in that: The total loss of the SMUNet reconstruction model is calculated as follows: In the formula, ||·||1 represents the 1-norm, Indicates visual loss, represents the feature loss, represents the total loss of the SMUNet reconstruction model, x represents the clean sample, and x r ' ec represents the reconstructed adversarial sample, φ(x) represents the clean sample feature, φ(x) r ' ec ) represents the reconstructed adversarial sample features.

7. The adversarial training method based on image reconstruction and feature distance optimization according to claim 1, characterized in that: The loss function of the basic adversarial training is In the formula, represents the cross entropy loss, x represents the clean sample, x′ represents the adversarial sample, and x rec Represents the reconstruction of clean samples, x r ' ec Represents the reconstructed adversarial sample, θ represents the parameters of the adversarial training model, and y represents the label of the scene category.

8. The adversarial training method based on image reconstruction and feature distance optimization according to claim 1, characterized in that: The formula for calculating the underfitting threshold based on the mean and standard deviation of all feature distances is: l=L mean -2L std Where l is the underfitting threshold, L mean is the mean of all feature distances, L std is the standard deviation of all feature distances.

9. An adversarial training device based on image reconstruction and feature distance optimization, comprising a memory and a processor, wherein the memory stores a computer program, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 8 are implemented.

Images