An image recognition anti-attack method and device based on double-flow mask

Abstract

The application discloses an image recognition and confrontation defense method based on a double-flow mask, and comprises the following steps: using a test image and a preset classification model, a preset attack method is used to generate an augmented first confrontation image, the first confrontation image is input into the preset classification model for confrontation training to obtain a first classification model; the first classification model is divided into a preset number of parts according to the size of an output feature map, and a double-flow mask module initialized with parameters is inserted at the end of each part to obtain a second classification model; using the test image and the second classification model, an augmented second confrontation image is generated, the second classification model is used to separate the feature map of the corresponding part into robust and non-robust features by using each double-flow mask module, and random masking is performed to obtain double-flow mask features; and the obtained double-flow mask features are used to train the second classification model to obtain a target classification model. The application can have good performance on invisible attack robustness and expected attack, considers multi-level defense capability, and can improve the confrontation defense capability of the model.

Description

Technical Field

[0001] This invention belongs to the field of machine learning and adversarial defense, specifically relating to an image recognition adversarial defense method and apparatus based on dual-stream masking. Background Technology

[0002] In recent years, machine learning security issues have received widespread attention, and more and more researchers have begun to explore effective adversarial defense methods. Existing adversarial defense methods mainly include adversarial training, preprocessing defense, gradient masking, and so on.

[0003] Adversarial training is a method to enhance the adversarial robustness of a model by using adversarial examples. It formulates the training process as a mini-maximum optimization problem, aiming to resist adversarial perturbations. Goodfellow et al. first proposed incorporating adversarial examples generated by FSGM (Fast Gradient Sign Method) into the training phase, but this method is vulnerable to iterative attacks. To overcome this drawback, Madry et al. proposed an adversarial training method based on PGD (Projection Gradient Descent), which can withstand stronger adversarial attacks. Furthermore, the training strategy also affects the effectiveness of adversarial training. For example, Zhang et al. proposed an adversarial training defense method (TRadeoff-inspired Adversarial DEfense via Surrogate-loss minimization, TREADS) based on the trade-off between natural accuracy and adversarial robustness. Wang et al. studied the impact of misclassified and correctly classified examples on the final robustness of adversarial training and proposed a misclassification-aware adversarial training (MART) algorithm. Huang et al. designed an Adaptive Adversarial Distillation (AdaAD) method to facilitate interaction between teacher and student models. Furthermore, data augmentation is also an effective means of improving robustness. Lin et al. combined data augmentation algorithms with adversarial training, further improving the robustness and accuracy of the model. Tack et al. found that data augmentation can alleviate the overfitting problem in adversarial training. They proposed a regularization method that requires the prediction distributions of two different augmentations attacking the same instance to be consistent.

[0004] Preprocessing defense is a defensive method that transforms the input layer data to reduce the impact of adversarial perturbations. Guesmi et al. proposed a Stochastic Input Transformation (SIT) method, which disrupts the structure of adversarial perturbations by randomly rotating, cropping, and scaling the input data. Raff et al. merged several weak defense methods into a single stochastic transformation defense method to increase the diversity of stochastic defenses. Xu et al. proposed a novel masking-based adversarial defense scheme, which reduces the effectiveness of adversarial perturbations by masking the input data. Sitawarin et al. proposed an attack method to evaluate stochastic transformation defenses. They also used novel attacks to adversarially train the defense network, achieving significant robustness gains. Blau et al. introduced a method to enhance classifier robustness through Test-Time Transformation (TETRA). This novel defense method utilizes perceptually aligned gradients (PAG) to improve the performance of the trained robust classifier. Xiang et al. performed two rounds of pixel masking on the input image to counteract the impact of adversarial patches.

[0005] Existing research commonly employs adversarial training (AT) as a defense method. It uses generated adversarial examples as data augmentation during model training. These methods can learn the features of adversarial examples, enhancing the model's robustness. However, this approach also has limitations. For example, manually selected adversarial examples may only cover a limited range of possible attack scenarios, resulting in poor generalization ability when facing unseen attacks. Furthermore, while methods based on simple preprocessing defenses possess some generalization ability, they are vulnerable to advanced attacks such as Expectation Over Transformation Projective Gradient Descent (EOTPGD). Additionally, these methods do not consider the defensive capabilities of different network layers.

[0006] In summary, existing adversarial defense methods do not simultaneously consider the generalization to unknown or anticipated attacks, and they do not implement multi-layered defense for the network, resulting in unsatisfactory adversarial defense effects. Summary of the Invention

[0007] To address the aforementioned problems in the prior art, this invention provides an image recognition adversarial defense method, apparatus, electronic device, and storage medium based on dual-stream masking. The technical problem to be solved by this invention is achieved through the following technical solution:

[0008] In a first aspect, embodiments of the present invention provide an image recognition adversarial defense method based on dual-stream masking, the method comprising:

[0009] Using the acquired test images with classification labels and a preset classification model, an augmented first adversarial image is generated using a preset attack method;

[0010] The preset classification model is trained adversarially using the first adversarial image to obtain a first classification model with a certain degree of robustness after training.

[0011] Based on the size of the output feature map, the first classification model is divided into a preset number of parts, and a parameter-initialized dual-stream mask module is inserted at the end of the structure of each part to obtain the second classification model;

[0012] Using the test image and the second classification model, an augmented second adversarial image is generated using the preset attack method;

[0013] The second adversarial image is input into the second classification model. The feature maps of the corresponding parts are separated into robust features and non-robust features using each dual-stream masking module. The two are then randomly masked to obtain the corresponding dual-stream mask features. The obtained dual-stream mask features are used to train the second classification model to obtain the trained target classification model, thereby achieving adversarial defense in the image classification process.

[0014] In one embodiment of the present invention, the preset classification model includes: the ResNet18 model.

[0015] In one embodiment of the present invention, the preset attack method includes: a projection gradient descent attack method.

[0016] In one embodiment of the present invention, the preset quantity is 4.

[0017] In one embodiment of the present invention, the formula used in the process of using the first adversarial image to perform adversarial training on the preset classification model to obtain a trained first classification model with a certain degree of robustness includes:

[0018]

[0019] Among them, L cls F represents the loss function; θLet represent the first classification model with parameter θ; x+δ represents the first adversarial image corresponding to the test image x; δ represents the adversarial noise corresponding to the first adversarial image; y represents the classification label of the test image; E (x,y)～D represents the expectation for the current dataset; D represents the training set containing the training images.

[0020] In one embodiment of the present invention, the second adversarial image is input into the second classification model, and the feature maps of corresponding parts are separated into robust features and non-robust features using each dual-stream masking module. The two are then randomly masked to obtain the corresponding dual-stream mask features, including:

[0021] The second adversarial image is input into the second classification model, and the feature separation network in each dual-stream masking module is used to separate the feature map of the corresponding part into robust features and non-robust features.

[0022] For each dual-stream masking module, the robust and non-robust features obtained from the dual-stream masking module are multiplied by the corresponding binary matrix, and the two multiplication results are summed to obtain the dual-stream masking features of the dual-stream masking module; wherein, the binary matrix is ​​composed of randomly selected 0s and 1s.

[0023] In one embodiment of the present invention, the formula used in training the second classification model using the obtained dual-stream mask features to obtain the trained target classification model includes:

[0024]

[0025] Where γ represents the parameters of the dual-stream masking module; x m δ' represents the obtained dual-stream mask features; δ' represents the adversarial noise corresponding to the second adversarial image.

[0026] Secondly, embodiments of the present invention provide an image recognition adversarial defense device based on dual-stream masking, the device comprising:

[0027] The first adversarial image generation module is used to generate an augmented first adversarial image using a pre-defined attack method based on the acquired test image with classification labels and a pre-defined classification model.

[0028] The first classification model generation module is used to perform adversarial training on the preset classification model using the first adversarial image to obtain a trained first classification model with a certain robustness.

[0029] The second classification model generation module is used to divide the first classification model into a preset number of parts according to the size of the output feature map, and insert a parameter-initialized dual-stream mask module at the end of the structure of each part to obtain the second classification model.

[0030] The second adversarial image generation module is used to generate an augmented second adversarial image using the test image and the second classification model, and employing the preset attack method.

[0031] The target classification model acquisition module is used to input the second adversarial image into the second classification model, use each dual-stream masking module to separate the feature map of the corresponding part into robust features and non-robust features, and randomly mask the robust features and non-robust features to obtain the corresponding dual-stream mask features; use the obtained dual-stream mask features to train the second classification model to obtain the trained target classification model, so as to realize adversarial defense in the image classification process.

[0032] Thirdly, embodiments of the present invention provide an electronic device, including a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus;

[0033] The memory is used to store computer programs;

[0034] When the processor executes the program stored in the memory, it implements the steps of the image recognition adversarial defense method based on dual-stream mask provided in the embodiments of the present invention.

[0035] Fourthly, embodiments of the present invention provide a computer-readable storage medium storing a computer program, wherein the computer program, when executed by a processor, implements the steps of the image recognition adversarial defense method based on dual-stream masking provided in the embodiments of the present invention.

[0036] The beneficial effects of this invention are:

[0037] The solution provided in this invention combines adversarial training with random preprocessing defense, considering generalization to unknown or expected attacks, enabling the method to perform well against both invisible and expected attacks. To enhance the model's adversarial robustness, a dual-stream masking module is specifically designed. This module separates the obtained features into robust and non-robust features before random masking, improving adversarial robustness while reducing the loss of useful information. Furthermore, multi-layered defense capabilities are considered, extending the dual-stream masking module to each layer of the network, effectively improving the network's adversarial robustness. The method in this invention increases feature diversity through dual-stream masking, ensures feature discriminability through the separation of robust and non-robust features, and provides good adversarial robustness, thus improving the model's adversarial defense capabilities. Attached Figure Description

[0038] Figure 1 This is a flowchart illustrating an image recognition adversarial defense method based on dual-stream masking provided in an embodiment of the present invention.

[0039] Figure 2 This is a schematic diagram illustrating the division of the first classification model obtained by the ResNet18 model into four parts in an embodiment of the present invention;

[0040] Figure 3 This is a schematic diagram of the structure of the second classification model in an embodiment of the present invention;

[0041] Figure 4 This is a schematic diagram illustrating the process by which the dual-stream mask module obtains dual-stream mask features in an embodiment of the present invention.

[0042] Figure 5 This is a schematic diagram of the structure of an image recognition adversarial defense device based on dual-stream masking provided in an embodiment of the present invention;

[0043] Figure 6 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0044] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0045] To facilitate understanding of the present invention, a brief description of the prior art and the inventive concept of the present invention will be provided first.

[0046] Existing deep neural networks are vulnerable to attacks from carefully designed adversarial instances, posing significant security risks to deep learning applications. Current adversarial training-based defense methods primarily focus on robustness against visible attacks, struggling to maintain satisfactory performance against invisible attacks. Another common and simple approach to improving adversarial robustness is to preprocess the input data directly during the inference phase; however, while these methods offer some generalization ability, they perform poorly against desired attacks.

[0047] The inventors of this application have discovered that appropriately increasing the diversity of feature distributions is beneficial to enhancing the adversarial robustness and adversarial transferability of the model. Based on this, the goal of this application is to design a defense method that can cause more perturbation to the image (features) without significantly affecting the model's natural accuracy. The basic intuition of this application is: 1. The attack strength of adversarial noise varies across different components; 2. Multi-level defense is beneficial to improving the model's adversarial robustness. Therefore, this application adaptively divides adversarial examples into robust and non-robust parts, and then masks both parts separately to interfere with the adversarial information between images to different degrees. This utilizes random masking to interfere with different types of adversarial noise, thus forming a masked feature preprocessing defense method. To improve the defense capability against different network layers, this application specifically designs the above-mentioned masked feature preprocessing defense method as a plug-in module—a dual-stream masking module—and embeds it between different depths of the network, ensuring that the extracted features maintain appropriate diversity and discriminability.

[0048] Specifically, embodiments of the present invention provide an image recognition adversarial defense method, apparatus, electronic device, and storage medium based on dual-stream masking.

[0049] It should be noted that the execution entity of the image recognition adversarial defense method based on dual-stream masking provided in this embodiment of the invention can be an image recognition adversarial defense device based on dual-stream masking, which can run in an electronic device. This electronic device can be a server or a terminal device, but is not limited to these.

[0050] In a first aspect, embodiments of the present invention provide an image recognition adversarial defense method based on dual-stream masking, such as... Figure 1 As shown, the method may include the following steps:

[0051] S1, using the acquired test images with classification labels and the preset classification model, generate the augmented first adversarial image using the preset attack method;

[0052] The test images in this embodiment of the invention are derived from a test set, which is a given classification dataset. An existing classification dataset can be used, represented as follows: Where, x i ∈R H×W×3 , represents the i-th test image in the test set, which is an RGB image; H and W represent the height and width of the test image, and 3 represents the number of channels; y i ∈{0,1,2,…,M} represents the classification label of the i-th test image in the test set, specifically, there are M categories; N represents the total number of test images in the test set.

[0053] In this embodiment of the invention, the preset classification model can be any existing model capable of classifying images. For example, in one optional implementation, the preset classification model may include the ResNet18 model. It is a commonly used network structure in the art. Of course, the preset classification model is not limited to this.

[0054] In this embodiment of the invention, the preset attack method can be any existing attack method, such as DDN (Decoupling Direction and Norm for Efficient Gradient-Based L2 Adversarial Attacks), CW (Carlini & Wagner Attack), AA (AutoAttack), etc. In an optional implementation, the preset attack method may include PGD (Projected Gradient Descent Attack). It is the most commonly used attack in the art and a standard test baseline.

[0055] For details on the process of generating the augmented first adversarial image using the acquired test image with classification labels and the preset classification model and the preset attack method, please refer to the prior art for understanding, and it will not be described in detail here.

[0056] The overall training objective of this invention is to find a robust classifier θ and its corresponding plug-in block γ, which has good performance for adversarial images.

[0057] S2, using the first adversarial image to perform adversarial training on the preset classification model, to obtain a first classification model with a certain robustness after training;

[0058] The process from S1 to S2 is the adversarial training process.

[0059] For S2, the first adversarial image is input into the preset classification model to obtain a predicted value. The classification loss is calculated by calculating the difference between the predicted value and the classification label (i.e., the true value) of the test image, and backpropagation is performed to update the model parameters. This process is repeated until the model loss reaches a minimum and convergence is achieved. For example, the model can be considered to have converged after repeating the loop several times (e.g., 120 times), and the first classification model with determined model parameters is obtained.

[0060] Since adversarial training can give the model better adversarial capabilities, S2 uses standard adversarial training to obtain the model parameters θ. AT Thus, the first classification model is determined. The formulas used in the process of using the first adversarial image to perform adversarial training on the preset classification model to obtain a trained first classification model with a certain degree of robustness include:

[0061]

[0062] Among them, L cls F represents the loss function; θ Let represent the first classification model with parameter θ, and let θ represent the initially obtained robust classifier; x+δ represents the first adversarial image corresponding to the test image x; δ represents the adversarial noise corresponding to the first adversarial image; y represents the classification label of the test image; E (x,y)～D represents the expectation for the current dataset; D represents the training set containing the training images.

[0063] S3. Based on the size of the output feature map, the first classification model is divided into a preset number of parts, and a parameter-initialized dual-stream mask module is inserted at the end of the structure of each part to obtain the second classification model.

[0064] The process from S3 to S6 is the fine-tuning training process.

[0065] The preset number of classification units can be set according to different preset classification models. For the ResNet18 model, the preset number is 4. For ease of understanding, the following explanation will use the ResNet18 model with a preset number of units of 4. Please refer to [link to relevant documentation]. Figure 2 , Figure 2 This diagram illustrates the division of the first classification model obtained by ResNet18 into four parts. The feature map size of the first part is 64×32×32, the feature map size of the second part is 128×16×16, the feature map size of the third part is 256×8×8, and the feature map size of the fourth part is 512×4×4. Conv represents a convolutional layer, BatchNorm represents batch normalization, and FC represents a fully connected layer.

[0066] In this embodiment of the invention, a dual-stream masking module can be pre-built, and the network parameters involved in the dual-stream masking module can be initialized. Then, the dual-stream masking module with the four initialized parameters can be inserted into the module. Figure 2 The second classification model obtained at the end of the structure of each part is as follows: Figure 3 As shown, Mask represents the inserted dual-stream mask module. The dual-stream mask module is inserted in each part because this application found that feature maps at different levels have different sensitivities to information masking. Adding a dual-stream mask module as a plug-in to each layer can effectively improve the network's adversarial robustness and increase the model's adversarial defense performance.

[0067] S4, using the test image and the second classification model, generate an augmented second adversarial image using the preset attack method;

[0068] This step is similar to S1, except that a different model is used. Please refer to S1 and related technologies for details, which will not be explained in detail here.

[0069] S5, input the second adversarial image into the second classification model, use each dual-stream mask module to separate the feature map of the corresponding part into robust features and non-robust features, and randomly mask the two to obtain the corresponding dual-stream mask features; use the obtained dual-stream mask features to train the second classification model to obtain the trained target classification model, so as to realize adversarial defense in the image classification process.

[0070] For S5, the second adversarial image is input into the second classification model to obtain the predicted value. The classification loss is calculated by calculating the difference between the predicted value and the classification label (i.e., the true value) of the test image, and backpropagation is performed to update the model parameters. This process is repeated until the model loss reaches a minimum and convergence is achieved. For example, the model can be considered to have converged after repeating the loop several times (e.g., 40 times), and the target classification model with determined model parameters is obtained.

[0071] In this process, for the first three dual-stream masking modules, the feature maps of their corresponding parts are separated into robust features and non-robust features, and these two types of features are randomly masked to obtain the corresponding dual-stream mask features, which are then input into the next part. The dual-stream mask features obtained by the fourth dual-stream masking module are used to obtain the final prediction value.

[0072] Specifically, the second adversarial image is input into the second classification model, and each dual-stream masking module separates the feature map of the corresponding part into robust features and non-robust features. The two are then randomly masked to obtain the corresponding dual-stream mask features, including:

[0073] (1) Input the second adversarial image into the second classification model and use the feature separation network in each dual-stream mask module to separate the feature map of the corresponding part into robust features and non-robust features;

[0074] The feature separation network, abbreviated as FS-Net, can be a simple three-layer network or implemented using any generative network. To improve processing speed, a lightweight network can be used.

[0075] In this embodiment of the invention, F is used γ Representing a dual-stream masking module, taking any dual-stream masking module as an example, its robustness characteristics can be expressed as:

[0076] C1 = F γ(x+δ') (2);

[0077] Where x+δ' represents the feature map received by the dual-stream masking module; for the first dual-stream masking module, x+δ' represents the feature map received by the first dual-stream masking module after the second adversarial image is input into the second classification model; for the second to fourth dual-stream masking modules, x+δ' represents the feature map received by the dual-stream masking module, which is output to the dual-stream masking module after the dual-stream masking features of the previous dual-stream masking module enter the part where the dual-stream masking module is located.

[0078] The non-robust feature is obtained by subtracting the robust feature from the feature map received by the dual-stream masking module, and can be expressed as:

[0079] C2=x+δ'-F γ (x+δ') (3);

[0080] It can be expressed in the form of a general formula as follows:

[0081]

[0082] Among them, f i F represents the feature map received in the i-th part; γi This is the i-th dual-stream mask module; The robustness feature obtained by the i-th dual-stream mask module; The non-robust feature obtained by the i-th dual-stream mask module; Let be the dual-stream mask feature obtained by the i-th dual-stream mask module.

[0083] (2) For each dual-stream mask module, the robust features and non-robust features obtained by the dual-stream mask module are multiplied by the corresponding binary matrix respectively, and the two multiplication results are summed to obtain the dual-stream mask features of the dual-stream mask module.

[0084] To randomly mask the robust and non-robust features obtained from each dual-stream masking module, this embodiment of the invention pre-designs corresponding binary matrices for the robust and non-robust features, denoted as m1 and m2 respectively. The dimension of m1 corresponds to the robust feature, and the dimension of m2 corresponds to the non-robust feature. The binary matrices corresponding to m1 and m2 are composed of randomly selected 0s and 1s. m1 and m2 are designed using corresponding parameters r1 and r2. Through preliminary experiments, this embodiment of the invention found that the specific information content of robust feature C1 and non-robust feature C2 depends on the value of the masking ratio r of each branch (i.e., r1 and r2).

[0085] r1 and r2 represent the proportions of 0 in binary matrix m1 and binary matrix m2, respectively, where the size and shape of m1 and m2 are consistent with the size of the input robust feature C1 and non-robust feature C2. These can be set as needed. In a preferred embodiment, r1 = 0.01 and r2 = 0.1, which achieves better performance.

[0086] Therefore, for any two-stream mask module, its two-stream mask characteristics can be expressed as:

[0087] x m =m1×C1+m2×C2 (5);

[0088] Where, x m In this context, m represents mask, and x represents the mask. m This represents the characteristics of a dual-stream mask.

[0089] It is understandable that, since m1 and m2 are binary matrices, after multiplying them with the corresponding robust or non-robust features, the original positional features can be preserved using the 1s in the binary matrix, while the original positional features can be masked using the 0s. Since the binary matrix is ​​randomly constructed, random positional masking can be achieved for both robust and non-robust features. This embodiment of the invention separates robust and non-robust features using a dual-stream masking module and performs random masking, which can improve adversarial robustness while reducing the loss of useful information. The dual-stream masking module, with its two branches, can increase feature diversity, and the separation of robust and non-robust features ensures feature discriminability.

[0090] For the process of obtaining the dual-stream mask features for any dual-stream mask module, please refer to [link to documentation]. Figure 4 As shown.

[0091] For S5, the second classification model is trained using the obtained dual-stream mask features to obtain the trained target classification model. The formulas used in this process include:

[0092]

[0093] Where γ represents the parameters of the dual-stream masking module; x m δ' represents the obtained dual-stream mask features; δ' represents the adversarial noise corresponding to the second adversarial image.

[0094] The overall training can also be viewed as a minimum-maximum training process.

[0095] Through S5, a trained target classification model is obtained, which can achieve adversarial defense when performing image classification.

[0096] The solution provided in this invention combines adversarial training with random preprocessing defense, considering generalization to unknown or expected attacks, enabling the method to perform well against both invisible and expected attacks. To enhance the model's adversarial robustness, a dual-stream masking module is specifically designed. This module separates the obtained features into robust and non-robust features before random masking, improving adversarial robustness while reducing the loss of useful information. Furthermore, multi-layered defense capabilities are considered, extending the dual-stream masking module to each layer of the network, effectively improving the network's adversarial robustness. The method in this invention increases feature diversity through dual-stream masking, ensures feature discriminability through the separation of robust and non-robust features, and provides good adversarial robustness, thus improving the model's adversarial defense capabilities.

[0097] To evaluate the effectiveness and generalization of the method of this invention, several widely used classification datasets were selected for experimental verification. The selected classification datasets include the CIFAR-10 dataset, the CIFAR-100 dataset, and the Tiny-ImageNet dataset. Among them:

[0098] The CIFAR-10 dataset is one of the most widely used machine learning research datasets. It consists of 50,000 32×32 training images and 10,000 test images. The CIFAR-10 dataset is divided into 10 classes, with 6,000 images in each class.

[0099] The CIFAR-100 database has 100 classes. Each class has 600 32×32 color images, of which 500 are used as the training set and 100 as the test set.

[0100] Tiny-ImageNet has 200 classes. Each class has 500 training images, 50 validation images, and 50 test images.

[0101] For the preset attack methods, adversarial examples are generated using state-of-the-art adversarial attack algorithms, including PGD (Projected Gradient Descent Attack), DDN (Decoupling Direction and Norm for Efficient Gradient-Based L2 Adversarial Attacks), CW (Carlini & Wagner Attack), and AA (Auto Attack), while Normal represents the original image without any adversarial noise.

[0102] Experiment 1:

[0103] To verify the effectiveness of the proposed dual-stream masking framework in improving adversarial robustness generalization, the method was evaluated on the CIFAR-10 dataset. Results on the CIFAR-10 dataset are presented. Specifically, several state-of-the-art methods were selected for comparative experiments: Misclassification-Aware Adversarial Training (MART), TRadeoff-inspired Adversarial Defense via Surrogate-loss Minimization (TREADS), Feature Separation and Recalibration (FSR), Adaptive Adversarial Distillation (AdaAD), and Standard Adversarial Training (AT). As shown in Table 1, the proposed method significantly improves the model's generalization ability to adversarial examples with minimal performance degradation. Specifically, under PGD attacks, the extended proposed method achieves even greater improvement. Compared to the original method, the improvement is 8.7 (AT). Furthermore, compared to the plug-in FSR method, the method of this invention also exhibits superior defensive performance. Only FSR, as a plug-in method, offers some resistance to unknown attacks (here, DDN attacks, CW attacks, and AA attacks). Moreover, the method of this invention performs significantly better than the comparative methods.

[0104] Table 1. Comparison of Accuracy (ACC) Performance on CIFAR10

[0105]

[0106] In Table 1, "Normal" represents natural samples (original images without adversarial noise). PGD, DDN, CW, and AA are different adversarial attack methods.

[0107] Secondly, corresponding to the above method embodiments, this invention also provides an image recognition adversarial defense device based on dual-stream masking, such as... Figure 5 As shown, the device includes:

[0108] The first adversarial image generation module 501 is used to generate an augmented first adversarial image using a pre-defined attack method based on an acquired test image with classification labels and a pre-defined classification model.

[0109] The first classification model generation module 502 is used to perform adversarial training on the preset classification model using the first adversarial image to obtain a trained first classification model with a certain robustness.

[0110] The second classification model generation module 503 is used to divide the first classification model into a preset number of parts according to the size of the output feature map, and insert a parameter-initialized dual-stream mask module at the end of the structure of each part to obtain the second classification model.

[0111] The second adversarial image generation module 504 is used to generate an augmented second adversarial image using the test image and the second classification model, and employing the preset attack method.

[0112] The target classification model acquisition module 505 is used to input the second adversarial image into the second classification model, use each dual-stream masking module to separate the feature map of the corresponding part into robust features and non-robust features, and randomly mask the robust features and non-robust features to obtain the corresponding dual-stream mask features; use the obtained dual-stream mask features to train the second classification model to obtain the trained target classification model, so as to realize adversarial defense in the image classification process.

[0113] For details on the specific processing procedures of each module of the device, please refer to the relevant content in the first section, which will not be repeated here.

[0114] Thirdly, embodiments of the present invention also provide an electronic device, such as... Figure 6 As shown, it includes a processor 601, a communication interface 602, a memory 603, and a communication bus 604, wherein the processor 601, the communication interface 602, and the memory 603 communicate with each other through the communication bus 604.

[0115] The memory is used to store computer programs;

[0116] When the processor executes the program stored in the memory, it implements the steps of any dual-stream mask-based image recognition adversarial defense method provided in the first aspect of the present invention.

[0117] The communication bus mentioned in the above electronic devices can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus can be divided into address bus, data bus, control bus, etc. For ease of illustration, only one thick line is used to represent it in the diagram, but this does not mean that there is only one bus or one type of bus.

[0118] The communication interface is used for communication between the aforementioned electronic devices and other devices.

[0119] The memory may include random access memory (RAM) or non-volatile memory (NVM), such as at least one disk storage device. Optionally, the memory may also be at least one storage device located remotely from the aforementioned processor.

[0120] The processors mentioned above can be general-purpose processors, including central processing units (CPUs), network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0121] The method provided in this invention can be applied to electronic devices. Specifically, the electronic device can be a desktop computer, a portable computer, a smart mobile terminal, a server, etc. No limitation is made herein; any electronic device that can implement this invention falls within the protection scope of this invention.

[0122] Fourthly, corresponding to the image recognition adversarial defense method based on dual-stream masking provided in the first aspect, this embodiment of the invention also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of any of the image recognition adversarial defense methods based on dual-stream masking provided in the first aspect of this invention.

[0123] For the embodiments of the device / electronic device / storage medium, since they are basically similar to the method embodiments, the description is relatively simple, and relevant parts can be referred to in the description of the method embodiments.

[0124] It should be noted that the device, electronic device and storage medium in the embodiments of the present invention are respectively the device, electronic device and storage medium for applying the above-mentioned image recognition adversarial defense method based on dual-stream mask. Therefore, all embodiments of the above-mentioned image recognition adversarial defense method based on dual-stream mask are applicable to the device, electronic device and storage medium, and can achieve the same or similar beneficial effects.

[0125] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this invention, "a plurality of" means two or more, unless otherwise explicitly specified.

[0126] The above description is merely a preferred embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention are included within the scope of protection of the present invention.

Claims

1. An image recognition adversarial defense method based on dual-stream masking, characterized in that, include: Using the acquired test images with classification labels and a preset classification model, an augmented first adversarial image is generated using a preset attack method; The preset classification model is trained adversarially using the first adversarial image to obtain a first classification model with a certain degree of robustness after training. Based on the size of the output feature map, the first classification model is divided into a preset number of parts, and a parameter-initialized dual-stream mask module is inserted at the end of the structure of each part to obtain the second classification model; Using the test image and the second classification model, an augmented second adversarial image is generated using the preset attack method; The second adversarial image is input into the second classification model. The feature maps of the corresponding parts are separated into robust features and non-robust features using each dual-stream masking module. The two are then randomly masked to obtain the corresponding dual-stream mask features. The obtained dual-stream mask features are used to train the second classification model to obtain the trained target classification model, thereby achieving adversarial defense in the image classification process.

2. The image recognition adversarial defense method based on dual-stream masking according to claim 1, characterized in that, The preset classification model includes: model ResNet18.

3. The image recognition adversarial defense method based on dual-stream masking according to claim 1, characterized in that, The preset attack method includes: the projected gradient descent attack method.

4. The image recognition adversarial defense method based on dual-stream masking according to claim 2, characterized in that, The preset quantity is 4.

5. The image recognition adversarial defense method based on dual-stream masking according to claim 2, characterized in that, The formulas used in the process of adversarial training of the preset classification model using the first adversarial image to obtain a first classification model with a certain degree of robustness include: Among them, L cls F represents the loss function; θ Let represent the first classification model with parameter θ; x+δ represents the first adversarial image corresponding to the test image x; δ represents the adversarial noise corresponding to the first adversarial image; y represents the classification label of the test image; E (x,y)～D represents the expectation for the current dataset; D represents the training set containing the training images.

6. The image recognition adversarial defense method based on dual-stream masking according to claim 5, characterized in that, The second adversarial image is input into the second classification model. Each dual-stream masking module separates the feature map of the corresponding part into robust features and non-robust features, and then randomly masks both to obtain the corresponding dual-stream mask features, including: The second adversarial image is input into the second classification model, and the feature separation network in each dual-stream masking module is used to separate the feature map of the corresponding part into robust features and non-robust features. For each dual-stream masking module, the robust and non-robust features obtained from the dual-stream masking module are multiplied by the corresponding binary matrix, and the two multiplication results are summed to obtain the dual-stream masking features of the dual-stream masking module; wherein, the binary matrix is ​​composed of randomly selected 0s and 1s.

7. The image recognition adversarial defense method based on dual-stream masking according to claim 6, characterized in that, The formulas used in training the second classification model using the obtained dual-stream mask features to obtain the trained target classification model include: Where γ represents the parameters of the dual-stream masking module; x m δ' represents the obtained dual-stream mask features; δ' represents the adversarial noise corresponding to the second adversarial image.

8. An image recognition adversarial defense device based on dual-stream masking, characterized in that, include: The first adversarial image generation module is used to generate an augmented first adversarial image using a pre-defined attack method based on the acquired test image with classification labels and a pre-defined classification model. The first classification model generation module is used to perform adversarial training on the preset classification model using the first adversarial image to obtain a trained first classification model with a certain robustness. The second classification model generation module is used to divide the first classification model into a preset number of parts according to the size of the output feature map, and insert a parameter-initialized dual-stream mask module at the end of the structure of each part to obtain the second classification model. The second adversarial image generation module is used to generate an augmented second adversarial image using the test image and the second classification model, and employing the preset attack method. The target classification model acquisition module is used to input the second adversarial image into the second classification model, use each dual-stream masking module to separate the feature map of the corresponding part into robust features and non-robust features, and randomly mask the robust features and non-robust features to obtain the corresponding dual-stream mask features; use the obtained dual-stream mask features to train the second classification model to obtain the trained target classification model, so as to realize adversarial defense in the image classification process.

9. An electronic device, characterized in that, It includes a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus; The memory is used to store computer programs; When the processor executes the program stored in the memory, it implements the steps of the method described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the method described in any one of claims 1-7.

Images