Method for improving robustness of image classification model based on adversarial contrast learning

Abstract

The application discloses a method for improving the robustness of an image classification model based on adversarial contrast learning. First, the application constructs a dataset for performing an adversarial contrast learning task and a TrCL network model. Second, the application generates three different augmented images from an original image through a data enhancement strategy. Two of the augmented images are added with disturbance to generate corresponding adversarial samples through an instance-level attack. Then, the two augmented images with adversarial and one augmented image without adversarial are put into the network model for training, so as to obtain a pre-training model with robust features. Finally, the encoder part of the trained pre-training model is extracted, and a fully connected layer is connected for classification task training, that is, a robust classifier is formed. The application can simultaneously consider the robust precision and clean precision of the model, and ensure that the model has robustness while also ensuring that the influence of the classification precision of the model on clean samples is small.

Description

Technical Field

[0001] This invention relates to the field of adversarial examples, and more specifically to a method for improving the robustness of image classification models based on adversarial contrastive learning. Background Technology

[0002] Deep neural networks are now widely used in various fields such as object detection, image classification, and autonomous driving. However, in recent years, researchers have discovered that neural networks are easily affected by a type of perturbation called adversarial examples, which can cause a well-trained network model to incorrectly identify an image as another species with a very high probability.

[0003] There are many ways to defend against adversarial examples, but most methods focus on supervised learning, with less use in the field of self-supervised learning. Supervised learning relies on labels, which can lead to significant resource overhead, a problem that self-supervised learning can address. However, previous methods using self-supervised approaches to solve adversarial example problems are limited, and training with self-supervised methods also faces challenges such as requiring large amounts of data, needing pseudo-labels, or making the construction of instance-level attacks too complex. Summary of the Invention

[0004] This invention addresses the shortcomings of existing technologies by providing a method for improving the robustness of image classification models based on adversarial contrastive learning.

[0005] The technical solution adopted in this invention includes the following steps:

[0006] 1) Construct a dataset and TrCL network model for adversarial contrastive learning tasks;

[0007] 2) Generate three different augmented images from the original image using data augmentation strategies;

[0008] 3) By using an instance-level attack, perturbations are added to two of the augmented images to generate corresponding adversarial examples;

[0009] 4) The two generated adversarial augmented images and one non-adversarial augmented image are fed into the TrCL network model for training;

[0010] in:

[0011] The first part of the model loss function is viewed as an instance-level classification task, which makes the two features of clean samples and adversarial samples as close as possible in the feature space;

[0012] The second part of the model loss function does not require negative sample pairs and uses gradient truncation to prevent gradient backpropagation; it is used to optimize the feature space distance between adversarial samples of one augmented image and another augmented image.

[0013] The third part of the model loss function is the contrastive loss, which is generated during the generation of adversarial examples, that is, when adversarial examples are compared with clean examples.

[0014] 5) Feed the dataset into the model to train it fully, thereby obtaining a pre-trained model with robust features;

[0015] 6) Extract the encoder part of the pre-trained model, connect it to a fully connected layer for classification task training, thus forming a robust classifier.

[0016] The beneficial effects of this invention are as follows: This invention designs a network structure consisting of a backbone network, a projection head, and a prediction head, and uses three loss functions for network optimization. Furthermore, the method for generating adversarial perturbations in this invention does not require labels and is simpler than previous methods. Through adversarial training, a robust pre-trained model can be obtained. When the encoder part of this model is extracted and a fully connected layer is added as a linear classifier, it is found that the features initialized by the pre-trained model can simultaneously ensure both robust accuracy and clean accuracy, ensuring the model's robustness while minimizing the impact on the classification accuracy of clean samples. Attached Figure Description

[0017] Figure 1 This is a schematic diagram of the TrCL network model structure of the present invention. Detailed Implementation

[0018] A method for improving the robustness of image classification models based on adversarial contrastive learning includes the following steps:

[0019] 1) Construct a dataset and TrCL network model for adversarial contrastive learning tasks.

[0020] 2) Through a series of data augmentation strategies such as random cropping, changing image attributes, horizontal flipping, and grayscale, three different augmented images are generated from the original image.

[0021] 3) By using a custom instance-level attack, perturbations are added to two augmented images to generate corresponding adversarial examples. The custom instance-level attack does not require any labels; it generates the desired adversarial perturbations simply by using the data-augmented images and its iteratively updated samples.

[0022] 4) Feed the generated two adversarial augmented images and one non-adversarial augmented image into the model for training. The two adversarial images are obtained through steps 2) and 3), while the non-adversarial augmented image is obtained solely through step 2).

[0023] The first part of the model loss function can be viewed as an instance-level classification task. The goal is to make the two features of clean samples and adversarial samples as similar as possible in the feature space.

[0024] The second part of the model's loss function does not require negative sample pairs. It employs gradient truncation to prevent gradient backpropagation. It is used to optimize the feature space distance between adversarial examples of one augmented image and another.

[0025] The third part of the model loss function is the contrastive loss, which is generated during the generation of adversarial examples, that is, when adversarial examples are compared with clean examples.

[0026] 5) Feed the dataset into the model to train it fully, thereby obtaining a pre-trained model with robust features;

[0027] 6) Extract the encoder part of the pre-trained model, connect it to a fully connected layer, and train it for a classification task to obtain a robust classifier.

[0028] In some embodiments, in step 1), the CIFAR-10 and CIFAR-100 datasets are primarily used as benchmarks, selecting only portions of their data without labels to achieve self-supervised, label-free training. A TrCL model is constructed using ResNet18 as the base encoder, with an additional projector h and a predictor p. Both the projector and predictor are two-layer MLP structures.

[0029] In some embodiments, the specific expression for generating adversarial examples in step 3) is as follows:

[0030]

[0031] Where: q() represents the image obtained by the augmentation strategy described in step 2), i represents the training round, S represents the maximum perturbation generated cannot exceed this boundary, Π is used to map the result to the target range, a is a hyperparameter, specifically set to 2.0 / 255, L is the mean squared error loss, and the sign function sgn is used to calculate the gradient direction, thereby controlling the perturbation distance.

[0032] The gradient direction obtained by combining the adversarial sample from the i-th round with the sample after image enhancement is added to the adversarial sample from the i-th round, thereby generating the adversarial sample from the (i+1)-th round.

[0033] In some embodiments, the first part of the model loss function is as follows:

[0034]

[0035] Where: D(x,y) represents the result of multiplying and normalizing vectors x and y, h() represents the features output by the projector, x represents a clean sample that has only undergone augmentation transformation, + represents a positive sample with respect to x, - represents a negative sample with respect to x, adv represents an adversarial sample generated from image x, and exp represents the exponential function e x t is the temperature coefficient, which is 0.5 here.

[0036] When optimizing this loss function, in order to ensure that the loss function continuously decreases, that is, to make the numerator smaller and the denominator larger, we need to reduce the distance between the image x and its adversarial samples in the feature space to make the numerator smaller, and to make the denominator larger, we need to reduce the distance between the image x and its negative sample pairs to achieve the optimization goal.

[0037] In some embodiments, the second part of the model loss function is as follows:

[0038]

[0039] Where: a and b represent two images obtained after a series of enhancements to the same image x in step 2), and h() and p() represent the features output by the projection head and prediction head, respectively. stopgrad() is a gradient truncation operation that prevents gradients from being backpropagated. The gradient truncation operation prevents the truncated parameters from being backpropagated to update the network parameters.

[0040] The third part of the model loss function is the contrastive loss, which is the mean squared error loss between the augmented image of the input image and the adversarial sample generated using an instance-level attack on that image.

[0041] Example: Taking the CIFAR-10 dataset as an example, this dataset mainly consists of 60,000 images across 10 categories. A ResNet18 backbone network is used, followed by a projector and predictor to construct the TrCL network model. When the CIFAR-10 dataset is fed into the network, certain processing is performed: for each input image, an image augmentation strategy is applied to generate three augmented images, which are then fed into the network for training. During training, two of these images are used to generate corresponding adversarial examples through a custom instance-level attack. The generation process is iterative; the adversarial examples in the next round are determined by the gradient direction of the loss between the adversarial examples generated in the current round and the augmented images.

[0042] Based on this, two adversarial examples and one augmented image are generated. These images are then fed into the network for training. The network parameters are updated using the three loss functions mentioned above and gradient backpropagation, resulting in a fully trained pre-trained model. The encoder portion of this pre-trained model is extracted and connected to a fully connected layer for downstream classification tasks, yielding a superior classification model capable of resisting certain adversarial perturbations.

[0043] Please refer to the diagram for explanation, such as Figure 1 As shown, x is the original image, which is augmented by image augmentation method t to generate three augmented images of image x. Two of these images are used to generate adversarial examples using a custom instance-level attack. Therefore, the two augmented adversarial examples and a clean image that has only been augmented are fed into the network for training. The network structure in this embodiment consists of three parts: encoder f, projector h, and predictor p. h1, h2, and h3 are the feature vectors output by the projector h for each of the three images. p1, p2, and p3 are the feature vectors output by the predictor p for each of the three images. In this embodiment, L is applied to the two features h1 and h2. s1 Operations are performed on the four features h2, h3, p2, and p3 using L... s2 operate.

[0044] The experimental results on the CIFAR-10 dataset are shown in the table below. All the models below are based on l ∞ The attack was performed using a pre-trained model with an attack strength of 8.0 / 255. The TrCL pre-trained model of this invention effectively addresses the shortcomings of previous techniques, significantly improves model robustness, and maintains high clean accuracy.

[0045] Method Name Cleanliness accuracy (%) Robust accuracy (%) Selfie 86.02 51.05 Rotation 85.66 50.40 Jigsaw 83.74 48.83 AdvSS 86.04 54.64 ACL 82.19 52.82 RoCL 80.43 47.69 TrCL 87.53 62.27

[0046] In summary, this invention presents a label-free method for training robust models. A custom instance-level attack can be used to obtain perturbations and train the network. Furthermore, three custom loss functions allow the network model to be fully trained and learn more robust features. Based on the adversarial example domain, this invention ensures the network's robustness through self-supervised contrastive learning. Compared with previous methods (as shown in the table above), it can be seen that using the TrCL robust pre-trained model allows the network to be fully trained, significantly improving the model's robustness and achieving higher clean accuracy.

Claims

1. A method for improving the robustness of image classification models based on adversarial contrastive learning, characterized in that... The method includes the following steps: 1) Construct a dataset and TrCL network model for adversarial contrastive learning tasks; 2) Generate three different augmented images from the original image using data augmentation strategies; 3) By using an instance-level attack, perturbations are added to two of the augmented images to generate corresponding adversarial examples; 4) The two generated adversarial augmented images and one non-adversarial augmented image are fed into the TrCL network model for training; in: The first part of the model loss function is viewed as an instance-level classification task, which makes the two features of clean samples and adversarial samples as close as possible in the feature space; The second part of the model loss function does not require negative sample pairs and uses gradient truncation to prevent gradient backpropagation; it is used to optimize the feature space distance between adversarial samples of one augmented image and another augmented image. The third part of the model loss function is the contrastive loss, which is generated during the generation of adversarial examples, that is, when adversarial examples are compared with clean examples. 5) Feed the dataset into the model to train it fully, thereby obtaining a pre-trained model with robust features; 6) Extract the encoder part of the pre-trained model, connect it to a fully connected layer, and train it for a classification task to form a robust classifier; The TrCL network model described in step 1) uses ResNet18 as the base model encoder, and adds a projection head h and a predictor p. The first part of the model loss function is as follows: Where: D(x,y) represents the result of multiplying and normalizing vectors x and y, h() represents the features output by the projector, + represents positive samples with respect to x, - represents negative samples with respect to x, adv represents adversarial samples generated from image x, and exp represents the exponential function e x τ represents the temperature coefficient; The second part of the model loss function is as follows: Where: a and b represent two images obtained after augmenting the same image x, p() represents the features output by the prediction head, and stopgrad() represents the gradient truncation operation to prevent gradient backpropagation.

2. The method for improving the robustness of image classification models based on adversarial contrastive learning according to claim 1, characterized in that: The dataset mentioned in step 1) is based on the CIFAR-10 and CIFAR-100 datasets. Only their data portions are selected without labels, in order to achieve the purpose of self-supervised learning and label-free training.

3. The method for improving the robustness of image classification models based on adversarial contrastive learning according to claim 1, characterized in that: The specific expression for generating adversarial examples in step 3) is as follows: Where: q() represents the image obtained by the data augmentation strategy described in step 2), i represents the training round, S represents that the maximum perturbation generated cannot exceed this boundary, Π represents mapping the result to the target range, α represents the hyperparameter, L is the mean squared error loss, and sgn represents the gradient direction, thereby controlling the perturbation distance.

4. The method for improving the robustness of image classification models based on adversarial contrastive learning according to claim 1, characterized in that: The contrast loss is the mean squared error loss between the augmented image of the input image and the adversarial sample generated using an instance-level attack on that image.

Images