An image classification method based on generative model

Abstract

The present invention discloses an image classification method based on a generative model, comprising the following steps: step 10, obtaining an image to be classified and a preprocessed training set; step 20, constructing a generative model based on a U-Net network structure and a residual network; step 30, iteratively training the generative model and the classifier according to the training set to obtain a trained generative model and a trained classifier; step 40, inputting the image to be classified into the trained generative model to obtain an adversarial sample to be classified; step 50, inputting the adversarial sample to be classified into the trained classifier to obtain a classification result. The present invention constructs a generative model by combining a U-Net network and a residual module (Resblock) to generate adversarial perturbations. At the same time, the input data of the generative model provides the generative model with richer gradient information, so that the generative model can extract more effective information of the input, thereby generating more powerful adversarial samples, thereby improving the robustness and accuracy of the classifier.

Description

Technical Field

[0001] The present invention belongs to the technical field of image classification, and in particular relates to an image classification method based on a generative model. Background Art

[0002] In recent years, deep learning has been widely used in various scenarios. However, most deep learning models are highly vulnerable to adversarial attacks. Specifically, by injecting a carefully designed, tiny perturbation into a normal sample, an unprotected, trained classifier can be caused to misclassify the input image. We call such input images adversarial examples. They are semantically indistinguishable from normal samples, but they can deceive deep learning models and undermine the security of deep learning. PGD adversarial training, proposed by Madry et al. in 2018, is one of the most effective adversarial defense techniques. Its main idea is to use adversarial examples to enhance the training data during each training cycle. Unlike other defense strategies, adversarial training inherently enhances the robustness of the model.

[0003] For a long time, scholars at home and abroad have conducted extensive research in the field of adversarial training for deep learning. From an optimization perspective, adversarial training is essentially solving a non-convex minimax optimization problem. However, due to the lack of concave-convex structure, such minimax problems are difficult to solve. By leveraging the ability of neural networks to fit functions, generative models are considered a good choice for exploring adversarial examples in the input space. Jiang et al. used a general L2L framework for adversarial training. They designed a generative network and superimposed the perturbation on the original image to form an adversarial example, which was then used in classifier training. The existing network structure and input design of the generative model do not fully utilize existing information to generate more powerful perturbations, resulting in low robustness and accuracy. Moreover, after the initial learning rate decrease, the classifier will overfit, and with continued training, the model's classification accuracy will decline in the face of attacks. Summary of the Invention

[0004] In order to solve the above problems existing in the prior art, the present invention provides an image classification method based on a generative model. The technical problem to be solved by the present invention is achieved through the following technical solutions:

[0005] An image classification method based on a generative model comprises the following steps:

[0006] Step 10: Obtain the image to be classified and the preprocessed training set;

[0007] Step 20: constructing a generative model based on the U-Net network structure and the residual network; wherein the downsampling layer of the generative model is a residual module, and random noise is input into each downsampling layer;

[0008] Step 30, iteratively training the generative model and the classifier according to the training set to obtain a trained generative model and a trained classifier; wherein the input data of the generative model is the spliced together training set, the second gradient information, and the third gradient information;

[0009] The second gradient information is the gradient information of the cross entropy loss of the second output data and the correct label obtained after the training set is input into the classifier with respect to the training set;

[0010] The third gradient information is the gradient information of the cross entropy loss of the third output data and the correct label obtained after the original adversarial sample is input into the classifier to the original adversarial sample; wherein the original adversarial sample is generated by performing an FGSM attack on the training set;

[0011] Step 40: input the image to be classified into the trained generative model to obtain an adversarial sample to be classified;

[0012] Step 50: input the adversarial sample to be classified into the trained classifier to obtain a classification result.

[0013] In one embodiment of the present invention, the specific steps of step 30 include:

[0014] Step 31, calculating the input data of the generation model based on the training set;

[0015] Step 32: inputting the input data of the generative model into the generative model to obtain adversarial disturbance data;

[0016] Step 33: superimpose the adversarial perturbation data onto the images in the training set to obtain training adversarial samples;

[0017] Step 34: input the training adversarial sample into the classifier, calculate the classification loss, and iteratively update the parameters of the classifier and the parameters of the generative model;

[0018] Step 35: When the number of iterative updates reaches a maximum, a trained generative model and a trained classifier are obtained.

[0019] In one embodiment of the present invention, the specific steps of step 34 include:

[0020] The training adversarial sample is input into the classifier. When the number of iterations is less than the preset number of iterations, the classification loss is calculated using a first loss function and the parameters of the classifier and the parameters of the generative model are iteratively updated. The expression of the first loss function is:

[0021]

[0022] in, Represents the cross entropy loss between the classifier’s output data and the correct label;

[0023] When the number of iterations is greater than or equal to the preset number of iterations, the classification loss is calculated using a second loss function and the parameters of the classifier and the parameters of the generative model are iteratively updated; wherein the expression of the second loss function is:

[0024]

[0025] Among them, w i =max j t i,j , p i Represents the output data obtained after the training adversarial sample is input into the classifier.

[0026] In one embodiment of the present invention, there is a skip connection between downsampling and upsampling of the generative model.

[0027] In one embodiment of the present invention, the downsampling and upsampling of the generation model are further connected via a connection module.

[0028] Beneficial effects of the present invention:

[0029] The present invention constructs a generative model by combining a U-Net network and a residual module (Resblock) to generate adversarial perturbations. The input data of the generative model provides the generative model with richer gradient information, enabling the generative model to extract more effective information from the input, thereby generating more powerful adversarial samples, thereby improving the robustness and accuracy of the classifier.

[0030] The present invention will be further described in detail below with reference to the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 A schematic diagram of a flow chart of an image classification method based on a generative model provided in an embodiment of the present invention;

[0032] Figure 2 A schematic diagram of the network structure of the generation model provided by an embodiment of the present invention;

[0033] Figure 3 This is a graph showing how the robustness of a classifier changes with the number of training times during the training process provided in the background technology and in the embodiments of the present invention. DETAILED DESCRIPTION

[0034] The present invention will be further described in detail below with reference to specific examples, but the embodiments of the present invention are not limited thereto.

[0035] Example 1

[0036] like Figure 1 As shown, an image classification method based on a generative model includes the following steps:

[0037] Step 10: Obtain the image to be classified and the preprocessed training set.

[0038] Obtain the original image dataset to be trained on the classifier, randomly crop each image, randomly horizontally flip it, and then normalize the data to obtain a preprocessed training set. This example uses the CIFAR10 dataset as the original image dataset. The image to be classified is any image in the CIFAR10 dataset.

[0039] Step 20: Build a generative model based on the U-Net network structure and the residual network.

[0040] Among them, the downsampling layer of the generative model is a residual module, and random noise is input into each downsampling layer.

[0041] In order to better extract the input information, the network structure of the generative model is constructed based on the network structure that combines the U-Net network structure and the residual module (Resblock). Among them, the U-Net network structure performs downsampling in the first half and upsampling in the second half. At the same time, there is a jump connection between downsampling and upsampling. Due to the connection between layers, this network structure is conducive to retaining part of the input information to the output. The Resblock network is a network structure in which the input directly introduces a short connection to the output of the non-linear layer. It has been proven to be capable of fitting higher-dimensional functions and is suitable for passing input information to the output. The generative model of this embodiment replaces the downsampling convolution layer of the U-Net network structure with a residual network. The network structure is simple, and the generative model can extract more input information, which is conducive to the generative model generating stronger disturbances. In addition, random noise z is added to each downsampling layer to make the generated adversarial samples more likely.

[0042] In this embodiment, the network structure of the generated model is as follows Figure 2As shown in the figure, the main network structure of each noise encoder is two linear layers and a ReLU layer. Each residual block ResBlock first uses a convolutional layer to downsample to obtain the result x. The result of x after passing through two convolutional layers is then added to x, and then sent to the ReLU layer, and finally passed through the maximum pooling layer. The upsampling and downsampling connection module Connection Block mainly consists of two convolutional layers. Each upsampling block UpBlock first has an upsampling layer, which is then spliced with a skip connection, and then connected to two convolutional layers. Finally, it passes through a convolutional layer and a tanh layer to keep the output perturbation range in the range [-1, 1].

[0043] The number of residual modules and upsampling modules can be set as needed. Preferably, the number of residual modules is three, and the number of upsampling modules is three.

[0044] In this example, a network structure combining U-Net and a residual module (Resblock) is introduced into a generative model to generate adversarial perturbations. This generative model can extract more input information, which helps the generative model generate stronger perturbations. Random noise z is added to each downsampling layer, creating a larger perturbation space and exploring more possibilities for generating perturbations.

[0045] Step 30: Iteratively train the generative model and the classifier according to the training set to obtain a trained generative model and a trained classifier.

[0046] The purpose of this step is to jointly generate the model and the classifier for iterative training. Specifically, step 30 includes steps 31 to 35:

[0047] Step 31: Calculate the input data of the generated model based on the training set.

[0048] In this embodiment, the training set and the second gradient information and the third gradient information are spliced together as the input data of the generative model, and the gradient information is spliced into the input vector of the generative model to provide the generative model with richer gradient information, thereby generating more powerful perturbations.

[0049] The second gradient information is the gradient of the cross entropy loss of the correct label and the second output data obtained after the training set is input into the classifier. The third gradient information is the gradient of the cross entropy loss of the correct label and the original adversarial sample generated by the FGSM attack on the training set.

[0050] In this embodiment, the second gradient information and the third gradient information are both data corresponding to the images in the training set.

[0051] Preferably, before step 31, the basic parameters of training are set, such as the number of training iterations, batch size, model learning rate, and optimizer. The learning rates of the generative model and classifier and the optimizer can be different. The decay strategy of the classifier's learning rate is generally multi-step decay. In the embodiment, the batch size is set to 64, the classifier uses the WRN-34-10 network model, the learning rate is set to 0.1, and the learning rate decays to one-tenth of the original after 30, 45, and 60 iterations. The optimizer uses SGD; the learning rate of the generative model is set to 3e-4, and the optimizer uses Adam.

[0052] Step 32: Input the input data of the generative model into the generative model to obtain adversarial disturbance data.

[0053] In step 33, the adversarial perturbation data is superimposed on the corresponding images in the training set to obtain training adversarial samples.

[0054] In step 34, the training adversarial sample is input into the classifier, the classification loss is calculated, and the parameters of the classifier and the parameters of the generative model are iteratively updated.

[0055] The classifier and generative model are trained alternately. In one iteration of training, the generator is first fixed and the classifier is updated. Then, the classifier is fixed and the generator is updated. The two models compete with each other. The goal of the generative model is to update its parameters so that the adversarial perturbation it generates maximizes the first and second loss functions. The goal of the classifier is to update its parameters so that it minimizes the first and second loss functions.

[0056] Specifically, the training adversarial sample is input into the classifier. When the number of iterations is less than the preset number of iterations, the classification loss is calculated by the first loss function and the parameters of the classifier and the generation model are iteratively updated. The expression of the first loss function is:

[0057]

[0058] in, represents the cross entropy loss between the classifier’s output data and the correct label; x i represents the i-th training adversarial sample (i-th input data), y i Represents x i The corresponding classification label, generating model pair x i The perturbation is δ i , f θ is a generator with parameter θ.

[0059] In this embodiment, in the early stage of training, the number of iterations is less than 30, that is, before the learning rate of the classifier decreases for the first time, the classifier has not overfitted, and the loss function used is the cross entropy loss between the classifier output data and the correct label. Right now

[0060] When the number of iterations is greater than or equal to the preset number of iterations, the classification loss is calculated using the second loss function and the parameters of the classifier and the generation model are iteratively updated; wherein the expression of the second loss function is:

[0061]

[0062] Among them, w i =max j t i,j , p i =softmax(f θ (x i +δ i )) represents the output data obtained after the training adversarial sample is input into the classifier, which is also the prediction of the classifier output result. The dimension is the number of categories in the training set. For example, the cifar10 dataset has 10 dimensions. i,j Then it means p i Predict the probability that the input is the jth category. i,j represents the predicted label of the i-th training adversarial sample (i-th input data) classified into the j-th category.

[0063] In the later stage of training, when the number of iterations is greater than or equal to 30, the adaptive training is performed after the learning rate of the classifier decreases for the first time. In order to alleviate the overfitting problem, the accumulated prediction t i To replace the original label y i In this way, the network's prediction of the training samples during the training process is used to correct the training labels, thereby improving the generalization performance of the model. i =αt i +(1-α)p i , where p i The output data obtained after the training adversarial sample formed by generating adversarial perturbation data for the generative model is input into the classifier, and the initial value t0 is set to the original label y i α is a custom constant used to set t i At the same time, using sample reweighting, the loss function is set to:

[0064]

[0065] In this embodiment, an adaptive training method is introduced into the adversarial training based on the generative model. First, the overfitting problem is solved. After the first learning rate decreases, the accuracy of the model facing attacks does not decrease. Second, the present invention makes the trained network more robust.

[0066] Step 35: When the number of iterative updates reaches a maximum, a trained generative model and a trained classifier are obtained.

[0067] In step 40, the image to be classified is input into the trained generative model to obtain an adversarial sample to be classified.

[0068] Step 50: Input the adversarial sample to be classified into the trained classifier to obtain the classification result.

[0069] The present invention constructs a generative model by combining a U-Net network and a residual module (Resblock) to generate adversarial perturbations. The input data of the generative model provides the generative model with richer gradient information, enabling the generative model to extract more effective information from the input, thereby generating more powerful adversarial samples, thereby improving the robustness and accuracy of the classifier.

[0070] The generative model of the present invention can extract more effective information from the input, thereby generating more powerful adversarial samples, thereby improving the robustness of the classifier; and uses an adaptive training method in the second half of model training to train a classifier with stronger generalization ability and higher classification accuracy in the face of different attacks, while also avoiding the robust overfitting problem that is common in adversarial training methods based on generative models.

[0071] Although deep neural networks have achieved advanced performance in various applications, adversarial attack algorithms for generating adversarial perturbations have continued to emerge. The existence of these attack algorithms poses a serious threat to the current artificial intelligence systems based on deep neural networks that have not taken any defensive measures. At the same time, in order to further protect deep learning models, scholars have proposed many effective defense strategies that can improve the robustness of the model or help the model resist attacks from adversarial samples. The present invention constructs a generative model network structure suitable for generating adversarial perturbations, optimizes the input information of the generative model, and uses an adaptive training method in the later stage of training to make the trained network more robust and avoid the influence of adversarial samples on the classification and prediction of deep neural networks as much as possible, thereby making the classifier model and even deep learning technology have a wider range of usage scenarios.

[0072] Example 2

[0073] This embodiment is a test result obtained by testing the classification method in the first embodiment.

[0074] In the field of classifier adversarial defense, the robustness of a model is usually measured by the model's accuracy in the face of different attacks. Common attacks include: FGSM, a simple gradient attack; the PGD method, which can be seen as a multi-step iterative version of FGSM, where the number after the PGD method represents the number of iterations; and the CW method, an optimization-based attack with excellent attack effectiveness.

[0075] Compared with the existing technology Chinese patent CN114881875A, the classification accuracy of this model in the face of attacks is improved. The classification accuracy of the existing technology is 85.84% when there is no attack, the classification accuracy is 62.46% when facing PGD10 attack, the classification accuracy is 47.72% when facing PGD100 attack, and the classification accuracy is 53.50% when facing CW attack.

[0076] (1) The generative model network structure used in the present invention effectively improves the classification accuracy of the model when facing attacks. The classification accuracy is 84.88% when there is no attack, 62.96% when facing PGD10 attack, 50.83% when facing PGD100 attack, and 56.45% when facing CW attack.

[0077] (2) After splicing the gradient information of the adversarial samples generated by the FGSM attack on the images in the training set into the input vector of the generative model, the classification accuracy of the model in the face of attack is further improved. The classification accuracy is 85.71% when there is no attack, 65.72% when facing PGD10 attack, 53.81% when facing PGD100 attack, and 59.54% when facing CW attack.

[0078] (3) After introducing the adaptive training method in the later stage of training model, the overfitting phenomenon of adversarial training method based on generative model was successfully alleviated, as shown in the attached figure. Figure 3 As shown, L2L is a common adversarial training method of the L2L framework in the background technology, and L2L-SAT is the adversarial training method of the present invention. Figure 3 The horizontal axis represents the number of training epochs, with one epoch equal to training once using all samples in the training set. The vertical axis represents the robustness of the classifier, measured by the classification accuracy when subjected to the PGD10 attack. Furthermore, the model's classification accuracy under attack was further improved, reaching 87.13% in the absence of an attack, 66.54% when subjected to the PGD10 attack, 53.46% when subjected to the PGD100 attack, and 60.91% when subjected to the CW attack.

[0079] In summary, the above data and the effects of basic defense methods in this field are summarized in Table 1. It can be seen that the final effect of the present invention is not only improved in the classification accuracy when attacked, but also in the classification accuracy when not attacked, compared with the benchmark.

[0080]

[0081] Table 1

[0082] In the description of the present invention, it should be understood that the terms "center", "longitudinal", "lateral", "length", "width", "thickness", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", "clockwise", "counterclockwise" and the like to indicate orientations or positional relationships based on the orientations or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be understood as limiting the present invention.

[0083] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of the technical features being referred to. Thus, a feature identified as "first" or "second" may explicitly or implicitly include one or more of the features. In the description of the present invention, "plurality" means two or more, unless otherwise specifically defined.

[0084] In the present invention, unless otherwise expressly specified or limited, the terms "mounted," "connected," "connect," "fixed," etc. should be understood broadly. For example, they may refer to fixed connection, detachable connection, or integration; mechanical connection or electrical connection; direct connection or indirect connection through an intermediate medium; internal communication between two components or interaction between two components. Those skilled in the art will understand the specific meanings of the above terms in the present invention based on specific circumstances.

[0085] In the present invention, unless otherwise expressly specified or limited, a first feature being "above" or "below" a second feature may include the first and second features being in direct contact, or may include the first and second features being in contact not directly but through another feature between them. Furthermore, a first feature being "above," "above," and "above" a second feature may include the first feature being directly above or obliquely above the second feature, or may simply mean that the first feature is higher in level than the second feature. A first feature being "below," "below," and "below" a second feature may include the first feature being directly below or obliquely below the second feature, or may simply mean that the first feature is lower in level than the second feature.

[0086] In the description of this specification, the reference terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" mean that the specific features, structures, materials, or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described can be combined in any appropriate manner in any one or more embodiments or examples. In addition, those skilled in the art can combine and combine different embodiments or examples described in this specification.

[0087] The above is a further detailed description of the present invention in conjunction with specific preferred embodiments, and the specific implementation of the present invention should not be considered to be limited to these descriptions. For those skilled in the art of the present invention, without departing from the concept of the present invention, several simple deductions or substitutions can be made, which should be considered to fall within the scope of protection of the present invention.

Claims

1. An image classification method based on a generative model, characterized in that: The following steps are involved: Step 10: Obtain the image to be classified and the preprocessed training set; Step 20: constructing a generative model based on the U-Net network structure and the residual network; wherein the downsampling layer of the generative model is a residual module, and random noise is input into each downsampling layer; Step 30, iteratively training the generative model and the classifier according to the training set to obtain a trained generative model and a trained classifier; wherein the input data of the generative model is the spliced together training set, the second gradient information, and the third gradient information; The second gradient information is the gradient information of the cross entropy loss of the second output data and the correct label obtained after the training set is input into the classifier with respect to the training set; The third gradient information is the gradient information of the cross entropy loss of the third output data and the correct label obtained after the original adversarial sample is input into the classifier to the original adversarial sample; wherein the original adversarial sample is generated by performing an FGSM attack on the training set; The specific steps of step 30 include: Step 31, calculating the input data of the generation model based on the training set; Step 32: inputting the input data of the generative model into the generative model to obtain adversarial disturbance data; Step 33: superimpose the adversarial perturbation data onto the images in the training set to obtain training adversarial samples; Step 34: input the training adversarial sample into the classifier, calculate the classification loss, and iteratively update the parameters of the classifier and the parameters of the generative model; Step 35, when the number of iterative updates reaches a maximum, obtaining the trained generative model and the trained classifier; Step 40: input the image to be classified into the trained generative model to obtain an adversarial sample to be classified; Step 50: input the adversarial sample to be classified into the trained classifier to obtain a classification result.

2. The image classification method based on a generative model according to claim 1, characterized in that: The specific steps of step 34 include: The training adversarial sample is input into the classifier. When the number of iterations is less than the preset number of iterations, the classification loss is calculated using a first loss function and the parameters of the classifier and the parameters of the generative model are iteratively updated. The expression of the first loss function is: in, represents the cross entropy loss between the classifier’s output data and the correct label, represents the i-th training adversarial sample, express The corresponding classification labels, Represents the generative model The disturbance, Indicates that the parameter is Generator of; When the number of iterations is greater than or equal to the preset number of iterations, the classification loss is calculated using a second loss function and the parameters of the classifier and the parameters of the generative model are iteratively updated; wherein the expression of the second loss function is: in, , represents the output data obtained after the training adversarial sample is input into the classifier, express Predict the probability that the input is the jth category, represents the predicted label of the i-th training adversarial example classified into the j-th class.

3. The image classification method based on a generative model according to claim 1, characterized in that: There are skip connections between downsampling and upsampling in the generative model.

4. The image classification method based on a generative model according to claim 1, characterized in that: The downsampling and upsampling of the generation model are also connected through a connection module.

Images