An image classification method based on curriculum learning and adversarial training
By introducing course learning and adversarial training methods into deep learning models, adversarial samples are gradually added to train the model, which solves the problem of misclassification of deep learning models when facing adversarial samples, and maintains high accuracy without perturbation, achieving a balance between adversarial robustness and recognition accuracy.
Patent Information
- Application Number
- CN202210369277.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-04-08
- Publication Date
- 2025-04-29
- Estimated Expiration
- 2042-04-08
AI Technical Summary
Existing deep learning models are prone to misclassification when facing adversarial samples, and methods to defend against adversarial samples usually sacrifice image classification accuracy without perturbation.
The image classification method based on course learning and adversarial training is adopted, and the model is trained by gradually adding adversarial samples, combined with game optimization ideas and regularization methods, a robust visual classification model is generated.
The classification accuracy of the model when facing adversarial samples is improved, and the image classification accuracy is maintained without perturbation, achieving a balance between adversarial robustness and recognition accuracy.
Smart Images
Figure CN114743049B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of deep learning and machine vision, and in particular to an image classification method based on curriculum learning and adversarial training. Background Art
[0002] Existing visual deep learning can be disturbed by adversarial samples. During the back-propagation process, attackers can obtain the gradient change direction of the network model and find the maximum value of the gradient change. They can then add single-step or multi-step perturbations in this direction to form adversarial noise. When adversarial noise is added to the original image, an adversarial sample is generated. This process does not change the semantic information of the image, but can induce "misclassification" of the neural network. The existence of adversarial samples is a major obstacle to building a reliable and secure deep learning process.
[0003] In order to defend against adversarial samples, academia and industry have proposed a variety of methods. Among all these methods, the most important one is the adversarial training strategy based on game optimization. In this training strategy, it is necessary to maximize the adversarial perturbation and minimize the expected loss as much as possible. This method can correct the incorrect decision boundaries of deep learning for image classification and extract interpretable features. However, these methods usually sacrifice the accuracy of image classification in the absence of perturbations. Deep learning needs to achieve a balance between adversarial robustness and recognition accuracy in the absence of perturbations.
[0004] From the data dimension, the construction of machine intelligence can imitate the construction of human intelligence. Children usually learn from simple knowledge and then transition to difficult knowledge. The knowledge mastered before will pave the way for learning later knowledge. Based on this idea, computer scientists proposed the concept of "curriculum learning". The data used for machine learning can be arranged in the order of "easy first and difficult later". The concept of "curriculum learning" is also in line with the idea of iterative development in software engineering - continuously improving system performance while meeting basic functions. Summary of the invention
[0005] The purpose of the present invention is to provide an image classification method based on curriculum learning and adversarial training in order to overcome the defects of the above-mentioned prior art that adversarial samples affect the accuracy of image classification and require sacrificing the accuracy of image classification under undisturbed conditions.
[0006] The purpose of the present invention can be achieved by the following technical solutions:
[0007] An image classification method based on curriculum learning and adversarial training specifically includes the following steps:
[0008] S1, obtain the original image data set and obtain the initial classification model through model initialization;
[0009] S2, training the initial classification model according to the original image data set, judging whether the classification accuracy of the initial classification model reaches the limit value, if so, taking the initial classification model corresponding to the limit value as the first classification model and going to step S3;
[0010] S3, adding adversarial samples of the single-step iterative FGSM attack method to the original image data set to obtain a single-step adversarial image data set, performing adversarial training on the first classification model according to the single-step adversarial image data set, and judging whether the classification accuracy of the first classification model reaches the limit value, if so, taking the first classification model corresponding to the limit value as the second classification model and going to step S4;
[0011] S4, adding adversarial samples of the multi-step iterative PGD attack method to the single-step adversarial image dataset to obtain a multi-step adversarial image dataset, performing adversarial training on the second classification model according to the multi-step adversarial image dataset, and judging whether the classification accuracy of the second classification model reaches the limit value, if so, retaining the model parameters to generate the final robust visual classification model;
[0012] S5. Input the image data set to be classified into the final robust visual classification model to obtain the image classification result.
[0013] The optimization process of the present invention is based on the idea of game theory, and is composed of an internal problem and an external problem. The internal problem needs to generate an adversarial disturbance and the larger the adversarial disturbance, the better. The external problem needs to achieve "empirical risk minimization" based on the adversarial disturbance.
[0014] The method for judging whether the classification accuracy has reached the limit value in the steps S2 to S4 is specifically to detect whether the classification accuracy does not increase after a preset number of training rounds.
[0015] Furthermore, the preset number of training rounds is preferably 4 groups of training rounds.
[0016] The step S1 also includes adapting the original image data set according to the scale of the convolutional neural network and performing data enhancement on the original image data set.
[0017] Furthermore, the data enhancement operations of the original image data set include cropping, rotation and exposure.
[0018] The calculation formula of the adversarial sample in steps S3 and S4 is specifically as follows:
[0019]
[0020] Among them, x is the image feature, E is the process function, and δ is to satisfy certain constraints adversarial perturbation, x+δ is the generated adversarial sample, y is the data label, θ is the model parameter, D is the data distribution, and L is the loss function. When the inner layer reaches the optimal state, the optimal solution to the internal problem of adversarial training is approximately the optimal solution to the external problem.
[0021] Furthermore, in the step S3, the single-step adversarial image dataset is fitted with adversarial samples generated based on the Fast Gradient Sign Method (FGSM), and the formula for attacking the model is as follows:
[0022]
[0023] Among them, x * is the adversarial sample generated during adversarial training, and ∈ is the perturbation budget.
[0024] Furthermore, in step S4, the multi-step adversarial image dataset is fitted with adversarial samples generated based on the Projected Gradient Descent (PGD) method, and the formula for attacking the model is as follows:
[0025]
[0026] in, is the cutoff coefficient, proj is the projection function, which is used to ensure that each dimension of the input is projected within a valid range, and a certain number of iteration steps is set. is the process parameter, Adversarial samples generated during adversarial training.
[0027] The deep learning model used in the model training in steps S2 to S4 is a wide residual neural network. The wide residual neural network (WideResNet) has the advantage of "overparameterization". In the process of stochastic gradient descent optimization, a sufficiently wide deep neural network can be equivalent to fitting a data set that satisfies the Gaussian distribution. Compared with models of other structures, the wide residual neural network forgets less during "course learning".
[0028] The steps S3 and S4 also include using a regularization method to smooth the process of model adversarial training, specifically a regularization method called "Curriculum Dropout", which can smooth the training process and help eliminate the "overfitting" phenomenon that may occur during adversarial training by randomly discarding some neurons.
[0029] Compared with the prior art, the present invention has the following beneficial effects:
[0030] 1. The present invention gradually adds adversarial samples generated by the fast gradient sign method and the projected gradient descent method to the original clean image data set, so that the final image classification model still has high classification accuracy when the artificial intelligence is interfered by the adversarial samples, thereby improving the robustness of the classification model.
[0031] 2. The strategy design of the "course defense" algorithm of the present invention meets the idea of "baby learning to walk". "Memory activation" is achieved by adding knowledge related to the previous data set (course) into the latter data set (course). The adversarial sample is associated with the previously learned clean sample through fitting, which reduces "forgetting" in the course learning process. It has good accuracy in different test sets and effectively improves the test accuracy of the model on normal samples. BRIEF DESCRIPTION OF THE DRAWINGS
[0032] Figure 1 It is a structural schematic diagram of the present invention. DETAILED DESCRIPTION
[0033] The present invention is described in detail below in conjunction with the accompanying drawings and specific embodiments. This embodiment is implemented based on the technical solution of the present invention, and provides a detailed implementation method and specific operation process, but the protection scope of the present invention is not limited to the following embodiments.
[0034] Example
[0035] like Figure 1 As shown, an image classification method based on curriculum learning and adversarial training specifically includes the following steps:
[0036] S1, obtain the original image data set and obtain the initial classification model through model initialization;
[0037] S2, training the initial classification model according to the original image data set, judging whether the classification accuracy of the initial classification model reaches the limit value, if so, taking the initial classification model corresponding to the limit value as the first classification model and going to step S3;
[0038] S3, adding adversarial samples of the single-step iterative FGSM attack method to the original image data set to obtain a single-step adversarial image data set, performing adversarial training on the first classification model according to the single-step adversarial image data set, and judging whether the classification accuracy of the first classification model reaches the limit value, if so, taking the first classification model corresponding to the limit value as the second classification model and going to step S4;
[0039] S4, adding adversarial samples of the multi-step iterative PGD attack method to the single-step adversarial image dataset to obtain a multi-step adversarial image dataset, performing adversarial training on the second classification model according to the multi-step adversarial image dataset, and judging whether the classification accuracy of the second classification model reaches the limit value, if so, retaining the model parameters to generate the final robust visual classification model;
[0040] S5. Input the image data set to be classified into the final robust visual classification model to obtain the image classification result.
[0041] The optimization process of the present invention is based on the idea of game theory, and is composed of an internal problem and an external problem. The internal problem needs to generate an adversarial disturbance and the larger the adversarial disturbance, the better. The external problem needs to achieve "empirical risk minimization" based on the adversarial disturbance.
[0042] Step S4 also includes adding more adversarial samples with higher difficulty to perform adversarial training on the model and retaining the model parameters when the limit value is reached.
[0043] The method for determining whether the classification accuracy has reached the limit value in steps S2 to S4 is specifically to detect whether the classification accuracy does not increase after a preset number of training rounds.
[0044] The preset number of training rounds is preferably 4 sets of training rounds.
[0045] Step S1 also includes adapting the original image data set according to the scale of the convolutional neural network and performing data enhancement on the original image data set. In this embodiment, the image size is 32×32.
[0046] The data augmentation operations on the original image dataset include cropping, rotation, and exposure.
[0047] The calculation formula of adversarial samples in steps S3 and S4 is as follows:
[0048]
[0049] Among them, x is the image feature, E is the process function, and δ is to satisfy certain constraints adversarial perturbation, x+δ is the generated adversarial sample, y is the data label, θ is the model parameter, D is the data distribution, and L is the loss function. When the inner layer reaches the optimal state, the optimal solution to the internal problem of adversarial training is approximately the optimal solution to the external problem.
[0050] In step S3, the single-step adversarial image dataset is fitted with adversarial samples generated based on the Fast Gradient Sign Method (FGSM). The formula for attacking the model is as follows:
[0051]
[0052] Among them, x * is the adversarial sample generated during adversarial training, and ∈ is the perturbation budget.
[0053] In step S4, the multi-step adversarial image dataset is fitted with adversarial samples generated based on the Projected Gradient Descent (PGD) method. The formula for attacking the model is as follows:
[0054]
[0055] in, is the cutoff coefficient, proj is the projection function, which is used to ensure that each dimension of the input is projected within a valid range, and a certain number of iteration steps is set. is the process parameter, For adversarial samples generated during adversarial training, in this embodiment, the effective range is specifically [0, 1], and the number of iteration steps is specifically 10 or 20 steps.
[0056] The deep learning model used in model training in steps S2 to S4 is a wide residual neural network. The wide residual neural network (WideResNet) has the advantage of "overparameterization". In the process of stochastic gradient descent optimization, a sufficiently wide deep neural network can be equivalent to fitting a data set that satisfies the Gaussian distribution. Compared with models of other structures, the wide residual neural network forgets less during "course learning".
[0057] Steps S3 and S4 also include using a regularization method to smooth the process of model adversarial training, specifically a regularization method called "Curriculum Dropout". This method can smooth the training process and help eliminate the "overfitting" phenomenon that may occur during adversarial training by randomly discarding some neurons.
[0058] In this embodiment, the specific process of the “Course Defense” algorithm is as follows:
[0059] Input: training data distribution D, learning rate η, number of attack methods N (this method only involves the case of N = 2, and the number of N can be increased in practice), initial model parameters θ, attack method The difficulty increases in sequence;
[0060]
[0061] Returns the robust visual classification model parameters θ robust .
[0062] In the specific implementation, the perturbation budget is set to ε = 0.031, a 10-step PGD iterative attack method is used in training, and a 20-step PGD iterative attack method is used in verification. The abandonment rate of the "course abandonment" operation is as follows:
[0063]
[0064] Among them, γ is the growth rate of the control "abandonment rate", θ curriculum as well as is the changing model parameter, and γ is calculated as follows:
[0065] γ=10 / T
[0066] Among them, the maximum number of gradient descents is taken as T, and γ can be roughly determined by empirical rules.
[0067] In addition to the FGSM and PGD attack methods used in the aforementioned training, the MIM attack method is similar to the PGD attack method. This method performs momentum iteration during the gradient descent process to escape the local saddle point. The specific formula is as follows:
[0068]
[0069]
[0070] Among them, g t and g t+1 is the accumulated momentum value, μ is the attenuation coefficient, ε is the perturbation constraint, and T is the number of iterations.
[0071] Another test metric is based on the C&W attack, which assumes that the distance metric is not completely differentiable and that standard gradient descent cannot solve this problem well. The form of the optimization objective is as follows:
[0072] minimize c·f(x+δ)+|δ| ∞
[0073] Among them, f(x+δ) is the process function, |δ| ∞ is the generated adversarial sample;
[0074] Since gradient descent produces poor results, we can use an iterative attack to solve this problem, which takes the following form:
[0075]
[0076] Among them, c and τ represent the coefficients of the attack. After each iteration, for any perturbation δ, if all i have δ i <τ, the τ value can be reduced to 0.9; otherwise, the search is terminated.
[0077] The robust image classification experiment is carried out on the final robust visual classification model, and the test results on the CIFAR ten-class image dataset are shown in Table 1. Table 1 is as follows:
[0078] Table 1 Test results of CIFAR ten-class image dataset
[0079]
[0080] It can be seen from the results in Table 1 that the final robust visual classification model of the present invention not only has a high classification accuracy for the clean original data set, but also can maintain a high value for adversarial samples, which effectively solves the problem that adversarial samples affect the accuracy of image classification and need to sacrifice the accuracy of image classification in the undisturbed case.
[0081] In addition, it should be noted that the specific embodiments described in this specification may be named differently, and the above contents described in this specification are merely examples of the structure of the present invention. Any equivalent changes or simple changes made based on the structure, features and principles of the present invention are included in the protection scope of the present invention. Those skilled in the art of the present invention may make various modifications or supplements to the specific examples described or adopt similar methods, as long as they do not deviate from the structure of the present invention or exceed the scope defined by the claims, they should all fall within the protection scope of the present invention.
Claims
1. An image classification method based on curriculum learning and adversarial training, characterized in that: The specific steps include: S1, obtain the original image data set and obtain the initial classification model through model initialization; S2, training the initial classification model according to the original image data set, judging whether the classification accuracy of the initial classification model reaches the limit value, if so, taking the initial classification model corresponding to the limit value as the first classification model and going to step S3; S3, adding adversarial samples of the single-step iterative FGSM attack method to the original image data set to obtain a single-step adversarial image data set, performing adversarial training on the first classification model according to the single-step adversarial image data set, and judging whether the classification accuracy of the first classification model reaches the limit value, if so, taking the first classification model corresponding to the limit value as the second classification model and going to step S4; S4, adding adversarial samples of the multi-step iterative PGD attack method to the single-step adversarial image dataset to obtain a multi-step adversarial image dataset, performing adversarial training on the second classification model according to the multi-step adversarial image dataset, and judging whether the classification accuracy of the second classification model reaches the limit value, if so, retaining the model parameters to generate the final robust visual classification model; S5, inputting the image data set to be classified into the final robust visual classification model to obtain the image classification result; The calculation formula of the adversarial sample in steps S3 and S4 is specifically as follows: in, is the image feature, is a process function, To meet certain constraints The adversarial disturbance, is the generated adversarial sample, is the data label, are model parameters, is the data distribution, is the loss function; The formula of the attack mode of the single-step adversarial image dataset on the model in step S3 is as follows: in, is the adversarial sample generated during adversarial training. budget for disturbances; The formula of the attack method of the multi-step adversarial image dataset on the model in step S4 is as follows: in, is the cutoff coefficient, is the projection function, is the process parameter, Adversarial samples generated during adversarial training.
2. According to claim 1, the image classification method based on curriculum learning and adversarial training is characterized in that: The method for judging whether the classification accuracy has reached the limit value in the steps S2 to S4 is specifically to detect whether the classification accuracy does not increase after a preset number of training rounds.
3. The image classification method based on curriculum learning and adversarial training according to claim 2 is characterized in that: The preset number of training rounds is preferably 4 sets of training rounds.
4. The image classification method based on curriculum learning and adversarial training according to claim 1, characterized in that: The step S1 also includes adapting the original image data set according to the scale of the convolutional neural network and performing data enhancement on the original image data set.
5. The image classification method based on curriculum learning and adversarial training according to claim 4, characterized in that: The data augmentation operations of the original image dataset include cropping, rotation and exposure.
6. The image classification method based on curriculum learning and adversarial training according to claim 1, characterized in that: The deep learning model used in the model training in steps S2 to S4 is a wide residual neural network.
7. The image classification method based on curriculum learning and adversarial training according to claim 1, characterized in that: The steps S3 and S4 also include using a regularization method to smooth the process of model adversarial training.