Decoupling consistency regularization-based adversarial training method and image classification method

Abstract

The invention discloses an adversarial training method, and particularly relates to an adversarial training method and an image classification method based on decoupling consistency regularization. The training method comprises the following steps: performing data enhancement on a target sample image twice to obtain a first clean sample image and a second clean sample image; performing adversarial attack on the first clean sample image and the second clean sample image to obtain a first adversarial sample image and a second adversarial sample image; the first clean sample image and the second clean sample image are used to carry out iterative training on the first sub-learning device, the first adversarial sample image and the second adversarial sample image are used to carry out iterative training on the second sub-learning device, and loss values comprise consistency comparison loss and classification loss; and updating the network parameters of the image classification model based on the network parameters of the first sub-learner and the network parameters of the second sub-learner. The method provided by the embodiment of the invention can improve the classification accuracy of the image classification model.

Description

Technical Field

[0001] The present invention relates to an adversarial training method, and specifically relates to an adversarial training method and an image classification method based on decoupling consistency regularization. Background Art

[0002] Deep learning has attracted widespread attention due to its revolutionary applications in many fields, but its rise has also exposed some security flaws. Among a series of security risks, adversarial samples are the most widespread. Adversarial samples induce high-confidence misclassification by introducing subtle and imperceptible perturbations into clean sample images.

[0003] For example, in the field of autonomous driving, in most cases, the autonomous driving system can normally identify traffic signs and road markings. However, if the road surface is painted with inconspicuous interference information (equivalent to adversarial samples), if the vehicle cannot correctly identify the adversarial samples, it may make wrong decisions based on these erroneous information, such as driving into the opposite lane, posing a serious threat to road safety. In the field of face recognition, after adding noise to the original sample without adding disturbance to obtain the adversarial sample, the face recognition model may not be able to correctly classify the face, which will cause security problems in application scenarios such as face payment and face recognition unlocking.

[0004] Adversarial training enhances the model's ability to defend against such potential threats by integrating adversarial examples into the training process, thereby providing them with resilience to future unknown attacks. This approach can be viewed as a min-max optimization effort that aims to discover the most adverse state of adversarial examples through attacks and subsequently train the model to withstand these worst-case scenarios. Although adversarial training has been recognized as a favorable defense against adversarial examples, it still faces some limitations. One prominent obstacle is how to balance the classification accuracy of unattacked samples with the robustness of the model. The reason for the compromise between classification accuracy and robustness is very intuitive: in order to prevent the model from being deceived and to keep the classification accuracy from decreasing, the perturbations added during training must be subtle and imperceptible. However, if the training data is too "clean", the model will not be able to learn sufficiently, thereby compromising the model's ability to generalize to more complex datasets. The two diametrically opposed optimization directions of classification accuracy and model robustness result in classification accuracy and robust accuracy being like two ends of a scale, and improving one side will always have an adverse effect on the other. In addition to the trade-off between generalization and robustness, the stability of training poses another challenge: a significant decrease in robust accuracy is often observed during the test phase, indicating overfitting.

[0005] In the prior art, both single-step adversarial training and multi-step adversarial training train clean samples and adversarial samples in a coupled network. However, training two different types of data sets at the same time is detrimental, especially for the classification of natural data sets. First, data contamination is an inherent challenge. If adversarial samples are mistakenly regarded as clean samples, the training data will be contaminated, causing the model to produce inaccurate results when processing natural data. Secondly, this will also lead to a decrease in the generalization ability of the model, which may affect the generalization ability of the network after training when it receives the input of adversarial samples and produces the correct response. In the testing phase, when faced with anomalies that have never been seen before but may occur, the model may make wrong judgments.

[0006] It can be seen from this that the model obtained by the adversarial training method in the prior art is difficult to accurately identify adversarial samples, resulting in low classification accuracy of the model. Summary of the invention

[0007] The technical problem to be solved by the present invention is that the model obtained by the adversarial training method in the prior art is difficult to accurately identify adversarial samples, resulting in low classification accuracy of the model. In order to solve the above problem, the present invention provides an adversarial training method and an image classification method based on decoupled consistency regularization.

[0008] The content of the present invention includes:

[0009] In a first aspect, an embodiment of the present invention provides an adversarial training method based on decoupled consistency regularization, comprising:

[0010] Performing data enhancement twice on the target sample image to obtain a first clean sample image and a second clean sample image;

[0011] Performing an adversarial attack on the first clean sample image and the second clean sample image to obtain a first adversarial sample image and a second adversarial sample image;

[0012] Iteratively training a first sub-learner using the first clean sample image and the second clean sample image, and iteratively training a second sub-learner using the first adversarial sample image and the second adversarial sample image, wherein a loss value of the iterative training includes a consistency contrast loss and a classification loss, wherein the consistency contrast loss is determined based on a similarity between two output images, and the classification loss is determined based on a loss caused by a category prediction error in a classification task;

[0013] The network parameters of the image classification model are updated based on the network parameters of the first sub-learner and the network parameters of the second sub-learner.

[0014] Optionally, the iteratively training a first sub-learner using the first clean sample image and the second clean sample image, and the iteratively training a second sub-learner using the first adversarial sample image and the second adversarial sample image includes:

[0015] Iteratively training a first sub-learner using the first clean sample image and the second clean sample image to obtain the loss value corresponding to the first sub-learner, and iteratively training a second sub-learner using the first adversarial sample image and the second adversarial sample image to obtain the loss value corresponding to the second sub-learner;

[0016] Parameters of the first sub-learner are adjusted based on the loss value corresponding to the first sub-learner, and parameters of the second sub-learner are adjusted based on the loss value corresponding to the second sub-learner.

[0017] Optionally, after updating the network parameters of the image classification model based on the network parameters of the first sub-learner and the network parameters of the second sub-learner, the method further includes:

[0018] Initializing the first sub-learner and the second sub-learner based on the updated network parameters of the image classification model according to the target frequency;

[0019] The target frequency is used to represent the frequency with which the image classification model initializes the first sub-learner and the second sub-learner.

[0020] Optionally, the initializing the first sub-learner and the second sub-learner based on the updated network parameters of the image classification model according to the target frequency includes:

[0021] When the current training cycle t is equal to the target frequency c, in the next training cycle, the first sub-learner and the second sub-learner are initialized based on the updated network parameters of the image classification model, and t and c are both positive integers.

[0022] Optionally, performing an adversarial attack on the first clean sample image and the second clean sample image to obtain a first adversarial sample image and a second adversarial sample image includes:

[0023] An adversarial attack is performed on the first clean sample image and the second clean sample image to obtain a first adversarial sample image and a second adversarial sample image, wherein the adversarial attack is performed under the guidance of the second sub-learner.

[0024] Optionally, the consistency contrast loss of the first sub-learner is satisfy:

[0025]

[0026] The consistency contrast loss of the second sub-learner satisfy:

[0027]

[0028] Among them, JS[·||·] represents the measure of the similarity between two images. represents the output of the trained network, x1 is the first clean sample image, x2 is the second clean sample image, is the first adversarial sample image, is the second adversarial sample image, and τ is the temperature hyperparameter.

[0029] Optionally, the classification loss of the first sub-learner satisfy:

[0030]

[0031] The classification loss of the second sub-learner satisfy:

[0032]

[0033] Among them, θ clean is the parameter of the first sub-learner, θ robust is the parameter of the second sub-learner, y1 is the label of the first clean sample image, y2 is the label of the second clean sample image, is the label of the first adversarial sample image, is the label of the second adversarial sample image;

[0034] The loss value of the first sub-learner satisfy:

[0035]

[0036] The loss value of the second sub-learner satisfy:

[0037]

[0038] Among them, μ is a preset parameter used to characterize the importance of the consistency contrast loss in the model training process.

[0039] In a second aspect, an embodiment of the present invention provides an image classification method, comprising:

[0040] Input the image to be processed into the image classification model for image classification processing to obtain the target category;

[0041] Wherein, the image classification model is trained using the adversarial training method based on decoupled consistency regularization as described in the first aspect.

[0042] In a third aspect, an embodiment of the present invention provides an adversarial training device based on decoupled consistency regularization, comprising:

[0043] A data enhancement module, used for performing data enhancement twice on the target sample image to obtain a first clean sample image and a second clean sample image;

[0044] An adversarial attack module, configured to perform an adversarial attack on the first clean sample image and the second clean sample image to obtain a first adversarial sample image and a second adversarial sample image;

[0045] an iterative training module, configured to iteratively train a first sub-learner using the first clean sample image and the second clean sample image, and iteratively train a second sub-learner using the first adversarial sample image and the second adversarial sample image, wherein a loss value of the iterative training includes a consistency contrast loss and a classification loss, wherein the consistency contrast loss is determined based on a similarity between two output images, and the classification loss is determined based on a loss generated due to a category prediction error in a classification task;

[0046] An updating module is used to update the network parameters of the image classification model based on the network parameters of the first sub-learner and the network parameters of the second sub-learner.

[0047] In a fourth aspect, an embodiment of the present invention provides an image classification device, including:

[0048] The classification module is used to input the image to be processed into the image classification model for image classification processing to obtain the target category;

[0049] Wherein, the image classification model is trained using the adversarial training method based on decoupled consistency regularization as described in the first aspect.

[0050] In a fifth aspect, an embodiment of the present invention provides an electronic device, comprising: a memory, a processor, and a program stored in the memory and executable on the processor; the processor is used to read the program in the memory to implement the steps of the adversarial training method based on decoupled consistency regularization as described in the first aspect, or to implement the steps of the image classification method as described in the second aspect.

[0051] In a sixth aspect, an embodiment of the present invention provides a readable storage medium for storing a program, which, when executed by a processor, implements the steps of the adversarial training method based on decoupled consistency regularization as described in the first aspect, or implements the steps of the image classification method as described in the second aspect.

[0052] In an embodiment of the present invention, during model training, two different data enhancement processes are first used to generate two groups of enhanced samples of clean samples, and then these enhanced samples are attacked to generate two groups of corresponding adversarial samples. Then, a decoupling method is adopted to use two sub-learners with a shared architecture to perform iterative training using clean sample images and adversarial sample images respectively, instead of training these samples in a single coupled network, thereby avoiding compromises between competing goals and preventing suboptimal results. In addition, in order to alleviate robust overfitting and accelerate convergence, the method also adds a consistency regularization loss to stabilize the training process, and alleviates harmful adversarial disturbances by controlling the data distribution distance of samples before and after the adversarial attack. The beneficial effect of the present invention is that while improving the accuracy and precision of the classification results of the trained image classification model, the robustness of the model can also be improved, and the problems of trade-offs and overfitting can be effectively handled. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] Attached Figure 1 A flowchart of an adversarial training method for decoupled consistency regularization provided by an embodiment of the present invention;

[0054] Attached Figure 2 The overall architecture of regularized decoupling adversarial training provided by the embodiment of the present invention;

[0055] Attached Figure 3a One of the Gaussian data distributions of the natural image and adversarial sample provided in the embodiment of the present invention;

[0056] Attached Figure 3b The second Gaussian data distribution of natural images and adversarial samples provided by the embodiment of the present invention;

[0057] Attached Figure 4 A schematic diagram of a consistency regularization module provided in an embodiment of the present invention;

[0058] Attached Figure 5 A schematic diagram of the effects of different methods on trade-offs;

[0059] Attached Figure 6a A diagram showing the change of clean accuracy and robust accuracy over training cycles when training with a decoupled network;

[0060] Attached Figure 6b A schematic diagram of the change of clean accuracy and robust accuracy with training cycles when the method provided by an embodiment of the present invention is used for training;

[0061] Attached Figure 7 A schematic diagram of the robust accuracy of different μ values ​​against various types of attacks provided by an embodiment of the present invention;

[0062] Attached Figure 8 A schematic diagram of an adversarial training device based on decoupled consistency regularization provided by an embodiment of the present invention;

[0063] Attached Fig. 9 A flowchart of an image classification method provided by an embodiment of the present invention;

[0064] Attached Fig.10 A structural diagram of an image classification device provided by an embodiment of the present invention;

[0065] Attached Fig.11 A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0066] In the embodiments of the present application, the term "and / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent three situations: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the associated objects before and after are in an "or" relationship. In the embodiments of the present application, the term "plurality" refers to two or more than two, and other quantifiers are similar.

[0067] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0068] See also Figure 1 , the embodiment of the present invention provides an adversarial training method based on decoupled consistency regularization, aiming to effectively solve the problems of overfitting and trade-off. For ease of understanding, the adversarial training involved in this solution is first described below.

[0069] Adversarial training is usually regarded as a minimum-maximum optimization process, and the optimization objective is shown in formula (1):

[0070]

[0071] Among them, θ is the model parameter to be trained, δ represents the perturbation added to the input image, and ε is the perturbation boundary size, usually expressed as l p Norm measure (p is 0, 1, 2, ∞), inner function Indicates that pThe most unfavorable state of the adversarial sample is found under the restriction of norm perturbation. The outer function argmin(·) represents the optimal solution for the current worst case.

[0072] Figure 1 A flowchart of an adversarial training method based on decoupled consistency regularization provided by an embodiment of the present invention, wherein the method specifically comprises the following steps:

[0073] Step 101: Perform data enhancement twice on a target sample image to obtain a first clean sample image and a second clean sample image.

[0074] Step 102: Perform an adversarial attack on the first clean sample image and the second clean sample image to obtain a first adversarial sample image and a second adversarial sample image.

[0075] Step 103, iteratively training the first sub-learner using the first clean sample image and the second clean sample image, and iteratively training the second sub-learner using the first adversarial sample image and the second adversarial sample image, wherein the loss value of the iterative training includes a consistency contrast loss and a classification loss, wherein the consistency contrast loss is determined based on the similarity between the two output images, and the classification loss is determined based on the loss caused by category prediction errors in the classification task.

[0076] Step 104: Update the network parameters of the image classification model based on the network parameters of the first sub-learner and the network parameters of the second sub-learner.

[0077] See also Figure 2 The embodiment of the present invention also provides a regularized decoupled adversarial training (RDAT) network architecture, which can be applied to Figure 1 In the method shown.

[0078] In an embodiment of the present invention, during model training, two different data enhancement processes are first used to generate two groups of enhanced samples of clean samples, and then these enhanced samples are attacked to generate two groups of corresponding adversarial samples. Then, a decoupling method is adopted to use two sub-learners with a shared architecture to perform iterative training using clean sample images and adversarial sample images respectively, instead of training these samples in a single coupled network, thereby avoiding compromises between competing goals and preventing suboptimal results. In addition, in order to alleviate robust overfitting and accelerate convergence, the method also adds a consistency regularization loss to stabilize the training process, and alleviates harmful adversarial disturbances by controlling the data distribution distance of samples before and after the adversarial attack. The beneficial effect of the present invention is that while improving the accuracy and precision of the classification results of the trained image classification model, the robustness of the model can also be improved, and the problems of trade-offs and overfitting can be effectively handled.

[0079] It should be understood that the first sub-learner and the second sub-learner can be classifier models, and their specific structures are not limited here. The first sub-learner and the second sub-learner have the same network architecture for data sets from different data distributions. In this embodiment, the first sub-learner is trained on a clean data set (containing only clean sample images), and the second sub-learner is trained on an adversarial data set (containing only adversarial sample images).

[0080] Optionally, in some embodiments, step 103 includes:

[0081] Iteratively training a first sub-learner using the first clean sample image and the second clean sample image to obtain the loss value corresponding to the first sub-learner, and iteratively training a second sub-learner using the first adversarial sample image and the second adversarial sample image to obtain the loss value corresponding to the second sub-learner;

[0082] Parameters of the first sub-learner are adjusted based on the loss value corresponding to the first sub-learner, and parameters of the second sub-learner are adjusted based on the loss value corresponding to the second sub-learner.

[0083] For the convenience of description, the clean data set is recorded as The adversarial dataset is denoted as Due to different training schemes, the parameters of the first sub-learner and the second sub-learner are not shared. Each sub-learner independently completes the training task using a suitable optimization strategy on its corresponding data set, that is, the parameters θ of the model classifier (i.e., the first sub-learner and the second sub-learner) are optimized on the clean sample image and the adversarial sample image respectively. * , the network parameters θ of the first sub-learner clean and the network parameters θ of the second sub-learner robustThe optimization equation is as shown in the following formula (2):

[0084]

[0085]

[0086] Among them, argmin represents the minimum value of the solution formula {·}, Opt represents the optimizer used in the training model, is the cross-entropy loss function used in training, which controls the model to predict the input image to be close to the label. (x, y) represents the input image x with label y. For training The training expectations of the dataset.

[0087] Since the first sub-learner and the second sub-learner are trained and optimized separately, the parameters finally trained are optimal for their respective tasks. In order to establish a connection between the first sub-learner and the second sub-learner, it is necessary to establish a global learner (i.e., an image classification model) as a link for information exchange between the first sub-learner and the second sub-learner.

[0088] It should be understood that in order to avoid transferring the parameters of the sub-learners to each other at the beginning, which may lead to incorrect optimization directions due to insufficient learning, RDAT transfers the network parameters in the global learner in proportion to γ ​​after a certain period of training. At this time, the two sub-learners have already been fully learned.

[0089] In addition, RDAT chooses to transmit the model every few cycles instead of every training cycle, which can prevent the adversarial perturbations in dirty data from destroying the learner of clean data. By initializing the two sub-learners with the parameters of the global learner, it can ensure that each sub-learner can recognize the other party's data as much as possible when training its own data, improving the generalization performance of the model.

[0090] The network parameter update of the image classification model is shown in formula (3):

[0091]

[0092] Among them, α′ is the attenuation factor, which represents the weight ratio of the parameters transferred between the first sub-learner training clean samples and the second sub-learner training adversarial samples.

[0093] Optionally, in some embodiments, after step 104, the method further includes:

[0094] Initializing the first sub-learner and the second sub-learner based on the updated network parameters of the image classification model according to the target frequency;

[0095] The target frequency is used to represent the frequency with which the image classification model initializes the first sub-learner and the second sub-learner.

[0096] In this embodiment, the network parameters of the first sub-learner and the network parameters of the second sub-learner are mixed to obtain the network parameters of the image classification model, so that the image classification model can simultaneously identify adversarial samples and clean samples. The first sub-learner and the second sub-learner are initialized based on the updated network parameters of the image classification model according to the target frequency, and the model can be further trained on the updated network parameters of the image classification model, thereby further improving the performance of the image classification model. As an optional implementation, the number of times the image classification model is updated can be used as a criterion for determining whether the training is completed.

[0097] Optionally, in some embodiments, initializing the first sub-learner and the second sub-learner based on the updated network parameters of the image classification model according to the target frequency includes:

[0098] When the current training cycle t is equal to the target frequency c, in the next training cycle, the first sub-learner and the second sub-learner are initialized based on the updated network parameters of the image classification model, and t and c are both positive integers.

[0099] After training for m cycles (at this time, the first sub-learner and the second sub-learner have been completely learned, and the value of m is the preset value), if the current training cycle t is equal to the frequency c of initializing the sub-learner of the global learner, then in the t+1 cycle, use Initialize the parameters of the two sub-learners. Otherwise, the parameters of the two sub-learners follow formula (2). The complete training process can be expressed by the following formula (4):

[0100]

[0101]

[0102] in, represents the parameters of the image classification model after the parameters of the first sub-learner and the second sub-learner are mixed in proportion, mod represents t divided by c, t represents the current training cycle, and c represents the frequency of initializing the first sub-learner and the second sub-learner by the image classification model.

[0103] It should be understood that before the 200th epoch, the first sub-learner and the second sub-learner can only recognize clean samples or adversarial samples. In order to obtain a model that can recognize both types of samples at the same time, it is necessary to build a "bridge" to realize the transfer of information learned between the two sub-learners (expressed as the transfer of parameters). The global learner (i.e., the image classification model) plays the role of a "bridge" here. However, if the two model parameters are mixed in each training cycle, it may affect the training of the first sub-learner and the second sub-learner; if the frequency is too low, the first sub-learner and the second sub-learner may not be able to fully learn each other's knowledge.

[0104] Optionally, in some embodiments, performing an adversarial attack on the first clean sample image and the second clean sample image to obtain a first adversarial sample image and a second adversarial sample image includes:

[0105] An adversarial attack is performed on the first clean sample image and the second clean sample image to obtain a first adversarial sample image and a second adversarial sample image, wherein the adversarial attack is performed under the guidance of the second sub-learner.

[0106] It should be understood that the second sub-learner also guides the generation process of adversarial sample images, because Figure 2 In Block B shown, the outputs of the two adversarial sample images after passing through the model are controlled to be similar, which will reduce or even eliminate the impact of the adversarial perturbation. In order to obtain a model that is more robust to adversarial attacks, it is necessary to use a stronger attack method during training to obtain adversarial samples with greater perturbations for training. Therefore, the results of the second sub-learner can be used to guide the generation process of adversarial samples.

[0107] It should be understood that the network parameters of the first sub-learner and the network parameters of the second sub-learner are initialized based on the updated network parameters of the image classification model, and then the initialized network parameters of the first sub-learner and the second sub-learner can be iteratively trained respectively.

[0108] In this embodiment, after updating the network parameters of the image classification model (i.e., the global learner) based on the network parameters of the first sub-learner and the network parameters of the second sub-learner, the two sub-learners can be initialized with the parameters of the global learner to ensure that each sub-learner can recognize the other party's data as much as possible when training its own data, thereby improving the generalization performance of the model.

[0109] Although configuring different sub-learners for each data distribution for training can avoid data contamination, it also reduces the complexity of the training data. Because the training data is fixed and unchanged, blindly expanding the training iteration cycle cannot solve the problem of incomplete training, and in severe cases it may even cause overfitting. By expanding the training data, the sample complexity can be increased, thereby obtaining a more effective model.

[0110] In an embodiment of the present application, two different data enhancement processes are first used to generate two groups of enhanced samples of clean samples, and then these enhanced samples are attacked to generate two groups of corresponding adversarial samples.

[0111] For example, in a specific embodiment, the 10-step PGD attack method is used to add adversarial perturbations to two random images in CIFAR-10, and the Gaussian data distribution histograms of the two clean images and the adversarial images after the attack are shown. Figure 3a and Figure 3b As shown in the figure. The X-axis represents the normalized pixel value, and the Y-axis represents the number of pixels of the pixel value in the image. The dotted lines marked as B and D represent the average value of the image pixels, and the dotted lines marked as A and C represent the standard deviation lines of the image pixels. Compared with the original clean image, the Gaussian distribution of the adversarial sample has changed significantly, and the number of pixels for each pixel value has changed.

[0112] For example, first, the clean sample data is distributed An image in is randomly and independently augmented twice. The augmented images x1 and x2 obey Then, for the adversarial training channel, adversarial attacks are performed on x1 and x2 to obtain data distribution Adversarial examples and This operation allows the training data to have a more diverse Gaussian distribution to make up for the lack of training data complexity faced by decoupled training. Similar to other conventional training methods, the predicted values ​​are compared with the true labels, and the classification losses of clean samples and adversarial samples are calculated separately. In addition, given the limitations of computational complexity and the fact that adversarial samples after data enhancement will change the Gaussian distribution more dramatically than natural data, this may increase the time required for training, as standard adversarial training methods are inherently longer-lasting than other adversarial defense methods. In this case, RDAT adds a regularization module to impose constraints on the enhanced data. RDAT adds consistency regularization to the enhanced adversarial samples, and at the same time, performs the same operation on the two enhanced images of the clean samples to maximize training efficiency. For details, see Figure 4 .

[0113] exist Figure 4 In the figure, the clean sample is the target image, the two enhanced samples obtained by performing two random data enhancements on the target image are the first clean sample image and the second clean sample image, the adversarial samples obtained by performing adversarial attacks on the two enhanced samples are the first adversarial sample image and the second adversarial sample image, sub-learner-1 is the first sub-learner, and sub-learner-2 is the second sub-learner.

[0114] Optionally, in some embodiments, Figure 4 As shown, JS divergence is used to measure the similarity of the logarithmic distribution of two images through the model.

[0115] The consistency contrast loss of the first sub-learner satisfy:

[0116]

[0117] The consistency contrast loss of the second sub-learner satisfy:

[0118]

[0119] Among them, JS[·||·] represents the measure of the similarity between two images. represents the output of the trained network, x1 is the first clean sample image, x2 is the second clean sample image, is the first adversarial sample image, is the second adversarial sample image, and τ is the temperature hyperparameter.

[0120] Optionally, in some embodiments, in order to save training time, the training of clean accuracy and robust accuracy is performed simultaneously. The classification loss in the second sub-learner is calculated in the same way as the classification loss in the first sub-learner, both of which are standard cross entropy loss (CE loss). In addition, since x1 and x2 are classified at the same time, and For classification, the classification loss function contains two images.

[0121] The classification loss of the first sub-learner satisfy:

[0122]

[0123] The classification loss of the second sub-learner satisfy:

[0124]

[0125] θ is the model parameter to be trained, (x, y) is used to represent the input image with label y;

[0126] Specifically, θ clean is the parameter of the first sub-learner, θ robust is the parameter of the second sub-learner, y1 is the label of the first clean sample image, y2 is the label of the second clean sample image, is the label of the first adversarial sample image, is the label of the second adversarial sample image;

[0127] The loss value of the first sub-learner satisfy:

[0128]

[0129] The loss value of the second sub-learner satisfy:

[0130]

[0131] Among them, μ is a preset parameter used to characterize the importance of the consistency contrast loss in the model training process.

[0132] The effectiveness of this method will be illustrated below through experimental results and analysis. In this embodiment, ResNet-18 is used to conduct extensive experiments on CIFAR-10, CIFAR-100 and SVHN, including comparative experiments and hyperparameter setting experiments, to verify the effectiveness of the method provided in the embodiment of this application.

[0133] Basic settings: In this example, ResNet-18 is used as the overall network architecture, and PGD-10 is used to add adversarial perturbations. For other training settings, the hyperparameters are as follows: the maximum perturbation added is 8 / 255, the attack step size is 2 / 255, and the perturbation is l ∞ norm constraint, set the total number of training epochs to 300 and the batch size to 128.

[0134] Decoupled training: Since the decoupled network allows clean samples and adversarial samples to be trained separately, the training details can be found in the Generalist method, and different settings of the Stochastic Gradient Descent (SGD) optimizer are used to search for more suitable network parameters for natural training and adversarial training. Specifically, the momentum of the two optimizers is set to 0.9, and the weight decay is set to 5e-4 for natural training, with an initial learning rate of 0.1; the weight decay of adversarial training is set to 3.5e-3, with an initial learning rate of 0.01, and the learning rate is decayed to 1 / 10 at the 200th and 300th cycles of training using interpolation. After independent training for m cycles to learn sufficient knowledge, the parameters of the two sub-learners are mixed at a ratio of γ and passed to the global learner at a frequency interval of 5 cycles.

[0135] Data augmentation: Attack on two images augmented with AutoAugment, which have greater diversity and randomness than traditional data augmentation, allowing the network to learn more robust features that are more conducive to classification.

[0136] Model evaluation: In order to accurately evaluate the effectiveness of the method provided in the embodiment of the present application, the following attack modes are followed in this embodiment, namely PGD, MIM, C&W, AutoAttack (AA, regarded as a black box attack) and all its variants (APGD ce , APGD dlr , APGD t ,FAB t , and Squareattacks).

[0137] The experimental results are as follows:

[0138] Table 1 lists the main results of the experiment on CIFAR-10 using the network architecture of ResNet-18 (the best result is marked in bold, and the second best result is underlined). The method provided by this application (ie, RDAT) has been significantly improved in model robustness. More precisely, in Table 1, RDAT improves the defense capability against various attacks, including PGD, MIM, C&W, and AA (and its variants). For the ten listed attack methods, the robust accuracy of RDAT is 4.47% higher than the SOTA method for each attack on average.

[0139] Table 1 Evaluation results of different attack types using ResNet-18 architecture on CIFAR-10

[0140]

[0141] First, Natural training (NT) is a standard classification model training method and can be regarded as the upper bound of clean accuracy. However, the NT model trained only on clean samples does not have any adversarial robustness and cannot withstand any form of adversarial attacks. Secondly, Adversarial Training (AT) is the basic method for most current methods aimed at optimizing adversarial training strategies. In tests, RDAT has higher classification accuracy and model robustness than AT. Then, at the method level, Generalist also adopts a decoupling method similar to RDAT. Although Generalist achieves a clean accuracy second only to NT, it does not solve the problem of low robustness and is surpassed by APGD. ce The robust accuracy after the attack is even 6.94% lower than TRADES (γ=6). RDAT imposes constraints on the network based on decoupled training to limit the adverse effects of adversarial samples. Although the clean accuracy has dropped slightly, the robustness of the model has been greatly improved, and the robust accuracy has increased by 7.11% on average. Compared with the SOTA method TRADES, which is currently widely used for comparison, the method provided in the embodiment of the present application also has better performance. The clean accuracy of the method provided in the embodiment of the present application is slightly lower than TRADES (γ=1), but the robust accuracy has increased by 11.74% on average.

[0142] In addition, AA is a comprehensive attack method that includes multiple attack types such as PGD, MIM, Square Attack, etc. It generates adversarial samples by minimizing the distance between input and output to destroy the performance of the model. Since the attack strategy and technology can be adaptively adjusted according to different scenarios to improve the attack efficiency, it is usually used to evaluate the performance of the model in a black box environment. The robust accuracy of RDAT on AA is as high as 51.72%, which is of great significance for the research on enhancing the model to resist unseen attacks.

[0143] Table 2 Evaluation results of different attack types using ResNet-18 architecture on CIFAR-100

[0144]

[0145] Experiments on the CIFAR-100 dataset (as shown in Table 2) show similar results to those on the CIFAR-10 dataset. Compared with the SOTA methods listed for various attacks, RDAT's robust accuracy is improved by an average of 3.37%. Decoupled training reduces the complexity of training data, resulting in insufficient model training. This effect is more serious on relatively simple datasets. RDAT makes up for the lack of simple training data, so the improvement is more obvious on the CIFAR-10 dataset with fewer dataset types.

[0146] For the SVHN dataset, since existing methods are usually tested against PGD attack and AA attack, we choose to use PGD 20 The adversarial training method proposed in this chapter is evaluated by AA in order to better compare it with existing methods. As shown in Table 3, compared with the SOTA defense method, the model robustness of RDAT is greatly improved, among which the model robustness of PGD is significantly improved. 20 The robust accuracy of TRADES is improved by 5.28%, and the robust accuracy of AA is improved by 4.22%, while the classification accuracy decreases by less than 1%.

[0147] Following the evaluation metrics proposed by the MTARD method, we choose the sum of clean accuracy and robust accuracy as the evaluation criterion for the trade-off problem. Figure 5 The improvement effect of different methods on the trade-off is intuitively shown, where the white band represents clean accuracy and the gray band represents robust accuracy. It can be observed from the figure that the difference in clean accuracy between different models is not obvious, but RDAT is the only method whose sum of clean accuracy and robust accuracy exceeds 140, proving that it has achieved the best effect in the trade-off between classification accuracy and model robustness.

[0148] Table 3 Comparison using ResNet-18 architecture on SVHN dataset

[0149]

[0150] The experiment found that when attacking the best model checkpoint (this embodiment selects the maximum sum of clean accuracy and robust accuracy as a metric), there is a significant gap between the results of the Generalist method and the final model checkpoint. As shown in Table 4, the average gap between the results of the two checkpoints after the attack is 5.55% on CIFAR-10 and 7.34% on CIFAR-100. This significant difference indicates the presence of overfitting. Although the decoupled architecture can focus more on subtasks, the relative simplicity of the samples is not conducive to training the network. In contrast, the RDAT method is more stable in training because the average gap between the training results of the two model checkpoints on the CIFAR-10 and CIFAR-100 datasets is only 0.11% and 0.02%. In this case, it can be considered that the final checkpoint and the best checkpoint of the model are the same point, and the model successfully avoids overfitting.

[0151] exist Figure 6a and Figure 6b The accuracy curves of Generalist and RDAT are shown in Figure 2. Figure 6aAs shown in the figure, after 180 cycles of training, the clean accuracy is still rising slowly (line A), but its robust accuracy begins to drop significantly (line B), indicating that robust overfitting begins to appear. This is a disaster for training unknown models because we cannot know the appropriate training time and training cycle. RDAT solves this problem by adding a regularization module to impose restrictions on the trained network architecture. Figure 6b In Figure 2, after 180 epochs of training, the clean accuracy of RDAT also improves slightly, and the robust accuracy remains stable.

[0152] Table 4 Results of attacking the best checkpoint and final checkpoint of the model using different attack methods on the CIFAR-10 and CIFAR-100 datasets

[0153]

[0154]

[0155] Regarding the adjustment of hyperparameters:

[0156] The parameter m determines the degree of independent training of the two sub-learners for training clean samples and adversarial samples. In other words, the parameters of the global learner begin to be returned to the two sub-learners at a fixed frequency after m iterations to initialize them, so as to meet the information exchange needs between the two sub-learners. In order to study the optimal m value that can enable the sub-learners to fully learn, this section sets several sets of m values ​​on the CIFAR-10 dataset (the other hyperparameters are fixed to μ = 0.8 at this time), and the experimental results are shown in Table 5.

[0157] Table 5 records the results of 6 values ​​from 50 to 250, and the best results are bolded. It can be seen that as m increases, the accuracy is gradually increasing, and the best performance is achieved when m = 200. When m = 250, the classification accuracy and robust accuracy of the model will decrease. This phenomenon can be explained: when the training cycle is less than 200, the two sub-learners independently train their own tasks. As the training time increases, the model learns more and more fully, and the performance of the model is gradually improving. When the training cycle exceeds 200 times (m = 250), due to overfitting of their respective training data, the generalization ability of the two sub-learners is greatly reduced, resulting in a decrease in model performance. In most cases, the best effect is achieved when m = 200, so this paper chooses to start the information exchange between the two sub-learners after 200 training cycles.

[0158] Table 5 Accuracy corresponding to different m values ​​on the CIFAR-10 dataset

[0159]

[0160] For the two loss functions involved in training the network architecture, RDAT chooses to assign a weight to the consistency loss to balance the importance of the two losses instead of simply assigning the classification loss and consistency loss The value of m is fixed to 200, and the ResNet-18 architecture is used on the CIFAR-10 dataset to train the value of μ from 0 to 5 and observe the experimental results. Figure 7 The robustness of the model under various types of attacks corresponding to different μ values ​​is demonstrated.

[0161] When μ = 0, it means that the consistency regularization loss is not applied. At this time, the performance of the model is poor, which also confirms the effectiveness of the consistency regularization module in RDAT. When μ = 5, the consistency regularization loss is dominant at this time, and the importance of classification loss is reduced, which is not conducive to the classification accuracy of the model, resulting in unsatisfactory final classification accuracy. When μ ≤ 2, the performance of the model is insensitive to the value of μ. Considering that the clean accuracy is highest when μ = 0.8, and the robust performance against most attack methods is optimal, it is finally determined that μ = 0.8 is used to balance the classification loss and consistency loss.

[0162] Through the analysis of the above experimental results, it can be seen that the method provided in this embodiment first applies two independent enhancements to an input clean sample, and uses regularization to keep the data distribution of the two enhanced samples consistent after the attack. Subsequently, a dual-channel network architecture is used to train clean samples and adversarial samples separately. Extensive experiments have shown that RDAT not only achieves a better trade-off between classification accuracy and model robustness, but also solves the overfitting problem and stabilizes the adversarial training process.

[0163] See also Figure 8 , an embodiment of the present invention provides an adversarial training device 800 based on decoupled consistency regularization, comprising:

[0164] A data enhancement module 801 is used to perform data enhancement twice on the target sample image to obtain a first clean sample image and a second clean sample image;

[0165] An adversarial attack module 802 is used to perform an adversarial attack on the first clean sample image and the second clean sample image to obtain a first adversarial sample image and a second adversarial sample image;

[0166] an iterative training module 803, configured to iteratively train a first sub-learner using the first clean sample image and the second clean sample image, and iteratively train a second sub-learner using the first adversarial sample image and the second adversarial sample image, wherein the loss value of the iterative training includes a consistency contrast loss and a classification loss, wherein the consistency contrast loss is determined based on the similarity between the two output images, and the classification loss is determined based on a loss caused by a category prediction error in a classification task;

[0167] An updating module 804 is used to update the network parameters of the image classification model based on the network parameters of the first sub-learner and the network parameters of the second sub-learner.

[0168] Optionally, the iterative training module 803 includes:

[0169] an iterative training unit, configured to iteratively train a first sub-learner using the first clean sample image and the second clean sample image to obtain the loss value corresponding to the first sub-learner, and iteratively train a second sub-learner using the first adversarial sample image and the second adversarial sample image to obtain the loss value corresponding to the second sub-learner;

[0170] A parameter adjustment unit is used to adjust the parameters of the first sub-learner based on the loss value corresponding to the first sub-learner, and to adjust the parameters of the second sub-learner based on the loss value corresponding to the second sub-learner.

[0171] Optionally, the adversarial training device 800 based on decoupled consistency regularization further includes:

[0172] An initialization module, configured to initialize the first sub-learner and the second sub-learner based on the updated network parameters of the image classification model according to a target frequency;

[0173] The target frequency is used to represent the frequency with which the image classification model initializes the first sub-learner and the second sub-learner.

[0174] Optionally, the initialization module is specifically used to:

[0175] When the current training cycle t is equal to the target frequency c, in the next training cycle, the first sub-learner and the second sub-learner are initialized based on the updated network parameters of the image classification model, and t and c are both positive integers.

[0176] Optionally, the adversarial attack module 802 is specifically used for:

[0177] An adversarial attack is performed on the first clean sample image and the second clean sample image to obtain a first adversarial sample image and a second adversarial sample image, wherein the adversarial attack is performed under the guidance of the second sub-learner.

[0178] Optionally, the consistency contrast loss of the first sub-learner is satisfy:

[0179]

[0180] The consistency contrast loss of the second sub-learner satisfy:

[0181]

[0182] Among them, JS[·||·] represents the measure of the similarity between two images. represents the output of the trained network, x1 is the first clean sample image, x2 is the second clean sample image, is the first adversarial sample image, is the second adversarial sample image, and τ is the temperature hyperparameter.

[0183] Optionally, the classification loss of the first sub-learner satisfy:

[0184]

[0185] The classification loss of the second sub-learner satisfy:

[0186]

[0187] θ is the model parameter to be trained, (x, y) is used to represent the input image with label y;

[0188] Specifically, θ clean is the parameter of the first sub-learner, θ robust is the parameter of the second sub-learner, y1 is the label of the first clean sample image, y2 is the label of the second clean sample image, is the label of the first adversarial sample image, is the label of the second adversarial sample image;

[0189] The loss value of the first sub-learner satisfy:

[0190]

[0191] The loss value of the second sub-learner satisfy:

[0192]

[0193] Among them, μ is a preset parameter used to characterize the importance of the consistency contrast loss in the model training process.

[0194] The adversarial training device 800 based on decoupled consistency regularization provided in the embodiment of the present application can perform the above-mentioned Figure 1 The implementation principle and technical effect of the method embodiment shown are similar, and this embodiment will not be repeated here.

[0195] like Fig. 9 As shown, the embodiment of the present invention also provides an image classification method, including:

[0196] Step 901, inputting the image to be processed into the image classification model for image classification processing to obtain the target category;

[0197] Among them, the image classification model is trained using the above-mentioned adversarial training method based on decoupled consistency regularization.

[0198] Since in this embodiment, the image classification model obtained by model training using the above-mentioned adversarial training method based on decoupled consistency regularization is used for image classification processing, the image classification model in this embodiment can better identify clean sample images and adversarial sample images and has better classification accuracy, the details of which will not be elaborated here.

[0199] like Fig.10 As shown, the embodiment of the present invention further provides an image classification device 1000, comprising:

[0200] The classification module 1001 is used to input the image to be processed into the image classification model for image classification processing to obtain the target category;

[0201] Among them, the image classification model is trained using the above-mentioned adversarial training method based on decoupled consistency regularization.

[0202] The image classification device 1000 provided in the embodiment of the present application can perform the above-mentioned Fig. 9 The implementation principle and technical effect of the method embodiment shown are similar, and this embodiment will not be repeated here.

[0203] It should be noted that the division of units in the embodiments of the present application is schematic and is only a logical function division. There may be other division methods in actual implementation. In addition, each functional unit in each embodiment of the present application may be integrated into a processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0204] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a processor-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) or a processor (processor) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), disk or optical disk and other media that can store program codes.

[0205] like Fig.11 As shown, an embodiment of the present application provides an electronic device 1100, comprising: a memory 1102, a processor 1101, and a program stored in the memory 1102 and executable on the processor 1101; the processor 1101 is used to read the program in the memory 1102 to implement the steps in the adversarial training method based on decoupled consistency regularization as described above or to implement the steps in the image classification method as described above.

[0206] An embodiment of the present application also provides a readable storage medium, on which a program is stored. When the program is executed by a processor, it implements the various processes of the steps in the above-mentioned adversarial training method embodiment based on decoupled consistency regularization or the aforementioned image classification method, and can achieve the same technical effect. To avoid repetition, it will not be repeated here. Among them, the readable storage medium can be any available medium or data storage device that can be accessed by the processor, including but not limited to magnetic storage (such as floppy disk, hard disk, magnetic tape, magneto-optical disk (MO), etc.), optical storage (such as compact disk (CD), digital video disc (DVD), Blu-ray Disc (BD), high-definition versatile disc (HVD), etc.), and semiconductor memory (such as read-only memory (ROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read only memory (EEPROM), non-volatile memory (NAND FLASH), solid-state drive (Solid State Disk or Solid State Drive, SSD)), etc.

[0207] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element.

[0208] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, disk, CD), and includes a number of instructions for a terminal (which can be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in each embodiment of the present application.

[0209] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present application, ordinary technicians in this field can also make many forms without departing from the purpose of the present application and the scope of protection of the claims, all of which are within the protection of the present application.

Claims

1. An adversarial training method based on decoupled consistency regularization, characterized in that: include: Performing data enhancement twice on the target sample image to obtain a first clean sample image and a second clean sample image; Performing an adversarial attack on the first clean sample image and the second clean sample image to obtain a first adversarial sample image and a second adversarial sample image; Iteratively training a first sub-learner using the first clean sample image and the second clean sample image, and iteratively training a second sub-learner using the first adversarial sample image and the second adversarial sample image, wherein a loss value of the iterative training includes a consistency contrast loss and a classification loss, wherein the consistency contrast loss is determined based on a similarity between two output images, and the classification loss is determined based on a loss caused by a category prediction error in a classification task; The network parameters of the image classification model are updated based on the network parameters of the first sub-learner and the network parameters of the second sub-learner.

2. The method according to claim 1, characterized in that: The iterative training of the first sub-learner using the first clean sample image and the second clean sample image, and the iterative training of the second sub-learner using the first adversarial sample image and the second adversarial sample image include: Iteratively training a first sub-learner using the first clean sample image and the second clean sample image to obtain the loss value corresponding to the first sub-learner, and iteratively training a second sub-learner using the first adversarial sample image and the second adversarial sample image to obtain the loss value corresponding to the second sub-learner; Parameters of the first sub-learner are adjusted based on the loss value corresponding to the first sub-learner, and parameters of the second sub-learner are adjusted based on the loss value corresponding to the second sub-learner.

3. The method according to claim 2, characterized in that: After updating the network parameters of the image classification model based on the network parameters of the first sub-learner and the network parameters of the second sub-learner, the method further includes: Initializing the first sub-learner and the second sub-learner based on the updated network parameters of the image classification model according to the target frequency; The target frequency is used to represent the frequency with which the image classification model initializes the first sub-learner and the second sub-learner.

4. The method according to claim 3, characterized in that: Initializing the first sub-learner and the second sub-learner based on the updated network parameters of the image classification model according to the target frequency includes: When the current training cycle t is equal to the target frequency c, in the next training cycle, the first sub-learner and the second sub-learner are initialized based on the updated network parameters of the image classification model, and t and c are both positive integers.

5. The method according to claim 3, characterized in that: The performing an adversarial attack on the first clean sample image and the second clean sample image to obtain a first adversarial sample image and a second adversarial sample image includes: An adversarial attack is performed on the first clean sample image and the second clean sample image to obtain a first adversarial sample image and a second adversarial sample image, wherein the adversarial attack is performed under the guidance of the second sub-learner.

6. The method according to any one of claims 1 to 5, characterized in that: The consistency contrast loss of the first sub-learner satisfy: The consistency contrast loss of the second sub-learner satisfy: Among them, JS[·||·] represents the measure of the similarity between two images. represents the output of the trained network, x1 is the first clean sample image, x2 is the second clean sample image, is the first adversarial sample image, is the second adversarial sample image, and τ is the temperature hyperparameter.

7. The method according to claim 6, characterized in that: The classification loss of the first sub-learner satisfy: The classification loss of the second sub-learner satisfy: Among them, θ clean is the parameter of the first sub-learner, θ robust is the parameter of the second sub-learner, y1 is the label of the first clean sample image, y2 is the label of the second clean sample image, is the label of the first adversarial sample image, is the label of the second adversarial sample image; The loss value of the first sub-learner satisfy: The loss value of the second sub-learner satisfy: Among them, μ is a preset parameter used to characterize the importance of the consistency contrast loss in the model training process.

8. An image classification method, characterized in that: include: Input the image to be processed into the image classification model for image classification processing to obtain the target category; Wherein, the image classification model is trained using the adversarial training method based on decoupled consistency regularization described in any one of claims 1 to 7.

9. An adversarial training device based on decoupled consistency regularization, characterized in that: include: A data enhancement module, used for performing data enhancement twice on the target sample image to obtain a first clean sample image and a second clean sample image; An adversarial attack module, configured to perform an adversarial attack on the first clean sample image and the second clean sample image to obtain a first adversarial sample image and a second adversarial sample image; an iterative training module, configured to iteratively train a first sub-learner using the first clean sample image and the second clean sample image, and iteratively train a second sub-learner using the first adversarial sample image and the second adversarial sample image, wherein a loss value of the iterative training includes a consistency contrast loss and a classification loss, wherein the consistency contrast loss is determined based on a similarity between two output images, and the classification loss is determined based on a loss generated due to a category prediction error in a classification task; An updating module is used to update the network parameters of the image classification model based on the network parameters of the first sub-learner and the network parameters of the second sub-learner.

10. An image classification device, characterized in that: include: The classification module is used to input the image to be processed into the image classification model for image classification processing to obtain the target category; Wherein, the image classification model is trained using the adversarial training method based on decoupled consistency regularization described in any one of claims 1 to 7.

11. An electronic device, comprising: A memory, a processor, and a program stored in the memory and executable on the processor; wherein the processor is used to read the program in the memory to implement the steps of the adversarial training method based on decoupled consistency regularization as described in any one of claims 1 to 7, or to implement the steps of the image classification method as described in claim 8.

12. A readable storage medium for storing a program, characterized in that: When the program is executed by a processor, the steps in the adversarial training method based on decoupled consistency regularization as described in any one of claims 1 to 7 are implemented, or the steps in the image classification method as described in claim 8 are implemented.

Images