Neural network adversarial training method, device, computer equipment and storage medium

Abstract

The present application discloses a neural network adversarial training method, apparatus, computer equipment and storage medium, and relates to the field of computer technology. The method comprises: inputting a first original sample into a policy network, obtaining a first attack strategy corresponding to the first original sample output by the policy network, wherein the first attack strategy refers to a method of perturbing the first original sample; based on the first attack strategy and the network parameters of the target neural network, perturbing the first original sample to obtain a first adversarial sample; based on the first adversarial sample and the first sample annotation, adversarially training the target neural network and the policy network, wherein the first sample annotation is the sample annotation corresponding to the first original sample. The policy network generates different attack strategies at different adversarial training stages, and the target neural network improves its robustness to different attack strategies at different adversarial training stages.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of computer technology, and in particular to a method, apparatus, computer device, and storage medium for adversarial training of a neural network. Background Art

[0002] With the continuous development of computer technology, deep learning has been widely used in different technical fields, such as image classification, target detection, machine translation and other fields.

[0003] During the research of deep learning, it was discovered that there are adversarial samples, and machine learning methods are very sensitive to these adversarial samples. These adversarial samples are very similar to clean samples, but the trained recognition models or classification models can easily misclassify the categories of these adversarial samples. For example, the original recognition model determines that the image is a panda with a confidence level of 57.7%, but a small perturbation (i.e., an adversarial sample) is added to the image. The model believes with a confidence level of 99.3% that the perturbed image is a gibbon. Related technologies usually defend against adversarial sample attacks by conducting adversarial training on the model.

[0004] Obviously, in the adversarial training process, how to generate adversarial samples to improve the effectiveness of adversarial samples is the key to determining the robustness of the model. Summary of the Invention

[0005] The present invention provides a method, apparatus, computer device, and storage medium for adversarial training of a neural network. The technical solution is as follows:

[0006] According to one aspect of the present application, a method for adversarial training of a neural network is provided, the method comprising:

[0007] Inputting a first original sample into a policy network, obtaining a first attack strategy corresponding to the first original sample output by the policy network, wherein the first attack strategy refers to a processing method of perturbing the first original sample;

[0008] Based on the first attack strategy and network parameters of the target neural network, perturb the first original sample to obtain a first adversarial sample;

[0009] Based on the first adversarial sample and the first sample annotation, the target neural network and the policy network are adversarially trained, where the first sample annotation is the sample annotation corresponding to the first original sample.

[0010] According to another aspect of the present application, a neural network adversarial training apparatus is provided, the apparatus comprising:

[0011] A first strategy generation module is configured to input a first original sample into a strategy network and obtain a first attack strategy corresponding to the first original sample output by the strategy network, wherein the first attack strategy refers to a method of perturbing the first original sample;

[0012] a first perturbation processing module, configured to perform perturbation processing on the first original sample based on the first attack strategy and network parameters of the target neural network to obtain a first adversarial sample;

[0013] A first training module is configured to perform adversarial training on the target neural network and the policy network based on the first adversarial sample and a first sample annotation, where the first sample annotation is a sample annotation corresponding to the first original sample.

[0014] According to another aspect of the present application, a computer device is provided, comprising a processor and a memory, wherein the memory stores at least one program, and the at least one program is loaded and executed by the processor to implement the adversarial training method of a neural network as described above.

[0015] According to another aspect of the present application, a computer-readable storage medium is provided, wherein the storage medium stores at least one instruction, at least one program, a code set, or an instruction set, and the at least one instruction, the at least one program, the code set, or the instruction set is loaded and executed by a processor to implement the adversarial training method of the neural network as described above.

[0016] According to another aspect of the present application, a computer program product or computer program is provided, comprising computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the adversarial training method for a neural network provided in the aforementioned optional implementation.

[0017] The beneficial effects of the technical solutions provided in the embodiments of the present application include at least:

[0018] The concept of learnable attack strategies is introduced in the adversarial training process, thus providing a new adversarial training framework. The adversarial training framework contains a policy network and a target neural network. During the adversarial training process, the network parameters of the target neural network and the policy network are updated with different adversarial training stages, so that the policy network generates different attack strategies in different adversarial training stages. The target neural network improves its robustness to different attack strategies in different adversarial training stages, while improving the robustness of the target neural network and avoiding the use of a fixed attack strategy that limits the generalization performance of the target neural network. In addition, the original samples are introduced in the process of generating the attack strategy, so that the generation of the attack strategy also depends on the specific sample information of the original samples, thereby improving the personalization of the generated attack strategy. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0020] Figure 1 A flowchart of an adversarial training method for a neural network provided by an exemplary embodiment of the present application is shown;

[0021] Figure 2 is a comparative diagram of two adversarial training frameworks shown in an exemplary embodiment of the present application;

[0022] Figure 3 A flowchart of an adversarial training method for a neural network provided by another exemplary embodiment of the present application is shown;

[0023] Figure 4 A schematic diagram of the network structure of a policy network shown in an exemplary embodiment of the present application is shown;

[0024] Figure 5 A schematic diagram of the network structure of a policy network according to another exemplary embodiment of the present application is shown;

[0025] Figure 6 A flowchart of an adversarial training method for a neural network provided by another exemplary embodiment of the present application is shown;

[0026] Figure 7 A schematic diagram of the adversarial training process shown in an exemplary embodiment of the present application is shown;

[0027] Figure 8 This is a structural block diagram of a neural network adversarial training device provided by an exemplary embodiment of the present application;

[0028] Figure 9 A schematic structural diagram of a computer device provided in one embodiment of the present application is shown. DETAILED DESCRIPTION

[0029] In order to make the objectives, technical solutions and advantages of this application clearer, the implementation methods of this application will be further described in detail below with reference to the accompanying drawings.

[0030] First, the nouns involved in the embodiments of this application are introduced:

[0031] (1) Artificial Intelligence (AI): It is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technology in computer science that attempts to understand the essence of intelligence and produce a new type of intelligent machine that can respond in a similar way to human intelligence. Artificial intelligence is also the study of the design principles and implementation methods of various intelligent machines, so that machines have the functions of perception, reasoning and decision-making. The basic technologies of artificial intelligence generally include technologies such as sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing technology, operation / interaction systems, mechatronics, etc. Artificial intelligence software technology mainly includes computer vision technology, speech processing technology, natural language processing technology and machine learning / deep learning. It should be noted that the embodiments of this application mainly involve the field of machine learning technology in the field of artificial intelligence technology.

[0032] Please refer to Figure 1 , which shows a flowchart of a neural network adversarial training method provided by an exemplary embodiment of the present application. This embodiment uses a computer device as an example to illustrate the method, and the method includes:

[0033] In step 101, a first original sample is input into a policy network to obtain a first attack strategy corresponding to the first original sample output by the policy network. The first attack strategy refers to a method of performing perturbation processing on the first original sample.

[0034] Among them, the first original sample is a clean sample without added disturbance, and the first original sample can be obtained from the training sample set corresponding to the target neural network, or other clean samples that can be used to train the target neural network except the training sample set can be used; optionally, the specific sample content of the first original sample is determined by the prediction function of the target neural network. Schematically, if the target neural network is used for face recognition, the first original sample can be an image sample containing a face area; if the target neural network is used for object classification, the first original sample can be an image sample containing a target object; if the target neural network is used for text (word) recognition, the first original sample can be an image sample containing target text or target words.

[0035] Unlike related technologies that use a fixed attack strategy during the adversarial training of the target neural network, the embodiments of the present application introduce the concept of a "learnable attack strategy" by adding a policy network to the adversarial training framework so that the policy network can dynamically learn and generate a more effective attack strategy based on dynamic clean samples and the adversarial learning stage of the target neural network; in one possible implementation, the first original sample is input into the policy network to obtain the first attack strategy currently corresponding to the first original sample output by the policy network, so that the first original sample can be subsequently perturbation-processed based on the first attack strategy.

[0036] In the adversarial training framework proposed in this application, the policy network and the target neural network compete with each other during the training process. In the early stage of training, the policy network can successfully attack the target neural network by generating a less aggressive attack strategy, causing the target neural network to make prediction errors. As the target neural network gradually learns the adversarial samples, its robustness increases accordingly. The policy network needs to generate a stronger attack strategy to successfully attack the target neural network. That is, the policy network can dynamically generate different attack strategies in different adversarial training stages. That is, given the same first original sample (clean sample), the policy network will generate different attack strategies accordingly in different adversarial training stages.

[0037] Illustratively, the first attack strategy may be a PGD attack strategy, a Trades attack strategy, an Avmixup attack strategy, etc., or it may be a type of attack strategy customized by a developer. The embodiment of the present application does not limit the specific type of attack strategy.

[0038] Step 102: Based on the first attack strategy and the network parameters of the target neural network, the first original sample is perturbed to obtain a first adversarial sample.

[0039] Among them, network parameters refer to the parameters that the target neural network learns by itself during the training process, such as weights and biases. These network parameters are automatically updated during the training process.

[0040] After the policy network outputs the first attack strategy corresponding to the first original sample, in order to use the first attack strategy to attack the target neural network, the first attack strategy also needs to be used to process the first original sample (clean sample) to generate a first adversarial sample carrying a disturbance factor; and whether the adversarial sample can successfully attack the target neural network depends not only on the generated attack strategy, but also on the network parameters of the target neural network in the current training stage. Therefore, in one possible implementation, after the policy network generates the first attack strategy currently corresponding to the first original sample, the first original sample can be perturbation-processed based on the first attack strategy and the current network parameters of the target neural network. Since the current network parameters of the target neural network are referenced during the perturbation processing, an adversarial sample that is easier to attack the target neural network can be generated based on the current network parameters of the target neural network, thereby improving the effectiveness of the adversarial sample.

[0041] Step 103: Based on the first adversarial sample and the first sample annotation, adversarially train the target neural network and the policy network, where the first sample annotation is the sample annotation corresponding to the first original sample.

[0042] The significance of adversarial training is that, under the premise that the target neural network can recognize clean samples, when a small perturbation is added to the clean samples, an adversarial sample is obtained, and the target neural network cannot recognize the adversarial sample. Adversarial training is used to improve the robustness (or prediction accuracy) of the target neural network to adversarial samples. Therefore, in one possible implementation, after generating the first adversarial sample, the target neural network and the policy network can be adversarially trained based on the first adversarial sample and the first sample annotation, so that in the process of multiple rounds of training, the network parameters of the target neural network and the policy network can be updated with different adversarial training stages, thereby improving the prediction accuracy of the target neural network for adversarial samples.

[0043] It should be noted that in the process of adversarial training of the target neural network and the policy network, the target neural network and the policy network are not trained at the same time, but there is an iterative training process, that is, the network parameters of the target neural network are first updated using the first adversarial sample and the first sample annotation, and then the policy network is trained based on the trained target neural network; schematically, after the first adversarial sample is generated, the first adversarial sample is input into the target neural network to obtain the prediction result of the target neural network, and the prediction loss between the prediction result and the standard result (first sample annotation) is compared, and then the target neural network is trained based on the prediction loss; and then the policy network is trained using the trained target neural network.

[0044] Since in the process of generating the first adversarial sample based on the first original sample, only a tiny perturbation is added to the first original sample, and this tiny perturbation may be invisible to the naked eye, the sample annotation corresponding to the first adversarial sample should be consistent with the sample annotation corresponding to the first original sample. That is, in the adversarial training process, the target neural network is trained through the sample annotations corresponding to the first adversarial sample and the first original sample.

[0045] Optionally, the effectiveness of the strategy network generation strategy can be evaluated by evaluating the prediction accuracy of the trained target neural network for other adversarial samples, or by evaluating the prediction accuracy of the trained target neural network for other clean samples, or other customized effectiveness evaluation methods.

[0046] like Figure 2 As shown, it is a comparative diagram of two adversarial training frameworks shown in an exemplary embodiment of the present application. In the adversarial training framework using a fixed attack strategy, a fixed hand-crafted attack strategy 212 is artificially predefined during adversarial training. clean )211 is input into the Adversarial Example Generator 213, which performs perturbation processing on the clean sample 211 based on the fixed manual attack strategy 212 and the network parameters corresponding to the target neural network (Target Net) 214, and generates an adversarial example (X) corresponding to the clean sample 211. adv ) 215, and then train the target neural network 214 based on the adversarial sample 215. When the adversarial training framework using the dynamic attack strategy is used for adversarial training, the strategy network (Strategy Net) 222 learns and generates a dynamic attack strategy (Dynamic Attack Strategy) based on the clean sample 221. The adversarial sample generator 223 perturbs the clean sample 221 based on the dynamic attack strategy and the network parameters corresponding to the target neural network 224 to generate adversarial samples 225. The adversarial sample 225 is then used to train the target neural network 224, and the target neural network 224 provides feedback to update the strategy network 222.

[0047] To summarize, in the embodiments of the present application, the concept of a learnable attack strategy is introduced in the adversarial training process, thereby providing a new adversarial training framework. The adversarial training framework includes a policy network and a target neural network, so that during the adversarial training process, the network parameters of the target neural network and the policy network are updated with different adversarial training stages, so that the policy network generates different attack strategies in different adversarial training stages, and the target neural network improves its robustness to different attack strategies in different adversarial training stages, avoiding the use of fixed attack strategies and limiting the generalization performance of the target neural network; in addition, in the process of generating attack strategies, original samples are also introduced, so that the generation of attack strategies also depends on the specific sample information of the original samples, thereby improving the personalization of the generated attack strategies.

[0048] During adversarial training, the network parameters of the target neural network are updated by evaluating its prediction accuracy for adversarial samples. For the policy network, if it is necessary to evaluate whether the attack strategy generated by the policy network is effective, that is, it is necessary to evaluate whether the adversarial samples generated by the attack strategy are effective in improving the robustness of the target neural network, then it is necessary to evaluate the prediction accuracy of the updated target neural network for adversarial samples or clean samples.

[0049] Please refer to Figure 3 , which shows a flowchart of a neural network adversarial training method provided by another exemplary embodiment of the present application. This embodiment uses a computer device as an example to illustrate the method, and the method includes:

[0050] In step 301, a first original sample is input into a policy network to obtain a first attack strategy corresponding to the first original sample output by the policy network. The first attack strategy refers to a method of performing perturbation processing on the first original sample.

[0051] Taking a common PGD attack strategy as an example, this attack strategy includes three attack parameters: maximum perturbation intensity, attack step size, and attack iteration. Correspondingly, if the policy network needs to output this type of PGD attack strategy, the policy network needs to predict the specific values ​​of the three attack parameters. In an illustrative example, the process of the policy network generating a first attack strategy based on a first original sample can include steps 301A to 301C (i.e., step 301 can include steps 301A to 301C).

[0052] Step 301A: Input the first original sample into the policy network to obtain the predicted selection probability of each candidate parameter value output by the policy network.

[0053] Schematically, developers predefine the value space of attack strategies, where each attack strategy corresponds to at least one attack parameter, and each attack parameter corresponds to at least two candidate parameter values. During adversarial training, the strategy network predicts the specific parameter values ​​corresponding to each attack parameter based on the input original sample (clean sample); schematically, the attack strategy is set to correspond to three attack parameters {α, I, ∈}, and each attack parameter has K m There are candidate parameter values, for example, α can be 0.1, 0.2, 0.3, etc., I can be 10, 20, 30, etc., and ∈ can be 2, 6, 8, 10, etc.

[0054] In one possible implementation, the first original sample is input into the policy network, and the policy network predicts how to configure the attack parameters of the attack strategy, that is, predicts the predicted selection probability of each candidate parameter value corresponding to the attack strategy, and then selects the target parameter value based on the predicted selection probability. The combination of target parameter values ​​constitutes an attack strategy.

[0055] Optionally, the policy network can be composed of convolutional layers and fully connected layers, where the convolutional layers are used to extract sample features of the original samples, and the fully connected layers are used to predict target parameter values ​​corresponding to different attack parameters based on the extracted sample features. If the attack strategy generated by the policy network corresponds to multiple attack parameters, different fully connected layers are set, and different fully connected layers are used to predict target parameter values ​​corresponding to different attack parameters.

[0056] Optionally, the policy network can adopt an existing classification network, such as ResNet18, ResNet50, etc., or the developer can build the policy network by himself. The embodiment of the present application does not limit the network architecture of the policy network.

[0057] For the case where the policy network contains multiple fully connected layers, in one possible implementation, the first original sample is input into the policy network, and the predicted selection probability of each candidate parameter value corresponding to the mth attack parameter is output through the mth fully connected layer, where m is a positive integer.

[0058] Schematically, taking the policy network used to generate the PGD attack strategy as an example, since the PGD attack strategy contains three attack parameters, the corresponding policy network contains three fully connected layers. After the original sample is input into the policy network, the fully connected layers in the policy network predict the predicted selection probability of the candidate parameter values ​​corresponding to each attack parameter, and then determine the target parameter value of each attack parameter based on the predicted selection probability. The combination of target parameter values ​​is the first attack strategy.

[0059] like Figure 4As shown, it shows a schematic diagram of the network structure of a policy network shown in an exemplary embodiment of the present application. Taking the attack strategy containing three attack parameters as an example, the policy network is composed of a convolutional layer and three fully connected layers. The original sample 401 is input into the policy network 402, and the convolutional layer performs feature extraction to obtain sample features. Then, the fully connected layer predicts the target parameter values ​​corresponding to the maximum perturbation intensity (Preturbation), attack iteration (Iteration), and attack step size (Step Size).

[0060] Optionally, due to the diversity of attack strategies, in order to improve the defense performance of the target neural network against different types of attack strategies, in one possible implementation, the policy network can generate different types of attack strategies, and different types of attack strategies correspond to attack parameters of different parameter types and different parameter numbers, wherein the parameter type indicates the type of attack parameter contained in the attack strategy, and the parameter number indicates the number of attack parameters contained in the attack strategy. Schematically, if the policy network is used to generate a PGD attack strategy or a Trades attack strategy, the parameter types corresponding to this type of attack strategy are maximum perturbation intensity, attack step size, and attack iteration, and the number of parameters is three; if the policy network is used to generate an AVmixup attack strategy, the parameter types corresponding to this type of attack strategy are maximum perturbation intensity, attack step size, attack iteration, scaling factor, and two label smoothing factors, and the number of parameters is six. Optionally, the attack parameters corresponding to different types of attack strategies may also contain the same number of parameters, but different parameter types.

[0061] Optionally, since the network structure of the policy network is related to the attack parameters corresponding to the attack strategy, that is, the number of attack parameters affects the number of fully connected layers in the policy network, therefore, during the adversarial training process, the same policy network can be used to generate attack strategies with the same number of attack parameters. Schematically, if the policy network is used to generate the PGD attack strategy or the Trades attack strategy, the policy network contains three fully connected layers, and the attack parameters included in the first attack strategy are: maximum perturbation intensity, attack step size, and attack iteration; if the policy network is used to generate the AVmixup attack strategy, the policy network contains six fully connected layers, and the attack parameters included in the first attack strategy are: maximum perturbation intensity, attack step size, attack iteration, scaling factor, and two label smoothing factors.

[0062] like Figure 5As shown, it shows a schematic diagram of the network structure of a policy network shown in another exemplary embodiment of the present application. Taking the attack strategy containing six attack parameters as an example, the policy network is composed of a convolutional layer and six fully connected layers. The original sample 501 is input into the policy network 502, and the convolutional layer performs feature extraction to obtain sample features. Then, the fully connected layer predicts the target parameter values ​​corresponding to the maximum perturbation intensity, attack iteration, attack step size, scaling factor, first label smoothing factor, and second label smoothing factor, respectively.

[0063] Optionally, during the adversarial training process, in order to enable the target neural network to accurately predict adversarial samples generated by different types of attack strategies, after the same type of attack strategy is trained, a different strategy network can be built to generate another type of attack strategy and continue training the target neural network.

[0064] Step 301B: determine the candidate parameter value with the highest predicted selection probability among the same attack parameters as the target parameter value.

[0065] Since the policy network outputs the predicted selection probability corresponding to each candidate parameter value in the same attack parameter, it is also necessary to determine the target parameter from multiple candidate parameter values ​​based on the predicted selection probability. In one possible implementation, the candidate parameter value with the highest predicted selection probability in the same attack parameter can be determined as the target parameter value.

[0066] In principle, if the attack strategy includes three attack parameters: maximum perturbation intensity, attack step, and attack iteration, the candidate parameter values ​​corresponding to the maximum perturbation intensity are: 2, 4, 6, 7, and 8. The predicted selection probabilities obtained by the strategy network are: P2=0.1, P4=0.05, P6=0.05, P7=0.1, and P8=0.8. Correspondingly, 8 is determined as the target parameter value corresponding to the maximum perturbation intensity, and the target parameter value corresponding to the attack step is 2, and the target parameter value corresponding to the iteration step is 10.

[0067] Step 301C: Determine a set of target parameter values ​​corresponding to each attack parameter as a first attack strategy.

[0068] After obtaining the target parameter values ​​corresponding to each attack parameter, the set of target parameter values ​​is the generated first attack strategy. Schematically, if the target parameter value corresponding to the maximum disturbance intensity is 8, the target parameter value corresponding to the attack step is 2, and the target parameter value corresponding to the iteration step is 10, the first attack strategy is (8, 2, 10).

[0069] Step 302: Based on the first attack strategy and the network parameters of the target neural network, the first original sample is perturbed to obtain a first adversarial sample.

[0070] Schematically, the process of generating adversarial examples can be defined as:

[0071] x adv =g(x,a,w)=x+δ

[0072] Among them, x represents the first original sample, x adv Denotes the first adversarial example, δ represents the generated perturbation, a represents the first attack strategy generated by the policy network, and w represents the network parameters of the target neural network. The generation process of this adversarial example is equivalent to finding the optimal perturbation that maximizes the prediction loss of the target neural network given the attack strategy a. It should be noted that the generation process of this adversarial example is a function of a and w.

[0073] The implementation of step 302 may refer to step 102 and will not be described in detail in this embodiment.

[0074] Step 303: Train the target neural network based on the first adversarial sample and the first sample annotation.

[0075] In each round of training, both the target neural network and the policy network need to be trained. Due to the differences in the functions of the target neural network and the policy network, the losses used to evaluate the target neural network and the policy network are also different. Adversarial training is to improve the prediction accuracy of the target neural network for adversarial samples. Therefore, when evaluating the target neural network, the prediction accuracy of the target neural network for the first adversarial sample is used as an evaluation indicator. In one possible implementation, the target neural network is first trained based on the first adversarial sample and the first sample annotation, wherein the first sample annotation is the sample annotation of the first adversarial sample corresponding to the first original sample.

[0076] In an exemplary example, the process of training the target neural network based on the first adversarial sample and the first sample annotation (step 303) may include steps 303A to 303C.

[0077] Step 303A: input the first adversarial sample into the target neural network to obtain a first predicted label output by the target neural network.

[0078] In order to enable the target neural network to learn the characteristics of the adversarial sample and improve the prediction accuracy of the adversarial sample, it is necessary to train the target neural network based on the first adversarial sample and the first sample annotation. In one possible implementation, the first adversarial sample is input into the target neural network, and the first prediction annotation output by the target neural network can be obtained. Then, based on the first prediction annotation and the first sample annotation, the prediction loss of the target neural network for the first adversarial sample is determined.

[0079] Step 303B: Determine a first prediction loss based on the first prediction label and the first sample label. The first prediction loss is used to evaluate the prediction accuracy of the target neural network for the adversarial sample.

[0080] In one possible implementation, based on the first prediction label and the first sample label, the prediction loss of the target neural network for the first adversarial sample, i.e., the first prediction loss, can be calculated to evaluate the prediction accuracy of the target neural network for the adversarial sample, and then the network parameters of the target neural network are reversely updated based on the first prediction loss.

[0081] Schematically, the first prediction loss can be expressed as:

[0082] L0(w,θ)=L(f(x adv ,w),y0)

[0083] Among them, L0(w,θ) represents the first prediction loss, x adv represents the first adversarial sample, w represents the network parameters of the target neural network, f(x adv ,w) represents the first predicted label obtained after the first adversarial sample is input into the target neural network, y0 represents the first sample label, L(f(x adv ,w),y0) represents the difference between the predicted label and the true label of the adversarial example.

[0084] Optionally, when determining the first prediction loss based on the first prediction label and the first sample label, a cross-entropy loss function, a regularized proxy loss, or other loss functions can be used. The embodiments of the present application do not limit the loss function for calculating the loss.

[0085] Optionally, in different application scenarios, the target neural network has different prediction functions. Correspondingly, during the adversarial training process, there are also differences between the original samples and the sample annotations. Schematically, if the target neural network is used for image classification, the first original sample can be an image sample containing the target object, and the first sample annotation can be the object category. For example, if the target neural network is used to classify animal categories, the first original sample can be an image containing various animals, and the first sample annotation can be: animal categories such as pandas, birds, cats, and fish; if the target neural network is used for image recognition, the first original sample can also be an image sample containing the target object, and the first sample annotation is an object feature point. For example, if the target neural network is used for face recognition, the first original sample can be an image containing various faces, and the first sample annotation is facial feature points: facial contour, eye feature points, mouth feature points, etc.; if the target neural network is used for text recognition, the first original sample can be an image sample containing the target text, and the first sample annotation can be the text content; if the target neural network is used for emotion recognition, correspondingly, the first original sample can be various expression images, and the first sample annotation can be the emotion represented by the expression image: happy, disappointed, sad, etc.; or the first original sample can also be text expressing emotions, and the first sample annotation is the emotion expressed by the text.

[0086] Step 303C: train the target neural network based on the first prediction loss with the goal of improving the prediction accuracy of adversarial samples.

[0087] Since the purpose of adversarial training is to improve the prediction accuracy of the target neural network for adversarial samples, in one possible implementation, when training the target neural network based on the first prediction loss, it is also necessary to train with the goal of improving the prediction accuracy of adversarial samples, that is, to train the target neural network with the goal of reducing the first prediction loss. Schematically, after determining the first prediction loss, the error back propagation algorithm can be used to train the target neural network to update the network parameters of the target neural network and obtain the trained target neural network.

[0088] Step 304: train the policy network based on the trained target neural network.

[0089] Since the policy network is used to generate attack strategies for attacking the target neural network during adversarial training, when evaluating the policy network during adversarial training, it is necessary to evaluate whether the attack strategy generated by the policy network is effective for the target neural network. In other words, it is necessary to evaluate whether the adversarial samples generated by the attack strategy can enhance the robustness of the target neural network (or the prediction accuracy of the adversarial samples) after training the target neural network. Based on the characteristics of the evaluation policy network, that is, the trained target neural network is used as the evaluation basis, in one possible implementation, in each round of training, the target neural network and the policy network are both iteratively trained, that is, the first adversarial sample and the first sample are first used to label the target neural network. When the network parameters of the target neural network are updated, the policy network is trained based on the trained target neural network.

[0090] An effective attack strategy must not only improve the target neural network's prediction accuracy for adversarial examples, but also ensure that the target neural network maintains its accuracy for clean examples. Therefore, during the training of the strategy network, it is necessary to verify the target neural network's prediction accuracy for both adversarial examples and clean examples. For example, the process of training the strategy network (step 304) may include steps 304A and 304B.

[0091] Step 304A: Based on the trained target neural network, determine a second prediction loss or a third prediction loss. The second prediction loss is used to evaluate the prediction accuracy of the trained target neural network for the adversarial sample, and the third prediction loss is used to evaluate the prediction accuracy of the trained target neural network for the original sample.

[0092] In order to guide the learning of the policy network and enable the policy network to learn to generate more effective attack strategies, an indicator for evaluating the effectiveness of the attack strategy generated by the policy network is proposed. On the one hand, the trained target neural network can be used to predict other adversarial samples to evaluate the robustness of the trained target neural network to other adversarial samples, thereby indirectly evaluating whether the target neural network trained by the first adversarial sample has the effect of improving the robustness of the target neural network, and then evaluating whether the first attack strategy generated by the policy network has the effect of improving the robustness of the target neural network; on the other hand, the trained target neural network can be used to predict other clean samples to evaluate whether the trained target neural network has the performance of maintaining the prediction of clean samples, thereby indirectly evaluating whether the generated first attack strategy can maintain the performance of predicting clean samples while improving the robustness of the target neural network to adversarial samples; therefore, in one possible implementation, the policy network is trained by inputting the original sample or the adversarial sample into the trained target neural network to determine the second prediction loss or the third prediction loss to update the network parameters of the policy network.

[0093] In an illustrative example, the process of determining the second predicted loss and the third predicted loss may include the following steps (step 1 to step 4).

[0094] Step 1: Input the second adversarial sample into the trained target neural network to obtain a second predicted label output by the target neural network.

[0095] In order to evaluate whether the first adversarial sample generated by the first attack strategy can improve the robustness of the target neural network, in one possible implementation, a second adversarial sample is constructed. The second adversarial sample is different from the first adversarial sample. The second adversarial sample is input into the trained target neural network, and the trained target neural network predicts the second adversarial sample to obtain a second prediction annotation to evaluate whether the trained target neural network also has prediction accuracy for other adversarial samples.

[0096] The second adversarial sample may be a fixed adversarial sample or may be generated by a fixed attack strategy. In an exemplary example, the generation process of the second adversarial sample may include the following steps (steps 1 and 2):

[0097] 1. Get the i+1th network parameters corresponding to the trained target neural network.

[0098] Since the adversarial sample generation process also depends on the network parameters of the current target neural network, and the network parameters of the target neural network are updated with the progress of the adversarial training, the network parameters of the target neural network after the i-th adversarial training are set to the i-th network parameters, where i is a positive integer.

[0099] The second adversarial sample is used to evaluate the prediction accuracy of the trained target neural network for the adversarial sample. Accordingly, in the process of generating the second adversarial sample, it is necessary to obtain the i+1th network parameters, that is, the network parameters of the target neural network after training with the first adversarial sample, for the subsequent generation of the second adversarial sample.

[0100] 2. Based on the second attack strategy and the (i+1)th network parameters, the second original sample is perturbed to obtain a second adversarial sample. The second attack strategy is different from the first attack strategy.

[0101] The second attack strategy may be an attack strategy of the same type as the first attack strategy. Schematically, both the second attack strategy and the first attack strategy are PGD attack strategies, and only the attack parameters of the second attack strategy and the first attack strategy differ.

[0102] Optionally, the second attack strategy can be a fixed attack strategy, that is, at any training stage in the adversarial training process, the same second attack strategy is used to generate the second adversarial sample; optionally, the second attack strategy can also adopt a non-fixed attack strategy (dynamic attack strategy), for example, at least two second attack strategies are set in advance, and at any training stage in the adversarial training process, any second attack strategy is selected according to a preset probability to generate the second adversarial sample; or any second attack strategy is randomly selected to generate the second adversarial sample.

[0103] It should be noted that when the second attack strategy adopts a fixed attack strategy, this fixed attack strategy is a standard attack strategy commonly used in the field of adversarial training technology to detect whether the target neural network is robust (robustness to adversarial samples). That is to say, if the target neural network can resist the second attack strategy, it means that the adversarial training of the target neural network is completed. Schematically, the second attack strategy is the PGD attack strategy, and the attack parameters are: maximum perturbation intensity is 8, iteration step size is 10, and step size is 2. Optionally, no matter what attack strategy the policy network is used to generate, the second attack strategy can be used to generate a second adversarial sample to indirectly evaluate the effectiveness of the attack strategy generated by the policy network. Optionally, since the robustness of the target neural network will vary with different stages of adversarial training, schematically, in the early stage of training, the robustness of the target neural network is poor, and a weaker attack strategy can interfere with the prediction results of the target neural network; and as the adversarial training progresses, the robustness of the target neural network increases accordingly, and a stronger attack strategy is required in the later stage to interfere with the prediction results of the target neural network. Therefore, in order to avoid the weaker attack strategy being unable to provide effective interference in the later stage of training, the second attack strategy should be set to an attack strategy with greater interference intensity to achieve the effect of improving the robustness of the target neural network.

[0104] In one possible implementation, based on the second attack strategy and the (i+1)th network parameter, the second original sample is perturbed to obtain a second adversarial sample.

[0105] Step 2: Determine a second prediction loss based on a second prediction label and a second sample label, where the second sample label is a sample label of the second adversarial sample corresponding to the second original sample.

[0106] In order to evaluate the prediction accuracy of the trained target neural network for the second adversarial sample, in one possible implementation, a second sample annotation corresponding to the second original sample is obtained, and then a second prediction loss is determined based on the second prediction annotation and the second sample annotation, and then the strategy network is trained based on the second prediction loss.

[0107] In an illustrative example, the second prediction loss can be defined as:

[0108]

[0109] Among them, L1(θ) represents the second prediction loss, represents the second adversarial sample, w* represents the network parameters of the trained target neural network, It represents the second predicted label obtained after the second adversarial sample is input into the trained target neural network, and y1 represents the second sample label.

[0110] Step three: input the third original sample into the trained target neural network to obtain a third predicted label output by the target neural network.

[0111] In order to evaluate whether the trained target neural network can maintain the performance of predicting clean samples, in one possible implementation, a third original sample (clean sample) is obtained, the third original sample is input into the trained target neural network, and the third prediction label obtained by the target neural network output prediction is used for subsequent calculation of the prediction loss for the clean sample.

[0112] Step 4: Determine a third prediction loss based on the third prediction label and the third sample label, where the third sample label is the sample label corresponding to the third original sample.

[0113] In an illustrative example, the third prediction loss can be defined as:

[0114] L2(θ)=-L(f(x2,w*),y2)

[0115] Among them, L2(θ) represents the third prediction loss, x2 represents the third original sample, w* represents the network parameters of the trained target neural network, f(x2,w*) represents the third prediction label obtained after inputting the third original sample into the trained target neural network, and y2 represents the third sample label.

[0116] Optionally, after obtaining the third prediction label, a third prediction loss of the target neural network for the third original sample may be determined based on the third prediction label and the third sample label.

[0117] It should be noted that the first prediction loss, the second prediction loss and the third prediction loss can all be calculated using the same loss function, for example, the cross-entropy loss function is used, or the first prediction loss, the second prediction loss and the third prediction loss are calculated using different loss functions, for example, the first prediction loss uses a regularized proxy loss, and the second prediction loss and the third prediction loss can use a cross-entropy loss function. The embodiments of the present application do not constitute a limitation to this.

[0118] Optionally, when determining the second predicted loss and the third predicted loss, the second predicted loss can be determined first and then the third predicted loss; or the third predicted loss can be determined first and then the second predicted loss; the embodiment of the present application does not limit the order of determining the second predicted loss and the third predicted loss.

[0119] Step 304B: training the policy network based on the first prediction loss, and at least one of the second prediction loss and the third prediction loss, with the training goal of improving the prediction accuracy of the adversarial samples and the original samples.

[0120] Since the policy network aims to generate an attack strategy that can improve the robustness of the target neural network, and the robustness of the target neural network is determined by the prediction accuracy of the target neural network for adversarial samples and original samples, therefore, in the process of adversarial training of the policy network, the training goal should be to improve the prediction accuracy for adversarial samples and original samples, and the policy network should be trained based on the first prediction loss, and at least one of the second prediction loss and the third prediction loss.

[0121] In the adversarial training framework adopted in this application, the target neural network and the policy network are trained iteratively, that is, the target neural network is first trained using the first prediction loss, and then the second prediction loss and the third prediction loss are determined based on the updated target neural network, and then the first prediction loss, the second prediction loss and the third prediction loss are used to train the policy network.

[0122] Optionally, in the process of training the policy network, only one of the second prediction loss or the third prediction loss can be used for training. For example, the first prediction loss and the second prediction loss can be used to train the policy network, or the first prediction loss and the third prediction loss can be used to train the policy network.

[0123] In summary, the adversarial training formula proposed in the embodiments of this application can be defined as:

[0124]

[0125] Among them, w represents the network parameters of the target neural network, θ represents the network parameters of the policy network, D represents the training sample set, p(a|x;θ) represents the attack strategy, and E a～p(a|x；θ)Denotes the policy network, L0 denotes the first prediction loss, L1 denotes the second prediction loss, and L2 denotes the third prediction loss. This indicates that the adversarial training formula is a minimax problem, with internal optimization involving the policy network's parameters. The two networks compete to minimize or maximize the same objective. The target neural network (corresponding to the parameter w) learns to adjust its parameters to defend against adversarial examples generated by the attacking policy, while the policy network (corresponding to the parameter θ) learns to improve its attack strategy based on given examples to attack the target neural network. At the beginning of the training phase, the target neural network is fragile, and weak attacks can easily cause it to make prediction errors. Therefore, the policy network can easily generate effective attack strategies, and because both weak and strong attacks can succeed, the attack strategies can be diversified. As the adversarial training process progresses, the target neural network becomes more robust, and the policy network must learn to generate more aggressive attack strategies to create stronger adversarial examples. Therefore, the game mechanism can gradually improve the robustness of the target neural network as the policy network improves.

[0126] In this embodiment, iterative training is used in the adversarial training process, that is, the target neural network is first trained based on the first prediction loss, and then the prediction accuracy of the target neural network for clean samples and other adversarial samples is determined based on the trained target neural network, so that the effectiveness of the adversarial strategy generated by the policy network can be indirectly evaluated by evaluating the network performance of the trained target neural network. As the adversarial training progresses, the policy network can learn to generate more aggressive attack strategies to create stronger adversarial samples, and the dynamic update of the attack strategy can be realized cyclically.

[0127] In order to enable the target neural network to apply different types of attack strategies and improve the robustness of the target neural network under various types of attack strategies, during the adversarial training process of the neural network, multiple strategy networks can be set up at one time to generate different types of attack strategies. After completing the adversarial training of one type of attack strategy, the adversarial training process of another type of attack strategy can be continued.

[0128] Please refer to Figure 6 , which shows a flowchart of a neural network adversarial training method provided by another exemplary embodiment of the present application. This embodiment uses a computer device as an example to illustrate the method, and the method includes:

[0129] Step 601: Input a first original sample into a first strategy network to obtain a first attack strategy output by the first strategy network. The first attack strategy belongs to a first type of attack strategy.

[0130] In this embodiment, in the process of building an adversarial training framework, at least two strategy networks are built to generate at least two types of attack strategies. Schematically, a first strategy network is built in the adversarial training framework to generate a PGD attack strategy (the first strategy network contains three parallel fully connected layers for respectively predicting attack parameters such as maximum perturbation intensity, attack step size, and attack iteration). At the same time, a second strategy network is built to generate an AVmixup attack strategy (the second strategy network contains six parallel fully connected layers for respectively predicting attack parameters such as maximum perturbation intensity, attack step size, attack iteration, scaling factor, and two label smoothing factors).

[0131] In one possible implementation, a first strategy network is used to generate a first type of attack strategy to train a target neural network against it. Correspondingly, a first original sample is input into the first strategy network to obtain a first attack strategy output by the first strategy network. The first original sample is then perturbed according to the first attack strategy and the network parameters of the target neural network to obtain a first adversarial sample. The target neural network is then trained based on the first adversarial sample, and the network parameters of the strategy network are updated according to the trained target neural network.

[0132] Step 602: Based on the first attack strategy and the network parameters of the target neural network, the first original sample is perturbed to obtain a first adversarial sample.

[0133] Step 603: Based on the first adversarial sample and the first sample annotation, the target neural network and the policy network are adversarially trained, where the first sample annotation is the sample annotation corresponding to the first original sample.

[0134] The implementation of step 602 and step 603 can refer to the above embodiment, and will not be described in detail in this embodiment.

[0135] In step 604, in response to the target neural network completing adversarial training on the first type of attack strategy, the first original sample is input into the second strategy network to obtain a third attack strategy corresponding to the first original sample output by the second strategy network, and the third attack strategy belongs to the second type of attack strategy.

[0136] After the target neural network completes adversarial training for the first type of attack strategy, that is, under the premise that the target neural network has a certain prediction accuracy for the adversarial samples generated by the first type of attack strategy, in order to improve the defense performance of the target neural network against other types of attack strategies, the first original sample can be re-input into the second strategy network, and the second strategy network generates a third attack strategy corresponding to the second type of attack strategy, and then the adversarial samples generated by the third attack strategy are used to continue training the target neural network.

[0137] Step 605: Train the target neural network based on the adversarial samples generated by the second type of attack strategy.

[0138] In one possible implementation, after the second type of attack strategy is generated, the first original sample can be perturbed based on the second type of attack strategy and the network parameters of the target neural network to generate an adversarial sample to train the target neural network.

[0139] It should be noted that the process of using the second type of attack strategy to train the target neural network is similar to the training process of using the first type of attack strategy to train the target neural network in the above embodiment. Please refer to the above embodiment, and the embodiments of this application will not be repeated here.

[0140] In this embodiment, by building different strategy networks, during the adversarial training process, the strategy networks can be switched to generate different types of attack strategies, so that the trained target neural network can have defense performance against multiple attack strategies, further improving the prediction accuracy of the target neural network.

[0141] like Figure 7As shown, it shows a schematic diagram of the adversarial training process shown in an exemplary embodiment of the present application. In the embodiment of the present application, the adversarial training framework includes a policy network 701, an adversarial sample generator 702 and a target neural network 703; the adversarial training process involves the update of the corresponding network parameters of the two networks 701 and the target neural network 703, so the single adversarial training process can be divided into two stages. The first stage is to update the network parameters of the target neural network 703. The training process corresponding to this stage is: inputting the first original sample (a clean sample without adding disturbance) into the policy network 701 to obtain the first attack strategy output by the policy network 701, and then the adversarial sample generator 702 performs perturbation processing on the first original sample based on the first attack strategy and the i-th network parameter (the i-th network parameter is the network parameter updated by the target neural network after the i-th adversarial training) to obtain the first adversarial sample; in order to evaluate the prediction accuracy of the target neural network for the first adversarial sample, the first adversarial sample is input into the target neural network 703 to obtain the first predicted label output by the target neural network 703, and based on the first predicted label and the first sample label (the sample label corresponding to the first original sample), the first adversarial sample is generated. The first stage is to train the target neural network 703 with a prediction loss to update the network parameters of the target neural network 703 (i.e., update the network parameters from the i-th network parameters to the i+1-th network parameters); the other stage is to update the network parameters of the strategy network 701. The training process of this stage includes two prediction loss determination processes. The first is to evaluate the prediction accuracy of the trained target neural network for other adversarial samples, that is, to perturb the second original sample according to the second attack strategy and the i+1-th network parameters to obtain a second adversarial sample, and then input the second adversarial sample into the trained target neural network 703 to obtain a second prediction label, thereby determining the second prediction loss based on the second prediction label and the second sample label (the sample label corresponding to the second original sample); the second is to evaluate the prediction accuracy of the trained target neural network for other clean samples, that is, to input the third original sample into the trained target neural network 703 to obtain a third prediction label, thereby determining the third prediction loss based on the third prediction label and the third sample label; after determining the first prediction loss, the second prediction loss and the third prediction loss, the strategy network 701 can be trained according to the sum of the first prediction loss, the second prediction loss and the third prediction loss; subsequently, according to Figure 7 The process shown repeats the training to improve the robustness of the target neural network 703.

[0142] In an illustrative example, the adversarial training method for a neural network provided in the above embodiments can be used as a plug-and-play component and combined with existing adversarial training to jointly train a target neural network.

[0143] Table 1 shows the experimental comparison results of adversarial training using the CIFAR10 dataset. The CIFAR10 dataset contains 50,000 training images and 10,000 test images, covering 10 categories of images with a size of 32×32.

[0144] Table 1

[0145]

[0146]

[0147] As can be seen from Table 1, taking the attack strategy PGD to train the target neural network Resnet18 as an example, if the fixed PGD strategy is used for adversarial training, the prediction accuracy of the trained target neural network for clean samples is 56.23, and the prediction accuracy for adversarial samples (adversarial samples generated by PGD-10) is 52.85; while using the dynamic PGD attack strategy for adversarial training, the prediction accuracy of the trained target neural network for clean samples is 58.15, and the prediction accuracy for adversarial samples (adversarial samples generated by PGD-10) is 54.23. Regardless of whether it is a clean sample or an adversarial sample, the prediction accuracy of the target neural network is significantly improved; it can be seen that the adversarial training method provided in the embodiment of the present application can improve the prediction accuracy of the target neural network.

[0148] In another illustrative example, the experimental comparison results of adversarial training using the CIFAR100 dataset are shown in Table 2. The CIFAR100 dataset also contains 50,000 32×32 training images and 10,000 test images, but it covers 100 classes.

[0149] Table 2

[0150]

[0151]

[0152] As shown in Table 2, taking the target neural network Resnet18 trained with the AVmixup attack strategy as an example, if a fixed AVmixup strategy is used for adversarial training, the prediction accuracy of the trained target neural network for clean samples is 63.81%, and the prediction accuracy for adversarial samples (adversarial samples generated by PGD-10) is 31.62%. However, if a dynamic AVmixup attack strategy is used for adversarial training, the prediction accuracy of the trained target neural network for clean samples is 65.04%, and the prediction accuracy for adversarial samples (adversarial samples generated by PGD-10) is 34.20. The prediction accuracy of the target neural network is significantly improved for both clean and adversarial samples.

[0153] In another illustrative example, the experimental comparison results of adversarial training using the Tiny ImageNet dataset are shown in Table 3. The Tiny ImageNet database is a subset collected from the ImageNet database, which covers 200 classes, and each class has 600 images of size 64×64.

[0154] Table 3

[0155]

[0156] As shown in Table 3, taking the target neural network PreActResnet18 trained with the Trades attack strategy as an example, if the fixed Trades attack strategy is used for adversarial training, the prediction accuracy of the trained target neural network for clean samples is 39.16%, and the prediction accuracy for adversarial samples (adversarial samples generated by PGD-10) is 16.34%. However, if the dynamic Trades attack strategy is used for adversarial training, the prediction accuracy of the trained target neural network for clean samples is 43.20%, and the prediction accuracy for adversarial samples (adversarial samples generated by PGD-10) is 18.82. The prediction accuracy of the target neural network is significantly improved for both clean and adversarial samples.

[0157] The following are device embodiments of the present application. For details not described in detail in the device embodiments, reference may be made to the above method embodiments.

[0158] Figure 8 This is a block diagram of a neural network adversarial training device provided by an exemplary embodiment of the present application. The device includes:

[0159] A first strategy generation module 801 is configured to input a first original sample into a strategy network and obtain a first attack strategy corresponding to the first original sample output by the strategy network, where the first attack strategy refers to a method of perturbing the first original sample;

[0160] A first perturbation processing module 802 is configured to perform perturbation processing on the first original sample based on the first attack strategy and network parameters of the target neural network to obtain a first adversarial sample;

[0161] The first training module 803 is used to perform adversarial training on the target neural network and the policy network based on the first adversarial sample and the first sample annotation, where the first sample annotation is the sample annotation corresponding to the first original sample.

[0162] Optionally, the first training module 803 includes:

[0163] A first training unit, configured to train the target neural network based on the first adversarial sample and the first sample annotation;

[0164] The second training unit is used to train the policy network based on the trained target neural network.

[0165] Optionally, the first training unit is further configured to:

[0166] Inputting the first adversarial example into the target neural network to obtain a first predicted label output by the target neural network;

[0167] Determining a first prediction loss based on the first prediction label and the first sample label, wherein the first prediction loss is used to evaluate the prediction accuracy of the target neural network for the adversarial sample;

[0168] The target neural network is trained based on the first prediction loss with the goal of improving the prediction accuracy of adversarial samples.

[0169] Optionally, the second training unit is further used to:

[0170] Determining a second prediction loss or a third prediction loss based on the trained target neural network, wherein the second prediction loss is used to evaluate the prediction accuracy of the trained target neural network for the adversarial sample, and the third prediction loss is used to evaluate the prediction accuracy of the trained target neural network for the original sample;

[0171] The policy network is trained based on the first prediction loss and at least one of the second prediction loss and the third prediction loss, with improving the prediction accuracy of adversarial samples and original samples as a training goal.

[0172] Optionally, the second training unit is further used to:

[0173] Inputting a second adversarial example into the trained target neural network to obtain a second predicted label output by the target neural network;

[0174] The second prediction loss is determined based on the second prediction label and the second sample label, where the second sample label is a sample label of the second adversarial sample corresponding to the second original sample.

[0175] Optionally, the network parameter is the i-th network parameter corresponding to the target neural network after the i-th adversarial training, where i is a positive integer;

[0176] The device further comprises:

[0177] An acquisition module, configured to acquire the i+1th network parameter corresponding to the trained target neural network;

[0178] A second perturbation processing module is used to perform perturbation processing on the second original sample based on a second attack strategy and the (i+1)th network parameter to obtain the second adversarial sample, where the second attack strategy is different from the first attack strategy.

[0179] Optionally, the second training unit is further used to:

[0180] Inputting a third original sample into the trained target neural network to obtain a third predicted label output by the target neural network;

[0181] The third prediction loss is determined based on the third prediction label and the third sample label, where the third sample label is a sample label corresponding to the third original sample.

[0182] Optionally, the first attack strategy includes at least one attack parameter, and each attack parameter corresponds to at least two candidate parameter values;

[0183] The first strategy generation module 801 includes:

[0184] a prediction unit, configured to input the first original sample into the policy network and obtain a predicted selection probability of each candidate parameter value output by the policy network;

[0185] A first determining unit is configured to determine the candidate parameter value with the highest predicted selection probability among the same attack parameters as the target parameter value;

[0186] The second determining unit is configured to determine a set of attack parameters corresponding to the target parameter values ​​as the first attack strategy.

[0187] Optionally, the policy network is composed of a convolutional layer and a fully connected layer, and different fully connected layers are used to predict target parameter values ​​corresponding to different attack parameters;

[0188] The prediction unit is further configured to:

[0189] The first original sample is input into the policy network, and the predicted selection probability of each candidate parameter value corresponding to the mth attack parameter is output through the mth fully connected layer, where m is a positive integer.

[0190] Optionally, the strategy network is used to generate different types of attack strategies, wherein different types of attack strategies correspond to attack parameters of different parameter types and different numbers of parameters.

[0191] Optionally, different strategy networks are used to generate different types of attack strategies;

[0192] The first strategy generation module 801 includes:

[0193] a strategy generating unit, configured to input the first original sample into a first strategy network to obtain the first attack strategy output by the first strategy network, where the first attack strategy belongs to a first category of attack strategies;

[0194] The device further comprises:

[0195] a second strategy generating module, configured to, in response to the target neural network completing adversarial training on the first type of attack strategy, input the first original sample into a second strategy network, and obtain a third attack strategy corresponding to the first original sample output by the second strategy network, where the third attack strategy belongs to the second type of attack strategy;

[0196] The second training module is used to train the target neural network based on the adversarial samples generated by the second type of attack strategy.

[0197] Optionally, the target neural network is used for image classification, the first original sample is an image sample containing a target object, and the first sample annotation is an object category;

[0198] The target neural network is used for image recognition, the first original sample is an image sample containing a target object, and the first sample annotation is an object feature point;

[0199] The target neural network is used for text recognition, the first original sample is an image sample containing target text, and the first sample annotation is text content.

[0200] To summarize, in the embodiments of the present application, the concept of a learnable attack strategy is introduced in the adversarial training process, thereby providing a new adversarial training framework. The adversarial training framework includes a policy network and a target neural network, so that during the adversarial training process, the network parameters of the target neural network and the policy network are updated with different adversarial training stages, so that the policy network generates different attack strategies in different adversarial training stages, and the target neural network improves its robustness to different attack strategies in different adversarial training stages. While improving the robustness of the target neural network, it avoids adopting a fixed attack strategy and limiting the generalization performance of the target neural network; in addition, in the process of generating the attack strategy, the original sample is also introduced, so that the generation of the attack strategy also depends on the specific sample information of the original sample, thereby improving the personalization of the generated attack strategy.

[0201] Please refer to Figure 9, which shows a structural diagram of a computer device provided in one embodiment of the present application, which can be used to implement the image processing method performed by the computer device provided in the above embodiment. The computer device 900 includes a central processing unit (CPU) 901, a system memory 904 including a random access memory (RAM) 902 and a read-only memory (ROM) 903, and a system bus 905 connecting the system memory 904 and the central processing unit 901. The computer device 900 also includes a basic input / output system (I / O) 906 that helps transmit information between various devices in the computer, and a large-capacity storage device 907 for storing an operating system 913, application programs 914 and other program modules 915.

[0202] The basic input / output system 906 includes a display 908 for displaying information and an input device 909 such as a mouse and a keyboard for user input. The display 908 and the input device 909 are both connected to the central processing unit 901 via an input / output controller 910 connected to the system bus 905. The basic input / output system 906 may also include an input / output controller 910 for receiving and processing input from a variety of other devices such as a keyboard, a mouse, or an electronic stylus. Similarly, the input / output controller 910 also provides output to a display screen, a printer, or other types of output devices.

[0203] The mass storage device 907 is connected to the central processing unit 901 via a mass storage controller (not shown) connected to the system bus 905. The mass storage device 907 and its associated computer-readable medium provide non-volatile storage for the computer device 900. In other words, the mass storage device 907 may include a computer-readable medium (not shown) such as a hard disk or a CD-ROM (Compact Disc Read-Only Memory) drive.

[0204] Without loss of generality, the computer-readable medium may include computer storage media and communication media. Computer storage media include volatile and non-volatile, removable and non-removable media implemented by any method or technology for storing information such as computer-readable instructions, data structures, program modules or other data. Computer storage media include RAM, ROM, EPROM (Erasable Programmable Read-Only Memory), EEPROM (Electrically Erasable Programmable Read-Only Memory), flash memory or other solid-state storage technologies, CD-ROM, DVD (Digital Video Disc) or other optical storage, tape cassettes, magnetic tape, disk storage or other magnetic storage devices. Of course, those skilled in the art will appreciate that the computer storage media are not limited to the above-mentioned ones. The above-mentioned system memory 904 and mass storage device 907 can be collectively referred to as memory.

[0205] According to various embodiments of the present application, the computer device 900 may also be connected to a remote computer on a network such as the Internet for operation. That is, the computer device 900 may be connected to a network 912 via a network interface unit 911 connected to the system bus 905. Alternatively, the network interface unit 911 may be used to connect to other types of networks or remote computer systems (not shown).

[0206] The memory also includes one or more programs, which are stored in the memory and configured to be executed by the one or more central processing units 901 .

[0207] The present application also provides a computer-readable storage medium, wherein the computer-readable storage medium stores at least one instruction, at least one program, code set, or instruction set, and the at least one instruction, at least one program, code set, or instruction set is loaded and executed by a processor to implement the adversarial training method for a neural network provided by any of the above exemplary embodiments.

[0208] An embodiment of the present application provides a computer program product or computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the adversarial training method for a neural network provided in the above-mentioned optional implementation.

[0209] Those skilled in the art will understand that all or part of the steps to implement the above embodiments may be accomplished by hardware, or by a program to instruct the relevant hardware, and the program may be stored in a computer-readable storage medium, which may be a read-only memory, a disk, or an optical disk, etc.

[0210] The above description is merely an optional embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present application shall be included in the scope of protection of the present application.

Claims

1. A neural network adversarial training method, characterized in that: The method comprises: Inputting a first original sample into a policy network, obtaining a first attack strategy corresponding to the first original sample output by the policy network, wherein the first attack strategy refers to a processing method of perturbing the first original sample; Based on the first attack strategy and network parameters of the target neural network, perturb the first original sample to obtain a first adversarial sample; Based on the first adversarial sample and the first sample annotation, adversarially training the target neural network and the policy network, where the first sample annotation is the sample annotation corresponding to the first original sample; Wherein, when the target neural network is used for image classification, the first original sample is an image sample containing a target object, and the first sample label is an object category; In the case where the target neural network is used for image recognition, the first original sample is an image sample containing a target object, and the first sample annotation is an object feature point; When the target neural network is used for text recognition, the first original sample is an image sample containing target text, and the first sample annotation is text content.

2. The method according to claim 1, characterized in that The adversarial training of the target neural network and the policy network based on the first adversarial sample and the first sample annotation includes: Training the target neural network based on the first adversarial sample and the first sample annotation; The policy network is trained based on the trained target neural network.

3. The method according to claim 2, characterized in that The training of the target neural network based on the first adversarial sample and the first sample annotation includes: Inputting the first adversarial example into the target neural network to obtain a first predicted label output by the target neural network; Determining a first prediction loss based on the first prediction label and the first sample label, wherein the first prediction loss is used to evaluate the prediction accuracy of the target neural network for the adversarial sample; The target neural network is trained based on the first prediction loss with the goal of improving the prediction accuracy of adversarial samples.

4. The method according to claim 3, characterized in that The step of training the policy network based on the trained target neural network includes: Determining a second prediction loss or a third prediction loss based on the trained target neural network, wherein the second prediction loss is used to evaluate the prediction accuracy of the trained target neural network for the adversarial sample, and the third prediction loss is used to evaluate the prediction accuracy of the trained target neural network for the original sample; The policy network is trained based on the first prediction loss and at least one of the second prediction loss and the third prediction loss, with improving the prediction accuracy of adversarial samples and original samples as a training goal.

5. The method according to claim 4, characterized in that The determining the second prediction loss or the third prediction loss based on the trained target neural network includes: Inputting a second adversarial example into the trained target neural network to obtain a second predicted label output by the target neural network; The second prediction loss is determined based on the second prediction label and the second sample label, where the second sample label is a sample label of the second adversarial sample corresponding to the second original sample.

6. The method according to claim 5, characterized in that The network parameter is the i-th network parameter corresponding to the target neural network after the i-th adversarial training, where i is a positive integer; Before inputting the second adversarial example into the trained target neural network to obtain a second prediction label output by the target neural network, the method further includes: Obtaining the i+1th network parameter corresponding to the trained target neural network; Based on a second attack strategy and the (i+1)th network parameter, the second original sample is perturbed to obtain the second adversarial sample, where the second attack strategy is different from the first attack strategy.

7. The method according to claim 4, characterized in that The determining the second prediction loss or the third prediction loss based on the trained target neural network includes: Inputting a third original sample into the trained target neural network to obtain a third predicted label output by the target neural network; The third prediction loss is determined based on the third prediction label and the third sample label, where the third sample label is a sample label corresponding to the third original sample.

8. The method according to any one of claims 1 to 7, characterized in that: The first attack strategy includes at least one attack parameter, each attack parameter corresponding to at least two candidate parameter values; Inputting the first original sample into the strategy network and obtaining a first attack strategy corresponding to the first original sample output by the strategy network includes: Inputting the first original sample into the policy network to obtain the predicted selection probability of each candidate parameter value output by the policy network; The candidate parameter value with the highest predicted selection probability among the same attack parameters is determined as the target parameter value; A set of target parameter values ​​corresponding to each attack parameter is determined as the first attack strategy.

9. The method according to claim 8, characterized in that The strategy network consists of convolutional layers and fully connected layers, and different fully connected layers are used to predict target parameter values ​​corresponding to different attack parameters; Inputting the first original sample into the policy network to obtain the predicted selection probability of each candidate parameter value output by the policy network includes: The first original sample is input into the policy network, and the predicted selection probability of each candidate parameter value corresponding to the mth attack parameter is output through the mth fully connected layer, where m is a positive integer.

10. The method according to claim 8, characterized in that The strategy network is used to generate different types of attack strategies, wherein different types of attack strategies correspond to attack parameters of different parameter types and different parameter quantities.

11. The method according to any one of claims 1 to 7, characterized in that: Different strategy networks are used to generate different types of attack strategies; Inputting the first original sample into the strategy network and obtaining a first attack strategy corresponding to the first original sample output by the strategy network includes: Inputting the first original sample into a first strategy network to obtain the first attack strategy output by the first strategy network, where the first attack strategy belongs to a first category of attack strategies; After training the policy network based on the trained target neural network, the method further includes: In response to the target neural network completing adversarial training on the first type of attack strategy, inputting the first original sample into a second strategy network to obtain a third attack strategy corresponding to the first original sample output by the second strategy network, where the third attack strategy belongs to the second type of attack strategy; The target neural network is trained based on adversarial samples generated based on the second type of attack strategy.

12. A neural network adversarial training device, characterized in that: The device comprises: A first strategy generation module is configured to input a first original sample into a strategy network and obtain a first attack strategy corresponding to the first original sample output by the strategy network, wherein the first attack strategy refers to a method of perturbing the first original sample; a first perturbation processing module, configured to perform perturbation processing on the first original sample based on the first attack strategy and network parameters of the target neural network to obtain a first adversarial sample; A first training module is configured to perform adversarial training on the target neural network and the policy network based on the first adversarial sample and a first sample annotation, where the first sample annotation is a sample annotation corresponding to the first original sample; Wherein, when the target neural network is used for image classification, the first original sample is an image sample containing a target object, and the first sample label is an object category; In the case where the target neural network is used for image recognition, the first original sample is an image sample containing a target object, and the first sample annotation is an object feature point; When the target neural network is used for text recognition, the first original sample is an image sample containing target text, and the first sample annotation is text content.

13. A computer device, characterized in that: The computer device includes a processor and a memory, wherein the memory stores at least one program, and the at least one program is loaded and executed by the processor to implement the adversarial training method of the neural network as described in any one of claims 1 to 11.

14. A computer-readable storage medium, characterized in that The readable storage medium stores at least one program, and the at least one program is loaded and executed by the processor to implement the adversarial training method of the neural network as described in any one of claims 1 to 11.

Images