Clean adversarial training based on diffusion processing, adversarial sample generation method and device

Abstract

The embodiment of the present application provides a method and device for clean adversarial training and adversarial sample generation based on diffusion processing, which relates to the field of artificial intelligence technology. The above-mentioned clean adversarial sample generation method includes: performing diffusion processing on the original sample and the first noise to obtain a first noisy sample; using the model to be trained to process the original sample to obtain a first result; using the model to be trained to process the first noisy sample to obtain a second result; based on the first result and the second result, obtaining the noise divergence of the first noisy sample relative to the original sample; based on the noise divergence and the first noisy sample, generating a second noise, superimposing the first noisy sample and the second noise to obtain a second noisy sample; performing inverse diffusion processing on the second noisy sample to remove separation noise from the second noisy sample; based on the noise removal result, obtaining an adversarial sample. Applying the solution provided in the embodiment of the present application, it is possible to generate adversarial samples needed in the process of adversarial training.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a method and device for clean adversarial training and adversarial sample generation based on diffusion processing. Background Art

[0002] When using a trained neural network model for data processing, the model may be attacked, resulting in erroneous output. Taking the image classification model as an example, the image to be input into the model may be tampered with by the attacker in a way that is difficult for the human eye to detect. The tampered image can be called an adversarial sample. In this way, after the adversarial sample is input into the model, the output generated by the model for the adversarial sample may be different from the output generated for the original image, that is, the model generates an erroneous output.

[0003] In related technologies, adversarial training can be used to reduce the probability of a model generating erroneous outputs. The principle of adversarial training is: during the model training phase, adversarial samples that are likely to cause the model to generate erroneous outputs are generated, and then the model is trained using the adversarial samples to improve the trained model's ability to resist adversarial samples, that is, to reduce the probability of the model generating erroneous outputs for adversarial samples and improve the robustness of the model.

[0004] In view of the above situation, it is necessary to provide an adversarial sample generation scheme to generate adversarial samples needed for adversarial training. Summary of the invention

[0005] The purpose of the embodiments of the present application is to provide a clean adversarial training and adversarial sample generation method and device based on diffusion processing to generate adversarial samples required for adversarial training. The specific technical solution is as follows:

[0006] In a first aspect, an embodiment of the present application provides a method for generating a clean adversarial sample based on diffusion processing, the method comprising:

[0007] Performing diffusion processing on the original sample and the first noise to obtain a first noisy sample;

[0008] Using the model to be trained to perform data processing on the original sample to obtain a first processing result;

[0009] Using the model to be trained to perform data processing on the first noisy sample to obtain a second processing result;

[0010] Based on the first processing result and the second processing result, obtaining a noise divergence of the first noisy sample relative to the original sample;

[0011] generating a second noise based on the noise divergence and the first noisy sample, and superimposing the first noisy sample and the second noise to obtain a second noisy sample;

[0012] performing inverse diffusion processing on the second noisy samples, predicting separation noise, and removing the separation noise from the second noisy samples;

[0013] Based on the noise removal results, adversarial samples are obtained.

[0014] In a second aspect, an embodiment of the present application provides a clean adversarial training method based on diffusion processing, the method comprising:

[0015] Inputting the adversarial sample into the model to be trained, and obtaining a third processing result output by the model to be trained after performing data processing on the adversarial sample, wherein the adversarial sample is: an adversarial sample generated according to the method described in the first aspect;

[0016] Determining a loss value generated by the to-be-trained model during data processing according to a difference between the third processing result and the data label of the adversarial sample;

[0017] Backward gradient propagation is performed according to the obtained loss value to update the weight coefficient of the model to be trained.

[0018] In a third aspect, an embodiment of the present application provides a clean adversarial sample generation device based on diffusion processing, the device comprising:

[0019] A first noise superposition module, used for performing diffusion processing on the original sample and the first noise to obtain a first noisy sample;

[0020] A first processing result obtaining module, used to perform data processing on the original sample using the model to be trained to obtain a first processing result;

[0021] A second processing result obtaining module, used to use the to-be-trained model to perform data processing on the first noisy sample to obtain a second processing result;

[0022] A noise divergence obtaining module, configured to obtain a noise divergence of the first noisy sample relative to the original sample based on the first processing result and the second processing result;

[0023] a second noise superposition module, configured to generate a second noise based on the noise divergence and the first noisy sample, and superimpose the first noisy sample and the second noise to obtain a second noisy sample;

[0024] a noise removal module, configured to perform inverse diffusion processing on the second noisy sample, predict separation noise, and remove the separation noise from the second noisy sample;

[0025] The adversarial sample acquisition module is used to obtain adversarial samples based on the noise removal results.

[0026] In a fourth aspect, an embodiment of the present application provides a clean confrontation training device based on diffusion processing, the device comprising:

[0027] a processing result obtaining module, configured to input the adversarial sample into the model to be trained, and obtain a third processing result output by the model to be trained after performing data processing on the adversarial sample, wherein the adversarial sample is an adversarial sample generated according to the clean adversarial sample generation method based on diffusion processing;

[0028] A loss value determination module, used to determine the loss value generated by the to-be-trained model in the data processing process according to the difference between the third processing result and the data label of the adversarial sample;

[0029] The weight coefficient updating module is used to perform reverse gradient propagation according to the obtained loss value to update the weight coefficient of the model to be trained.

[0030] In a fifth aspect, an embodiment of the present application provides an electronic device, including:

[0031] Memory, used to store computer programs;

[0032] The processor is used to implement the method described in the first aspect or the second aspect when executing the program stored in the memory.

[0033] In a sixth aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method described in the first aspect or the second aspect is implemented.

[0034] In a seventh aspect, an embodiment of the present application further provides a computer program product, including a computer program, which implements the method described in the first aspect or the second aspect when executed by a processor.

[0035] As can be seen from the above, the solution provided by the embodiment of the present application is to first diffuse the original sample to obtain a noisy sample, then predict the separation noise in the noisy sample, and finally remove the separation noise from the noisy sample to obtain an adversarial sample. That is, after adding noise to the original sample, the noisy sample is properly denoised by predicting and removing the noise, so that the generated adversarial sample will not contain excessive noise, that is, the generated adversarial sample contains less noise and has a higher cleanliness. In addition, the generated adversarial sample is different from the original sample, but the gap between the original sample and the original sample is not too large, which improves the quality of the obtained adversarial sample. It can be known to those skilled in the art that although adversarial training can improve the robustness of the trained model, it may cause the model to focus too much on learning the characteristics of the adversarial sample during the training stage. In other words, the model pays too much "attention" to the adversarial sample, while the attention to the general sample is relatively insufficient. This makes the processing performance of the trained model for general input data worse, which makes it easy to reduce the inference accuracy of the model for general input data, that is, the generalization ability of the model is poor. When the adversarial sample training model generated by the present application is used, since the generated adversarial samples are relatively clean and closer to the original samples, the characteristics of the adversarial samples are also similar to the original samples. In this way, clean training can be achieved based on adversarial samples. Even if the model pays more attention to adversarial samples during adversarial training, it will not significantly reduce the model's processing performance for general input data, thereby improving the generalization ability of the model.

[0036] In summary, adversarial training using adversarial samples generated by the solution provided in the embodiments of the present application can not only improve the model's ability to resist adversarial samples, but also will not significantly reduce the model's processing performance for general input data, taking into account both the robustness and generalization ability of the model.

[0037] Of course, implementing any product or method of the present application does not necessarily require achieving all of the advantages described above at the same time. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application, and for ordinary technicians in this field, other embodiments can also be obtained based on these drawings.

[0039] Figure 1 A schematic diagram of the process of the first method for generating clean adversarial samples based on diffusion processing provided in an embodiment of the present application;

[0040] Figure 2 A schematic diagram of a noise superposition process provided in an embodiment of the present application;

[0041] Figure 3 A schematic diagram of a flow chart of a second method for generating clean adversarial samples based on diffusion processing provided in an embodiment of the present application;

[0042] Figure 4 A schematic diagram of a clean adversarial sample generation process based on diffusion processing provided in an embodiment of the present application;

[0043] Figure 5 A flow chart of a clean adversarial training method based on diffusion processing provided in an embodiment of the present application;

[0044] Figure 6 A schematic diagram of the structure of a clean adversarial sample generation device based on diffusion processing provided in an embodiment of the present application;

[0045] Figure 7 A schematic diagram of the structure of a clean confrontation training device based on diffusion processing provided in an embodiment of the present application;

[0046] Figure 8 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0047] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field based on the present application belong to the scope of protection of the present application.

[0048] First, the execution subject of the solution provided in the embodiment of the present application is described.

[0049] The executor of the solution provided in the embodiment of the present application is: any electronic device with data processing, storage and other functions.

[0050] Some concepts involved in the solutions provided in the embodiments of the present application are introduced.

[0051] 1. Noise superposition

[0052] In the solution provided in the embodiment of the present application, the process of superimposing the first noise on the original sample to obtain the first noisy sample is called noise superposition.

[0053] 2. Noise Removal

[0054] In the solution provided in the embodiment of the present application, the process of predicting the separation noise from the first noisy sample and removing the separation noise is called noise removal.

[0055] 3. Diffusion treatment

[0056] In one embodiment of the present application, noise superposition can be achieved through diffusion processing. For example, the original sample and the first noise can be input into a pre-trained diffusion model so that the diffusion model performs an operation of superimposing the first noise onto the original sample. The above operation can be referred to as forward diffusion processing of the diffusion model, or diffusion processing of the model, or diffusion process of the model. The principle and specific method of diffusion processing by the diffusion model can be found in the description of subsequent embodiments, which will not be described in detail here.

[0057] In the present application, in addition to the diffusion model, other methods can also be used to perform diffusion processing on the original sample and the first noise. Regardless of whether the diffusion model is used to superimpose noise, the operation of superimposing noise on the original sample can be called diffusion processing.

[0058] 4. Reverse diffusion treatment

[0059] In one embodiment of the present application, noise removal can be achieved through inverse diffusion processing. For example, the first noisy sample can be input into the diffusion model, and the diffusion model can perform the operation of predicting the separation noise in the first noisy sample and removing the separation noise from the first noisy sample. The above operation can be called reverse diffusion processing of the diffusion model, or reverse diffusion processing of the model, or reverse diffusion process of the model. The principle and specific method of the inverse diffusion processing of the diffusion model can be found in the description of the subsequent embodiments, which will not be described in detail here.

[0060] In the present application, in addition to the diffusion model, other methods can also be used to perform inverse diffusion processing on the original sample and the first noise. Regardless of whether the diffusion model is used for noise removal, the operation of removing noise from the first noisy sample can be called inverse diffusion processing.

[0061] The following is a detailed introduction to the clean adversarial sample generation scheme based on diffusion processing provided in an embodiment of the present application in conjunction with the accompanying drawings.

[0062] See also Figure 1 , is a flow chart of a first method for generating clean adversarial samples based on diffusion processing provided in an embodiment of the present application. The method includes the following steps S101 to S107.

[0063] Step S101: performing diffusion processing on the original sample and the first noise to obtain a first noisy sample.

[0064] The above original samples are general samples used to train the model to be trained. Since these samples have not been specially processed, they can be called original samples. In this solution, adversarial samples for adversarial training of the model to be trained can be generated by processing the original samples.

[0065] The following is a detailed introduction to the training model and original samples.

[0066] The embodiments of the present application do not limit the network architecture and internal data processing flow of the model to be trained. The model to be trained can be a model set according to the actual needs of the application scenario.

[0067] For example, if the actual application scenario is an image processing scenario, such as an image classification scenario, the model to be trained may be an image classification model set by the staff; if the actual application scenario is an image segmentation or object recognition scenario, the model to be trained may be an image segmentation model or an object recognition model. It can be seen that in the image processing scenario, the solution provided by the embodiment of the present application can generate adversarial samples for use in the adversarial training stage of image processing models such as image classification models, image segmentation models, or object recognition models.

[0068] For another example, if the actual application scenario is an audio recognition scenario, the model to be trained may be a model set by a staff member for audio recognition.

[0069] Correspondingly, depending on the model to be trained, the original samples used to train the model to be trained are also different.

[0070] For example, if the model to be trained is the above-mentioned image classification model, image segmentation model or object recognition model, the original sample can be a sample image; for another example, if the model to be trained is a model for audio recognition, the original sample can be a sample audio, etc.

[0071] From the above concept introduction, it can be known that the diffusion processing is performed on the original sample and the first noise, that is, the first noise is superimposed on the original sample. In other words, the first noise is the noise to be added to the original sample. After the first noise is added to the original sample, the obtained first noisy sample can be distinguished from the original sample.

[0072] In one embodiment of the present application, when the first noise is superimposed on the original sample multiple times, the first noise (Gaussian noise) randomly sampled each time the noise is superimposed is different, so that the first noisy sample obtained by superimposing the first noise on the original sample each time is also different, so that each time the second noisy sample is generated subsequently, the first noisy sample used changes from a single point to a neighborhood centered on the point, thereby avoiding falling into a local optimal solution, improving the quality of the subsequently generated adversarial samples, and enhancing the generalization ability of the model obtained by training based on adversarial samples. The first noise can be randomly generated noise or pre-generated noise, which is not limited in the embodiment of the present application.

[0073] It should be noted that the first noise has the same data format as the original sample. For example, if the original sample is a sample audio, the first noise may be a noise audio; if the original sample is a sample image, the first noise may be a noise image.

[0074] In one case, if the original sample is a sample image, a Gaussian noise matrix with the same size as the sample image can be randomly generated as the first noise. The Gaussian noise matrix can also be called a noise image.

[0075] In addition, the number of the first noises to be added may be 1 or more. When the number of the first noises is more than one, the multiple first noises may be the same noise or different noises.

[0076] The following describes a method for performing diffusion processing on the original sample and the first noise.

[0077] In one implementation, the original sample and the first noise may be input into the aforementioned diffusion model, so that the diffusion model performs diffusion processing to obtain the first noisy sample.

[0078] In another implementation manner, the original sample and the first noise may be diffused in the following manners according to different situations:

[0079] 1. The number of the first noise is 1

[0080] In this case, the first noise may be superimposed on the original sample by data fusion.

[0081] For example, if the original sample is a sample audio and the first noise is a noise audio, the noise audio can be directly mixed with the sample audio to obtain a first noisy sample; for another example, if the original sample is a sample image and the first noise is a noise image, the noise image can be scaled to the same size as the sample image, and then the noise image and the sample image can be fused to obtain the first noisy sample.

[0082] Among them, when performing image fusion, the pixel values ​​at the same position in the noise image and the sample image can be directly added to obtain the first noisy sample, or the pixel values ​​at the same position in the noise image and the sample image can be weighted calculated according to the set weights to obtain the first noisy sample, which is reasonable.

[0083] 2. The number of the first noise is multiple

[0084] In this case, the target sample and the target noise can be superimposed to obtain an intermediate sample, and then the target sample can be updated to the intermediate sample, and the target noise can be updated to any first noise that has not been superimposed with noise, and the step of superimposing the target sample and the target noise can be returned to until all the first noises are superimposed with noise, and the latest intermediate sample is determined as the first noisy sample.

[0085] The target sample is initially an original sample, and the target noise is initially any first noise.

[0086] In this way, it is equivalent to sequentially accumulating multiple first noises to the original samples, and the samples obtained after each noise accumulation are used as the samples participating in the next noise superposition.

[0087] Combine the following Figure 2 From an intuitive explanation, it can be seen that the original sample is O, and the three first noises are the first noise N1, the first noise N2, and the first noise N3 respectively. Then, the original sample O can be used as the target sample T1, and N1 can be used as the target noise. O and N1 are superimposed to obtain the intermediate sample M1; then, the target sample is updated to M1, the target noise is updated to N2, M1 is superimposed to N2, and the intermediate sample M2 is obtained; finally, the target sample is updated to M2, the target noise is updated to N3, M2 is superimposed to N3, and the intermediate sample M3 is obtained. At this time, all the first noises are superimposed, so the latest intermediate sample M3 is the first noisy sample.

[0088] It should be noted that, in this case, the specific manner of each noise superposition may be the same as that described in the previous case, and will not be repeated here.

[0089] Step S102: Use the model to be trained to process the original sample data to obtain a first processing result.

[0090] Specifically, the original sample may be input into the model to be trained, so that the model to be trained performs data processing on the original sample and outputs a first processing result.

[0091] Of course, depending on the model to be trained and the original sample, the output first processing result may also be different. For example, if the model to be trained is an image segmentation model, the first processing result is the image segmentation result of the original image; if the model to be trained is an audio recognition model, the first processing result is the recognition result of the audio content, and so on. Examples will not be given one by one here.

[0092] In one case, the model to be trained may be an image classification model, and the first processing result may be the confidence that the original sample predicted by the model to be trained belongs to a set category.

[0093] Step S103: using the model to be trained to perform data processing on the first noisy sample to obtain a second processing result.

[0094] Similar to step S102, the first noisy sample is input into the model to be trained, and a second processing result output by the model to be trained after data processing of the original sample can be obtained. In one case, the model to be trained can be an image classification model, and the second processing result can be the confidence level that the first noisy sample predicted by the model to be trained belongs to a set category.

[0095] Step S104: based on the first processing result and the second processing result, obtain the noise divergence of the first noisy sample relative to the original sample.

[0096] The above noise divergence can be understood as the strength of the noise contained in the first noisy sample, or the distribution of the noise, which is used to reflect the difference between the second processing result and the first processing result, and can also reflect the extent of the impact of the first noise on the original sample.

[0097] The first noisy sample is a noisy sample generated by superimposing noise on the original sample. There is a certain difference between the two, but the first noisy sample at this time is just the original sample with random noise added. As a sample in the field of original samples, it is not aggressive yet. Therefore, the prediction results included in the processing results output by the trained model for the original sample and the first noisy sample are generally the same, but the specific values ​​of the confidence included in the prediction results may be different.

[0098] For example, if the model to be trained is an image classification model, the first processing result of the model to be trained on the original sample indicates that the category to which the original sample belongs may be category c1, c2, and c3, and the confidences corresponding to c1, c2, and c3 are p1-p3 respectively. The second processing result of the model to be trained on the first noisy sample indicates that the category to which the original sample belongs may also be category c1, c2, and c3, and the confidences corresponding to c1, c2, and c3 are p4-p6 respectively. It can be seen that the categories predicted by the model to be trained are the same, but the specific confidence values ​​p1 and p4 can be different, p2 and p5 can be different, and p3 and p6 can be different.

[0099] Furthermore, the difference between the first processing result and the second processing result can also reflect the strength of the noise contained in the first noisy sample, or the distribution of the noise, that is, the noise divergence.

[0100] Specifically, various difference calculation methods can be used to calculate the difference characterization information between the first processing result and the second processing result as the noise divergence of the first noisy sample relative to the original sample. The above difference calculation method can be a mean square error calculation method, a cross entropy calculation method, etc., which is not limited in the embodiments of the present application.

[0101] In one case, if the first processing result includes a first confidence that the original sample predicted by the to-be-trained model belongs to each set category, and the second processing result includes a second confidence that the first noisy sample predicted by the to-be-trained model belongs to each set category, then the confidence difference between the first confidence and the second confidence corresponding to each set category can be calculated, and then based on the obtained confidence difference, the noise divergence of the first noisy sample relative to the original sample can be determined.

[0102] For example, if the set categories are k, the first processing result includes k first confidences that the original sample belongs to the k set categories, and the second processing result includes k second confidences that the first noisy sample belongs to the k set categories. Then, for each set category, the confidence difference between the first confidence and the second confidence can be calculated to obtain k confidence differences, and then the noise divergence is determined based on the k confidence differences.

[0103] Specifically, the characteristic value of the obtained confidence difference can be calculated, and the obtained characteristic value is used as the above noise divergence. The above characteristic value can be a sum value, an average value, etc., which is not limited in the embodiment of the present application.

[0104] It can be seen that when the first processing result includes the first confidence that the original sample belongs to each set category, and the second processing result includes the second confidence that the first noisy sample belongs to each set category, the confidence difference between the first confidence and the second confidence corresponding to each set category can more comprehensively and accurately characterize the overall difference between the first processing result and the second processing result. Therefore, based on the obtained confidence difference, the noise divergence of the first noisy sample relative to the original sample can be determined more accurately and reasonably.

[0105] In one embodiment of the present application, the noise divergence of the first noisy sample relative to the original sample can be determined according to the following expression: :

[0106] ;

[0107] Among them, k represents the total number of set categories, Indicates the first confidence that the original sample belongs to the i-th set category, The second confidence level that the first noisy sample belongs to the i-th set category, represents the confidence difference between the first confidence and the second confidence corresponding to the i-th set category.

[0108] The above expression is equivalent to calculating the product of the first confidence and the confidence difference for each set category, and then accumulating the obtained products to obtain the noise divergence of the first noisy sample relative to the original sample. In this way, the noise divergence of the first noisy sample relative to the original sample can be determined more comprehensively and reasonably based on the difference between the first processing result and the second processing result.

[0109] Step S105: generating a second noise based on the noise divergence and the first noisy sample, and superimposing the first noisy sample and the second noise to obtain a second noisy sample.

[0110] The second noise can be understood as: after being superimposed with the first noisy sample, the noise divergence in the obtained second noisy sample can be larger. The role of the second noise is to further increase the difference between the second noisy sample and the original sample, so as to finally generate an adversarial sample that is different from the original sample. In the solution provided in the embodiment of the present application, the second noise can also be called adversarial disturbance.

[0111] Specifically, an adversarial training algorithm in the relevant technology can be used to find adversarial perturbations, and the obtained adversarial perturbations can be used as the second noise. The above algorithm can be an FGSM (Fast Gradient Sign Method) algorithm, PGD (Project Gradient Descent) algorithm, etc., which is not limited to the embodiments of the present application.

[0112] In one embodiment of the present application, the first noisy sample may be differentiated using noise divergence to obtain a gradient vector of the first noisy sample, and then the product of the gradient vector and a set coefficient may be calculated to obtain the second noise.

[0113] The above gradient vector is also the gradient of the noise divergence with respect to the first noisy sample, and the gradient vector reflects the rising direction of the noise divergence at each position of the first noisy sample.

[0114] The above-mentioned set coefficient can be understood as a set step size. The product of the gradient vector and the set coefficient represents the distance offset in the gradient ascending direction of the noise divergence. After offsetting the above distance on the basis of the first noisy sample, a second noisy sample with a larger noise divergence can be obtained. In this way, after superimposing the first noisy sample and the second noise, a second noisy sample with a higher noise divergence can be obtained, which is conducive to the subsequent acquisition of an adversarial sample that is somewhat different from the original sample, ensuring that the obtained adversarial sample has a certain degree of aggressiveness.

[0115] Step S106: performing inverse diffusion processing on the second noisy sample, predicting the separation noise, and removing the separation noise from the second noisy sample.

[0116] As can be seen from the foregoing, the second noisy sample is a noisy sample. In this step, the inverse diffusion process is used to predict the separation noise to be removed from the second noisy sample and remove the separation noise.

[0117] Specifically, the second noisy sample may be input into the diffusion model so that the diffusion model performs inverse diffusion processing on the second noisy sample, specifically including using a noise predictor to predict the separation noise and removing the separation noise from the second noisy sample.

[0118] In one embodiment of the present application, the separation noise may be removed from the second noisy sample according to the following expression:

[0119]

[0120] Above represents the second noisy sample, represents the second noisy sample after removing the separation noise, represents the separation noise, represents random parameters, represents a random disturbance.

[0121] Above represents the set first noise processing parameter. It should be noted that in the solution provided in the embodiment of the present application, multiple rounds of denoising can be performed on the second noisy sample. In each denoising process, the above The value of can be different.

[0122] Above represents the product of the first noise processing parameter and the used noise processing parameter.

[0123] For example, suppose the noise processing parameters set for the i-th noise removal are , then, when removing noise for the first time, for , when removing noise for the second time, for , When the noise is removed for the third time, for , , The product of , and so on.

[0124] In this way, after the separation noise to be removed from the second noisy sample is predicted, the separation noise can be removed from the second noisy sample conveniently and efficiently by using the above expression.

[0125] Step S107: Based on the noise removal result, an adversarial sample is obtained.

[0126] The second noisy sample is a sample obtained by superimposing noise on the original sample, which is significantly different from the original sample. In the above steps, the second noisy sample is not directly used as an adversarial sample, but the separation noise to be removed from the second noisy sample is predicted, and then the separation noise is removed from the second noisy sample, which is equivalent to first superimposing noise on the original sample and then appropriately denoising the noisy sample.

[0127] Therefore, in this step, the second noisy sample after noise removal is different from the original sample but not too different from the original sample. The second noisy sample after noise removal can be used as an adversarial sample.

[0128] In the embodiment of the present application, in this step, if it is determined that the iteration end condition is not met, the previous step can be returned to perform, thereby performing multiple rounds of denoising operations, and the second noisy sample after the last round of separation noise removal is determined as the adversarial sample. For specific implementation methods, please refer to the subsequent Figure 3 The illustrated embodiment will not be described in detail here.

[0129] As can be seen from the above, by applying the solution provided in the embodiment of the present application, the original sample and the first noise are diffused to obtain a first noisy sample, and then based on the first processing result and the second processing result generated by the model to be trained for the original sample and the first noisy sample, the noise divergence of the first noisy sample relative to the original sample is obtained, the noise divergence and the first noisy sample are used to generate the second noise, and the first noisy sample and the second noise are superimposed to obtain the second noisy sample, and finally the second noisy sample is inversely diffused to predict the separation noise, and the separation noise is removed from the second noisy sample, and based on the noise removal result, the adversarial sample used in the adversarial training stage can be successfully generated.

[0130] It can be seen that the solution provided by the embodiment of the present application is to first add noise to the original sample to obtain a noisy sample, then predict the separation noise in the noisy sample, and finally remove the separation noise from the noisy sample to obtain an adversarial sample. That is, after adding noise to the original sample, the noisy sample is properly denoised by predicting and removing the noise, so that the generated adversarial sample will not contain excessive noise, that is, the generated adversarial sample contains less noise and has a higher cleanliness. In addition, the generated adversarial sample is different from the original sample, but the gap between the original sample and the generated adversarial sample is not too large, which improves the quality of the obtained adversarial sample. It can be known to those skilled in the art that although adversarial training can improve the robustness of the trained model, it may cause the model to focus too much on learning the characteristics of the adversarial sample during the training stage. In other words, the model pays too much "attention" to the adversarial sample, while the attention to the general sample is relatively insufficient. This makes the processing performance of the trained model for general input data worse, which makes it easy to reduce the inference accuracy of the model for general input data, that is, the generalization ability of the model is poor. When the adversarial sample training model generated by the present application is used, since the generated adversarial samples are relatively clean and closer to the original samples, the characteristics of the adversarial samples are also similar to the original samples. In this way, clean training can be achieved based on adversarial samples. Even if the model pays more attention to adversarial samples during adversarial training, it will not significantly reduce the model's processing performance for general input data, thereby improving the generalization ability of the model.

[0131] In summary, adversarial training using adversarial samples generated by the solution provided in the embodiments of the present application can not only improve the model's ability to resist adversarial samples, but also will not significantly reduce the model's processing performance for general input data, taking into account both the robustness and generalization ability of the model.

[0132] In one embodiment of the present application, when there are multiple first noises, the original sample and the first noise may be diffused in the following manner to obtain a first noisy sample:

[0133] Specifically, the first noise may be accumulated to the original samples one by one according to the following expression, and the sample obtained after the last accumulation of the noise is used as the first noisy sample:

[0134]

[0135] in, represents the sample obtained after accumulating noise, represents the sample after the last noise accumulation, Initially, it is the original sample. represents the first noise processing parameter set, represents the product of the first noise processing parameter and the used noise processing parameter, Represents the first noise.

[0136] That is, each time noise superposition is performed, the above expression can be used to accumulate the first noise into the original sample.

[0137] It can be seen that when there are multiple first noises, the above expression can be used to conveniently and efficiently add each first noise to the original sample one by one to obtain the first noisy sample.

[0138] In one embodiment of the present application, the generated adversarial samples can also be used to train the training model to obtain a trained model. The specific process can refer to the model training process in the relevant technology, which will not be repeated here.

[0139] exist Figure 1 On the basis of the illustrated embodiment, when obtaining an adversarial sample based on the noise removal result, the first noisy sample can be updated to a second noisy sample after removing the separation noise when it is determined that the iteration end condition is met, and the steps of generating the second noise, superimposing the second noise, and removing the separation noise are returned to execute, that is, multiple rounds of separation noise removal, and the second noisy sample after the last round of separation noise removal is determined as an adversarial sample. In view of the above situation, the embodiment of the present application provides a second adversarial sample generation method.

[0140] See also Figure 3 , which is a flow chart of a second method for generating clean adversarial samples based on diffusion processing provided in an embodiment of the present application. The method includes the following steps S301-S308.

[0141] Step S301: performing diffusion processing on the original sample and the first noise to obtain a first noisy sample.

[0142] Step S302: Use the model to be trained to process the original sample data to obtain a first processing result.

[0143] Step S303: using the model to be trained to perform data processing on the first noisy sample to obtain a second processing result.

[0144] Step S304: based on the first processing result and the second processing result, obtain the noise divergence of the first noisy sample relative to the original sample.

[0145] Step S305: generating a second noise based on the noise divergence and the first noisy sample, and superimposing the first noisy sample and the second noise to obtain a second noisy sample.

[0146] Step S306: performing inverse diffusion processing on the second noisy sample, predicting the separation noise, and removing the separation noise from the second noisy sample.

[0147] The above steps S301 to S306 are the same as the above Figure 1 In the illustrated embodiment, steps S101 to S106 are the same and will not be described in detail here.

[0148] Step S307: determine whether the set iteration end condition is met. If not, update the first noisy sample to the second noisy sample after the separation noise is removed, and return to step S303. If yes, execute step S308.

[0149] The above-mentioned iteration end condition is used to determine whether the second noisy sample after removing the separation noise meets the requirements. If the iteration end condition is not met, it means that the second noisy sample does not meet the requirements. Therefore, the first noisy sample can be updated to the second noisy sample after removing the separation noise, and then return to step S303 for the next round of denoising and denoising; if the iteration end condition is met, it means that the second noisy sample after separating the noise has met the requirements, and step S308 can be directly executed to determine the second noisy sample after removing the separation noise as the adversarial sample.

[0150] The above iteration end condition can be understood as: the difference between the second noisy sample after the separation noise is removed and the original sample is within a certain range. The above iteration end condition is introduced in detail below.

[0151] The iteration end condition may include at least one of the following conditions:

[0152] 1. The number of times the separation noise has been removed from the second noisy sample reaches the set total number of iterations

[0153] The total number of iterations may be set by staff based on experience and / or actual needs, or may be obtained based on the number of times the original sample and the first noise are diffused, that is, based on the number of times the first noise is superimposed on the original sample.

[0154] For example, it may be greater than the number of times the first noise is superimposed in the original sample, or it may be less than the number of times the first noise is superimposed in the original sample, which is not limited in the embodiment of the present application.

[0155] In this way, when the set total number of iterations is reached, it can be considered that the difference between the second noisy sample after the separation noise is removed and the original sample is not too large or too small, so the iteration can be terminated.

[0156] In one case, the total number of iterations may be equal to the number of times the diffusion process is performed on the original sample and the first noise, that is, equal to the number of times the first noise is superimposed on the original sample.

[0157] That is, the number of times the noise is removed is equal to the number of times the first noise is superimposed, that is, the number of times the inverse diffusion process is equal to the number of times the diffusion process is performed.

[0158] Diffusion processing is used to add noise to the original sample. The more diffusion processing times are performed, the more noise is included in the noisy sample. Inverse diffusion processing is used to remove noise from the noisy sample. The more inverse diffusion processing times are performed, the more information is removed from the noisy sample.

[0159] Then, when the number of noise removal is equal to the number of superimposed first noise, that is, the number of inverse diffusion processing is equal to the number of diffusion processing, it can reduce the probability that the added noise cannot be removed well due to the number of diffusion processing being greater than the number of inverse diffusion processing, and also reduce the probability that the sample loses too much original information due to the number of diffusion processing being less than the number of inverse diffusion processing, thereby improving the rationality of the scheme and the quality of the final adversarial sample.

[0160] 2. A sample difference between the second noisy sample after the separation noise is removed and the first noisy sample is greater than a first set difference.

[0161] That is, the second noisy sample after the separation noise is removed is significantly different from the first noisy sample.

[0162] In this case, it means that more noise has been removed from the first noisy sample. It can be considered that the noise in the first noisy sample has been well removed. Therefore, stopping the iteration when the above conditions are met is helpful to prevent too many denoising times from causing the sample to lose too much original information.

[0163] 3. A sample difference between the second noisy sample after the separation noise is removed and the original sample is less than a second set difference.

[0164] That is, the second noisy sample after removing the separation noise is closer to the original sample.

[0165] In this case, it means that the second noisy sample obtained after noise removal is close to the original sample. It can be considered that the noise in the first noisy sample has been well removed. Therefore, stopping the iteration when the above conditions are met is helpful to prevent too many denoising times from causing the sample to lose too much original information.

[0166] It can be seen that in this embodiment, the iteration end condition can be flexibly set according to the actual situation, so that when removing the separation noise from the noisy sample, the strength of the noise removal is moderate, and finally the adversarial sample obtained is both aggressive and not too different from the original sample, thereby improving the sample quality.

[0167] Step S308: Determine the second noisy sample after the separation noise is removed as an adversarial sample.

[0168] When this step is performed under the iteration end condition, it means that the noise included in the second noisy sample after the noise is removed has met the requirement, so the second noisy sample after the separation noise is removed can be directly determined as the adversarial sample.

[0169] In this embodiment, after a round of noise superposition and noise removal, if it is determined that the iteration end condition is not met, the first noisy sample can be updated to the second noisy sample after the separation noise is removed, and the step of using the model to be trained to process the data of the first noisy sample is returned to perform the next round of noise divergence calculation, noise superposition and noise removal, until the second noisy sample after the separation noise is removed meets the iteration end condition. It can be seen that in this way, when the second noisy sample after the separation noise is removed does not meet the requirements, a second noisy sample that meets the requirements can be obtained through multiple rounds of iterative noise addition and denoising, and then the newly obtained second noisy sample is determined as an adversarial sample, thereby improving the quality of the obtained adversarial sample.

[0170] Based on the introduction of the foregoing embodiments, a specific implementation process of the embodiments of the present application in an image processing scenario is illustrated below. The original sample is referred to as the original image, the noisy sample is referred to as the noisy image, and the model to be trained is referred to as the image classification model.

[0171] From the above introduction, it can be known that the diffusion model can be used to perform diffusion processing on the original image to superimpose the first noise in the original image to obtain the first noisy sample, and the diffusion model can be used to perform inverse diffusion processing on the first noisy sample to remove the separation noise in the second noisy image. The diffusion processing and inverse diffusion processing methods of the diffusion model are described in detail below.

[0172] 1. Diffusion treatment

[0173] During the training phase of the diffusion model, diffusion processing must first be performed, that is, different Gaussian noises are superimposed and added to the original image according to the given hyperparameters to obtain a noisy image.

[0174] The diffusion process can be expressed as follows:

[0175]

[0176] In the above, s=1,2,...,num1, num1 represents the number of noise superpositions of the input model. represents the noisy image that participates in noise superposition for the sth time, Represents the image after the noise is superimposed for the sth time.

[0177] Above represents the random Gaussian noise participating in the noise superposition for the sth time, that is, the first noise, It represents the image after the noise is superimposed for the sth time, that is, the noisy image.

[0178] Above It indicates the noise processing parameters set when the sth noise is superimposed. , is a hyperparameter set artificially when the sth noise is superimposed. It represents the product of the noise processing parameters set for the sth noise superposition and the noise processing parameters used, that is, .

[0179] According to the above expression, the data distribution of the noisy image can be obtained as follows:

[0180]

[0181] in, () represents data distribution, I represents the unit matrix, Represents the image that participates in noise superposition for the first time, that is, the original image.

[0182] During the training phase of the diffusion model, a noise predictor is also used to predict the noise included in the noisy image. The training goal of the noise predictor is to make the predicted noise as close as possible to the noise superimposed in the original image.

[0183] 2. Reverse diffusion treatment

[0184] After the diffusion model is trained, inverse diffusion processing can be performed to gradually denoise the noisy image and eventually restore it to a new image that conforms to the data distribution.

[0185] The process of inverse diffusion processing includes noise prediction and noise separation.

[0186] Since the training goal of the noise predictor in the training stage of the diffusion model is to make the predicted noise as close as possible to the noise superimposed in the original image, the noise predictor can predict the noise included in the noisy image, that is, predict the separation noise to be removed from the noisy image.

[0187] Specifically, the noisy image and the number of denoising times are input into a noise predictor, and the noise predictor can output the separated noise.

[0188] Then, in the noise separation process, the noisy image is subjected to inverse diffusion processing to generate an image, that is, the first noisy image is gradually denoised, and finally a new image that conforms to the data distribution of the original image is restored. The process is as follows:

[0189]

[0190] Among them, s=num2, num2-1,...,1, num2 represents the number of noise removal of the input model, num2 can be equal to num1, represents the noisy image after the num2-s+1th noise removal. represents the separation noise when removing noise for the num2-s+1th time, represents the noisy image after removing the separation noise, represents the random parameters when removing noise for the num2-s+1th time, represents a random disturbance, Indicates the noise processing parameters set for the num2-s+1th noise removal. It represents the product of the noise processing parameter set in the num2-s+1th noise removal and the noise processing parameter used.

[0191] According to the above expression, the noisy image after removing the separation noise can be calculated The data distribution is as follows:

[0192]

[0193] in, () represents data distribution, represents the mean of the noisy image, Represents the variance of the noisy image.

[0194] Based on the understanding of the above diffusion processing and inverse diffusion processing, the adversarial sample generation method of the embodiment of the present application in the image processing scenario is described in detail below in conjunction with the accompanying drawings:

[0195] See also Figure 4 , is a schematic diagram of an adversarial sample generation process provided in an embodiment of the present application. The above process can be divided into the following stages:

[0196] Stage 1: Adding noise

[0197] Firstly, the original image and the first noise are input into the diffusion model for diffusion processing to obtain a first noisy image.

[0198] Phase 2: Finding Perturbations

[0199] Specifically, according to the first processing result generated by the image classification model for the original image and the second processing result output for the first noisy image, the noise divergence of the first noisy image relative to the original image can be calculated, so that according to the noise divergence, an adversarial perturbation that makes the noise divergence larger can be found. The adversarial perturbation is superimposed on the first noisy image to obtain the second noisy image.

[0200] Stage 3: Noise Separation

[0201] The diffusion model performs inverse diffusion processing on the second noisy image, predicts the separation noise, and removes the separation noise from the second noisy image to obtain the second noisy image after the noise is removed.

[0202] Phase 4: Multiple iterations

[0203] The first noisy image is updated to the second noisy image after the noise is removed, and the execution returns to stage 2, that is, the steps of finding disturbances and noise separation are continued until the set round is reached, and the newly obtained second noisy image after the noise is removed is used as the adversarial sample.

[0204] The generation process of the above adversarial samples can also be expressed by the following expression:

[0205]

[0206] In the above, t=1,2,...T, where T represents the set number of iterations. represents the forward diffusion process, corresponding to the aforementioned noise superposition step, represents the noisy image generated by the t-th step diffusion process. When t=1, Represents the original image.

[0207] Above represents the output of the image classification model for the noisy image generated in step t, corresponding to the aforementioned second data processing result; y represents the output of the image classification model for the original image, corresponding to the aforementioned first data processing result; Represents the noise divergence of the noisy image generated in the tth step relative to the original image.

[0208] Above represents the gradient vector obtained by derivation of the noise divergence with respect to the noisy image generated in step t, Represents the setting coefficient, also known as the attack step length.

[0209] Above Represents the product of the set coefficient and the gradient vector, corresponding to the aforementioned adversarial disturbance.

[0210] Above represents the inverse diffusion process, corresponding to the aforementioned noise removal step, Represents projection calculation, where R represents the data range, It means that the data after projection calculation is within the above data range.

[0211] In one embodiment of the present application, after obtaining the adversarial sample, the adversarial sample can be used to train an image classification model.

[0212] Based on the above description, in this embodiment, the framework of training the image classification model can be expressed by the following expression:

[0213]

[0214] Above represents the adversarial sample generated by the above method, x represents the original image, y represents the processing result generated by the image classification model for the original image, Represents the processing result of the image classification model for adversarial samples. is the cross entropy loss, Represents the cross entropy loss of the above processing results relative to the model's processing results for the original image.

[0215] Indicates that the original image and label belong to the set sample library and label library, represents the model parameters, It represents external minimization, that is, the goal of adversarial training is to minimize the loss of the image classification model for adversarial samples.

[0216] It can be seen that in this embodiment, for the goal of adversarial training, an external minimization strategy is adopted to pursue the robustness of the trained model; and for internal search for adversarial samples, this scheme does not seek adversarial samples with the largest loss, but generates adversarial samples that are somewhat aggressive but not too different from the original images, so as to improve the generalization of the trained model, thus taking into account both the robustness and generalization of the trained model.

[0217] Corresponding to the aforementioned clean sample generation method based on diffusion processing, the embodiment of the present application also provides a clean adversarial training method based on diffusion processing.

[0218] See also Figure 5 , is a flow chart of a clean adversarial training method based on diffusion processing provided in an embodiment of the present application, and the method includes the following steps S501-S503.

[0219] Step S501: input the adversarial sample into the model to be trained, and obtain a third processing result output by the model to be trained after performing data processing on the adversarial sample.

[0220] Among them, the above-mentioned adversarial samples are: adversarial samples generated according to the sample generation method based on diffusion processing introduced in the previous article, that is, the original samples are first diffused to obtain noisy samples, and then the separation noise in the noisy samples is predicted, and finally the separation noise is removed from the noisy samples to obtain adversarial samples.

[0221] Step S502: Calculate the loss value generated by the model to be trained during the data processing process based on the difference between the third processing result and the data label of the adversarial sample.

[0222] Step S503: Perform reverse gradient propagation according to the obtained loss value to update the weight coefficient of the model to be trained.

[0223] In this way, after executing the above steps S501-S503, the weight coefficient of the model is adjusted according to the processing results generated by the model for the adversarial samples, and a round of model training is completed.

[0224] In one embodiment of the present application, multiple rounds of training may be performed until a model that meets the requirements is obtained. Specifically, after executing step S503, the process may return to execute step S501 until the set training end condition is met, and the model to be trained after the weight coefficient is updated is determined as the trained model.

[0225] The above-mentioned training end condition can be that the model to be trained converges, or the number of training times reaches a set number, etc., which is not limited in the embodiments of the present application.

[0226] In this embodiment, the adversarial samples used in adversarial training of the training model are samples generated based on diffusion processing. Specifically, after adding noise to the original samples based on diffusion processing, the noisy samples are appropriately denoised through noise prediction and removal, so that the generated adversarial samples will not contain excessive noise, that is, the generated adversarial samples contain less noise and have higher cleanliness. In addition, the generated adversarial samples are different from the original samples, but the difference between them is not too large, which improves the quality of the obtained adversarial samples.

[0227] Furthermore, since the generated adversarial samples are clean and close to the original samples, the characteristics of the adversarial samples are also similar to those of the original samples. In this way, clean training can be achieved based on adversarial samples. Even if the model pays more attention to adversarial samples during adversarial training, it will not significantly reduce the model's processing performance for general input data, thereby improving the generalization ability of the model.

[0228] In summary, using the above-mentioned adversarial samples for adversarial training can not only improve the model's ability to resist adversarial samples, but also will not significantly reduce the model's processing performance for general input data, taking into account the model's robustness and generalization ability.

[0229] The following compares the model performance of the model trained with adversarial samples generated by this solution with the model trained with adversarial samples generated by related technologies.

[0230] First, some preparations and settings for the experiment are explained:

[0231] 1. Target model: ResNet18 network model

[0232] 2. Defense method: The defense method refers to the method adopted by adversarial training. The defense methods used in the experiment include AT (Adversarial Training) method, FAT (Friendly Adversarial Training) method, CAT (Customized Adversarial Training) method and the adversarial training method provided in this embodiment (i.e., using the adversarial samples generated by this application to train the network).

[0233] 3. Attack methods:

[0234] In order to make the test performance more comprehensive, this solution conducts white box experiments against white box attacks and black box experiments against black box attacks. Among them, white box experiments refer to experiments to defend against white box attacks, that is, to conduct white box attacks on the trained model to test the model performance; black box experiments refer to experiments to defend against black box attacks, that is, to conduct black box attacks on the trained model to test the model performance.

[0235] A white-box attack is an attack performed when the attacker knows all the parameters of the model. It is an idealized attack. A black-box attack is an attack performed when the attacker obtains a small number of parameters or output results of the model. It is closer to the real scene and therefore more realistic.

[0236] The white-box attack methods used in the experiment include: FGSM algorithm, PGD-20 algorithm, AA algorithm and C&W algorithm. The black-box attack methods used in the experiment include: FGSM algorithm, PGD-20 algorithm, TI-FGSM algorithm, MI-FGSM algorithm and DI-FGSM algorithm.

[0237] 4. Evaluation criteria for model performance:

[0238] The evaluation criteria include the clean accuracy and adversarial accuracy of the model. Clean accuracy refers to the classification accuracy of the model when it is not attacked, and adversarial accuracy refers to the classification accuracy of the model when it is attacked.

[0239] For white box attacks:

[0240] First, refer to the following Table 1. The first column in Table 1 indicates the data set used for the test, the second column indicates the different defense methods used by the model, the third column indicates the clean accuracy of the model when it is not attacked, and the fourth to seventh columns indicate the adversarial accuracy of the model when it is attacked by various attack methods.

[0241] Table 1

[0242]

[0243] It can be seen intuitively from Table 1 that in terms of clean accuracy of the model, this scheme has achieved the best among all defense methods. This shows that when this scheme generates adversarial samples, it has the least impact on the original samples, so the generalization ability of the model trained based on the adversarial samples is stronger than other defense methods; in terms of adversarial accuracy, although this scheme is not as good as the CAT method, it still exceeds the AT method and the FAT method overall.

[0244] For black box attacks:

[0245] Refer to Table 2 below. The first column in Table 2 indicates the data set used for the test, the second column indicates the different defense methods used by the model, the third column indicates the clean accuracy of the model when it is not attacked, and columns 4-7 indicate the adversarial accuracy of the model when it is attacked by various attack methods.

[0246] Table 2

[0247]

[0248] It can be seen intuitively from Table 2 that, except for a few data indicators that are slightly lower than the FAT method, this scheme has achieved the best performance among the defense methods selected in the experiment when resisting black-box attacks. This can also further show that the model trained based on the adversarial samples generated by this scheme has strong generalization ability and can better resist the impact of black-box attacks.

[0249] In the above two experiments, the purpose of the white-box experiment is to test the defensive performance of the defense method (the robustness of the trained model), while the purpose of the black-box experiment is to test the actual performance of the defense method (the generalization of the trained model).

[0250] The experimental results show that this scheme has good defense effect against both white-box attacks and black-box attacks, and the defense effect against black-box attacks is more obvious, which shows that this scheme has good practical significance.

[0251] Specifically, for white-box attacks, this solution can successfully improve the clean accuracy of the model while ensuring that the model has good robustness, that is, improve the generalization of the model. For black-box attacks, this solution has achieved excellent results in the experiment of resisting black-box migration attacks, and can significantly improve the generalization ability of the model. In summary, according to the above experimental results, it can be seen that this solution can not only make the trained model have better defense capabilities against attacks, but also make the generalization of the model surpass most defense methods. In addition, this solution also has the characteristics of plug-and-play, which is convenient for use with other defense methods, making the effect of adversarial training further than the original method.

[0252] In the technical solution of this application, the operations involved in obtaining, storing, using, processing, transmitting, providing and disclosing user personal information are all carried out with the user's authorization.

[0253] It should be noted that the two-dimensional face images in this embodiment come from a public data set.

[0254] Corresponding to the above-mentioned method for generating a clean sample based on diffusion processing, an embodiment of the present application further provides a device for generating a clean sample based on diffusion processing.

[0255] See also Figure 6 , is a schematic diagram of the structure of a clean adversarial sample generation device based on diffusion processing provided in an embodiment of the present application, the device includes the following modules:

[0256] A first noise superposition module 601 is used to perform diffusion processing on the original sample and the first noise to obtain a first noisy sample;

[0257] A first processing result obtaining module 602 is used to perform data processing on the original sample using the model to be trained to obtain a first processing result;

[0258] A second processing result obtaining module 603 is used to use the to-be-trained model to perform data processing on the first noisy sample to obtain a second processing result;

[0259] A noise divergence obtaining module 604, configured to obtain a noise divergence of the first noisy sample relative to the original sample based on the first processing result and the second processing result;

[0260] A second noise superposition module 605 is used to generate a second noise based on the noise divergence and the first noisy sample, and superimpose the first noisy sample and the second noise to obtain a second noisy sample;

[0261] A noise removal module 606, configured to perform inverse diffusion processing on the second noisy samples, predict separation noise, and remove the separation noise from the second noisy samples;

[0262] The adversarial sample obtaining module 607 is used to obtain an adversarial sample based on the noise removal result.

[0263] As can be seen from the above, by applying the solution provided in the embodiment of the present application, the original sample and the first noise are diffused to obtain a first noisy sample, and then based on the first processing result and the second processing result generated by the model to be trained for the original sample and the first noisy sample, the noise divergence of the first noisy sample relative to the original sample is obtained, the noise divergence and the first noisy sample are used to generate the second noise, and the first noisy sample and the second noise are superimposed to obtain the second noisy sample, and finally the second noisy sample is inversely diffused to predict the separation noise, and the separation noise is removed from the second noisy sample, and based on the noise removal result, the adversarial sample used in the adversarial training stage can be successfully generated.

[0264] It can be seen that the solution provided by the embodiment of the present application is to first add noise to the original sample to obtain a noisy sample, then predict the separation noise in the noisy sample, and finally remove the separation noise from the noisy sample to obtain an adversarial sample. That is, after adding noise to the original sample, the noisy sample is properly denoised by predicting and removing the noise, so that the generated adversarial sample will not contain excessive noise, that is, the generated adversarial sample contains less noise and has a higher cleanliness. In addition, the generated adversarial sample is different from the original sample, but the gap between the original sample and the generated adversarial sample is not too large, which improves the quality of the obtained adversarial sample. It can be known to those skilled in the art that although adversarial training can improve the robustness of the trained model, it may cause the model to focus too much on learning the characteristics of the adversarial sample during the training stage. In other words, the model pays too much "attention" to the adversarial sample, while the attention to the general sample is relatively insufficient. This makes the processing performance of the trained model for general input data worse, which makes it easy to reduce the inference accuracy of the model for general input data, that is, the generalization ability of the model is poor. When the adversarial sample training model generated by the present application is used, since the generated adversarial samples are relatively clean and closer to the original samples, the characteristics of the adversarial samples are also similar to the original samples. In this way, clean training can be achieved based on adversarial samples. Even if the model pays more attention to adversarial samples during adversarial training, it will not significantly reduce the model's processing performance for general input data, thereby improving the generalization ability of the model.

[0265] In summary, adversarial training using adversarial samples generated by the solution provided in the embodiments of the present application can not only improve the model's ability to resist adversarial samples, but also will not significantly reduce the model's processing performance for general input data, taking into account both the robustness and generalization ability of the model.

[0266] In one embodiment of the present application, the adversarial sample acquisition module 607 is specifically used to determine whether the set iteration end condition is met; if not, the first noisy sample is updated to a second noisy sample after the separation noise is removed, and the step of using the model to be trained to perform data processing on the first noisy sample to obtain a second processing result is returned; otherwise, the second noisy sample after the separation noise is removed is determined as an adversarial sample.

[0267] In this embodiment, after a round of noise superposition and noise removal, if it is determined that the iteration end condition is not met, the first noisy sample can be updated to the second noisy sample after the separation noise is removed, and the step of using the model to be trained to process the data of the first noisy sample is returned to perform the next round of noise divergence calculation, noise superposition and noise removal, until the second noisy sample after the separation noise is removed meets the iteration end condition. It can be seen that in this way, when the second noisy sample after the separation noise is removed does not meet the requirements, a second noisy sample that meets the requirements can be obtained through multiple rounds of iterative noise addition and denoising, and then the newly obtained second noisy sample is determined as an adversarial sample, thereby improving the quality of the obtained adversarial sample.

[0268] In one embodiment of the present application, the iteration end condition includes at least one of the following conditions:

[0269] The number of times the separation noise has been removed from the second noisy sample reaches a set total number of iterations, wherein the total number of iterations is obtained based on the number of times the original sample and the first noise are diffused; the sample difference between the second noisy sample after the separation noise is removed and the first noisy sample is greater than a first set difference; the sample difference between the second noisy sample after the separation noise is removed and the original sample is less than a second set difference.

[0270] It can be seen that in this embodiment, the iteration end condition can be flexibly set according to the actual situation, so that when removing the separation noise from the noisy sample, the strength of the noise removal is moderate, and finally the adversarial sample obtained is both aggressive and not too different from the original sample, thereby improving the sample quality.

[0271] In one embodiment of the present application, the total number of iterations is equal to the number of times the diffusion process is performed on the original sample and the first noise.

[0272] The number of times the original sample and the first noise are diffused, that is, the number of times the noise is removed is equal to the number of times the first noise is superimposed, that is, the number of inverse diffusion processing is equal to the number of diffusion processing, can reduce the probability that the added noise cannot be removed well due to the number of diffusion processing being greater than the number of inverse diffusion processing, and also reduce the probability that the sample loses too much original information due to the number of diffusion processing being less than the number of inverse diffusion processing, thereby improving the rationality of the scheme and the quality of the adversarial sample finally obtained.

[0273] In one embodiment of the present application, the noise removal module 606 is specifically configured to perform inverse diffusion processing on the second noisy sample, predict separation noise, and remove the separation noise from the second noisy sample according to the following expression:

[0274]

[0275] in, represents the second noisy sample, represents the second noisy sample after removing the separation noise, represents the first noise processing parameter set, represents the product of the first noise processing parameter and the used noise processing parameter, represents the separation noise, represents random parameters, represents a random disturbance.

[0276] In this way, after the separation noise to be removed from the second noisy sample is predicted, the separation noise can be removed from the second noisy sample conveniently and efficiently by using the above expression.

[0277] In one embodiment of the present application, the first processing result includes: a first confidence level that the original sample predicted by the to-be-trained model belongs to each set category, and the second processing result includes: a second confidence level that the first noisy sample predicted by the to-be-trained model belongs to each set category.

[0278] The noise divergence obtaining module 604 includes: a confidence difference calculating submodule, used to calculate the confidence difference between the first confidence and the second confidence corresponding to each set category; and a noise divergence determining submodule, used to determine the noise divergence of the first noisy sample relative to the original sample based on the obtained confidence difference.

[0279] It can be seen that when the first processing result includes the first confidence that the original sample belongs to each set category, and the second processing result includes the second confidence that the first noisy sample belongs to each set category, the confidence difference between the first confidence and the second confidence corresponding to each set category can more comprehensively and accurately characterize the overall difference between the first processing result and the second processing result. Therefore, based on the obtained confidence difference, the noise divergence of the first noisy sample relative to the original sample can be determined more accurately and reasonably.

[0280] In one embodiment of the present application, the noise divergence determination submodule is specifically used to determine the noise divergence of the first noisy sample relative to the original sample according to the following expression: : ; where k represents the total number of set categories, Indicates the first confidence that the original sample belongs to the i-th set category, A second confidence level indicating that the first noisy sample belongs to the i-th set category, represents the confidence difference between the first confidence and the second confidence corresponding to the i-th set category.

[0281] The above expression is equivalent to calculating the product of the first confidence and the confidence difference for each set category, and then accumulating the obtained products to obtain the noise divergence of the first noisy sample relative to the original sample. In this way, the noise divergence of the first noisy sample relative to the original sample can be determined more comprehensively and reasonably based on the difference between the first processing result and the second processing result.

[0282] In one embodiment of the present application, the second noise superposition module 605 is specifically used to use the noise divergence to derive the first noisy sample to obtain a gradient vector of the first noisy sample; calculate the product of the gradient vector and a set coefficient to obtain a second noise; and superimpose the first noisy sample and the second noise to obtain a second noisy sample.

[0283] In this way, after superimposing the first noisy sample and the second noise, a second noisy sample with higher noise divergence can be obtained, which is conducive to subsequently obtaining an adversarial sample that is somewhat different from the original sample, ensuring that the obtained adversarial sample has a certain degree of aggressiveness.

[0284] In one embodiment of the present application, the number of the first noise is greater than 1,

[0285] The first noise superposition module 601 is specifically configured to accumulate the first noise to the original samples one by one according to the following expression, and use the sample obtained after the last noise accumulation as the first noisy sample: in, represents the sample obtained after accumulating noise, represents the sample after the last noise accumulation, Initially, it is the original sample, represents the first noise processing parameter set, represents the product of the first noise processing parameter and the used noise processing parameter, represents the first noise.

[0286] It can be seen that when there are multiple first noises, the above expression can be used to conveniently and efficiently add each first noise to the original sample one by one to obtain the first noisy sample.

[0287] In one embodiment of the present application, the original sample is an image, and the model to be trained is: an image classification model, an image segmentation model or an object recognition model.

[0288] It can be seen that in the image processing scenario, the solution provided by the embodiment of the present application can generate adversarial samples for use in the adversarial training stage of image processing models such as image classification models, image segmentation models or object recognition models.

[0289] Corresponding to the above-mentioned clean adversarial training method based on diffusion processing, an embodiment of the present application also provides a clean adversarial training device based on diffusion processing.

[0290] See also Figure 7 , is a schematic diagram of the structure of a clean adversarial training device based on diffusion processing provided in an embodiment of the present application, the device includes the following modules:

[0291] The processing result obtaining module 701 is used to input the adversarial sample into the model to be trained, and obtain a third processing result output by the model to be trained after performing data processing on the adversarial sample, wherein the adversarial sample is an adversarial sample generated according to the above-mentioned clean adversarial sample generation method based on diffusion processing;

[0292] A loss value determination module 702, configured to determine a loss value generated by the to-be-trained model during data processing according to a difference between the third processing result and the data label of the adversarial sample;

[0293] The weight coefficient updating module 703 is used to perform reverse gradient propagation according to the obtained loss value to update the weight coefficient of the model to be trained.

[0294] In this embodiment, the adversarial samples used in adversarial training of the training model are samples generated based on diffusion processing. Specifically, after adding noise to the original samples based on diffusion processing, the noisy samples are appropriately denoised through noise prediction and removal, so that the generated adversarial samples will not contain excessive noise, that is, the generated adversarial samples contain less noise and have higher cleanliness. In addition, the generated adversarial samples are different from the original samples, but the difference between them is not too large, which improves the quality of the obtained adversarial samples.

[0295] Furthermore, since the generated adversarial samples are clean and close to the original samples, the characteristics of the adversarial samples are also similar to those of the original samples. In this way, clean training can be achieved based on adversarial samples. Even if the model pays more attention to adversarial samples during adversarial training, it will not significantly reduce the model's processing performance for general input data, thus achieving clean adversarial training and improving the generalization ability of the model.

[0296] In summary, using the above-mentioned adversarial samples for adversarial training can not only improve the model's ability to resist adversarial samples, but also will not significantly reduce the model's processing performance for general input data, taking into account the model's robustness and generalization ability.

[0297] The present application also provides an electronic device, such as Figure 8 As shown, including:

[0298] Memory 801, used for storing computer programs;

[0299] The processor 802 is used to execute the program stored in the memory 801, and implements the following steps:

[0300] Furthermore, the electronic device may further include a communication bus and / or a communication interface, and the processor 802, the communication interface, and the memory 801 communicate with each other via the communication bus.

[0301] The communication bus mentioned in the above electronic device can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The communication bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus.

[0302] The communication interface is used for communication between the above electronic device and other devices.

[0303] The memory may include a random access memory (RAM) or a non-volatile memory (NVM), such as at least one disk memory. Optionally, the memory may also be at least one storage device located away from the aforementioned processor.

[0304] The above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0305] In another embodiment provided in the present application, a computer-readable storage medium is also provided, which stores a computer program. When the computer program is executed by a processor, the steps of the clean adversarial training method based on diffusion processing or the adversarial sample generation method based on diffusion processing are implemented.

[0306] In another embodiment provided in the present application, a computer program product is also provided, including a computer program, which, when executed by a processor, implements any clean adversarial training method based on diffusion processing or adversarial sample generation method based on diffusion processing in the above embodiments.

[0307] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from a website site, computer, server or data center to another website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated. The available medium may be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a solid-state hard disk (SSD), etc.

[0308] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0309] Each embodiment in this specification is described in a related manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device, electronic device, storage medium and program product embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiments.

[0310] The above description is only a preferred embodiment of the present application and is not intended to limit the protection scope of the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application are included in the protection scope of the present application.

Claims

1. A method for generating clean adversarial samples based on diffusion processing, characterized in that: The method comprises: Performing diffusion processing on the original sample and the first noise to obtain a first noisy sample, wherein the original sample is: a sample image or a sample audio; Using the model to be trained to perform data processing on the original sample to obtain a first processing result; Using the model to be trained to perform data processing on the first noisy sample to obtain a second processing result; Based on the first processing result and the second processing result, obtaining a noise divergence of the first noisy sample relative to the original sample; generating a second noise based on the noise divergence and the first noisy sample, and superimposing the first noisy sample and the second noise to obtain a second noisy sample; performing inverse diffusion processing on the second noisy samples, predicting separation noise, and removing the separation noise from the second noisy samples; Based on the noise removal results, adversarial samples are obtained.

2. The method according to claim 1, characterized in that The step of obtaining an adversarial sample based on the noise removal result includes: Determine whether the set iteration end condition is met; If not, updating the first noisy sample to a second noisy sample after removing the separation noise, and returning to the step of using the to-be-trained model to perform data processing on the first noisy sample to obtain a second processing result; Otherwise, the second noisy sample after removing the separation noise is determined as an adversarial sample.

3. The method according to claim 2, characterized in that The iteration end condition includes at least one of the following conditions: The number of times the separation noise has been removed from the second noisy sample reaches a set total number of iterations, wherein the total number of iterations is obtained based on the number of times the diffusion process is performed on the original sample and the first noise; A sample difference between the second noisy sample after removing the separation noise and the first noisy sample is greater than a first set difference; A sample difference between the second noisy sample after the separation noise is removed and the original sample is smaller than a second set difference.

4. The method according to claim 2, characterized in that: The removing the separation noise from the second noisy sample comprises: The separation noise is removed from the second noisy sample according to the following expression: ； in, represents the second noisy sample, represents the second noisy sample after removing the separation noise, represents the first noise processing parameter set, represents the product of the first noise processing parameter and the used noise processing parameter, represents the separation noise, represents random parameters, represents a random disturbance.

5. The method according to claim 1, characterized in that The first processing result includes: a first confidence level that the original sample predicted by the to-be-trained model belongs to each set category; the second processing result includes: a second confidence level that the first noisy sample predicted by the to-be-trained model belongs to each set category; The obtaining, based on the first processing result and the second processing result, a noise divergence of the first noisy sample relative to the original sample, comprises: Calculating a confidence difference between a first confidence level and a second confidence level corresponding to each set category; Based on the obtained confidence difference, a noise divergence of the first noisy sample relative to the original sample is determined.

6. The method according to claim 5, characterized in that The determining, based on the obtained confidence difference, the noise divergence of the first noisy sample relative to the original sample comprises: The noise divergence of the first noisy sample relative to the original sample is determined according to the following expression: : ； Among them, k represents the total number of set categories, Indicates the first confidence that the original sample belongs to the i-th set category, A second confidence level indicating that the first noisy sample belongs to the i-th set category, represents the confidence difference between the first confidence and the second confidence corresponding to the i-th set category.

7. The method according to claim 1, characterized in that The step of generating a second noise based on the noise divergence and the first noisy sample comprises: The first noisy sample is derivatized by using the noise divergence to obtain a gradient vector of the first noisy sample; the product of the gradient vector and a set coefficient is calculated to obtain a second noise; and / or, The amount of the first noise is greater than 1, and the diffusion processing is performed on the original sample and the first noise to obtain the first noisy sample, including: The first noise is accumulated to the original samples one by one according to the following expression, and the sample obtained after the last accumulation of noise is used as the first noisy sample: in, represents the sample obtained after accumulating noise, represents the sample after the last noise accumulation, Initially, it is the original sample, represents the first noise processing parameter set, represents the product of the first noise processing parameter and the used noise processing parameter, represents the first noise.

8. A clean adversarial training method based on diffusion processing, characterized in that: The method comprises: Inputting the adversarial sample into the model to be trained, and obtaining a third processing result output by the model to be trained after performing data processing on the adversarial sample, wherein the adversarial sample is an adversarial sample generated according to any one of claims 1 to 7, and the adversarial sample is a sample image or a sample audio; Determining a loss value generated by the to-be-trained model during data processing according to a difference between the third processing result and the data label of the adversarial sample; Backward gradient propagation is performed according to the obtained loss value to update the weight coefficient of the model to be trained.

9. An electronic device, characterized in that: include: Memory, used to store computer programs; A processor, for implementing the method described in any one of claims 1 to 7 or 8 when executing a program stored in a memory.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the method according to any one of claims 1 to 7 or 8 is implemented.

Images