Adversarial sample generation method and system
By using NAG methods to generate adversarial samples on multiple alternative models, the problem of lack of diversity and generalization capabilities of adversarial samples in existing black box attack methods is solved, and the success rate of transfer attacks and the robustness of the model is improved.
Patent Information
- Application Number
- CN202411174480.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-26
- Publication Date
- 2024-12-10
AI Technical Summary
The existing black box attack method overfits the alternative model during iterative computing processing, resulting in a lack of diversity in adversarial samples and low success rate of transfer attacks, and only generates adversarial perturbations on one alternative model, low generalization ability, and failure to concentrate. On key features.
By training multiple alternative models, using NAG methods to introduce second-order guide information of gradients, generate significant graphs and masks, and gradually adjust the adversarial samples to successfully attack and generalize on multiple models, improving generalization and transfer attacks Success rate.
It improves the diversity and generalization ability of adversarial samples, enhances the robustness of deep neural networks, and improves the security and reliability of the model.
Smart Images
Figure CN119107517A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of artificial intelligence. Background Art
[0002] As an important branch of artificial intelligence, deep neural networks have achieved great success in many fields. Deep neural networks have been widely used in many important systems, such as autonomous driving, face recognition, surveillance systems, medical image processing, etc. The rapid development of information technology has created unprecedented conditions for the deep development and extensive use of information resources. Information has become a strategic resource as important as materials and energy. As an efficient and intelligent means of information processing, artificial intelligence technology has made great progress in recent years. Among them, intelligent visual perception technology represented by deep convolutional neural networks has brought huge economic and social benefits to mankind in the past few years, and has become an indispensable and powerful tool in daily life, scientific research and social production. As intelligent visual perception technology continues to penetrate into human society, its credibility issues have begun to attract the attention of more and more countries and well-known institutions. Deep neural networks are susceptible to small perturbations, which may affect the security of deployed neural networks. Adversarial attacks can help enhance the robustness of models and improve existing model training algorithms. In the fields of autonomous driving, image and speech recognition, using adversarial attack technology, researchers can find and fix vulnerabilities in models, thereby improving the security and reliability of models.
[0003] At present, in the related research in the field of adversarial sample generation, it can be divided into black-box attack and white-box attack according to the degree of the attacker's understanding of the target model. Black-box attack means that the attacker has no information about the target model, and the attack in which the attacker knows the structure and parameter details of the target model is called white-box attack. Since black-box attack requires the attacker to have no relevant information about the target model, it is more in line with reality. Existing black-box attack methods can be divided into the following categories: transfer-based attack methods, score-based attack methods, and decision-based attack methods. Since score-based attack methods and decision-based attack methods require a large number of query operations on the target model, which may lead to defensive behavior of the model owner, transfer-based attack methods that do not require query operations on the target model are the most widely used. Transferability is an important property of adversarial samples, that is, an adversarial attack that is effective on one model has a high probability of being effective on other similar task models.
[0004] In summary, there are several major problems with existing attack algorithms: First, in the iterative calculation process, the added perturbation is the accumulation of gradients in each iteration, which often overfits the alternative model. This will lead to a lack of diversity in the generated perturbations and reduce the success rate of transfer attacks on adversarial samples. Second, existing methods only make adversarial examples closer to misclassification, and stop calculating after the adversarial samples cross the classification boundary of the alternative model, which makes the samples not learn enough features of misclassification, resulting in a low success rate of transfer attacks. Third, existing methods only generate adversarial samples on one alternative model, and the generated adversarial perturbations may overfit the model and have low generalization ability. Fourth, the perturbations generated by existing adversarial attack algorithms do not focus on key features that have an important impact on model classification. Summary of the invention
[0005] The present invention is aimed at the problem that in the iterative calculation process in the prior art, the added perturbation is the accumulation of gradients in each iteration, which often overfits the alternative model. This will lead to the problem that the generated perturbation lacks diversity and reduces the success rate of transfer attacks on adversarial samples; secondly, the existing method only makes the adversarial example closer to the wrong classification, and stops the calculation after the adversarial sample crosses the classification boundary of the alternative model, which makes the sample not learn enough features of the wrong classification, resulting in a low success rate of transfer attacks; finally, the existing method only generates adversarial samples on one alternative model, and the generated adversarial perturbation may overfit the model and have low generalization ability; finally, the perturbation generated by the existing adversarial attack algorithm does not focus on the key features that have an important impact on model classification. In order to solve the above technical problems, the present invention is implemented through the following technical solutions:
[0006] The present invention proposes a method for generating adversarial samples, which comprises the following steps:
[0007] S1. Train at least one alternative model;
[0008] S2, setting the parameters of the substitution model described in S1;
[0009] S3, randomly select an alternative model from at least one alternative model in S1;
[0010] S4, input the original image into the alternative model selected in S3, and back propagate to generate a saliency map of the original image;
[0011] S5, sorting the pixels of the original image according to the significance of its saliency map; setting the pixels with a significance quantile greater than 95% as the foreground, and setting the pixels with a significance quantile less than 30% as the background;
[0012] S6, setting the maximum connected component of the foreground pixel as the segmented foreground object;
[0013] S7, generating a mask according to the foreground object segmented in S6, and obtaining a picture containing only the foreground object;
[0014] S8, inputting pictures with only foreground objects into the substitution models respectively and executing S9 to S13 respectively;
[0015] S9, calculating the image gradient of the foreground object;
[0016] S10, let the adversarial sample move forward in two directions, one direction of gradient increase and the other direction of gradient decrease. If the gradient decrease direction has passed the minimum point, change it to the gradient increase direction, where the forward direction is calculated by the NAG algorithm;
[0017] S11, determine whether the set number of iterations has been completed, if not, return to S4;
[0018] S12, judging whether the generated adversarial sample can successfully attack the adversarial sample according to the number of iterations set in S11, if only one adversarial sample successfully attacks the alternative model, then selecting the sample to proceed to the next step, if both adversarial samples successfully attack the alternative model, then selecting the adversarial sample with the smallest disturbance to proceed to the next step;
[0019] S13, calculate the adversarial sample gradient and make it move one step in the direction of gradient ascent and then return this image;
[0020] S14. The generated adversarial samples are obtained by weighted averaging the images returned by all alternative models.
[0021] Further, a preferred implementation is provided, in which the formula for calculating the forward direction using the NAG method in S10 is:
[0022]
[0023]
[0024] x t ' +1 =Clip x,ε {x t ′+α·g t+1}
[0025] In the formula, represents the current adversarial sample, represents the updated adversarial sample, α is the step size, μ is the decay rate, and g t is the current gradient, Represents the cross entropy of A and B, Clip X,ε {x t ′+α·g t+1} means limiting the disturbance amplitude to within ε.
[0026] Furthermore, a preferred embodiment is provided, wherein the alternative models described in S1 include ResNet50, Inc-v3, Inc-v4, IncRes-v2, MnasNet1_0, MobileNet, DenseNet121 and GoogleNet.
[0027] Furthermore, a preferred implementation is provided, wherein the parameters set in step 2 include the step size α, the number of iterations T, the maximum disturbance amplitude ∈ of a single pixel, and the attenuation rate μ.
[0028] Solution 2: A system for generating adversarial samples, the system comprising:
[0029] A training module for training at least one alternative model;
[0030] A parameter setting module, used to set the parameters of the replacement model in the training module;
[0031] A selection module, used for randomly selecting a replacement model from at least one replacement model in the training module;
[0032] A saliency map generation module is used to input the original image into the alternative model selected in the selection module, and back-propagate to generate a saliency map of the original image;
[0033] A sorting module is used to sort the pixels of the original image according to the significance of its saliency map; pixels with a significance quantile greater than 95% are set as foreground, and pixels with a significance quantile less than 30% are set as background;
[0034] A segmentation module, used for setting the maximum connected component of the foreground pixels as the segmented foreground object;
[0035] A mask generation module is used to generate a mask based on the foreground object segmented in the segmentation module to obtain a picture with only the foreground object;
[0036] A data input module, used to input images with only foreground objects into the substitution model and execute the calculation modules to the adversarial sample gradient calculation module respectively;
[0037] Image gradient calculation module, used to calculate the image gradient of foreground objects;
[0038] The direction judgment module is used to make the adversarial sample move in two directions, one direction of gradient increase and the other direction of gradient decrease. If the gradient decrease direction has passed the minimum point, it will be changed to move in the direction of gradient increase. The moving direction is calculated by the NAG algorithm.
[0039] The iteration coefficient judgment module is used to judge whether the set number of iterations has been completed. If not, it returns to the saliency map generation module;
[0040] The attack module is used to determine whether the generated adversarial sample can successfully attack the adversarial sample according to the number of iterations set in the iteration coefficient judgment module. If only one adversarial sample successfully attacks the alternative model, the sample is selected to proceed to the next step. If both adversarial samples successfully attack the alternative model, the adversarial sample with the smallest disturbance is selected to proceed to the next step.
[0041] The adversarial sample gradient calculation module is used to calculate the adversarial sample gradient and make it move one step in the direction of gradient ascent and then return this image;
[0042] The adversarial sample generation module is used to obtain the generated adversarial samples by weighted averaging the images returned by all alternative models.
[0043] Solution three: A computer device includes a memory and a processor, wherein the memory stores a computer program, and when the processor runs the computer program stored in the memory, the processor executes any one of the methods described in Solution one.
[0044] Solution 4: A computer-readable storage medium storing a computer program, wherein the computer program, when executed by a processor, implements the steps of the method described in any one of Solution 1.
[0045] The present invention is beneficial in that:
[0046] The present invention proposes an adversarial sample generation method and system. Compared with the adversarial sample generation method in the prior art, the method described in the present invention introduces the second-order derivative information of the gradient by using the NAG method, thereby accelerating the convergence speed and avoiding falling into poor local extreme values during the adversarial sample generation process. In addition, after the adversarial sample successfully attacks the alternative model, the method described in the present invention continues to allow the adversarial sample to move deeper into the misclassification, allowing the adversarial sample to learn more features of the misclassification and have a higher success rate of transfer attack. At the same time, the method described in the present invention improves the generalization of the generated adversarial perturbations by generating adversarial samples on multiple alternative models.
[0047] The adversarial attack described in the present invention can overcome the problem in the prior art that deep neural networks are easily affected by small perturbations, which may affect the security of deployed neural networks, and help enhance the robustness of models and improve existing model training algorithms. In the fields of autonomous driving, image and speech recognition, using adversarial attack technology, researchers can find and fix vulnerabilities in models, thereby improving the security and reliability of models.
[0048] The method described in the present invention can be applied to robustness evaluation before deep learning application, and can also be used as a comparison algorithm for other adversarial sample generation algorithms. BRIEF DESCRIPTION OF THE DRAWINGS
[0049] Figure 1 A flowchart of an adversarial sample generation method according to Implementation Method 1.
[0050] Figure 2 Schematic diagram of the foreground segmentation process in the adversarial sample generation method described in Implementation Method 1.
[0051] Figure 3 This is a schematic diagram of the process of obtaining a picture with only a foreground object from S3 to S7 in the first implementation mode.
[0052] Figure 4 This is a calculation flow chart of the gradient descent direction in the S10 calculation stage in the first implementation mode.
[0053] Figure 5 Schematic diagram of the iterative trajectory in S11 in the first implementation mode.
[0054] Figure 6 A schematic diagram of the comparison between the original image and the generated adversarial sample described in Implementation Example 8.
[0055] In the figure, Figure 6 (a) is the original picture, Figure 6 (b) is an adversarial example. DETAILED DESCRIPTION
[0056] In order to make the purpose, technical solutions and advantages of the implementation methods of the present application clearer, the technical solutions in the implementation methods of the present application will be clearly and completely described below in conjunction with the drawings in the implementation methods of the present application. Obviously, the described implementation methods are only part of the implementation methods of the present application, not all of the implementation methods.
[0057] Implementation method 1: This implementation method provides an adversarial sample generation method by introducing the second-order derivative information of momentum, limiting the range of features destroyed by adversarial perturbations, and using more alternative models. The method includes the following steps:
[0058] S1. Train at least one alternative model;
[0059] S2, setting the parameters of the substitution model described in S1;
[0060] S3, randomly select an alternative model from at least one alternative model in S1;
[0061] S4, input the original image into the alternative model selected in S3, and back propagate to generate a saliency map of the original image;
[0062] S5, sorting the pixels of the original image according to the significance of its saliency map; setting the pixels with a significance quantile greater than 95% as the foreground, and setting the pixels with a significance quantile less than 30% as the background;
[0063] S6, setting the maximum connected component of the foreground pixel as the segmented foreground object;
[0064] S7, generating a mask according to the foreground object segmented in S6, and obtaining a picture containing only the foreground object;
[0065] S8, inputting pictures with only foreground objects into the substitution models respectively and executing S9 to S13 respectively;
[0066] S9, calculating the image gradient of the foreground object;
[0067] S10, let the adversarial sample move forward in two directions, one direction of gradient increase and the other direction of gradient decrease. If the gradient decrease direction has passed the minimum point, change it to the gradient increase direction, where the forward direction is calculated by the NAG algorithm;
[0068] S11, determine whether the set number of iterations has been completed, if not, return to S4;
[0069] S12, judging whether the generated adversarial sample can successfully attack the adversarial sample according to the number of iterations set in S11, if only one adversarial sample successfully attacks the alternative model, then selecting the sample to proceed to the next step, if both adversarial samples successfully attack the alternative model, then selecting the adversarial sample with the smallest disturbance to proceed to the next step;
[0070] S13, calculate the adversarial sample gradient and make it move one step in the direction of gradient ascent and then return this image;
[0071] S14. The generated adversarial samples are obtained by weighted averaging the images returned by all alternative models.
[0072] Implementation method 2: This implementation method further limits the adversarial sample generation method described in implementation method 1. The formula for calculating the forward direction using the NAG method in S10 is:
[0073]
[0074]
[0075] x′ t+1 =Clip X,ε {x′ t +α·g t+1}
[0076] In the formula, represents the current adversarial sample, represents the updated adversarial sample, α is the step size, μ is the decay rate, and g t is the current gradient, Represents the cross entropy of A and B, Clip X,ε {x′ t +α·g t+1} means limiting the disturbance amplitude to within ε.
[0077] Implementation method three. This implementation method further limits the adversarial sample generation method described in implementation method one. The alternative models described in S1 include ResNet50, Inc-v3, Inc-v4, IncRes-v2, MnasNet1_0, MobileNet, DenseNet121 and GoogleNet.
[0078] Implementation method 4: This implementation method further limits the adversarial sample generation method described in implementation method 1. The parameters set in step 2 include step size α, number of iterations T, maximum perturbation amplitude ∈ of a single pixel, and attenuation rate μ.
[0079] Embodiment 5: This embodiment proposes a system for generating adversarial samples, the system comprising:
[0080] A training module for training at least one alternative model;
[0081] A parameter setting module, used to set the parameters of the replacement model in the training module;
[0082] A selection module, used for randomly selecting a replacement model from at least one replacement model in the training module;
[0083] A saliency map generation module is used to input the original image into the alternative model selected in the selection module, and back-propagate to generate a saliency map of the original image;
[0084] A sorting module is used to sort the pixels of the original image according to the significance of its saliency map; pixels with a significance quantile greater than 95% are set as foreground, and pixels with a significance quantile less than 30% are set as background;
[0085] A segmentation module, used for setting the maximum connected component of the foreground pixels as the segmented foreground object;
[0086] A mask generation module is used to generate a mask based on the foreground object segmented in the segmentation module to obtain a picture with only the foreground object;
[0087] A data input module, used to input images with only foreground objects into the substitution model and execute the calculation modules to the adversarial sample gradient calculation module respectively;
[0088] Image gradient calculation module, used to calculate the image gradient of foreground objects;
[0089] The direction judgment module is used to make the adversarial sample move in two directions, one direction of gradient increase and the other direction of gradient decrease. If the gradient decrease direction has passed the minimum point, it will be changed to move in the direction of gradient increase. The moving direction is calculated by the NAG algorithm.
[0090] The iteration coefficient judgment module is used to judge whether the set number of iterations has been completed. If not, it returns to the saliency map generation module;
[0091] The attack module is used to determine whether the generated adversarial sample can successfully attack the adversarial sample according to the number of iterations set in the iteration coefficient judgment module. If only one adversarial sample successfully attacks the alternative model, the sample is selected to proceed to the next step. If both adversarial samples successfully attack the alternative model, the adversarial sample with the smallest disturbance is selected to proceed to the next step.
[0092] The adversarial sample gradient calculation module is used to calculate the adversarial sample gradient and make it move one step in the direction of gradient ascent and then return this image;
[0093] The adversarial sample generation module is used to obtain the generated adversarial samples by weighted averaging the images returned by all alternative models.
[0094] Embodiment 6: This embodiment proposes a computer device, including a memory and a processor, wherein the memory stores a computer program, and when the processor runs the computer program stored in the memory, the processor executes the method described in any one of embodiments 1 to 4.
[0095] Embodiment 7: This embodiment proposes a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the steps of the method described in any one of Embodiments 1 to 4 are implemented.
[0096] Implementation Method 8: See Figures 1 to 6 To illustrate this embodiment, this embodiment provides the following examples, which are used to explain the above-mentioned embodiments 1 to 7, and the specific implementation process is as follows:
[0097] Step 1: Train multiple alternative models, the model structures are: ResNet50, Inc-v3, Inc-v4, IncRes-v2, MnasNet1_0, MobileNet, DenseNet121 and GoogleNet;
[0098] Step 2: Set parameters. Step size α = 1, where the larger the step size, the faster the theoretical convergence speed, but in the actual calculation process, too large a step size may cause the sample to oscillate continuously near the extreme point, which will lead to failure to converge; number of iterations T = 20, where 5-∞ can be taken. The more iterations, the better the effect, but the larger the number of iterations, the lower the profit growth; the maximum perturbation amplitude of a single pixel ∈ = 15, where the perturbation amplitude is invisible to the human eye when it is within 20 of each pixel. The larger the maximum perturbation amplitude, the better the interference effect on the model, and the attenuation rate μ = 0.9, where the larger the attenuation rate, the greater the influence of the previous gradient on the current update direction;
[0099] Step 3: Randomly select a model from all alternative models
[0100] Step 4: Input the original image into the model selected in step 3 and back propagate to generate the saliency map of the image.
[0101] Step 5: The pixels of the image are sorted according to their significance, and the pixels with a significance quantile greater than the 95% quantile are set as the foreground, and the pixels with a significance quantile less than the 30% quantile are set as the background.
[0102] Step 6: Set the maximum connected component of the foreground pixel as the segmented foreground object
[0103] Step 7: Generate a mask based on the foreground object segmented in step 6 to obtain a picture with only the foreground object
[0104] Step 8: Input the pictures with only foreground objects into the substitution model and execute steps 9 to 13 respectively;
[0105] Step 9: Calculate the image gradient;
[0106] Step 10: Let the adversarial sample move in two directions, one direction of gradient ascent and the other direction of gradient descent. If the gradient descent direction has passed the minimum point, change it to the gradient ascent direction. The specific moving direction is calculated by the NAG algorithm.
[0107] Step 11: Determine whether the set number of iterations has been completed. If not, return to step 4;
[0108] Step 12: Determine whether the generated adversarial sample can successfully attack the adversarial sample. If only one sample can successfully attack the alternative model, select that sample to proceed to the next step. If both samples can successfully attack the alternative model, select the adversarial sample with less perturbation to proceed to the next step.
[0109] Step 13: Calculate the adversarial sample gradient and return this image after taking a step forward in the direction of gradient ascent.
[0110] Step 14: Take the weighted average of the returned images of all alternative models to obtain the generated adversarial samples.
[0111] The formula for updating the image gradient in the NAG method in step 10 is:
[0112]
[0113]
[0114] x′ t+1 =Clip X,ε {x′ t +α·g t+1}
[0115] In the formula, represents the current adversarial sample, represents the updated adversarial sample, α is the step size, μ is the decay rate, and g t is the current gradient, Represents the cross entropy of A and B, Clip X,ε {x′ t +α·g t+1} means limiting the disturbance amplitude to within ε.
[0116] in, Figure 4 When the adversarial sample moves along this direction, the gradient first descends to the local minimum of the classification, and then the gradient rises.
[0117] Figure 5 The green track in the middle is the monotonic gradient ascending direction, i.e. gradient ascend, and the purple track is the track where the gradient first descends to the minimum value and then ascends, i.e. gradient decend.
[0118] Figure 6 (a) is the original image, and (b) is the adversarial image. It can be seen that the difference between the generated adversarial sample and the original image is invisible to the naked eye.
[0119] This embodiment is different from the traditional adversarial sample generation method. This method introduces the second-order derivative information of the gradient by using the NAG method, which accelerates the convergence speed and avoids falling into poor local extreme values during the adversarial sample generation process. After the adversarial sample successfully attacks the alternative model, this method continues to allow it to move deeper into the misclassification, allowing the adversarial sample to learn more features of the misclassification and have a higher success rate of transfer attack. At the same time, the method described in this embodiment improves the generalization of the generated adversarial perturbations by generating adversarial samples on multiple alternative models.
[0120] The adversarial attack described in this embodiment can overcome the problem that deep neural networks in the prior art are easily affected by small perturbations, which may affect the security of deployed neural networks, and help enhance the robustness of the model and improve the existing model training algorithm. In the fields of autonomous driving, image and speech recognition, using adversarial attack technology, researchers can find and fix vulnerabilities in the model, thereby improving the security and reliability of the model.
[0121] The method described in this embodiment can be applied to robustness evaluation before deep learning application, and can also be used as a comparison algorithm for other adversarial sample generation algorithms.
[0122] Figure 1 and Figure 2 Any process or method description in the flowchart described in or otherwise described herein can be understood as representing a module, fragment or portion of a code including one or more executable instructions for implementing the steps of a custom logic function or process, and the scope of the preferred embodiment of the present invention includes other implementations, in which the functions may be performed in a substantially simultaneous manner or in a reverse order according to the functions involved, which should be understood by a person skilled in the art of the present invention. The logic and / or steps represented in the flowchart or otherwise described herein illustrate the possible implementation architecture, functions and operations of the apparatus and methods according to various embodiments of the present disclosure. In this regard, each box in the flowchart or block diagram can represent a module, a program segment, or a part of a code, and the above-mentioned module, program segment, or a part of a code contains one or more executable instructions for implementing the specified logic function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in an order different from that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in reverse order, depending on the functions involved. It is also noted that each block in the block diagram or flowchart, and combinations of blocks in the block diagram or flowchart, may be implemented by a dedicated hardware-based system that performs the specified functions or operations, or may be implemented by a combination of dedicated hardware and computer instructions. For example, what may be considered as an ordered list of executable instructions for implementing a logical function may be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a system including a processor, or other system that can fetch instructions from and execute instructions in an instruction execution system, apparatus, or device).
[0123] As far as the tenth embodiment of this specification is concerned, "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit programs for use in an instruction execution system, device or equipment or in combination with these instruction execution systems, devices or equipment. More specific examples (non-exhaustive list) of computer-readable media include the following: an electrical connection portion (electronic device) with one or N wirings, a portable computer disk box (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and editable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium can even be paper or other suitable media on which the program can be printed, because the program can be obtained electronically, for example, by optically scanning the paper or other media, followed by editing, interpreting or processing in other suitable ways as necessary, and then stored in a computer memory. It should be understood that the various parts of the present invention can be implemented with hardware, software, firmware or a combination thereof. In the above embodiment, N steps or methods can be implemented with software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one of the following technologies known in the art or a combination thereof: a discrete logic circuit having a logic gate circuit for implementing a logic function on a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.
[0124] Those skilled in the art will appreciate that the above are only preferred embodiments of the present invention, and the various embodiments of the present disclosure and / or the features described in the claims may be combined or combined in various ways, even if such combinations or combinations are not explicitly described in the present disclosure. It is not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art may still modify the technical solutions described in the aforementioned embodiments, or perform equivalent substitutions on some of the technical features therein. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention shall be included in the protection scope of the present invention.
[0125] Although preferred embodiments of the present invention have been described, additional changes and modifications may be made to these embodiments by those skilled in the art once the basic inventive concepts are known. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present invention. Obviously, those skilled in the art may make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.
Claims
1. A method for generating adversarial samples, characterized in that: The method comprises the following steps: S1. Train at least one alternative model; S2, setting the parameters of the substitution model described in S1; S3, randomly select an alternative model from at least one alternative model in S1; S4, input the original image into the alternative model selected in S3, and back propagate to generate a saliency map of the original image; S5, sorting the pixels of the original image according to the significance of its saliency map; setting the pixels with a significance quantile greater than 95% as the foreground, and setting the pixels with a significance quantile less than 30% as the background; S6, setting the maximum connected component of the foreground pixel as the segmented foreground object; S7, generating a mask according to the foreground object segmented in S6, and obtaining a picture containing only the foreground object; S8, inputting pictures with only foreground objects into the substitution models respectively and executing S9 to S13 respectively; S9, calculating the image gradient of the foreground object; S10, let the adversarial sample move forward in two directions, one direction of gradient increase and the other direction of gradient decrease. If the gradient decrease direction has passed the minimum point, change it to the gradient increase direction, where the forward direction is calculated by the NAG algorithm; S11, determine whether the set number of iterations has been completed, if not, return to S4; S12, judging whether the generated adversarial sample can successfully attack the adversarial sample according to the number of iterations set in S11, if only one adversarial sample successfully attacks the alternative model, then selecting the sample to proceed to the next step, if both adversarial samples successfully attack the alternative model, then selecting the adversarial sample with the smallest disturbance to proceed to the next step; S13, calculate the adversarial sample gradient and make it move one step in the direction of gradient ascent and then return this image; S14. The generated adversarial samples are obtained by weighted averaging the images returned by all alternative models.
2. The adversarial sample generation method according to claim 1, characterized in that: The formula for calculating the forward direction using the NAG method in S10 is: x′ t+1 =Clip X,ε {x′ t +α·g t+1 } In the formula, represents the current adversarial sample, represents the updated adversarial sample, α is the step size, μ is the decay rate, and g t is the current gradient, Represents the cross entropy of A and B, Clip X,ε {x′ t +α·g t+11 } means limiting the disturbance amplitude to within ε.
3. The adversarial sample generation method according to claim 1, characterized in that: Alternative models described in S1 include ResNet50, Inc-v3, Inc-v4, IncRes-v2, MnasNet1_0, MobileNet, DenseNet121, and GoogleNet.
4. The adversarial sample generation method according to claim 1, characterized in that: The parameters set in step 2 include the step size α, the number of iterations T, the maximum perturbation amplitude ∈ of a single pixel, and the decay rate μ.
5. A system for generating adversarial samples, characterized in that: The system comprises: A training module for training at least one alternative model; A parameter setting module, used to set the parameters of the replacement model in the training module; A selection module, used for randomly selecting a replacement model from at least one replacement model in the training module; A saliency map generation module is used to input the original image into the alternative model selected in the selection module, and back-propagate to generate a saliency map of the original image; A sorting module is used to sort the pixels of the original image according to the significance of its saliency map; pixels with a significance quantile greater than 95% are set as foreground, and pixels with a significance quantile less than 30% are set as background; A segmentation module, used for setting the maximum connected component of the foreground pixels as the segmented foreground object; A mask generation module is used to generate a mask based on the foreground object segmented in the segmentation module to obtain a picture with only the foreground object; A data input module, used to input images with only foreground objects into the substitution model and execute the calculation modules to the adversarial sample gradient calculation module respectively; Image gradient calculation module, used to calculate the image gradient of foreground objects; The direction judgment module is used to make the adversarial sample move in two directions, one direction of gradient increase and the other direction of gradient decrease. If the gradient decrease direction has passed the minimum point, it will be changed to move in the direction of gradient increase. The moving direction is calculated by the NAG algorithm. The iteration coefficient judgment module is used to judge whether the set number of iterations has been completed. If not, it returns to the saliency map generation module; The attack module is used to determine whether the generated adversarial sample can successfully attack the adversarial sample according to the number of iterations set in the iteration coefficient judgment module. If only one adversarial sample successfully attacks the alternative model, the sample is selected to proceed to the next step. If both adversarial samples successfully attack the alternative model, the adversarial sample with the smallest disturbance is selected to proceed to the next step. The adversarial sample gradient calculation module is used to calculate the adversarial sample gradient and make it move one step in the direction of gradient ascent and then return this image; The adversarial sample generation module is used to obtain the generated adversarial samples by weighted averaging the images returned by all alternative models.
6. A computer device comprising a memory and a processor, characterized in that A computer program is stored in the memory. When the processor runs the computer program stored in the memory, the processor executes the method according to any one of claims 1 to 4.
7. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 4 are implemented.