An adversarial sample generation method and device, a storage medium and an electronic device

Abstract

The specification discloses an adversarial sample generation method, device, storage medium and electronic equipment. In the adversarial sample generation method provided in the specification, an original image and a label of the original image are obtained; the original image is taken as a target image, and is input into a pre-trained image classification model to obtain a classification result output by the image classification model; it is judged whether the classification result matches the label; if not, the target image is determined as an adversarial sample image; if yes, a difference between the classification result and the label is determined, a state is determined according to the difference, the state is input into a pre-trained sample generation model, an action value in the state is output by the sample generation model, a perturbation image is determined according to the target image and the action value, and the perturbation image is re-determined as a target image.

Description

TECHNICAL FIELD

[0001] The present specification relates to the technical field of computer technology, and particularly relates to an adversarial sample generation method and device, a storage medium and an electronic device. BACKGROUND

[0002] With the development of artificial intelligence technology, many carefully designed samples can deliberately make neural network models recognize errors after input, and this kind of sample is usually called an adversarial sample. The existence of adversarial samples makes many programs and applications based on neural network models have security risks, so the research on adversarial samples is very important.

[0003] However, at present, the existing method usually adopts fixed perturbation and fixed rounds to perturb the original sample when generating an adversarial sample. However, for different original samples, different perturbations are often needed to generate suitable adversarial samples. When the perturbation is too large, the generated adversarial sample may have too low similarity with the original sample, losing the significance as an adversarial sample; when the perturbation is too small, the generated adversarial sample may change too little to deceive the neural network model.

[0004] It can be seen that the adaptive ability of the existing method for generating adversarial samples is poor, and the quality of the generated adversarial samples is low.

[0005] To solve the above technical problems, the present specification provides an adversarial sample generation method with strong adaptive ability. SUMMARY

[0006] The present specification provides an adversarial sample production method, device, storage medium and electronic device to at least partially solve the above problems existing in the prior art.

[0007] The present specification adopts the following technical solutions:

[0008] The present specification provides an adversarial sample generation method, comprising:

[0009] obtaining an original image and a label of the original image;

[0010] inputting the original image as a target image into a pre-trained image classification model to obtain a classification result output by the image classification model;

[0011] determining whether the classification result matches the label;

[0012] if not, determining the target image as an adversarial sample image;

[0013] If yes, a difference between the classification result and the label is determined, a state is determined according to the difference, the state is input into a pre-trained sample generation model, an action value in the state is output by the sample generation model, a perturbed image is determined according to the target image and the action value, and the perturbed image is re-determined as the target image.

[0014] Optionally, the label of the original image is obtained, and specifically includes:

[0015] The original image is input into a pre-trained image classification model, and an output of the image classification model is determined as the label of the original image.

[0016] Optionally, a classification result output by the image classification model is obtained, and specifically includes:

[0017] The probability that the target image belongs to each category output by the image classification model is obtained.

[0018] Optionally, it is determined whether the classification result matches the label, and specifically includes:

[0019] According to the probability that the target image belongs to each category, a category to which the target image belongs is determined.

[0020] When the category is the same as the label, it is determined that the classification result matches the label.

[0021] When the category is different from the label, it is determined that the classification result does not match the label.

[0022] Optionally, a difference between the classification result and the label is determined, and specifically includes:

[0023] A cross-entropy loss between the classification result and the label is determined.

[0024] Optionally, a state is determined according to the difference, and specifically includes:

[0025] According to the cross-entropy loss between the classification result and the label, a gradient of the cross-entropy loss is determined.

[0026] The target image and the gradient are determined as the state.

[0027] Optionally, a perturbed image is determined according to the target image and the action value, and specifically includes:

[0028] According to the action value, an action corresponding to the action value is determined.

[0029] The action is taken as a perturbation factor, and the target image is perturbed to obtain a perturbed image.

[0030] Optionally, the perturbed image is re-determined as a target image, specifically comprising:

[0031] determining a current perturbation round, a value of the perturbation round increasing as a number of times of determining the perturbed image increases;

[0032] when the perturbation round is greater than a first specified threshold, determining the perturbed image as an adversarial sample image;

[0033] when the perturbation round is not greater than the first specified threshold, re-determining the perturbed image as a target image.

[0034] Optionally, the pre-training sample generation model, specifically comprising:

[0035] obtaining a sample original image and a sample label of the sample original image;

[0036] inputting the sample original image as a sample target image into a pre-trained image classification model to obtain a sample classification result output by the image classification model;

[0037] judging whether the sample classification result matches the sample label;

[0038] if yes, determining a reward as a first preset value, determining a difference between the sample classification result and the sample label, determining a sample state according to the difference, inputting the sample state into a to-be-trained sample generation model, outputting a to-be-optimized action value in the sample state by the sample generation model, determining a to-be-optimized perturbed image according to the sample target image and the to-be-optimized action value, and re-determining the to-be-optimized perturbed image as a sample target image;

[0039] if no, determining a reward according to the sample target image and the sample original image;

[0040] training the sample generation model with the maximum reward as an optimization target.

[0041] Optionally, determining the to-be-optimized perturbed image according to the sample target image and the to-be-optimized action value, specifically comprising:

[0042] determining an action corresponding to the to-be-optimized action value according to the to-be-optimized action value;

[0043] inputting the action as a perturbation factor to perturb the sample target image to obtain a to-be-optimized perturbed image.

[0044] Optionally, inputting the action as a perturbation factor, specifically comprising:

[0045] randomly generating a discrimination value in a specified interval;

[0046] if the discrimination value is not less than a second specified threshold value, the action is taken as a disturbance factor;

[0047] if the discrimination value is less than the second specified threshold value, a random action in a preset action interval is selected as a disturbance factor.

[0048] Optionally, the reward is determined according to the sample target image and the sample original image, specifically including:

[0049] the reward is determined according to the similarity between the sample target image and the sample original image, wherein the reward is positively correlated with the similarity.

[0050] Optionally, the first preset value is less than the minimum value of the value range of the similarity.

[0051] Optionally, before training the sample generation model, the method further includes:

[0052] determining a current sample disturbance round, the value of the sample disturbance round increasing with the number of times of re-determining the to-be-optimized disturbance image as a sample target image;

[0053] when the sample disturbance round is greater than a first specified threshold value, determining that the reward is a second preset value, the second preset value being less than the first preset value and less than the minimum value of the value range of the similarity.

[0054] The specification provides an apparatus for generating an adversarial sample, the apparatus comprising:

[0055] an acquisition module configured to acquire an original image and a label of the original image;

[0056] an input module configured to input the original image as a target image into a pre-trained image classification model to obtain a classification result output by the image classification model;

[0057] a judgment module configured to judge whether the classification result matches the label;

[0058] a determination module configured to, if not, determine the target image as an adversarial sample image;

[0059] a loop module configured to, if yes, determine a difference between the classification result and the label, determine a state according to the difference, input the state into a pre-trained sample generation model, output an action value in the state by the sample generation model, determine a disturbance image according to the target image and the action value, and re-determine the disturbance image as a target image.

[0060] The specification provides a computer readable storage medium, the storage medium stores a computer program, the computer program is executed by a processor to realize the above-mentioned adversarial sample generation method.

[0061] The specification provides an electronic device, including a memory, a processor and a computer program stored on the memory and executable on the processor, the processor executes the program to realize the above-mentioned adversarial sample generation method.

[0062] The above-mentioned at least one technical solution adopted by the specification can achieve the following beneficial effects:

[0063] In the adversarial sample generation method provided by the specification, an original image and a label of the original image are obtained; the original image is taken as a target image, and a pre-trained image classification model is input to obtain a classification result output by the image classification model; it is judged whether the classification result matches the label; if not, the target image is determined as an adversarial sample image; if yes, the difference between the classification result and the label is determined, the state is determined according to the difference, the state is input into a pre-trained sample generation model, the action value in the state is output by the sample generation model, the perturbation image is determined according to the target image and the action value, and the perturbation image is determined as a target image again.

[0064] When the adversarial sample generation method provided by the specification is used to generate an adversarial sample image, the original image can be perturbed for multiple rounds, and in each round of perturbation, a perturbation factor for perturbing the target image is adaptively determined, and at the same time, the image can be perturbed adaptively by using a suitable perturbation round, effectively avoiding the problem of low quality of the adversarial sample image caused by too large or too small perturbation in the process of generating the adversarial sample. BRIEF DESCRIPTION OF DRAWINGS

[0065] The drawings described herein are used to provide further understanding of the specification, constitute a part of the specification, the illustrative embodiments of the specification and the description thereof are used to explain the specification, and do not constitute improper limitation on the specification. In the drawings:

[0066] Figure 1 It is a flowchart of an adversarial sample generation method in the specification;

[0067] Figure 2 It is a schematic diagram of an adversarial sample generation device provided by the specification;

[0068] Figure 3 It is a schematic diagram of an electronic device provided by the specification corresponding to Figure 1 . DETAILED DESCRIPTION

[0069] In order to make the purpose, technical solutions and advantages of the present specification clearer, the technical solutions of the present specification will be described clearly and completely in the following with specific embodiments of the present specification and corresponding drawings. Obviously, the described embodiments are only part of the embodiments of the present specification, not all the embodiments. Based on the embodiments in the present specification, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of the present application.

[0070] The technical solutions provided by the embodiments of the present specification will be described in detail below with reference to the drawings.

[0071] Figure 1 For the flowchart of an adversarial sample generation method in the present specification, the specific steps include the following steps:

[0072] S100: Obtain an original image and a label of the original image.

[0073] All steps in the adversarial sample generation method provided by the present specification can be realized by any electronic device with computing function, such as terminal, server and the like.

[0074] In the neural network model technology, the adversarial sample refers to an input sample formed by deliberately adding subtle interference to the original sample, which causes the model to give a wrong output with high confidence. The adversarial sample generation method provided by the present specification is mainly used for generating adversarial samples for image classification tasks. Based on this, in this step, an original image used for generating an adversarial sample image and a label of the original image can be obtained. The original image can be obtained from an existing image dataset, such as CIFAR-10 image dataset; the label of the original image is the true class to which the original image belongs. The label of an original image can only contain one true class, or can contain multiple true classes. When determining the label of the original image, a manual determination method can be used, or the original image can be input into a pre-trained image classification model, and the output of the image classification model is determined as the label of the original image. The present specification does not make specific limitations thereto.

[0075] S102: Input the original image as a target image into a pre-trained image classification model to obtain a classification result output by the image classification model.

[0076] In the adversarial sample generation method provided by the present specification, the adversarial sample image is generated by a number of rounds of perturbation of the original image, and the target image is the target of each round of perturbation. It can be thought that before any perturbation, the original image can be determined as the target image for the first round of perturbation.

[0077] In the method for generating an adversarial sample provided in the present specification, the image classification model can be any neural network model with image classification function that has been pre-trained. The target model is input into the image classification model, and the classification result of the target image can be output by the image classification model. The classification result can be one or more categories to which the target image belongs.

[0078] More preferably, the classification result of the target image output by the image classification model can also be the probability of the target image belonging to each category. Thus, the category with the highest probability can be selected from the probabilities of the target image belonging to each category to determine the category of the target image; or the category with a probability greater than a specified percentage can be determined as the category of the target image from the probabilities of each category. Similarly, the category of the target image determined by the image classification model can be one or more, which is not specifically limited in the present specification. However, it should be noted that in order to ensure the consistency of the model, the mode output by the model should be the same as the labeled mode, i.e., if the model can only output one category, the label should also only contain one real category; if the model can output multiple categories, the label can also contain multiple real categories. In order to ensure the simplicity and ease of understanding of the present specification, only the case where the image classification model outputs only one category is taken as an example for illustration in the subsequent embodiments of the present specification.

[0079] S104: Determine whether the classification result matches the label.

[0080] In this step, it can be determined whether the classification result determined in step S102 matches the label obtained in step S100. Specifically, when the category is the same as the label, it can be determined that the classification result matches the label; when the category is different from the label, it can be determined that the classification result does not match the label.

[0081] For the two cases of the classification result matching the label and the classification result not matching the label, different processing can be performed respectively.

[0082] S106: If not, the target image is determined as an adversarial sample image.

[0083] When the classification result does not match the label, the target image can be determined as an adversarial sample image in this step.

[0084] Since the role of the adversarial sample is to induce the model to make a wrong judgment and make the model give a wrong output with high confidence, when the classification result does not match the label, it can be considered that the target image obtained after the scrambling in the previous rounds is sufficient to make the image classification model output a wrong classification. Based on this, the target image can be determined as an applicable adversarial sample image.

[0085] S108: If yes, determine the difference between the classification result and the label, determine a state according to the difference, input the state into a pre-trained sample generation model, output an action value in the state through the sample generation model, determine a perturbed image according to the target image and the action value, and determine the perturbed image as a target image again.

[0086] When the classification result matches the label, it means that the current target image is not enough to make the image classification model make a wrong judgment and cannot be used as an adversarial sample image. Therefore, the target image needs to be further perturbed.

[0087] Specifically, a pre-trained sample generation model can be used to determine an action value according to the difference between the classification result and the label, and the target image can be perturbed according to the action value to obtain a perturbed image.

[0088] The difference between the classification result and the label can be represented by the cross-entropy loss between the classification result and the label. For example, suppose the classification result is the probability that the target image belongs to each category, denoted as P = [p1, p2, p3, …, pn], where there are n items in total, and each item represents the probability that the target image belongs to a category. n represents the total number of categories; the label is the real category to which the original image without perturbation belongs, denoted as Q = [0, 0, 1, …, 0], where there are also n items in total, 1 corresponds to the real category, and 0 corresponds to other categories. It can be imagined that in addition to the value of the item corresponding to the real category being 1, the values of the items corresponding to other categories are all 0. Then the cross-entropy loss between the classification result and the label can be represented by H(P, Q), and the formula is as follows:

[0089]

[0090] Where P(i) represents the i-th item in the classification result, and Q(i) represents the i-th item in the label.

[0091] According to the determined difference, the state input to the sample generation model can be determined.

[0092] Specifically, the gradient of the cross-entropy loss between the classification result and the label can be determined according to the cross-entropy loss, and the target image and the gradient can be determined as the state.

[0093] The cross-entropy loss between the classification result and the label represents the size of the difference between the classification result and the label. In the process of generating an adversarial sample, the trend of the difference between the classification result and the label is desired. Therefore, for the determined cross-entropy loss, the gradient of the cross-entropy loss can be further determined. The determined gradient and the corresponding target image can be input into the sample generation model as the state.

[0094] The role of the sample generation model employed in the present specification is to output a corresponding action value according to an input state. According to the action value and the target image, a perturbed image, that is, a scrambled target image, can be determined. Specifically, according to the action value, an action corresponding to the action value can be determined; the action is used as a perturbation factor to scramble the target image to obtain a perturbed image.

[0095] In the adversarial sample generation method provided in the present specification, the action value corresponds to the action, any action value will have a corresponding action, and the correspondence between the action value and the action is predetermined. In this method, the action can be a perturbation factor used to scramble the target image. When the target image is scrambled using the perturbation factor, the specific formula can be as follows:

[0096]

[0097] wherein, represents the perturbed image after scrambling, represents the target image; a t represents the action, that is, the perturbation factor, represents the gradient of the cross-entropy loss between the classification result and the label; m represents a normalization coefficient, which can be set according to specific requirements, and can usually take a value such as 255.

[0098] Thus, the perturbed image can be determined. After the perturbed image is determined, the perturbed image can be re-determined as the target image, and steps S100-S108 are re-executed until the adversarial sample image is determined.

[0099] It should be noted that the role of the adversarial sample is to induce the model to output an incorrect result in a case very similar to the original sample. If too many rounds of perturbation are performed, the perturbed image obtained is likely to have deviated too much from the original image, and regardless of whether the classification result output by the image classification model matches the label, the perturbed image that deviates too much from the original image has already lost its value as an adversarial sample.

[0100] Therefore, in order to avoid the above situation, when the perturbed image is re-determined as the target image at the end of each round of scrambling, it can be additionally considered to directly jump out of the loop and directly determine the perturbed image as the adversarial sample image. Specifically, the current perturbation round can be determined, the value of the perturbation round increases as the number of times the perturbed image is determined increases; when the perturbation round is greater than a first specified threshold, the perturbed image is determined as the adversarial sample image; when the training round is not greater than the first specified threshold, the perturbed image is re-determined as the target image.

[0101] The first specified threshold value can be set according to specific requirements. In order to prevent too much scrambling from causing the disturbance image to change too much, if the disturbance has been performed for a certain number of rounds and the adversarial sample image has not been determined, the loop can be directly stopped, and the disturbance image in the current round can be directly determined as the adversarial sample image.

[0102] When the adversarial sample image is generated by using the adversarial sample generation method provided in the specification, the original image can be scrambled for multiple rounds, and in each round of scrambling, the disturbance factor used to scramble the target image is adaptively determined. At the same time, the image can be scrambled adaptively by using a suitable disturbance round, which effectively avoids the problem of low quality of the adversarial sample image caused by too much disturbance or too little disturbance in the process of generating the adversarial sample.

[0103] In addition, the sample generation model used in the specification can be pre-trained. Specifically, a sample original image and a sample label of the sample original image can be obtained; the sample original image is input into a pre-trained image classification model as a sample target image, and a sample classification result output by the image classification model is obtained; it is judged whether the sample classification result matches the sample label; if yes, it is determined that the reward is a first preset value, and the difference between the sample classification result and the sample label is determined, the sample state is determined according to the difference, the sample state is input into the sample generation model to be trained, the action value to be optimized in the sample state is output by the sample generation model, the sample target image and the action value to be optimized are used to determine a sample disturbance image to be optimized, and the sample disturbance image to be optimized is determined as a sample target image again; if not, the reward is determined according to the sample target image and the sample original image; the sample generation model is trained with the maximum reward as the optimization target.

[0104] The sample generation model used in the specification is a reinforcement learning model, which is trained by using a reinforcement learning method. Different rewards are formulated according to different sample classification results output by the image classification model for the sample target image, and the sample generation model is trained with the maximum reward as the optimization target.

[0105] In the training phase, the same as in the application, when the to-be-optimized perturbed image is determined, the action corresponding to the to-be-optimized action value can also be determined according to the to-be-optimized action value; the action is taken as a perturbation factor to perturb the sample target image to obtain the to-be-optimized perturbed image. However, the difference is that, in the training phase, in order to further enhance the learning ability of the sample generation model, an exploration process can be introduced in the process of determining the to-be-optimized perturbed image. Specifically, a discrimination value can be randomly generated in a specified interval; if the discrimination value is not less than a second specified threshold, the action is taken as a perturbation factor; if the discrimination value is less than the second specified threshold, a random action is selected as a perturbation factor in a preset action interval.

[0106] In this process, a random exploration process can be introduced by randomly generating a discrimination value. The specified interval and the second specified threshold can be set according to specific requirements, but it is necessary to ensure that the second specified threshold is in the specified interval. The specified interval and the second specified threshold can actually be regarded as determining a group of probabilities. For example, the specified interval can be set to [0, 1] and the second specified threshold can be set to 0.1, so that the probability of the randomly generated discrimination value in the specified interval being not less than the second specified threshold is 90% and the probability of the randomly generated discrimination value being less than the second specified threshold is 10%. The meaning of the above values is that, in the training process, 90% of the probability directly uses the action corresponding to the action value determined by the sample generation model as the perturbation factor and 10% of the probability randomly selects an action in the action interval as the perturbation factor. The action interval can be set according to requirements in advance and can be regarded as the value range of the perturbation factor in the training process, for example, it can be set to 0<γ≤8, where γ represents the perturbation factor.

[0107] In the reinforcement learning process of training the sample generation model, different rewards can be formulated according to different sample classification results output by the image classification model on the sample target image. Specifically, when the sample classification result matches the sample label, the reward can be determined as a first preset value; and when the sample classification result does not match the sample label, the reward can be determined according to the similarity between the sample target image and the sample original image, and at this time, the value of the reward is positively correlated with the size of the determined similarity. For example, the similarity between the sample target image and the sample original image can be directly determined as the reward. More preferably, the first preset value is less than the minimum value of the value range of the similarity.

[0108] As in the application process, in the training process, the disturbance round can also be limited by setting different rewards. Specifically, the current sample disturbance round can be determined, and the value of the sample disturbance round increases as the number of times the to-be-optimized disturbance image is re-determined as a sample target image increases; when the sample disturbance round is greater than a first specified threshold, it is determined that the reward is a second preset value, which is less than the first preset value and less than the minimum value of the value range of the similarity.

[0109] It can be seen that the size relationship between the rewards in the above three different cases is second preset value < first preset value < similarity.

[0110] In the training process, when the sample disturbance round is greater than the first specified threshold, it indicates that enough rounds of scrambling have been performed, but the image classification model still cannot judge the class to be different from the sample label, so the reward can be set to the smallest second preset value, and the sample generation model adjusts its parameters based on the reward; if the sample disturbance round does not exceed the first specified threshold, if the sample classification result matches the sample label, it means that the sample target image at this time still cannot make the image classification model judge the class to be different from the sample label, but the disturbance round has not reached the maximum, and there is still room for further scrambling, so the reward can be set to the median first preset value, and the sample generation model adjusts its parameters based on the reward; when the sample disturbance round is not greater than the first specified threshold, and the sample classification result does not match the sample label, it means that the target image at this time is sufficient to make the image classification model output an incorrect sample classification result, so the reward can be set to the similarity between the to-be-optimized sample image and the sample original image, that is, a larger reward, so that the sample generation model adjusts its parameters based on the reward. The higher the similarity, the closer the to-be-optimized sample image is to the original sample image, the more deceptive the image classification model is, the better the generated adversarial sample is, and the larger the corresponding reward is.

[0111] Wherein, the greater the reward, the better the generated adversarial sample, and the more optimal the parameters in the sample generation model. Therefore, it can be imagined that the smaller the reward, the greater the adjustment amplitude of the sample generation model when adjusting its parameters. The setting of the similarity, the first preset value and the second preset value can be reasonably set according to specific needs, and this specification does not make specific limitations. For example, the value range of the similarity can be set to (0, 1), the first preset value can be set to 0, and the second preset value can be set to a negative number such as -10.

[0112] Additionally, the training process described above is an online training process for the sample generation model. In addition to online training, the sample generation model can also be trained offline. Specifically, the trained image classification model and the untrained sample generation model can be used to obtain training data, and after a sufficient amount of training data is collected, the sample generation model can be trained.

[0113] In each piece of training data, the current state, the action, the next state, and the reward can be included. The present specification provides an embodiment of obtaining training data for reference. Specifically, first, similar to online training, the sample target image can be input into the image classification model, the sample classification result output by the image classification model can be obtained, the gradient of the sample classification result can be determined, and the sample target image and the obtained gradient parameter can be determined as the current state. The current state can be input into the sample generation model, the action corresponding to the current state can be determined according to the action value output by the sample generation model, and the sample target image can be disturbed using the action (perturbation factor) to obtain a sample perturbation image. The sample perturbation image can be re-input into the image classification model as a sample target image, the gradient corresponding to the image classification result can be determined, and the sample target image and the gradient at this time can be determined as the next state. The determination method of the reward is the same as that in online training, and will not be described here. The current state, the action, the next state, and the reward obtained through the above method can be saved as a piece of training data for application in training. It should be noted that, similar to online training, the number of perturbation rounds still needs to be counted. It is not difficult to think that when a new sample original image is perturbed and training data is collected, the sample original image can be input into the image classification model as a sample target image.

[0114] For each piece of training data, after the training data is obtained, it can be first determined whether a sufficient amount of training data has been collected. When a sufficient amount of training data has been collected, the generation of training data can be stopped, and training can be started. The number of collected training data can be set according to specific requirements, such as 10,000, and the present specification does not make specific limitations on this.

[0115] When a sufficient amount of training data has not been collected, it can be determined whether the current perturbation round is greater than a first specified threshold. If yes, a new sample original image can be obtained, and the above process of obtaining training data can be performed. If no, the next state in the above process can be taken as the current state, and the above process of obtaining training data can be performed again.

[0116] After collecting enough training data, the sample generation model can be trained. Specifically, a specified number of training data can be selected from the collected training data to train the sample generation model, and it is determined whether the sample generation model converges. If yes, it is determined that the sample generation model training is completed; if no, it is returned to the initial step to re-collect training data. The specified number can be set according to specific requirements, for example, 100, etc., which is not specifically limited in the specification.

[0117] The sample generation model can be a model designed based on a deep Q network (DQN). In the training process, the sample generation model can be trained by using, for example, a temporal difference algorithm (TD); the maximum number of iterations and the maximum number of perturbation rounds can be set according to specific requirements, for example, the maximum number of iterations can be set to 2000 times, and the maximum number of perturbation rounds can be set to 30 times, which is not specifically limited in the specification.

[0118] It is worth mentioning that, in order to obtain better training effect, the above online training method and offline training method provided by the specification can be used in combination when training the sample generation model to train the model together. Since offline training adjusts the model by a large amplitude according to the total reward during training, but does not change the parameters of the model during data collection; while online training can adjust the model parameters in real time, but the amplitude of the model adjustment is small each time, which is not suitable for training the model from scratch. Therefore, offline training and online training can be combined. Specifically, the model can be first trained by using the offline training method to adjust the model to a certain extent, and then changed to the online training method, and data for the next offline training is collected during online training of the model. Thus, the sample generation model with better training effect can be obtained by alternately combining offline training and online training.

[0119] The above is the adversarial sample generation method provided by the specification. Based on the same idea, the specification also provides a corresponding adversarial sample generation device, as shown in Figure 2

[0120] Figure 2 The adversarial sample generation device provided by the specification is shown in the figure, which specifically includes:

[0121] The acquisition module 200 is configured to acquire an original image and a label of the original image.

[0122] The input module 202 is configured to input the original image as a target image into a pre-trained image classification model to obtain a classification result output by the image classification model. ​

[0123] determining whether the classification result matches the label;

[0124] determining module 206, configured to determine the target image as an adversarial sample image if not;

[0125] cycling module 208, configured to determine a difference between the classification result and the label if yes, determine a state according to the difference, input the state into a pre-trained sample generation model, output an action value in the state through the sample generation model, determine a perturbed image according to the target image and the action value, and determine the perturbed image as a target image again.

[0126] Optionally, the obtaining module 200 is specifically configured to input the original image into a pre-trained image classification model, and determine an output of the image classification model as a label of the original image.

[0127] Optionally, the input module 202 is specifically configured to obtain probabilities that the target image output by the image classification model belongs to each category.

[0128] Optionally, the determining module 204 is specifically configured to determine a category to which the target image belongs according to the probabilities that the target image belongs to each category, determine that the classification result matches the label when the category is the same as the label, and determine that the classification result does not match the label when the category is different from the label.

[0129] Optionally, the cycling module 208 is specifically configured to determine a cross-entropy loss between the classification result and the label.

[0130] Optionally, the cycling module 208 is specifically configured to determine a gradient of the cross-entropy loss according to the cross-entropy loss between the classification result and the label, and determine the target image and the gradient as a state.

[0131] Optionally, the cycling module 208 is specifically configured to determine an action corresponding to the action value according to the action value, and perform perturbation on the target image by taking the action as a perturbation factor to obtain a perturbed image.

[0132] Optionally, the cycling module 208 is specifically configured to determine a current perturbation round, the value of the perturbation round increasing with an increase in a number of times of determining a perturbed image, determine the perturbed image as an adversarial sample image when the perturbation round is greater than a first specified threshold, and determine the perturbed image as a target image again when the perturbation round is not greater than the first specified threshold.

[0133] Optionally, the apparatus further includes a training module 210, specifically configured to obtain a sample original image and a sample label of the sample original image; input the sample original image as a sample target image into a pre-trained image classification model to obtain a sample classification result output by the image classification model; determine whether the sample classification result matches the sample label; if yes, determine that a reward is a first preset value, and determine a difference between the sample classification result and the sample label, determine a sample state according to the difference, input the sample state into a sample generation model to be trained, output an action value to be optimized in the sample state by the sample generation model, determine a perturbed image to be optimized according to the sample target image and the action value to be optimized, and determine the perturbed image to be optimized as the sample target image again; if no, determine a reward according to the sample target image and the sample original image; and train the sample generation model with the maximum reward as an optimization target.

[0134] Optionally, the training module 210 is specifically configured to determine an action corresponding to the action value to be optimized according to the action value to be optimized; and perform perturbation on the sample target image by taking the action as a perturbation factor to obtain the perturbed image to be optimized.

[0135] Optionally, the training module 210 is specifically configured to randomly generate a discrimination value in a specified interval; if the discrimination value is not less than a second specified threshold, take the action as the perturbation factor; and if the discrimination value is less than the second specified threshold, select a random action in a preset action interval as the perturbation factor.

[0136] Optionally, the training module 210 is specifically configured to determine a reward according to a similarity between the sample target image and the sample original image, where the reward is positively correlated with the similarity.

[0137] Optionally, the first preset value is less than a minimum value of a value range of the similarity.

[0138] Optionally, the training module 210 is specifically configured to determine a current sample perturbation round, and a value of the sample perturbation round increases with an increase in a number of times of determining the perturbed image to be optimized as the sample target image again; and when the sample perturbation round is greater than a first specified threshold, determine that the reward is a second preset value, the second preset value is less than the first preset value and less than the minimum value of the value range of the similarity.

[0139] The specification also provides a computer-readable storage medium storing a computer program, the computer program being used to execute the above Figure 1 The specification also provides an adversarial sample generation method.

[0140] The specification also providesFigure 3 A schematic structural diagram of an electronic device is shown. As Figure 3 At the hardware level, the electronic device includes a processor, an internal bus, a network interface, a memory, and a non-volatile memory, and can also include other hardware required by a business. The processor reads the corresponding computer program from the non-volatile memory into the memory and then runs to implement the above Figure 1 The method for generating an adversarial sample. Of course, in addition to the software implementation, the present specification does not exclude other implementation manners, such as a logic device or a combination of software and hardware, and the like, that is, the execution subject of the following processing flow is not limited to each logic unit, but can also be hardware or a logic device.

[0141] For a technical improvement, it can be obvious whether the improvement is in hardware (e.g., improvement of circuit structures of diodes, transistors, switches, etc.) or in software (e.g., improvement of method processes). However, with the development of technology, many improvements of method processes nowadays can be considered as direct improvements of hardware circuit structures. Designers almost always obtain the corresponding hardware circuit structures by programming the improved method processes into hardware circuits. Therefore, it cannot be said that an improvement of a method process cannot be implemented by a hardware entity module. For example, a programmable logic device (PLD) (e.g., a field programmable gate array (FPGA)) is an integrated circuit whose logic function is determined by user programming of the device. A designer programs a digital system "integrated" on a PLD by himself / herself, without having to ask a chip manufacturer to design and manufacture a special integrated circuit chip. Moreover, nowadays, instead of manually manufacturing integrated circuit chips, this programming is mostly implemented by "logic compiler" software, which is similar to a software compiler used when developing programs, and the original code before compilation also has to be written in a specific programming language, which is called a hardware description language (HDL), and there are many kinds of HDLs, such as ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, RHDL (Ruby Hardware Description Language), etc., and the most commonly used ones are VHDL (Very-High-Speed Integrated Circuit Hardware Description Language) and Verilog. It should be clear to those skilled in the art that only a little logical programming of the method processes in the above-mentioned hardware description languages and programming into integrated circuits can easily obtain hardware circuits implementing the logical method processes.

[0142] The controller can be implemented in any suitable way, for example, the controller can take the form of a microprocessor or processor and a computer readable medium storing computer readable program code, such as software or firmware, executable by the (micro)processor, logic gates, switches, an application specific integrated circuit (ASIC), a programmable logic controller and an embedded microcontroller, examples of which include but are not limited to the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20 and Silicone Labs C8051F320, the memory controller can also be implemented as part of the control logic of the memory. Those skilled in the art will also know that, in addition to being implemented in pure computer readable program code, the controller can equally well be implemented to perform the same functions using logic gates, switches, an application specific integrated circuit, a programmable logic controller and an embedded microcontroller, etc. by means of a logical programming of the method steps. The controller can thus be considered as a hardware component, and the means comprised therein for performing the various functions can be considered as structures within the hardware component. Alternatively, the means for performing the various functions can even be considered as both a software module implementing the method and a structure within the hardware component.

[0143] The systems, apparatuses, modules or units illustrated by the above embodiments can be specifically implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, the computer can be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

[0144] For the sake of description, the above apparatuses are described in functional division and are described respectively. Of course, the functions of the units can be implemented in the same or multiple software and / or hardware when implementing the present specification.

[0145] Those skilled in the art will understand that the embodiments of the present application can be provided as a method, a system, or a computer program product. Therefore, the present application can take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0146] The computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart block or blocks. Figure 1 one or more flow or blocks Figure 1 means for functionally implementing the steps listed in the flowchart block or blocks.

[0147] These computer program instructions can also be stored in a computer readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer readable memory produce an article of manufacture including instructions which implement the function specified in the flowchart block or blocks. Figure 1 one or more flow or blocks Figure 1 means for functionally implementing the steps listed in the flowchart block or blocks.

[0148] The computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart block or blocks. Figure 1 one or more flow or blocks Figure 1 means for functionally implementing the steps listed in the flowchart block or blocks.

[0149] In one typical configuration, the computing device includes one or more processors (CPUs), input / output interfaces, network interfaces, and memory.

[0150] The memory can include non-persistent memory and / or persistent memory, such as flash memory, read-only memory (ROM), and / or volatile or non-volatile random access memory (RAM), among others. The memory is an example of computer readable media.

[0151] Computer-readable media includes permanent and non-permanent, movable and non-movable media that can be implemented by any method or technology to store information. The information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassette, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information accessible to a computing device. According to the definition herein, computer-readable media does not include transitory media such as modulated data signals and carriers.

[0152] It should also be noted that the terms "comprising", "containing", or any other variant thereof are intended to cover non-exclusive inclusion, such that a process, method, article or apparatus that comprises a list of elements does not only include those elements, but can also include other elements not expressly listed or inherent to such process, method, article or apparatus. Without more limitations, the element defined by the statement "comprising a" does not exclude the presence of additional identical elements in the process, method, article or apparatus that includes the element.

[0153] Those skilled in the art will appreciate that embodiments of the present specification can be provided as methods, systems or computer program products. Therefore, the present specification can take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Moreover, the present specification can take the form of a computer program product implemented on one or more computer-usable storage media (including, but not limited to, magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0154] The present specification can be described in the general context of computer-executable instructions, such as program modules, executed by computers. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform particular tasks or implement particular abstract data types. The present specification can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules can be located in both local and remote computer storage media including storage devices.

[0155] The various embodiments described in this specification are described using a numbering of embodiments approach: these are each individually related to one another in order to build the overall number of embodiments described in this specification. Each individual embodiment described, however, can stand on its own as a separate inventive concept whether or not it is considered by the applicant to be the best or only embodiment. Each individual embodiment relates to one another and to the entire description herein as indicating different points of emphasis of the overall description. For example, the system embodiments are described with less detail than the method embodiments because they are substantially similar to the method embodiments. The system embodiments are therefore cross-referenced to the method embodiments for relevant portions of the description.

[0156] The above description is embodied in the form of only a few examples of the present description and is not intended to limit the present description. Various modifications and changes can be made by those skilled in the art to which the present description pertains. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present description shall be included in the scope of the claims of the present application.

Claims

1. An adversarial sample generation method, characterized in that, The method comprises the following steps: obtaining an original image and a label of the original image; inputting the original image as a target image into a pre-trained image classification model to obtain a classification result output by the image classification model; determining whether the classification result matches the label; if not, determining the target image as an adversarial sample image; if yes, determining a difference between the classification result and the label, determining a state according to the difference, inputting the state into a pre-trained sample generation model, outputting an action value in the state by the sample generation model, determining a perturbation image according to the target image and the action value, and re-determining the perturbation image as a target image; wherein, determining the perturbation image according to the target image and the action value comprises: determining an action corresponding to the action value according to the action value, wherein the action value and the action have a predetermined corresponding relationship; adding the action as a perturbation factor to the target image to obtain a perturbation image.

2. The method of claim 1, wherein, The label of the original image is obtained, specifically comprising: inputting the original image into a pre-trained image classification model, and determining the output of the image classification model as the label of the original image.

3. The method of claim 1, wherein, The classification result output by the image classification model is obtained, specifically comprising: obtaining the probability of the target image belonging to each category output by the image classification model.

4. The method of claim 3, wherein, Determining whether the classification result matches the label, specifically comprising: determining the category to which the target image belongs according to the probability of the target image belonging to each category; when the category is the same as the label, it is determined that the classification result matches the label; when the category is different from the label, it is determined that the classification result does not match the label.

5. The method of claim 1, wherein, Determining the difference between the classification result and the label, specifically comprising: determining the cross-entropy loss between the classification result and the label.

6. The method of claim 5, wherein, Determining the state according to the difference, specifically comprising: determining the gradient of the cross-entropy loss according to the cross-entropy loss between the classification result and the label; determining the target image and the gradient as the state.

7. The method of claim 1, wherein, Re-determining the perturbation image as a target image, specifically comprising: determining the current perturbation round, wherein the value of the perturbation round increases with the number of times the perturbation image is determined; when the perturbation round is greater than a first specified threshold, determining the perturbation image as an adversarial sample image; when the perturbation round is not greater than the first specified threshold, re-determining the perturbation image as a target image.

8. The method of claim 1, wherein, Pre-training the sample generation model, specifically comprising: obtaining a sample original image and a sample label of the sample original image; inputting the sample original image as a sample target image into a pre-trained image classification model to obtain a sample classification result output by the image classification model; determining whether the sample classification result matches the sample label; If yes, it is determined that the reward is a first preset value, and the difference between the sample classification result and the sample label is determined, the sample state is determined according to the difference, the sample state is input into the sample generation model to be trained, the action value to be optimized in the sample state is output through the sample generation model, the disturbance image to be optimized is determined according to the sample target image and the action value to be optimized, and the disturbance image to be optimized is determined as the sample target image again; If no, the reward is determined according to the sample target image and the sample original image; The sample generation model is trained with the maximum reward as an optimization target.

9. The method of claim 8, wherein, The disturbance image to be optimized is determined according to the sample target image and the action value to be optimized, and specifically includes: According to the action value to be optimized, an action corresponding to the action value to be optimized is determined; The action is taken as a disturbance factor to disturb the sample target image to obtain the disturbance image to be optimized.

10. The method of claim 9, wherein, The action is taken as a disturbance factor, and specifically includes: A discrimination value is randomly generated in a specified interval; If the discrimination value is not less than a second specified threshold value, the action is taken as a disturbance factor; If the discrimination value is less than the second specified threshold value, a random action is selected as a disturbance factor in a preset action interval.

11. The method of claim 8, wherein, The reward is determined according to the sample target image and the sample original image, and specifically includes: The reward is determined according to the similarity between the sample target image and the sample original image, and the reward is positively correlated with the similarity.

12. The method of claim 11, wherein, The first preset value is less than the minimum value of the value range of the similarity.

13. The method of claim 11, wherein, Before the sample generation model is trained, the method further includes: A current sample disturbance round is determined, and the value of the sample disturbance round increases with the number of times that the disturbance image to be optimized is determined as the sample target image again; When the sample disturbance round is greater than a first specified threshold value, it is determined that the reward is a second preset value, the second preset value is less than the first preset value, and is less than the minimum value of the value range of the similarity.

14. An adversarial sample generation apparatus, comprising: It includes: An acquisition module is configured to acquire an original image and a label of the original image; An input module is configured to input the original image as a target image into a pre-trained image classification model to obtain a classification result output by the image classification model; A judgment module is configured to judge whether the classification result matches the label; A determination module is configured to determine the target image as an adversarial sample image if the classification result does not match the label; A loop module is configured to determine the difference between the classification result and the label, determine a state according to the difference, input the state into a pre-trained sample generation model, output an action value in the state through the sample generation model, determine an action corresponding to the action value according to the action value, the action value and the action have a predetermined corresponding relationship, disturb the target image by taking the action as a disturbance factor, determine a disturbance image, and determine the disturbance image as the target image again if the classification result matches the label.

15. A computer-readable storage medium, characterized in that, The storage medium stores a computer program, and the computer program is executed by a processor to implement the method in any one of claims 1-13.

16. An electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, The processor executes the program to implement the method in any one of claims 1-13.

Images