Artificial intelligence actor security attack methods and systems
By generating toxic samples in AI actors and training them with Gaussian noise, and using generative adversarial networks to simulate adversarial examples, the problem of easy detection of adversarial examples in existing technologies is solved, and the effectiveness of attack training and the robustness of the model are improved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- UESTC (SHENZHEN) ADVANCED RES INST
- Filing Date
- 2023-04-07
- Publication Date
- 2026-05-29
AI Technical Summary
The current technology's limitations on the intensity of adversarial example attack training are far from achieving the effect of making attacks undetectable, resulting in adversarial example attacks being easily detected by defensive measures and thus losing the effectiveness of attack training.
By obtaining the attack-specified output of the target model in the artificial intelligence actor, a normal sample set is determined, and the normal samples are slightly modified to generate toxic samples, which are mixed into the real sample set. The target model is then trained with Gaussian noise, and a generative adversarial network is used to generate triggers to simulate adversarial samples, thereby increasing the difficulty of attack training.
It effectively improves the attack training effect of artificial intelligence actors, making attack samples difficult to detect and enhancing the robustness and security of deep learning models.
Smart Images
Figure CN116527339B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of computer technology, specifically to methods and systems for security attacks on artificial intelligence actors. Background Technology
[0002] With the increasing prevalence of artificial intelligence technology, the security and reliability issues of deep learning models in AI actors are gradually being exposed, attracting widespread attention from academia and industry. For deep learning models, the most significant security concern is adversarial examples, which are input samples created by deliberately adding subtle perturbations to a dataset.
[0003] Adversarial attacks degrade the performance of deep learning models, causing them to output incorrect results with high confidence. Attackers often use adversarial attacks to manipulate AI agents deployed in various scenarios, threatening their secure operation. Therefore, training deep learning models with adversarial examples can help improve their robustness, enhance their security, and reduce erroneous outputs.
[0004] Currently, the limitation on the intensity of attack training only restricts the number of perturbations, which is far from achieving the effect of making the attack undetectable. As a result, the attack of adversarial examples can be easily detected by current defense methods, thus losing the effect of attack training. Summary of the Invention
[0005] To improve the training effect of AI actors in attack, this application provides a method and system for security attacks on AI actors.
[0006] The first aspect of this application provides a method for security attacks on artificial intelligence actors, specifically including:
[0007] Obtain the attack-specified output result of the target model in the artificial intelligence actor; determine the input normal sample set based on the attack-specified output result, where the attack-specified output result is the result of the sample to be attacked being input into the target model's specified output; modify any normal sample in the normal sample set to obtain a toxic sample, and add the toxic sample to the normal sample set to obtain a real sample set, where the toxic sample is a subtle interference of the sample to be attacked; obtain Gaussian noise, and perform attack training on the target model based on the real sample set and the Gaussian noise.
[0008] By adopting the above technical solution, after determining the sample to be attacked, in order to ensure that the AI agent outputs the corresponding attack-specified output result after inputting the sample, a set of normal samples—that is, samples unaffected by the attack—is determined based on this attack-specified output result. Next, a normal sample is arbitrarily selected from this set and slightly modified to obtain a toxic sample that differs from the normal sample. This toxic sample is then mixed into the normal sample set to obtain a real sample set, thus interfering with the sample to be attacked. Finally, Gaussian noise is added to the real sample set for attack training of the target model in the AI agent, resulting in better attack training performance for the AI agent.
[0009] Optionally, obtaining the attack-specified output result of the target model in the artificial intelligence actor, and determining the normal sample set based on the attack-specified output result, specifically includes:
[0010] Obtain the attack-specified output result of the target model in the artificial intelligence actor, and determine the input sample corresponding to the attack-specified output result;
[0011] Filter the set of normal samples corresponding to the specified output result of the attack from the input samples.
[0012] By adopting the above technical solution, after determining the specified output result of the attack, the input sample corresponding to the artificial intelligence agent is traced back based on the specified output result of the attack. Since there may be abnormal samples that are misjudged in the input sample, a set of normal samples is selected from the input sample. That is, the normal samples in the set of normal samples are input into the artificial intelligence agent to output the specified output result of the attack normally. This makes it convenient for the artificial intelligence agent to not be able to distinguish between the sample to be attacked and the normal sample, so as to achieve the goal of the artificial intelligence agent misjudging and outputting the specified output result of the attack when the sample to be attacked is input.
[0013] Optionally, modifying any one of the normal samples in the set of normal samples to obtain a toxic sample specifically includes:
[0014] A toxic sample is obtained by modifying any normal sample in the normal sample set according to a preset first formula, wherein the first formula is:
[0015]
[0016] In the formula, p represents a toxic sample, t represents a sample to be attacked, α represents a coefficient factor, b represents a normal sample, f() represents the objective function, x represents the parameters for minimizing the objective function, and argmin represents the variable value that minimizes the objective function.
[0017] By adopting the above technical solution, after determining the normal sample set, a normal sample is selected from it for modification. The parameters x for minimizing the objective function f(), the sample to be attacked t, and the normal sample b from the normal sample set are input into the first formula to obtain the variable value that minimizes the objective function, i.e., the toxic sample p. This makes the output result of inputting the toxic sample p into the artificial intelligence agent as consistent as possible with the output result of inputting the sample to be attacked, further interfering with the sample to be attacked.
[0018] Optionally, modifying any one of the normal samples in the set of normal samples to obtain a toxic sample specifically includes:
[0019] When the AI agent has undergone data preprocessing, any normal sample in the normal sample set is modified according to a preset second formula to obtain a toxic sample, wherein the second formula is:
[0020]
[0021] In the formula, Let f() represent the data preprocessing of the artificial intelligence agent, p represent toxic samples, t represent samples to be attacked, α represent coefficient factors, b represent normal samples, f() represent the objective function, x represent the parameters for minimizing the objective function, and argmin represent the variable values that minimize the objective function.
[0022] By adopting the above technical solution, after the normal sample set is determined, a normal sample is selected from it and modified to obtain a toxic sample. Since there is a sample data preprocessing step before the sample data is input into the artificial intelligence agent, this situation needs to be taken into account when determining the toxic sample. This mainly involves inputting the parameters for minimizing the objective function into the corresponding function of data preprocessing. The results are then input into the objective function f(), and finally the toxic samples are determined by the second formula, thus avoiding the impact of data preprocessing on the preparation of toxic samples.
[0023] Optionally, the step of obtaining Gaussian noise, and training the target model based on the real sample set and the Gaussian noise, specifically includes:
[0024] Gaussian noise is obtained, input into the generator, and constrained by a first loss function to obtain a trigger; the trigger is added to the real sample set to obtain a mixed sample.
[0025] The mixed sample and the real sample set are respectively input into the discriminator and constrained by the second loss function to obtain the corresponding output result. The discriminator is used to determine whether the input sample is a real sample.
[0026] Based on the output results and the generator, determine the target generator;
[0027] The Gaussian noise is input into the target generator to obtain the target trigger, and the discriminator cannot distinguish between the target trigger and the real sample set.
[0028] The target trigger and the set of real samples are input into the target model, and a third loss function is used to constrain the target model for attack training.
[0029] By employing the above technical solution, after determining the real sample set, Gaussian noise is input into the generator. The generator generates triggers, which represent subtle interferences corresponding to the real sample set. These triggers and the real sample set are then mixed to obtain a mixed sample. Next, the mixed sample and the real sample set are input into a discriminator, which determines whether each is a real sample and outputs the result. Based on the output, the generator is adjusted to obtain the target generator. This results in the target trigger generated by the target generator being indistinguishable from the real sample set by the discriminator, indicating that the target trigger is now infinitely close to the real samples in the real sample set. Finally, the target trigger and the real sample set are input into the target model in the AI agent. This allows for attack training of the target model by distinguishing between the target trigger and the real sample set, which is more difficult to differentiate, thereby improving the attack training effect of the AI agent.
[0030] Optionally, determining the target generator based on each of the output results and the generator specifically includes:
[0031] Based on each output result, determine whether the discriminator distinguishes between the trigger and the real sample set. If so, adjust the generator until the discriminator no longer distinguishes between the trigger and the real sample set, and determine the adjusted generator as the target generator.
[0032] If not, then the generator is determined as the target generator.
[0033] By adopting the above technical solution, after the output results corresponding to the real sample set and the mixed sample are determined, if the output results show that both the real sample set and the mixed sample are real samples, it means that the discriminator cannot distinguish between the real sample set and the trigger. In this case, the generator is determined as the target generator. If the output results show that the real sample set is real samples and the mixed sample is not real samples, it means that the discriminator can distinguish between the trigger and the real sample set in the mixed sample. The generator needs to be adjusted and trained until the discriminator can no longer distinguish between the trigger and the real sample set, so that the attack effect of the training samples input into the target model is better.
[0034] Optionally, the first loss function is:
[0035] The second loss function is: The third loss function is:
[0036] In the formula, L G L represents the first loss value. D L represents the second loss value. p E represents the third loss value. x This represents the expected value of x. Indicates a trigger, z represents Gaussian noise, and X represents a trigger. i Let Y represent the real sample set. i This represents the output result corresponding to the real sample set. This represents a mixed sample, where α and β are both coefficient factors, D represents the discriminator, and L... D Let L represent the second loss value, λ represent the hyperparameter, and L represent the second loss value. C The loss function of the classifier is represented. Indicates an indicator function.
[0037] By adopting the above technical solution, when Gaussian noise is input into the generator, and samples are input into the discriminator and the target model, the generator is constrained by the first loss function, the discriminator is constrained by the second loss function, and the target model is constrained by the third loss function, thereby achieving the purpose of model convergence and continuously optimizing the model.
[0038] A second aspect of this application provides a security attack system for artificial intelligence actors, specifically including:
[0039] The normal sample determination module (11) is used to obtain the attack-specified output result of the target model in the artificial intelligence actor, and determine the input normal sample set according to the attack-specified output result, wherein the attack-specified output result is the result of the sample to be attacked being input into the target model's specified output.
[0040] The real sample determination module (12) is used to modify any normal sample in the normal sample set to obtain a toxic sample, and add the toxic sample to the normal sample set to obtain a real sample set.
[0041] The adversarial attack training module (13) is used to acquire Gaussian noise and perform attack training on the target model based on the real sample set and the Gaussian noise.
[0042] By employing the above technical solution, the normal sample determination module determines the set of normal samples from the input samples based on the specified attack output. Then, the real sample determination module modifies any normal sample in the normal sample set to obtain a poisoned sample, which is then added to the normal sample set to obtain the real sample set. Finally, the adversarial attack training module acquires Gaussian noise and uses this noise, along with the real sample set, to train the target model in the AI agent, thereby increasing the difficulty of the adversarial samples and ensuring the effectiveness of the attack training.
[0043] In summary, this application includes at least one of the following beneficial technical effects:
[0044] After identifying the target sample, to ensure that the AI agent outputs the specified attack result upon inputting the target sample, a set of normal samples—samples unaffected by the attack—is determined, guided by this specified attack output. Next, a normal sample is arbitrarily selected from this set and slightly modified to obtain a toxic sample. This toxic sample is then mixed into the normal sample set to obtain a set of real samples, thus interfering with the target sample. Finally, Gaussian noise is added to the set of real samples, and the target model in the AI agent is trained for attack, resulting in better attack training performance. Attached Figure Description
[0045] Figure 1 This is a flowchart illustrating a method for security attacks on artificial intelligence actors provided in an embodiment of this application;
[0046] Figure 2 This is a schematic diagram illustrating the principle of a generative adversarial network provided in an embodiment of this application;
[0047] Figure 3 This is a flowchart illustrating another method for security attacks on artificial intelligence actors provided in this application embodiment;
[0048] Figure 4 This is a schematic diagram of a process for attack training a target model provided in an embodiment of this application;
[0049] Figure 5 This is a schematic diagram of the structure of an artificial intelligence actor security attack system provided in an embodiment of this application.
[0050] Figure labeling: 11. Normal sample determination module; 12. Real sample determination module; 13. Adversarial attack training module. Detailed Implementation
[0051] To enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments.
[0052] In the description of the embodiments in this application, words such as "illustrative," "for example," or "for example" are used to indicate examples, illustrations, or explanations. Any embodiment or design described as "illustrative," "for example," or "for example" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or designs. Rather, the use of words such as "illustrative," "for example," or "for example" is intended to present the relevant concepts in a specific manner.
[0053] In the description of the embodiments of this application, the term "and / or" is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, B existing alone, or A and B existing simultaneously. Furthermore, unless otherwise stated, the term "multiple" means two or more. For example, multiple systems refer to two or more systems, and multiple screen terminals refer to two or more screen terminals. In addition, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the indicated technical features. Thus, a feature defined with "first" or "second" may explicitly or implicitly include one or more of that feature. The terms "comprising," "including," "having," and their variations all mean "including but not limited to," unless otherwise specifically emphasized.
[0054] See Figure 1 This application discloses a flowchart illustrating a method for attacking artificial intelligence actors, which can also be implemented using a computer program or run on an artificial intelligence actor security attack device based on the von Neumann architecture. The computer program can be integrated into an application or run as a standalone utility application, specifically including:
[0055] S101: Obtain the attack-specified output result of the target model in the artificial intelligence actor, determine the set of normal input samples based on the attack-specified output result, and the attack-specified output result is the result of inputting the sample to be attacked into the target model's specified output.
[0056] In one feasible implementation, the attack-specified output result of the target model in the artificial intelligence actor is obtained, and the input sample corresponding to the attack-specified output result is determined.
[0057] Filter the set of normal samples that correspond to the specified output result of the attack from the input samples.
[0058] Specifically, an AI agent has four elements: perception of the external environment, an internal deep learning model, a self-driving mechanism, and interactive behavior. After perceiving the external environment, the internal algorithm model makes decisions, which are then executed through the driving mechanism, thereby achieving autonomy for the AI agent. There are many types of AI agents, such as self-driving robots, self-driving cars, and AI weapons. The target model uses a deep learning model, which can be a Convolutional Neural Network (CNN). In other embodiments, the deep learning model can also be a Recurrent Neural Network (RNN).
[0059] The sample to be attacked is the sample intended for attack. It is manually selected, and once determined, the specified output result corresponding to attacking this sample is also manually determined. The purpose is to ensure that when the sample to be attacked is input into the AI agent, it outputs the corresponding specified attack output result—that is, the false positive result the attacker wants the AI agent to output. From the historical input-output data of this AI agent, input samples corresponding to the specified attack output result are matched, and a set of normal samples, excluding the sample to be attacked, is selected from the input samples. Normal samples are those input into the AI agent that will not result in a false positive. In other embodiments, the input samples corresponding to the specified attack output result can all be normal samples, i.e., samples that have not been attacked.
[0060] S102: Modify any normal sample in the normal sample set to obtain a toxic sample, and add the toxic sample to the normal sample set to obtain the real sample set.
[0061] In another feasible implementation, a toxic sample is obtained by modifying any normal sample in the normal sample set according to a preset first formula, wherein the first formula is:
[0062]
[0063] In the formula, p represents a toxic sample, t represents a sample to be attacked, α represents a coefficient factor, b represents a normal sample, f() represents the objective function, x represents the parameters for minimizing the objective function, and argmin represents the variable value that minimizes the objective function. It should be noted that the objective function is actually the function corresponding to the target model.
[0064] Specifically, after the normal sample set is determined, an arbitrary normal sample is selected from it and modified using the first formula to obtain a toxic sample. The toxic sample is a subtle perturbation to the sample to be attacked. Further, the argmin function is represented as argmin f(x), which represents the variable value that minimizes the objective function f(). f(t) represents the output of the AI agent after inputting the sample to be attacked t, and f(x) represents the output of the AI agent after inputting x. Finally, this toxic sample is mixed into the normal sample set to obtain the real sample set, thus facilitating subsequent backdoor attacks on the target model using a generative adversarial network.
[0065] In another feasible implementation, when data preprocessing is performed on the AI agent, any normal sample in the normal sample set is modified according to a preset second formula to obtain a toxic sample, wherein the second formula is:
[0066]
[0067] In the formula, Let f() represent the data preprocessing of the artificial intelligence agent, p represent toxic samples, t represent samples to be attacked, α represent coefficient factors, b represent normal samples, f() represent the objective function, x represent the parameters for minimizing the objective function, and argmin represent the variable values that minimize the objective function.
[0068] Specifically, since sample data undergoes preprocessing before being input into the AI agent—for example, if the sample data is an image sample, the preprocessing mainly involves cropping and flipping—if the impact of data preprocessing is not considered when identifying toxic samples, the toxic samples may be affected after being input into the AI agent. Therefore, given that the AI agent undergoes data preprocessing, toxic samples are identified using the second formula mentioned above. The second formula is similar to the first formula, except that before x is input into the objective function f(), it undergoes preprocessing... The function performs preprocessing to prevent toxic samples from being affected by data preprocessing after being input into the AI agent.
[0069] It's important to note that, for example, if a toxic sample is mixed into the set of normal samples, and both toxic and normal samples are labeled with the same output A, then during the training phase of the AI agent, the AI agent will learn that the output of the toxic sample is A, and will establish a mapping from toxic samples to A. According to the first formula, when identifying toxic samples, the goal is to make the output corresponding to the target sample C as consistent as possible with the output corresponding to the toxic sample. Therefore, after inputting the target sample, the AI agent will output a misaligned A, meaning it will misjudge the target sample as A. A healthy AI agent, after inputting the target sample, will output B. This makes it difficult for the AI agent to distinguish between the target sample and normal samples. Therefore, the real sample set, composed of the normal sample set and the toxic sample set, can be understood as the attack on the target sample C; that is, the real sample set is the interference with the target sample.
[0070] In other embodiments, a preset sample change algorithm can be used to calculate the corresponding change for this normal sample. Based on this change, the normal sample can be modified to obtain a toxic sample. This modification is imperceptible to the human eye, thus making the change between the toxic sample and the normal sample undetectable to the naked eye. The sample change algorithm can employ the FGSM algorithm.
[0071] S103: Obtain Gaussian noise, and perform attack training on the target model based on the real sample set and Gaussian noise.
[0072] Specifically, Gaussian noise refers to a type of noise whose probability density function follows a Gaussian distribution (normal distribution). After determining the real sample set, Gaussian noise is obtained. One feasible way to obtain Gaussian noise is to generate it using the `randn()` function in MATLAB. After obtaining the Gaussian noise, it is then used to train an attack on the target model in the artificial intelligence agent based on the Gaussian noise and the real sample set. Details are as follows:
[0073] like Figure 2As shown, a Generative Adversarial Network (GAN) consists of a generator (model) and a discriminator (model). The generator is responsible for capturing the distribution of sample data, while the discriminator is typically a binary classifier that distinguishes between real and generated input data. The generator's goal is to prevent the discriminator from identifying the generated samples, while the discriminator's goal is to distinguish between generated and real samples. Through adversarial training of the generator and discriminator, a steady state is achieved, ultimately enabling the generator to generate a highly realistic sample. During training, one model (discriminator or generator) is fixed while the parameters of the other model are updated, alternating iteratively. It's important to note that both the generator and discriminator are neural network models.
[0074] In this embodiment, similarly, by using a generative adversarial network, the generated samples obtained by inputting Gaussian noise into the generator cannot be distinguished from the real sample set by the discriminator. This makes the generated samples extremely similar to the real sample set, making it difficult to distinguish them. At this time, using the generated samples and the real sample set to attack and train the target model in the artificial intelligence actor is effective.
[0075] See Figure 3 This application discloses a flowchart illustrating another method for attacking the security of artificial intelligence actors. This method can also be implemented using a computer program and can run on an AI actor security attack device based on the von Neumann architecture. The computer program can be integrated into an application or run as a standalone utility application, specifically including:
[0076] S201: Obtain the attack-specified output result of the target model in the artificial intelligence actor, determine the set of normal input samples based on the attack-specified output result, and the attack-specified output result is the result of inputting the sample to be attacked into the target model's specified output.
[0077] S202: Modify any normal sample in the normal sample set to obtain a toxic sample, and add the toxic sample to the normal sample set to obtain the real sample set.
[0078] For details, please refer to steps S101-S102, which will not be repeated here.
[0079] S203: Obtain Gaussian noise, input the Gaussian noise into the generator, and constrain it through the first loss function to obtain the trigger.
[0080] Specifically, such as Figure 4As shown, after the Gaussian noise is determined, it is input into the generator to train the generator. Each training iteration constrains the generator using a first loss function, thereby optimizing the generator, which ultimately outputs a trigger. The role of the loss function is well-known and will not be elaborated here. In this embodiment, the trigger can be understood as the generated sample corresponding to the Gaussian noise, and the first loss function is:
[0081]
[0082] In the formula, L G Identify the first loss value, E x This represents the expected value of x. Indicates a trigger, z represents Gaussian noise, and X represents a trigger. i Let Y represent the real sample set. i This represents the output result corresponding to the real sample set. This represents a mixed sample consisting of trigger and real sample sets, where α and β are both coefficient factors, and D represents the discriminator. This represents the indicator function. Further discussion of the meaning of each term in the first loss function, the first term: The meaning is to add the triggers generated by the generator to real samples so that the discriminator cannot distinguish them. The second item: This means outputting the labels of the mixed samples as Y. i This allows for targeted attacks. Third item: The meaning is to make the Euclidean norm of the trigger as close to 0 as possible, so that it cannot be distinguished by the naked eye.
[0083] S204: Add the trigger to the real sample set to obtain a mixed sample.
[0084] S205: Input the mixed sample and real sample sets into the discriminator respectively, and constrain it through the second loss function to obtain the corresponding output results.
[0085] Specifically, after the generator produces a trigger, it mixes the trigger into the real sample set to obtain a mixed sample, which is a real sample with the trigger. Then, the real sample set and the mixed sample are respectively input into the discriminator. The discriminator is used to determine whether the input sample is a real sample. After being input into the discriminator, the loss is constrained by the second loss function. Finally, the discriminator produces an output result, which includes the probability value that the input sample is a real sample. That is, the probability value is used to determine whether the input sample is a real sample.
[0086] The second loss function is:
[0087]
[0088] In the formula, L D Let's represent the second loss value, and further discuss the meaning of each term in the second loss function. The first term: E x logD((X i ,Y i The meaning of )) is to make the input real sample be identified as a real sample. The second item: The meaning is to try to make the mixed input samples as real samples as possible, even if the discriminator cannot distinguish between the triggers in the mixed samples and the set of real samples.
[0089] S206: Determine the target generator based on the output results and the generator.
[0090] In one feasible implementation, it is determined whether the discriminator distinguishes between triggers and the real sample set based on each output result. If so, the generator is adjusted until the discriminator no longer distinguishes between triggers and the real sample set, and the adjusted generator is determined as the target generator.
[0091] If not, then the generator will be determined as the target generator.
[0092] Specifically, after the discriminator's output is determined, if the probability value in the output corresponding to the mixed sample is close to the probability value in the output corresponding to the real sample set, it indicates that the discriminator classifies the input mixed sample as a real sample, meaning it cannot distinguish the trigger from the real sample set. This further indicates that the trigger generated by the generator is very close to the real sample and difficult to distinguish, so the generator is designated as the target generator. Conversely, if the discriminator can distinguish the trigger from the real sample, the generator's model parameters are adjusted, and steps S204-S206 are repeated until the discriminator can no longer distinguish the trigger from the real sample set. The adjusted generator is then designated as the target generator. In other embodiments, if the difference between the probability value in the mixed sample output and the probability value in the real sample set output is less than a difference threshold, it indicates that the two probability values are close.
[0093] S207: Input Gaussian noise into the target generator to obtain the target trigger.
[0094] S208: Input the target trigger and the set of real samples into the target model, and use a third loss function to constrain it in order to train the target model for attack.
[0095] Specifically, after the target generator is determined, Gaussian noise is input into it to obtain a target trigger. This target trigger cannot be distinguished from the real sample set by the discriminator, indicating that it is quite similar to the real sample set and difficult to distinguish. Therefore, this target trigger and the real sample set are used as inputs to the target model for attack training, resulting in better attack performance. If the target trigger and the real sample set were easily distinguishable, the attack performance on the target model would be poor, failing to significantly improve the robustness of the target model. During the attack training process, a third loss function is used to constrain the loss of the target model, thereby continuously optimizing it. This is existing technology and will not be elaborated further. It should be noted that the third loss function is expressed as:
[0096]
[0097] In the formula, L D Let L represent the second loss value, λ represent the hyperparameter, and L represent the second loss value. C This represents the loss function of the classifier. To further clarify, X represents the input sample of the target model, and Y represents the output result of the target model.
[0098] The implementation principle of the AI agent security attack method in this application is as follows: After determining the sample to be attacked, in order to ensure that the AI agent outputs the corresponding attack-specified output result after the sample is input, a set of normal samples that also output the attack-specified output result after input is determined, i.e., samples that have not been interfered with by the attack. Next, the sample to be attacked is slightly modified to obtain a toxic sample that differs from the normal samples. This toxic sample is then mixed into the set of normal samples to obtain a set of real samples, thus interfering with the sample to be attacked. Finally, Gaussian noise is added to the set of real samples for interference, and the target model in the AI agent is trained for attack, resulting in a better attack training effect for the AI agent.
[0099] The following are system embodiments of this application, which can be used to execute the method embodiments of this application. For details not disclosed in the system embodiments of this application, please refer to the method embodiments of this application.
[0100] Please see Figure 5 This is a schematic diagram of the structure of the AI actor security attack system provided in this application embodiment. This AI actor security attack system can be implemented as all or part of a system through software, hardware, or a combination of both. System 1 includes a normal sample determination module 11, a real sample determination module 12, and an adversarial attack training module 13.
[0101] The normal sample determination module 11 is used to obtain the attack-specified output result of the target model in the artificial intelligence actor, and determine the input normal sample set based on the attack-specified output result. The attack-specified output result is the result of inputting the sample to be attacked into the target model and specifying the output.
[0102] The real sample determination module 12 is used to modify any normal sample in the normal sample set to obtain a toxic sample, and add the toxic sample to the normal sample set to obtain the real sample set;
[0103] Adversarial attack training module 13 is used to acquire Gaussian noise and perform attack training on the target model based on the real sample set and Gaussian noise.
[0104] Optional, the normal sample determination module 11 is specifically used for:
[0105] Obtain the specified attack output of the target model in the artificial intelligence actor, and determine the input sample corresponding to the specified attack output;
[0106] Filter the set of normal samples that correspond to the specified output result of the attack from the input samples.
[0107] Optional, the real sample determination module 12 is specifically used for:
[0108] A toxic sample is obtained by modifying any normal sample in the normal sample set according to a preset first formula, whereby the first formula is:
[0109]
[0110] In the formula, p represents a toxic sample, t represents a sample to be attacked, α represents a coefficient factor, b represents a normal sample, f() represents the objective function, x represents the parameters for minimizing the objective function, and argmin represents the variable value that minimizes the objective function.
[0111] Optionally, the real sample determination module 12 is also used for:
[0112] In the case of data preprocessing by the AI actor, a toxic sample is obtained by modifying any normal sample in the normal sample set according to a preset second formula, whereby the second formula is:
[0113]
[0114] In the formula, Let f() represent the data preprocessing of the artificial intelligence agent, p represent toxic samples, t represent samples to be attacked, α represent coefficient factors, b represent normal samples, f() represent the objective function, x represent the parameters for minimizing the objective function, and argmin represent the variable values that minimize the objective function.
[0115] Optional, adversarial attack training module 13, specifically used for:
[0116] Gaussian noise is obtained, input into the generator, and constrained by the first loss function to obtain a trigger; the trigger is added to the real sample set to obtain a mixed sample.
[0117] The mixed sample and real sample sets are respectively input into the discriminator and constrained by the second loss function to obtain the corresponding output results. The discriminator is used to determine whether the input sample is a real sample.
[0118] Based on the output results and the generator, determine the target generator;
[0119] When Gaussian noise is input into the target generator, a target trigger is obtained. The discriminator cannot distinguish between the target trigger and the real sample set.
[0120] The target trigger and the set of real samples are input into the target model, and a third loss function is used to constrain the target model for attack training.
[0121] Optionally, the adversarial attack training module 13 is also used for:
[0122] Based on each output result, determine whether the discriminator distinguishes between triggers and the real sample set. If so, adjust the generator until the discriminator no longer distinguishes between triggers and the real sample set, and then determine the adjusted generator as the target generator.
[0123] If not, then the generator will be determined as the target generator.
[0124] It should be noted that the AI actor security attack system provided in the above embodiments is only illustrated by the division of the above functional modules when executing the AI actor security attack method. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the AI actor security attack system and the AI actor security attack method embodiment provided in the above embodiments belong to the same concept, and the implementation process is detailed in the method embodiment, which will not be repeated here.
[0125] This application also discloses a computer-readable storage medium storing a computer program, wherein when the computer program is executed by a processor, it employs an artificial intelligence actor security attack method as described in the above embodiments.
[0126] The computer program can be stored in a computer-readable medium. The computer program includes computer program code, which can be in the form of source code, object code, executable file, or certain middleware. The computer-readable medium includes any entity or device capable of carrying computer program code, recording media, USB flash drive, portable hard drive, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the computer-readable medium includes, but is not limited to, the above-mentioned components.
[0127] The above-described method for security attacks on artificial intelligence actors is stored in the computer-readable storage medium and loaded and executed on the processor to facilitate the storage and application of the method.
[0128] This application also discloses an electronic device in which a computer program is stored in a computer-readable storage medium. When the computer program is loaded and executed by a processor, the above-mentioned method for security attack on artificial intelligence actors is employed.
[0129] The electronic device can be a desktop computer, a laptop computer, or a cloud server, and the electronic device includes, but is not limited to, a processor and a memory. For example, the electronic device may also include input / output devices, network access devices, and buses.
[0130] The processor can be a central processing unit (CPU). Of course, depending on the actual use, it can also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), off-the-shelf programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor, etc., and this application does not limit it in this regard.
[0131] The memory can be an internal storage unit of an electronic device, such as a hard disk or RAM, or an external storage device, such as a plug-in hard disk, smart memory card (SMC), secure digital card (SD), or flash memory card (FC) equipped on the electronic device. Furthermore, the memory can be a combination of an internal storage unit and an external storage device of the electronic device. The memory is used to store computer programs and other programs and data required by the electronic device. The memory can also be used to temporarily store data that has been output or will be output. This application does not limit this.
[0132] In this electronic device, the artificial intelligence actor security attack method of the above embodiment is stored in the memory of the electronic device and loaded and executed on the processor of the electronic device for convenient use.
[0133] The foregoing description is merely an exemplary embodiment of this disclosure and should not be construed as limiting the scope of this disclosure. Any equivalent changes and modifications made in accordance with the teachings of this disclosure shall still fall within the scope of this disclosure. Other embodiments of this disclosure will be readily apparent to those skilled in the art upon consideration of the specification and practice of the disclosure herein. This application is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not described herein. The specification and embodiments are to be considered exemplary only, and the scope and spirit of this disclosure are defined by the claims.
Claims
1. A method for security attacks on artificial intelligence actors, characterized in that, The method includes: Obtain the attack-specified output result of the target model in the artificial intelligence actor; determine the set of normal input samples based on the attack-specified output result; the attack-specified output result is the result of inputting the sample to be attacked into the target model; the sample to be attacked is the sample to be attacked; wherein, the input sample corresponding to the attack-specified output result is matched from the historical input-output data of the artificial intelligence actor; and the set of normal samples other than the sample to be attacked is filtered out from the input samples. Modifying any normal sample in the set of normal samples to obtain a toxic sample includes: modifying any normal sample in the set of normal samples according to a preset first formula to obtain a toxic sample, wherein the first formula is: ； In the formula, p represents the toxic sample, and t represents the sample to be attacked. Let denot _{r}_ represent the coefficient factor, _{b}_ represent the normal sample, _{f()}_ represent the objective function, _{x}_ represent the parameters to be minimized in the objective function, and _{argmin}_ represent the parameters to be minimized. x This represents the value of the variable that minimizes the objective function. The toxic sample is then added to the normal sample set to obtain the real sample set, where the toxic sample is a subtle interference with the sample to be attacked. Gaussian noise is obtained, and the target model is trained for attack based on the real sample set and the Gaussian noise.
2. The method for security attacks on artificial intelligence actors according to claim 1, characterized in that, The step of obtaining the specified attack output of the target model in the artificial intelligence actor, and determining the normal sample set based on the specified attack output, specifically includes: Obtain the attack-specified output result of the target model in the artificial intelligence actor, and determine the input sample corresponding to the attack-specified output result; Filter the set of normal samples corresponding to the specified output result of the attack from the input samples.
3. The method for security attacks on artificial intelligence actors according to claim 1, characterized in that, The step of modifying any normal sample in the set of normal samples to obtain a toxic sample specifically includes: When the AI agent has undergone data preprocessing, any normal sample in the normal sample set is modified according to a preset second formula to obtain a toxic sample, wherein the second formula is: ； In the formula, This represents the data preprocessing of the AI agent, where p represents a toxic sample and t represents a sample to be attacked. Let denot _{r}_ represent the coefficient factor, _{b}_ represent the normal sample, _{f()}_ represent the objective function, _{x}_ represent the parameters to be minimized in the objective function, and _{argmin}_ represent the parameters to be minimized. x This represents the value of the variable that minimizes the objective function.
4. The method for security attacks on artificial intelligence actors according to claim 1, characterized in that, The acquisition of Gaussian noise, based on the real sample set and the Gaussian noise, and the attack training of the target model, specifically includes: Gaussian noise is obtained, input into the generator, and constrained by a first loss function to obtain a trigger; The trigger is added to the real sample set to obtain a mixed sample; The mixed sample and the real sample set are respectively input into the discriminator and constrained by the second loss function to obtain the corresponding output result. The discriminator is used to determine whether the input sample is a real sample. Based on the output results and the generator, determine the target generator; The Gaussian noise is input into the target generator to obtain the target trigger, and the discriminator cannot distinguish between the target trigger and the real sample set. The target trigger and the set of real samples are input into the target model, and a third loss function is used to constrain the target model for attack training.
5. The method for security attacks on artificial intelligence actors according to claim 4, characterized in that, The step of determining the target generator based on the output results and the generator specifically includes: Based on each output result, determine whether the discriminator distinguishes between the trigger and the real sample set. If so, adjust the generator until the discriminator no longer distinguishes between the trigger and the real sample set, and determine the adjusted generator as the target generator. If not, then the generator is determined as the target generator.
6. A security attack system for artificial intelligence actors, used to implement the security attack method for artificial intelligence actors according to any one of claims 1 to 5, characterized in that, include: The normal sample determination module (11) is used to obtain the attack-specified output result of the target model in the artificial intelligence actor, and determine the input normal sample set according to the attack-specified output result, wherein the attack-specified output result is the result of the sample to be attacked being input into the target model's specified output. The real sample determination module (12) is used to modify any normal sample in the normal sample set to obtain a toxic sample, and add the toxic sample to the normal sample set to obtain a real sample set. The adversarial attack training module (13) is used to acquire Gaussian noise and perform attack training on the target model based on the real sample set and the Gaussian noise.
7. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is loaded and executed by the processor, it employs the method described in any one of claims 1-5.
8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and capable of running on the processor, characterized in that, When the processor loads and executes the computer program, it employs the method described in any one of claims 1-5.