Confrontation sample generation method and system based on trigger condition

Abstract

The invention discloses an adversarial sample generation method and system based on a trigger condition, belongs to the field of artificial intelligence safety, and solves the problems that an adversarial sample generated by an existing adversarial sample generation method has adversarial resistance to a target model under any condition, and meanwhile, the adversarial sample may have adversarial resistance to other models due to mobility of the adversarial sample, so that the adversarial sample cannot be generated. Therefore, targeted robustness test and evaluation on the target model are not facilitated. According to the method, a data set is obtained, two disturbances are initialized for each sample in the data set, and the two disturbances are a trigger condition and a trigger disturbance respectively; and detecting each sample and the two disturbances based on the target model, if the requirements are met, obtaining the sum of the sample and the two disturbances, namely a conditional adversarial sample, and if the requirements are not met, updating the disturbances and then executing again. The method is used for generating the adversarial sample.

Description

technical field [0001] A method and system for generating adversarial samples based on triggering conditions are used for generating adversarial samples and belong to the field of artificial intelligence security. Background technique [0002] Existing research shows that deep learning models based on deep neural networks are vulnerable to adversarial example attacks. Adversarial sample attacks refer to the attackers making small modifications to the input samples that are invisible to the human eye, causing the deep learning model to respond incorrectly. The existence of adversarial samples on the one hand threatens the application of deep learning models in safety-related scenarios such as autonomous driving and smart medical care, and on the other hand promotes research on the interpretability and robustness of deep learning models. The adversarial samples generated by the existing adversarial sample generation methods are antagonistic to the target model under any condi...

AI Technical Summary

Problems solved by technology

[0006]Aiming at the problems of the above research, the purpose of the present invention is to provide a method and system for generating adversarial samples based on trigger conditions, so as to solve the problems generated by existing adversarial sample generation methods. The samples are adversarial to the target model under any conditions. At the same time, due to the migration of adversarial samples, they may also be adversarial to other models, which is not conducive to the targeted robustness testing and evaluation of the target model.

Images