Adversarial example defense method based on two-stage adversarial knowledge migration

Abstract

The invention belongs to the technical field of artificial intelligence safety, and discloses an adversarial example defense method based on two-stage adversarial knowledge migration. The method comprises the steps: firstly migrating adversarial knowledge from data to a large-scale complex DNN (deep neural network) through heterogeneous multi-source adversarial training, and completing the first-stage adversarial knowledge migration; then, a soft label of an adversarial sample is adopted, an adversarial extraction technology is provided, adversarial knowledge is migrated from a complex DNN toa simple DNN, and adversarial knowledge migration in the second stage is achieved. According to the two-stage adversarial knowledge migration method provided by the invention, the simple neural network on the edge equipment can obtain robustness close to that of a large complex network, and the problem of robustness of the simple network which cannot be solved by pure adversarial training is better solved. The adversarial extraction proposed by the invention has good algorithm convergence, can stably improve the performance and robustness of a simple network model and accelerate convergence, and well solves the problem of instability of the model performance and robustness in integrated adversarial training.

Description

technical field [0001] The invention belongs to the technical field of artificial intelligence security, and in particular relates to an adversarial example defense method based on two-stage adversarial knowledge transfer. Background technique [0002] At present, the closest existing technology: Deep Neural Networks (DNN, Deep Neural Networks) has been widely used in image recognition, natural language processing and other fields recently, but studies have shown that if some well-designed, imperceptible Perturbation of the deep neural network can cause misclassification. Such examples with malicious noise added are called adversarial examples. The emergence of adversarial examples limits the application of deep neural networks in security-sensitive fields, such as autonomous driving and facial payment. Researchers have done a lot of work on defense against adversarial examples, among which adversarial examples are used as training data, and confrontation training of DNN i...

AI Technical Summary

Problems solved by technology

At present, various model compression methods have been proposed, such as pruning, parameter quantization, and knowledge extraction, or direct confrontation training on simple DNNs on edge devices, but these existing technologies are not effective.

The reasons are: (1) Simple DNN is more difficult to train than large DNN to obtain high classification accuracy and robustness

(2) Existing methods such as model compression only focus on improving the classification accuracy of simple DNNs, but cannot improve the defense capabilities of simple DNNs against examples

Images