Adversarial sample detection method based on deep learning model neural pathway activation features

Abstract

The invention discloses an adversarial sample detection method based on neural pathway activation features of a deep learning model. The method comprises the following steps: (1) acquiring an image data set; (2) constructing a detection model based on a deep learning network and performing training; (3) representing the maximum k neurons of a given input sample in the ith layer by Top-k (x, i), and defining a Top-k neural pathway for a set of the k neurons with the maximum activation value in each layer; (4) randomly selecting k samples in the test set of each category of the image data set, inputting the k samples into the trained detection model, and sequentially connecting Top-k neurons of each layer as a neural pathway of the sample in the model; (5) counting neural pathways of a large number of benign samples as comparison pathways of the neural pathways of the adversarial samples; and (6) inputting a to-be-tested sample, and if the generated neural pathway does not meet the benign sample pathway range, determining that the to-be-tested sample is an attack sample. According to the invention, the purpose of confrontation sample detection and rejection is achieved, and the detection rate is high.

Description

technical field [0001] The invention belongs to the field of artificial intelligence security, and in particular relates to an adversarial sample detection method based on neural pathway activation characteristics of a deep learning model. Background technique [0002] Deep neural network (DNN) is a machine learning technology, which is the key to the rapid development of artificial intelligence technology. With its powerful feature extraction ability, deep neural network has been widely used. In recent years, DNNs have achieved great success in various generative and discriminative learning tasks in fields including image processing, speech recognition, natural language processing, drug discovery, and biogenetic science. [0003] We have seen widespread applications of DNNs in many safety-critical scenarios. However, numerous studies have shown that state-of-the-art deep learning systems have various vulnerabilities that can lead to severe consequences when applied to prac...

AI Technical Summary

Problems solved by technology

[0006] For the detection of adversarial samples, at present, the size of the neuron activation value and the neuron coverage rate under a fixed neuron activation threshold have been detected, and through quantitative analysis, the detection of adversarial samples and the protection of the deep learning model have been achieved. purpose, but no further studies have been conducted on the transmission of neuron activations between model layers

Images