A face adversarial sample detection method based on spatial sensitivity

Abstract

Aiming at the security problem of face recognition model, the present invention discloses a face adversarial sample detection method based on spatial sensitivity. The method first obtains a face image dataset through video stream and frame sampling, then calculates the spatial domain sensitivity between the face image dataset and the face adversarial sample set generated based on the attack method as the sample feature, and trains a binary classifier composed of a fully connected FC layer and a batch normalization BN layer to distinguish adversarial samples from normal samples in face images. The method of the present invention can filter adversarial samples from face image data and strengthen the security of face recognition system.

Description

Technical Field

[0001] The present invention relates to the field of defense technology against adversarial samples, and in particular to security issues in face recognition systems. A face adversarial sample detection method based on spatial sensitivity is proposed. Background Art

[0002] Benefiting from the emergence of big data and the improvement of large-scale computing power, face detection and recognition technology based on deep learning has achieved good performance, far surpassing traditional algorithms, and is widely used in real-world scenarios such as financial verification, reconnaissance and security, and mobile phone login. However, its deep learning model has a problem of slightly adjusted input samples leading to output errors, which is called adversarial samples. The existence of adversarial samples will cause deep learning models to pose a huge security threat in real-world scenarios, becoming a key problem that face recognition models need to solve further.

[0003] There are two main forms of face adversarial sample attacks: physical and digital. Physical adversarial sample attacks come from the camera that collects face images obtaining physical adversarial sample factors, such as glasses and hats. Digital adversarial sample attacks come from the hijacking of data packets in data transmission and replacing them with adversarial samples. Today's adversarial sample defense methods are mainly divided into two categories: detection only and full defense. To a certain extent, both have achieved the effect of resisting adversarial attacks. However, in the field of security-sensitive face recognition applications, the full defense method requires a large number of face sample training and has poor feasibility. The detection-only defense method can obtain the difference features between adversarial samples and normal samples on a small amount of sample data through an unsupervised method, thereby obtaining a classifier for face adversarial samples with good performance. Summary of the invention

[0004] In order to reduce the impact of adversarial samples on the misjudgment of face recognition systems, the present invention provides a face adversarial sample detection method based on spatial sensitivity, which can accurately detect adversarial samples from real-time face data and solve the problem of challenging small perturbation face adversarial sample attacks. The technical solution of the present invention includes the following steps:

[0005] Step 1: Set the face detection area, obtain the real-time video stream through the camera, and obtain the face image dataset X through frame sampling;

[0006] Step 2: Normalize and preprocess the face image dataset X to obtain the original sample dataset X data , divide the training set and the test set at the same time, select the neural network model M for training and testing, and obtain the basic model M with good classification performance data and its prediction confidence m data ;

[0007] Step 3: Based on the iterative adversarial sample attack method, gradient optimization is used to generate a face adversarial sample set X with subtle changes on the feature values ​​of the original face image dataset X. adv ;

[0008] Step 4: For the face image dataset X, use the linear dimensionality reduction method to transform the face image data into the spatial domain to obtain the sample dataset X in the spatial transform domain WT tran ;

[0009] Step 5: Use sample data X tran Train and test the neural network model M to obtain the same data The domain transfer model M that predicts the same label wt ;

[0010] Step 6: Use face adversarial sample set X adv In the domain conversion model M wt We train again on the , and get the domain conversion model M after the adversarial sample attack awt and its prediction confidence m awt ;

[0011] Step 7: Calculate x for the input face image data i The spatial sensitivity value S(x i ), where 1≤i≤n, the formula is as follows:

[0012] S(x i )={m data (x i )-m awt (x i )}

[0013] Step 8: Build a binary classifier consisting of a fully connected FC layer and a batch normalization BN layer, and convert the spatial sensitivity value S(x i ) is used as the input of the deep neural network binary classifier, and the binary classification model D is obtained by training;

[0014] Step 9: Calculate the spatial sensitivity value of the newly input face image sample dataset, and determine whether the newly input face image sample is a normal sample or an adversarial sample based on the constructed binary classification model D. If the detector result label = 0, it means that the sample is an adversarial sample, otherwise label = 1, and the sample is an original sample. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Attached Figure 1 This is a flow chart of a face adversarial sample detection method based on spatial sensitivity of the present invention. DETAILED DESCRIPTION

[0016] The present invention implements a face adversarial sample detection method based on spatial sensitivity, which uses the difference in spatial sensitivity between adversarial samples and normal samples of face images as the identification feature, has a higher detection accuracy, can effectively resist threats under various scenarios, and enhance the security performance of face recognition applications. Figure 1 Taking the Resnet neural network as an example, the face adversarial sample detection method based on spatial sensitivity proposed in the present invention is described in detail.

[0017] Step 1: Set the face detection area, obtain the real-time video stream through the camera, intercept the video stream into a face image sequence according to the frame frequency, and obtain the real-time face image dataset X;

[0018] Step 2: The CIFAR-10 dataset is used to identify universal objects. It contains 60,000 color images divided into 10 categories. The CIFAR-10 image dataset is normalized and preprocessed to be divided into corresponding training sets and test sets for training and testing. The Resnet neural network model is selected and trained for 10 epochs to obtain a basic model M with an accuracy of 98.7%. data and its prediction confidence m data ;

[0019] Step 3: Use the iterative adversarial sample attack method BIM (Basic Iterative Method) to perform cropping on the image dataset by iteratively finding the perturbation of each image pixel point to ensure that each pixel of the adversarial sample is within the neighborhood of X with a step size of ∈, and generate a subtle adversarial sample set X adv ; The formula is as follows:

[0020]

[0021] Step 4: For the CIFAR-10 image dataset, use the wavelet transform method to transform the data in the spatial domain to obtain the sample dataset X in the spatial transform domain WT tran To flatten the high curve area that is sensitive to adversarial samples, the formula is as follows:

[0022] X train =w1X ll +w2X lh +w3X hl +w4X hh

[0023] X ll ,X lh ,X hl ,X hh They are the four sub-bands of wavelet transform, w1, w2, w3, w4 are the weights for balancing different sub-bands;

[0024] Step 5: Use sample data X tran Train and test on the Resnet neural network to obtain the same model as the base model M data The domain transfer model M that predicts the same label wt ;

[0025] Step 6: Use the generated adversarial sample set X adv In the domain conversion model M wt We train again on the , and get the domain conversion model M after the adversarial sample attack wt and its prediction confidence m awt ;

[0026] Step 7: Input face image data x i , calculate x i The spatial sensitivity value S(x i ), where 1≤i≤n, the formula is as follows:

[0027] S(x i )={m data (x i )-m awt (x i )}

[0028] For x i , in the basic model M data and domain transformation model M wt There are certain differences in the confidence of the predicted values, which is due to the inconsistency in the sensitivity of the adversarial samples and the original samples in the spatial domain.

[0029] Step 8: AUC is the standard for judging the performance of the binary classification model. Build a binary classifier consisting of a fully connected FC (Fully Connected) layer and a batch normalization BN (Batch Normalization) layer, and convert the spatial sensitivity value S (x i ) is used as the input of the deep neural network binary classifier, and the training obtains the AUC=97.35% binary classification model D.

[0030] Step 9: Calculate the spatial sensitivity value of the newly input face image sample dataset, and determine whether the newly input face image sample is a normal sample or an adversarial sample based on the constructed binary classification model D. If the detector result label = 0, it means that the sample is an adversarial sample, otherwise label = 1, and the sample is an original sample.

[0031] Through the above process, a face adversarial sample detection method based on spatial sensitivity can be realized. The specific flow chart is as follows: Figure 1As shown in the figure, it is divided into two stages: acquisition and detection. The main task of the acquisition stage is to obtain the original face samples. The main task of the detection stage is to detect the adversarial samples in the collected face images by using the inconsistent spatial sensitivity features between normal samples and adversarial samples in the spatial transformation domain. In the actual face recognition application process, unknown types of face adversarial samples can be well detected, so as to achieve the purpose of filtering out adversarial samples to protect the face recognition model.

[0032] For the Resnet model, the iterative BIM attack algorithm was used to conduct experiments on the CIFAR-10 dataset, and the AUC index was finally used as the standard for the effectiveness of the detection method. Table 1 shows the AUC index values ​​of the proposed adversarial sample detection method based on spatial sensitivity (SD, Sensitivity Domain) and the adversarial sample detection method based on local intrinsic dimension (LID, Local Intrinsic Dimensionality) and contrastive learning (MD, Mahalanobis Distance) on the CIFAR-10 and LFW (Labled Faces in the Wild) datasets. It can be seen from Table 1 that the detection method SD proposed in the present invention has the highest AUC value compared with the LID and MD detection methods, and can more effectively detect adversarial samples. The LFW dataset is a face image dataset from natural scenes in life, with a total of 5749 people. Taking 100 face image data in the LFW dataset as an example, the performance of the adversarial sample detection method (SD) based on spatial sensitivity is verified. It can be seen from Table 1 that the AUC value of the adversarial sample detection method (SD) proposed in the present invention reaches 96.9%, and it can still effectively detect face adversarial samples.

[0033] Table 1 AUC comparison of different adversarial sample detection methods on CIFAR-10 and LFW datasets

[0034]

Claims

1. A face adversarial sample detection method based on spatial sensitivity, characterized by The following steps are involved: Step 1: Set the face detection area, obtain the real-time video stream through the camera, and obtain the face image dataset X through frame sampling; Step 2: Normalize and preprocess the face image dataset X to obtain the original sample dataset X data , divide the training set and the test set at the same time, select the neural network model M for training and testing, and obtain the basic model M with good classification performance data and its prediction confidence m data ; Step 3: Based on the iterative adversarial sample attack method, gradient optimization is used to generate a face adversarial sample set X with subtle changes on the feature values ​​of the original face image dataset X. adv ; Step 4: For the face image dataset X, use the linear dimensionality reduction method to transform the face image data into the spatial domain to obtain the sample dataset X in the spatial transform domain WT tran ; Step 5: Use sample data X tran Train and test the neural network model M to obtain the same data The domain transfer model M that predicts the same label wt ; Step 6: Use face adversarial sample set X adv In the domain conversion model M wt Retrain on it to obtain the domain conversion model M after adversarial sample attack awt and its prediction confidence m awt ; Step 7: Input face image data x i , calculate x i The spatial sensitivity value S(x i ), where 1≤i≤n, the formula is as follows: S(x i )={m data (x i )-m awt (x i )} Step 8: Build a binary classifier consisting of a fully connected FC layer and a batch normalization BN layer, and convert the spatial sensitivity value S(x i ) is used as the input of the deep neural network binary classifier, and the binary classification model D is obtained by training; Step 9: Calculate the spatial sensitivity value of the newly input face image sample dataset, and determine whether the newly input face image sample is a normal sample or an adversarial sample based on the constructed binary classification model D; if the detector result label = 0, it means that the sample is an adversarial sample, otherwise label = 1, and the sample is an original sample.

Images