Face adversarial attack method based on generative adversarial network

Abstract

The application discloses a face adversarial attack method based on a generative adversarial network, and belongs to the field of adversarial attack. The method is characterized in that a normalized attention generator and an image reconstruction generator, as well as a true-false verification discriminator and an identity information discriminator, are designed, so that the adversarial sample can achieve a high attack success rate while having a good visual quality. Meanwhile, a traditional attack method is added as a fusion attack module for adversarial training, so as to strengthen the robustness of the discriminator D1 in the model, thereby promoting the stable convergence of the generator and obtaining an adversarial sample with a more natural visual quality. Different types of face recognition models are integrated as the identity discriminator, and the adversarial sample generated by the generator is subjected to size scaling, Gaussian blur and other transformations to enhance the sample diversity, promote the model to learn the common areas focused by various face recognition models, so that the adversarial sample has good transferability, thereby improving the attack performance.

Description

TECHNICAL FIELD

[0001] The present application belongs to the field of adversarial attacks, and more particularly, to a face adversarial attack method based on a generative adversarial network. BACKGROUND

[0002] With the development of computer technology, deep learning models are gradually applied to various aspects of people's life and social production, such as face recognition, natural language processing, and financial risk control. At present, face recognition models are used as identity verification tools and can also be used to protect important network resources and private data. However, the emergence of adversarial attacks allows some malicious attackers to improperly access face recognition models to seek improper benefits. Current deep neural networks, including some face recognition systems, have been proven to be vulnerable to digital and physical adversarial samples. Adversarial samples are generated by adding subtle perturbations to the original input samples, which look no different to the human visual system from the original input samples. Therefore, the imperceptible adversarial samples have the ability to make the most advanced face recognition systems make wrong judgments.

[0003] Adversarial attacks on deep learning models have many classification angles. For example, according to the scene of implementing the adversarial attack, it can be classified as white-box attack or black-box attack, and black-box attack is usually more challenging than white-box attack. According to the application scene of the attack, adversarial attacks can also be classified as physical attacks or digital attacks. Finally, according to whether the attack has a specific target, adversarial attacks can be classified as targeted attacks or non-targeted attacks. Targeted attacks expect the target model to misclassify adversarial samples as a certain specified class, and the attack fails if the target model classifies them as other classes. Non-targeted attacks do not specify the target class, and the attack is successful as long as the target model classifies the adversarial samples incorrectly. This is usually referred to as impersonation attack or evasion attack in the field of face adversarial attack. Evasion attack aims to make the input face image be recognized as any other individual in the face database, while impersonation attack aims to make the input face image be recognized as a specific attack target in the database.

[0004] At present, there are a large number of research results on the adversarial attack method designed for the face recognition model, and the research focus is concentrated on the black box attack direction which has more practical application significance. The current adversarial sample generation method for the face recognition task has some limitations, and the gradient-based attack method is difficult to achieve the ideal balance between the attack strength of the adversarial sample and the visual perception change. The adversarial patch attack method generates an adversarial disturbance pattern that is obvious and easy to attract attention, and is also easy to be identified by the current adversarial sample detection method, so the application scene is limited. And in view of the access frequency limit of most face recognition systems, the query-based attack method is difficult to apply to the real scene. The existing attack method based on the generative adversarial network also has some room for improvement, such as the attack success rate of the adversarial attack in the black box scene still has room for improvement, and the adversarial sample based on the makeup transfer has a relatively obvious abnormal pattern.

[0005] Since the attack and defense are closely related and promote each other, the research on the face recognition adversarial attack method can also enable researchers to explore the potential vulnerabilities of the face recognition model, so as to improve the face recognition model and make it have the ability to resist the adversarial attack, which has a positive significance for the improvement of the security and robustness of the face recognition model. SUMMARY

[0006] In view of the above defects or improvement needs of the prior art, the present application provides a face adversarial attack method based on a generative adversarial network, thereby solving the problem that the current face recognition adversarial attack method has weak transferability, poor attack performance and unsatisfactory visual quality, thereby resulting in poor attack performance.

[0007] To achieve the above purpose, according to the first aspect of the present application, a face adversarial attack method based on a generative adversarial network is provided, comprising:

[0008] Training stage:

[0009] S1, input the latent code z of the target face image y and the original face image x to the adversarial sample generator G1 together to synthesize the first adversarial sample G1(x,z); input G1(x,z) to the image reconstruction generator G2 to obtain the reconstructed original face image (G2(G1(x,z)); input y and x to the attacker module to generate the second adversarial sample Att(x,y);

[0010] S2, input x, G1(x, z) and Att(x, y) into the true and false verification discriminator D1 respectively, obtain the difference d1 between the true and false judgment results of D1 on G1(x, z) and Att(x, y), the difference d2 between the true and false judgment results of x and G1(x, z); input G1(x, z) and y into the identity information discriminator D2 together, obtain the cosine distance d3 between the feature vector of G1(x, z) and the feature vector of y;

[0011] S3, training G1, G2, D1 and D2 aiming at minimizing the difference between G2(G1(x, z)) and x and d1, d2 and d3;

[0012] Application stage:

[0013] input the face image to be attacked and z into the trained G1 to obtain the corresponding adversarial sample g; input g and y into the face recognition model, if the cosine distance between the feature vector of g calculated by the face recognition model and the feature vector of y is greater than a threshold, the attack fails, otherwise the attack succeeds.

[0014] According to the second aspect of the present application, a face adversarial attack method system based on a generative adversarial network is provided, characterized in that it comprises a computer readable storage medium and a processor.

[0015] The computer readable storage medium is used to store executable instructions.

[0016] The processor is used to read the executable instructions stored in the computer readable storage medium and execute the method according to the first aspect.

[0017] According to the third aspect of the present application, a computer readable storage medium is provided, characterized in that the computer readable storage medium stores computer instructions, and the computer instructions are used to make the processor execute the method according to the first aspect.

[0018] Overall, compared with the prior art, the above technical solutions conceived by the present application can achieve the following beneficial effects:

[0019] 1. The method provided by the present application can achieve a higher attack success rate of the adversarial sample while having a better visual quality by designing a normalized attention generator and an image reconstruction generator, as well as a true and false verification discriminator and an identity information discriminator; at the same time, a traditional attack method is added as a fusion attack module for adversarial training, which strengthens the robustness of the discriminator D1 in the model, thereby promoting the stable convergence of the generator and obtaining an adversarial sample with more natural visual quality

[0020] 2、The method provided by the application provides a transferability enhancement module (namely, an identity discriminator D2) based on the idea of integrated learning and data enhancement, the module integrates different types of face recognition models, and performs size scaling, Gaussian blur and other transformations on the adversarial samples generated by the generator to enhance sample diversity, promote the model to learn the common areas focused on by various face recognition models, so that the adversarial samples have good transferability, thereby improving the attack performance.

[0021] To sum up, the method provided by the application can be used for both black-box face adversarial attack tasks and white-box face attack tasks, has good attack performance and good transferability, and can ensure a certain attack success rate while maintaining good visual quality. BRIEF DESCRIPTION OF DRAWINGS

[0022] Figure 1 A face adversarial attack method based on a generative adversarial network provided by an embodiment of the application is shown in the flowchart;

[0023] Figure 2 A structure diagram of an adversarial sample generator provided by an embodiment of the application is shown in the diagram;

[0024] Figure 3 A structure diagram of a true-false verification discriminator provided by an embodiment of the application is shown in the diagram. DETAILED DESCRIPTION

[0025] In order to make the purpose, technical scheme and advantages of the application clearer, the application is further described in detail below with reference to the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the application and not to limit the application. In addition, the technical features involved in each embodiment of the application described below can be combined with each other as long as they do not conflict with each other.

[0026] An embodiment of the application provides a face adversarial attack method based on a generative adversarial network, comprising:

[0027] Training phase:

[0028] S1, input the latent encoding z of the target face image y and the original face image x to the adversarial sample generator G1 together to synthesize the first adversarial sample G1(x, z); input G1(x, z) to the image reconstruction generator G2 to obtain the reconstructed original face image G2(G1(x, z)); input y and x to the attacker module to generate the second adversarial sample Att(x, y);

[0029] S2, input x, G1(x, z) and Att(x, y) into the true-false verification discriminator D1 respectively, obtain the difference d1 between the true-false judgment results of D1 on G1(x, z) and Att(x, y), the difference d2 between the true-false judgment results of x and G1(x, z); input G1(x, z) and y into the identity information discriminator D2 together, obtain the cosine distance d3 between the feature vectors of G1(x, z) and y;

[0030] S3, jointly train G1, G2, D1 and D2 by minimizing the difference between G2(G1(x, z)) and x and d1, d2 and d3;

[0031] Application stage:

[0032] input the face image to be attacked and z into the trained G1 to obtain the corresponding adversarial sample g; input g and y into the face recognition model together, if the cosine distance between the feature vectors of g and y calculated by the face recognition model is greater than a threshold, the attack fails, otherwise the attack succeeds.

[0033] Preferably, the total loss function of the training stage is

[0034] wherein, L att =E x [D1(G1(x))]-E x,y [D1(Att(x,y))], F(y) and F(G1(x, z)) are the feature vectors of y and G1(x, z) respectively.

[0035] Preferably, the adversarial sample generator G1 comprises an encoder and a decoder, and the encoder comprises a normalized attention module.

[0036] Preferably, the true-false verification discriminator D1 is a Markov discriminator based on a convolutional layer structure, and the identity information discriminator D2 is an integrated plurality of pre-trained face recognition models.

[0037] Preferably, the attacker module generates the second adversarial sample based on a gradient attack method.

[0038] Specifically, the face recognition adversarial attack method (Adversarial Attacked Face GAN, AAF-GAN) based on the generated adversarial network provided by the embodiment of the present application comprises: Figure 1 as shown, including: an adversarial sample pre-generation stage and an identity information and true-false double identification stage;

[0039] The adversarial sample pre-generation stage comprises:

[0040] A pre-trained VAE encoder is used to explore the latent space of the target face image, map the target face image into a low-dimensional latent space, and obtain the corresponding latent code, which retains the personalized face identity information. Then the feature map of the input image and the latent code of the target face image are combined to enable the adversarial sample generator to generate a face image with a specific target identity.

[0041] The adversarial sample generator G1 combined with the normalized attention takes the feature map of the source image and the latent feature of the target image as input to synthesize the adversarial sample.

[0042] The image reconstruction generator G2 is responsible for ensuring the consistency of the content information and most of the feature representation of the generated adversarial sample with the input image during training.

[0043] An attacker module based on gradient optimization, the attack method selected by the attack module is an iterative attack method PGD, which uses the data distribution of the adversarial sample generated by the GAN learning attack module, and then performs adversarial training on the learned distribution to obtain a more robust discriminator D1.

[0044] The identity information and authenticity dual discrimination stage includes:

[0045] The authenticity verification discriminator D1 uses a Markov discriminator structure based on a convolutional layer structure, and is responsible for determining whether the image generated by the generator G1 belongs to the real distribution of images, i.e., whether it is a real image.

[0046] The identity information discriminator D2 uses multiple pre-trained face recognition models for integration, and is responsible for minimizing the cosine distance between the adversarial sample and the feature vector of the target face image.

[0047] It can be understood that the face recognition model used in the application stage can be any one of the face recognition models in the identity information discriminator D2 (such as Figure 1 It can also be other types of face recognition models, such as VGG-Face, OpenFace, etc.

[0048] Preferably, the structure of the adversarial sample generator G1 combined with the normalized attention is divided into an encoder and a decoder. After the encoder accepts the image input, it uses ReflectionPad2d, an image enhancement method, to symmetrically expand the image edges to increase the resolution of the image, Figure 2 The CIL contains convolution operations, IN regularization, and Leaky ReLU activation functions. Between the encoder and the decoder are residual blocks, a total of 9 repeated modules, which are used to restore and enhance the data. Figure 2The CTIR of the middle decoder includes deconvolution, IN standardization, ReLU activation function, is responsible for restoring the size of the image, uses the ReflectionPad2d again to increase the image resolution, and finally restores the image to the original size through the convolution operation, effectively solving the problem of object edge information.

[0049] That is, the encoder of G1 includes a normalization attention module; the normalization attention module includes a channel attention submodule and a spatial attention submodule.

[0050] Further, the generated adversarial samples of the generator G1 are subjected to size scaling, Gaussian blur and other transformations to enhance sample diversity and promote the model to learn the common areas of attention of various face recognition models, so that the adversarial samples have good transferability, thereby improving the attack performance.

[0051] Preferably, in order to make the generator focus on important features of the face image and suppress unimportant features such as background, the present application adds a normalization-based attention module (NAM) in the adversarial sample generator G1 as a face key area feature enhancement mechanism. NAM enables the model to focus on important feature information and suppress features with low reference value such as background, which helps to guide the generator to generate higher-quality adversarial samples.

[0052] Preferably, the real-fake verification discriminator is used to distinguish the distribution of the fake images generated by the generator and the real images. It uses the structure of a Markov discriminator based on a convolutional layer structure. The Markov discriminator is a discriminant model, which is different from most classification models based on convolutional neural networks. The last layer of most models is a fully connected layer responsible for output judgment. It is completely composed of convolutional layers, and finally outputs an n*n matrix, and the output is generated by taking the mean of the output matrix. This structure not only enables the discriminator to focus on image details to improve the quality of generated images, but also improves the stability of network training.

[0053] Preferably, in the present application, the adversarial samples based on the gradient attack method are used in the robust training process, which provides more information about the data distribution of the adversarial samples to the generator, and a more robust discriminator can be obtained, which can be better generalized to unseen data. Another advantage of the robust discriminator is that it helps to stabilize and accelerate the entire training process. Specifically, the L att = E x [D1(x)]-E x,y [D1(Att(x,y))] calculates the loss of adversarial attack training.

[0054] Preferably, in the present application, a weighted average model integration strategy is adopted to integrate multiple face recognition models. Due to the overfitting of adversarial samples to a specific model architecture, adversarial samples often show poor transferability between different face recognition models. Since these models have different training data, loss functions, model architectures, and training strategies, the focus of the region during face recognition is different. Therefore, when the integrated model is used as an identity information discriminator, the adversarial samples that can meet the requirements of a successful attack have stronger transferability and can achieve good attack performance on other recognition models that have not participated in the training.

[0055] As shown in Figure 1 AAF-GAN is composed of an adversarial sample generator G1 and an image reconstruction generator G2, a true-false verification discriminator D1 and an identity information discriminator D2, an attacker module and an encoder of a pre-trained VAE. During training, the generator G1 generates false images and provides them to the true-false verification discriminator D1 and the identity information discriminator D2; at the same time, the original images sampled from the training data and their adversarial samples generated by the attacker module are provided to the true-false verification discriminator D1. The identity information discriminator D2 is integrated by multiple pre-trained face recognition models. Given a target face image, the encoder is used to learn the latent features of the target face image, the generator G1 takes the source image feature map and the latent features of the target image as input to synthesize adversarial samples, and the generator G2 is responsible for ensuring that the content information and most of the feature representation of the generated adversarial samples are consistent with the input image (source image). The true-false verification discriminator D1 is responsible for determining whether the image synthesized by the generator G1 belongs to the real distribution of images. In order to make the generated image indistinguishable from the real image, the AAF-GAN adopts an adversarial loss to train the generator and the discriminator.

[0056] The method shown in Figure 1 includes two stages, namely the pre-generation stage of adversarial samples and the dual identification stage of identity information and true-false, which will be described in detail below.

[0057] (A) The pre-generation stage of adversarial samples includes:

[0058] In order to obtain an adversarial sample that can be identified as the identity of the target face and avoid random sampling of latent codes z, the present application uses an encoder to map the target face image into a low-dimensional latent code z, which retains the personalized identity information of the target face. Inject z into the original face image feature map to prompt the adversarial sample generator to generate a face image with a specific target identity. The loss function of the encoder during the training process is as shown in the formula .

[0059] To obtain a latent code close to the prior distribution and capable of describing some features of the input face image, the distribution of the latent code extracted by the encoder needs to fit a random Gaussian distribution. In equation (1), the KL divergence is used to measure the similarity between two probability distributions and to calculate the loss, where y refers to the target face image, where is a random Gaussian distribution, E(y) is the real distribution that needs to be matched.

[0060] In the initial training stage, the generator G1 is not well trained temporarily, and the quality of the generated image samples is poor, which will be rejected by the discriminator D1 with a high confidence probability, which will cause the generator to be unable to provide sufficient gradients for updating during training. Therefore, the maximum E x,z [D1(G1(x,z))] is selected instead of minimizing E x,z (1-D1(G1(x,z))) to update the parameters of G1. The definition of the adversarial loss is as shown in equation (2), where E(·) refers to the expected value.

[0061] Under the attention mechanism in the adversarial sample generator G1, the output feature of the channel attention submodule is denoted as M c = sigmoid(W γ (BN(F1))) represents the output feature. γ is the scale factor of each channel, and the weight is The output feature of the corresponding spatial attention submodule is denoted as M s = sigmoid(W λ (BN s (F2))), and λ is the scale factor, and the weight is The ability of the model to focus on the key areas of the face is strengthened by integrating NAM, which improves the quality of the adversarial sample images generated by the generator. The latent code of the target face image is combined with the input face image in the channel dimension and participates in the subsequent training process of the generator. The channel attention submodule in NAM can better remember the latent code information and prevent the loss of latent code information during the adversarial training process of the generator and the discriminator. Thanks to the spatial attention submodule in NAM, the generator focuses on the dependence of important features and spatial relative positions of facial features, which strengthens the quality of the generated images.

[0062] The generator G2 takes the image generated by the generator G1 as input to reconstruct the original input image x, and the loss is calculated using the L1 norm. By backpropagating these two mappings, the cycle-consistency loss between the two mappings can be calculated. The reconstruction loss ​​It can be ensured that the generated adversarial samples are closer to the original input image in terms of image content representation. x is the original image, G1(x, z) is the sample image generated by the generator G1, and G2(G1(x, z)) is the image obtained after inputting G1(x, z) to G2

[0063] The adversarial samples based on the gradient attack method are used in the robust training process, which provides more information about the data distribution of the adversarial samples for the generator, so that a more robust discriminator can be obtained, which can better generalize to unseen data. At the same time, another advantage of the robust discriminator is that it helps to stabilize and accelerate the entire training process. L att x [D1(x)]-E x,y [D1(Att(x,y))] calculates the loss of adversarial attack training. Wherein Att(x, y) represents the targeted adversarial sample obtained by the PGD attack algorithm, and D1 is the true and false verification discriminator.

[0064] (B) The identity information and true and false dual discrimination stage includes:

[0065] The identity information discriminator uses multiple pre-trained face recognition models for integration, such as IRSE50, IR152, Facenet, and Mobileface. The invention adopts a weighted average model integration method. For example Figure 1 The right side of the migration enhancement module is shown, and then the calculation results of multiple models are weighted and averaged to obtain the final output result of the integrated model. Therefore, after applying the model integration strategy, the cosine loss is as shown in formula . Wherein k refers to the number of substitute models, F k refers to the integrated substitute model, and the cosine distance is selected as the distance measurement method of the feature vector of the face image. The cosine distances calculated by the k different models are weighted and averaged. The invention adopts the method of minimizing the cosine distance between vectors, and minimizes the feature vector distance between the adversarial sample and the target image through adversarial training.

[0066] The true and false verification discriminator is used to judge whether the generated sample is real or generated, and its loss is calculated by . Wherein, is the adversarial loss of the adversarial sample generator G1. As shown in formula Figure 3 , c refers to the number of output channels, and s refers to the step size. The size of each convolution kernel is 4x4. The final output of the discriminator D1 is a feature map, wherein each value is equivalent to the score of the corresponding region of the discriminated image. Finally, the average value of these scores is taken as the true and false classification score of the discriminated image.

[0067] ​The embodiment of the present application provides a face adversarial attack method system based on a generative adversarial network, comprising a computer readable storage medium and a processor.

[0068] The computer readable storage medium is used for storing executable instructions.

[0069] The processor is used for reading the executable instructions stored in the computer readable storage medium, and executing the method according to any one of the above embodiments.

[0070] The embodiment of the present application provides a computer readable storage medium, the computer readable storage medium stores computer instructions, and the computer instructions are used for making the processor execute the method according to any one of the above embodiments.

[0071] Those skilled in the art can easily understand that the above description is only the preferred embodiment of the present application, and is not used to limit the present application, and any modification, equivalent replacement and improvement made within the spirit and principle of the present application should be included in the protection scope of the present application.

Claims

1. A face adversarial attack method based on a generative adversarial network, characterized in that, Comprising: Training phase: S1, target face image Latent encoding Compared with the original face image The common inputs are fed into the adversarial sample generator G1 to synthesize the first adversarial sample. ;Will Input to image reconstruction generator Obtain the reconstructed original face image ;Will and The common input is fed into the attacker module to generate the second adversarial sample. ; S2, inputting the real and false verification discriminator and respectively to the real and false verification discriminator , obtaining the difference d1 between the real and false judgment results of and , the difference d2 between the real and false judgment results of and ; inputting and to the identity information discriminator together, obtaining the cosine distance d3 between the feature vector of and the feature vector of ; S3, to minimize with G1, G2, D1, D2 are trained with the goal of minimizing the difference between Application phase: The face image to be attacked and are input into the trained G1 to obtain a corresponding adversarial sample g; g and y are input into the face recognition model, if the cosine distance between the feature vector of g and the feature vector of y calculated by the face recognition model is greater than a threshold, the attack fails, otherwise the attack succeeds; the total loss function of the training phase ; wherein , , , ; , are the feature vectors of , respectively.

2. The method of claim 1, wherein, The adversarial sample generator G1 comprises an encoder and a decoder, and the encoder comprises a normalized attention module.

3. The method of claim 1 or 2, wherein, The true and false verification discriminator The identity information discriminator is a Markov discriminator based on a convolutional layer structure The integrated multiple pre-trained face recognition models 4. The method of claim 1, wherein, The attacker module generates a second adversarial sample based on a gradient attack method.

5. A face adversarial attack method system based on a generative adversarial network, characterized in that, Comprising: Computer readable storage medium and processor; The computer readable storage medium is used to store executable instructions; The processor is used to read the executable instructions stored in the computer readable storage medium, and execute the method according to any one of claims 1-4.

6. A computer-readable storage medium, characterized in that, The computer readable storage medium stores computer instructions for causing the processor to execute the method according to any one of claims 1-4.

Images