Face recognition attack and defense method based on gan

Abstract

The application discloses a face recognition attack and defense method based on GAN, comprising the following steps: inputting an original face image and a face facial expression unit into a global GAN generator, performing adversarial training by using a discriminator, and obtaining an original adversarial example; cutting the original face image to obtain a facial feature image, and constructing a global GAN generator and a facial feature GAN generator; inputting the original face image and the facial feature image and the face facial expression unit into four generators to obtain generated images; cutting the original adversarial example to obtain a facial feature adversarial image; performing DCT transformation on the generated images and the facial feature adversarial image to remove high-frequency components, and calculating a loss function; cutting a target overall image to be detected into a facial feature target image, performing DCT transformation on the original adversarial example and the facial feature adversarial image, the target overall image and the facial feature target image to remove low-frequency components, and calculating a loss function; constructing a face recognition model, performing probability transformation on the original adversarial example by using a Tp function, attacking the face recognition model, and obtaining a loss function; updating parameters of the global GAN generator and the discriminator by using three kinds of loss functions, and inputting the original face image to generate a final adversarial example; and the method guarantees that the generated face image can be normally used on social media, and effectively protects the face image from being infringed by a non-personal face recognition system.

Description

Technical Field

[0001] This invention relates to the field of facial recognition technology, and in particular to a method for defending against adversarial attacks on facial recognition based on GAN. Background Technology

[0002] The rapid development of facial recognition technology has brought about significant social impacts. The collection of facial data in scenarios such as social media and public place surveillance has become commonplace; however, this also makes individual privacy more vulnerable to infringement. Facial recognition systems can be used to identify social media profiles and track user relationships through large-scale photo analysis. This excessive surveillance of users urgently requires an effective method to help individuals protect their facial images from unauthorized attacks by facial recognition systems.

[0003] Currently, the main methods for protecting facial privacy include: obfuscation techniques, data poisoning attacks, and using adversarial examples to initiate evasion.

[0004] Adversarial examples are a mainstream method for protecting facial privacy. They involve intentionally creating deceptive inputs to trick facial recognition systems into producing incorrect results in real-world applications. The primary method for adversarial example evasion attacks is impersonation, where the generated adversarial example is identified as a specified target. Adversarial examples are generated by making small but targeted perturbations to the input data. These perturbations are small enough to be imperceptible to the human eye, but sufficient to cause machine learning models to produce incorrect outputs. For example, the recently popular AMT-GAM (Adversarial Methods-Entanglement) architecture uses a feature-based deentanglement regularization module to improve the visual quality of adversarial images. A joint training pipeline is then developed to train the generator, discriminator, and regularization module. The generator can then simultaneously perform two tasks: makeup transmission and adversarial attacks, establishing robust mappings between different data manifolds. It has shown good performance on various commercial facial recognition APIs. However, the generation of adversarial examples relies excessively on the makeup reference image, and the generated makeup-wearing adversarial examples often appear unnatural, especially when generating adversarial examples from male facial images. Most existing methods for circumventing adversarial examples suffer from problems such as generating unnatural adversarial examples and a low success rate in having the adversarial examples recognized as the target identity by face recognition models. Summary of the Invention

[0005] This invention provides a GAN-based method for defending against adversarial attacks on face recognition, overcoming the problems of unnatural generated adversarial examples, low attack success rate, and low transferability of attack models in most existing face privacy protection methods.

[0006] To achieve the above objectives, the technical solution of the present invention is as follows:

[0007] A GAN-based method for defending against adversarial attacks on face recognition includes:

[0008] S1: Input the original face image and the facial expression unit into the global GAN ​​generator, and use the discriminator to perform adversarial training to obtain the original adversarial example image that matches the facial expression unit;

[0009] S2: Crop the original face image to obtain images of the eyes, nose, and mouth; construct an overall GAN ​​generator, an eye GAN generator, a nose GAN generator, and a mouth GAN generator; and input the original face image and the images of the eyes, nose, and mouth, along with the facial expression unit, into the overall GAN ​​generator, the eye GAN generator, the nose GAN generator, and the mouth GAN generator to obtain the generated overall, eye, nose, and mouth images;

[0010] S3: Crop the original adversarial example image to obtain adversarial images of the eyes, nose, and mouth; perform DCT transformation on the generated overall, eye, nose, and mouth images and the original adversarial example image, eye, nose, and mouth adversarial images, and remove high-frequency components; calculate the naturalness loss function of the generated overall, eye, nose, and mouth images after DCT transformation and high-frequency component removal compared with the original adversarial example image, eye, nose, and mouth adversarial images.

[0011] S4: Input the overall image of the target to be tested and crop the overall image of the target to be tested into target images of eyes, nose and mouth. Perform DCT transformation on the original adversarial example image, the adversarial images of eyes, nose and mouth, the overall image of the target to be tested and the target images of eyes, nose and mouth respectively, and remove low frequency components. Calculate the attack success rate loss function of the original adversarial example image, the adversarial images of eyes, nose and mouth after DCT transformation and removal of low frequency components, and the overall image of the target to be tested and the target images of eyes, nose and mouth.

[0012] S5: Construct k pre-trained face recognition models, perform probability transformation on the original adversarial example image through the input diversity function Tp, attack the face recognition model, and obtain the diversity attack loss function of the adversarial example processed by the input diversity function Tp and the feature image obtained by the face recognition model of the overall image of the target under test;

[0013] S6: Update the parameters of the global GAN ​​generator and the discriminator using the three loss functions, input the original face image into the updated global GAN ​​generator and discriminator, and generate the final adversarial example.

[0014] Furthermore, the original face image and facial expression units are input into the global GAN ​​generator, and adversarial training is performed using a discriminator to obtain original adversarial example images that match the facial expression units, including:

[0015] S11. Input the original face image and facial expression unit into the global GAN ​​generator to obtain the original adversarial example image;

[0016] S12. The original face image and the original adversarial example image are evaluated using a discriminator, and adversarial training is performed against the global GAN ​​generator. The adversarial training process is shown in formulas (1) and (2).

[0017]

[0018] in, Indicates discriminator loss. Let λ represent the loss of the global GAN ​​generator, D represent the discriminator, G represent the global GAN ​​generator, and λ represent the loss of the global GAN ​​generator. c λ represents the discriminative loss weight. gp Indicates the gradient penalty loss weights. a0 represents the original face image, and a0 represents the facial expression unit of the original face image itself. i The input facial expression unit, x * This represents the random interpolation distribution between the original face image and the generated image.

[0019] Further, S3 crops the original adversarial example image to obtain adversarial images of the eyes, nose, and mouth; performs DCT transform on the generated overall, eye, nose, and mouth images and the original adversarial example image and the adversarial images of the eyes, nose, and mouth, and removes high-frequency components; calculates the naturalness loss function between the generated overall, eye, nose, and mouth images after DCT transform and high-frequency component removal and the adversarial images of the eyes, nose, and mouth, including:

[0020] S31. Crop the original adversarial example image to obtain adversarial images of the eyes, nose and mouth;

[0021] S32. Perform DCT transformation on the generated overall, eye, nose and mouth images and the adversarial images of the eyes, nose and mouth to obtain DCT components;

[0022] S33. Set the value greater than the DCT component to be a high-frequency component, reset the high-frequency component to 0, and remove the high-frequency component.

[0023] S34. Calculate the naturalness loss function for the overall image, the eye, nose, and mouth images after removing high-frequency components, and the adversarial image of the eye, nose, and mouth. The naturalness loss function includes SSIM loss and MSE loss. The overall image is subjected to SSIM loss, and the local features are subjected to MSE loss. The naturalness loss function is shown in formula (3).

[0024]

[0025]

[0026] in, λ represents the DCT component obtained after the image undergoes DCT transformation and high-frequency components are removed. g G represents the global natural loss weight. p Denotes local GAN ​​generators, namely eye GAN generator, nose GAN generator, and mouth GAN generator, λ l Crop represents the local natural loss weights. p The adversarial images of the eyes, nose, and mouth are obtained by cropping the adversarial examples.

[0027] Further, S4 inputs the overall image of the target to be tested and crops it into target images of the eyes, nose, and mouth. The original adversarial example image, the adversarial images of the eyes, nose, and mouth, the overall image of the target to be tested, and the target images of the eyes, nose, and mouth are all subjected to DCT transformation, and low-frequency components are removed. The attack success rate loss function is calculated between the original adversarial example image, the adversarial images of the eyes, nose, and mouth after DCT transformation and low-frequency component removal, and the overall image of the target to be tested and the target images of the eyes, nose, and mouth. This includes:

[0028] S41. Input the overall image of the target to be tested and crop the overall image of the target to be tested into target images of eyes, nose and mouth;

[0029] S42. The original adversarial example image, the adversarial images of the eyes, nose and mouth, the overall image of the target to be tested and the target images of the eyes, nose and mouth are respectively subjected to DCT transformation to obtain DCT components;

[0030] S43. Set the value less than the DCT component to a low-frequency component, reset the low-frequency component to 0, and remove the low-frequency component.

[0031] S44. Calculate the attack success rate loss function of the original adversarial example image after removing the low-frequency components, the adversarial image of the eyes, nose and mouth, and the overall image of the target and the target images of the eyes, nose and mouth. The attack success rate loss function is calculated using MSE loss, as shown in formula (4).

[0032]

[0033] in, The x represents the DCT component obtained after the image undergoes DCT transformation and low-frequency components are removed. t Let λ represent the target image, λ1 represent the global attack loss weight, λ2 represent the local attack loss weight, and m represent the median of the image after DCT transformation.

[0034] Furthermore, S5 constructs k pre-trained face recognition models, performs probability transformation on the original adversarial example image through the input diversity function Tp, attacks the face recognition model, and obtains a diversity attack loss function for the adversarial example processed by the input diversity function Tp and the feature image obtained by the face recognition model from the overall image of the target under test, including:

[0035] S51. Construct k pre-trained face recognition models;

[0036] S52. The original adversarial example image is subjected to probability transformation through the input diversity function Tp, and the adversarial example and target image processed by the input diversity function Tp are input into the k pre-trained face recognition models to obtain k feature images of the adversarial example and target image;

[0037] S53. Calculate the diversity attack loss function for the k feature images of the adversarial example and the k feature images of the target image, as shown in formula (5).

[0038]

[0039] Where, λ trans Let C represent the transferable augmentation loss weights, K represent the k face recognition models, and C represent the weights. k (.) represents the feature image obtained by the face recognition model k.

[0040] Furthermore, S6 updates the parameters of the global GAN ​​generator and the discriminator using the three loss functions, inputs the original face image into the updated global GAN ​​generator and discriminator, and generates the final adversarial example, including:

[0041] S61. Based on the three loss functions, use the backpropagation algorithm to calculate the gradient values ​​of the generation parameters of the global GAN ​​generator and the discrimination parameters of the discriminator, and calculate new generation parameters and discrimination parameters based on the gradient values ​​and the learning rate in the GMAA method.

[0042] S62. Update the global GAN ​​generator generation parameters and the discriminator discrimination parameters using the new generation parameters and discrimination parameters, generate and save the GAN generator weight file, configure the new global GAN ​​generator according to the GAN generator weight file, and obtain the updated global GAN ​​generator and discriminator.

[0043] S63. Input the original face image into the updated global GAN ​​generator to generate adversarial examples, and use the updated discriminator to evaluate the original face image and the adversarial examples to obtain the evaluated loss function;

[0044] S64. Based on the loss function, perform adversarial training on the updated global GAN ​​generator and the updated discriminator to obtain the minimum loss function and generate the final adversarial example.

[0045] Compared to traditional obfuscation techniques, this invention achieves higher attack accuracy by performing DCT transformation on images, generating more natural-looking face images that can attack trained face recognition models, thus increasing the diversity of attackable adversarial examples. This method ensures that the generated face images can be used normally on social media while effectively protecting them from malicious face recognition systems. Attached Figure Description

[0046] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0047] Figure 1 This is a schematic diagram of the face recognition adversarial attack defense method based on GAN of the present invention;

[0048] Figure 2 This is a flowchart illustrating the overall processing procedure of the present invention;

[0049] Figure 3 This is a simulation result diagram of one embodiment of the present invention. Detailed Implementation

[0050] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0051] This embodiment provides a GAN-based method for defending against adversarial attacks on face recognition, such as... Figure 1 As shown, it includes:

[0052] S1: Input the original face image and the facial expression unit into the global GAN ​​generator, and use the discriminator to perform adversarial training to obtain the original adversarial example image that matches the facial expression unit;

[0053] S2: Crop the original face image to obtain images of the eyes, nose, and mouth; construct an overall GAN ​​generator, an eye GAN generator, a nose GAN generator, and a mouth GAN generator; and input the original face image and the images of the eyes, nose, and mouth, along with the facial expression unit, into the overall GAN ​​generator, the eye GAN generator, the nose GAN generator, and the mouth GAN generator to obtain the generated overall, eye, nose, and mouth images;

[0054] S3: Crop the original adversarial example image to obtain adversarial images of the eyes, nose, and mouth; perform DCT transformation on the generated overall, eye, nose, and mouth images and the original adversarial example image, eye, nose, and mouth adversarial images, and remove high-frequency components; calculate the naturalness loss function of the generated overall, eye, nose, and mouth images after DCT transformation and high-frequency component removal compared with the original adversarial example image, eye, nose, and mouth adversarial images.

[0055] S4: Input the overall image of the target to be tested and crop the overall image of the target to be tested into target images of eyes, nose and mouth. Perform DCT transformation on the original adversarial example image, the adversarial images of eyes, nose and mouth, the overall image of the target to be tested and the target images of eyes, nose and mouth respectively, and remove low frequency components. Calculate the attack success rate loss function of the original adversarial example image, the adversarial images of eyes, nose and mouth after DCT transformation and removal of low frequency components, and the overall image of the target to be tested and the target images of eyes, nose and mouth.

[0056] S5: Construct k pre-trained face recognition models, perform probability transformation on the original adversarial example image through the input diversity function Tp, attack the face recognition model, and obtain the diversity attack loss function of the adversarial example processed by the input diversity function Tp and the feature image obtained by the face recognition model of the overall image of the target under test;

[0057] S6: Update the parameters of the global GAN ​​generator and the discriminator using the three loss functions, input the original face image into the updated global GAN ​​generator and discriminator, and generate the final adversarial example.

[0058] Specifically, this invention first inputs the original face image and facial expression units into a global GAN ​​generator, and uses a discriminator for adversarial training to obtain original adversarial example images that conform to the expression features of the facial expression units. Training the GAN allows for continuous parameter optimization to improve its generation capabilities. The discriminator can distinguish between real and generated images, prompting the generator to produce more realistic adversarial examples. Next, the original face image is cropped to obtain images of the eyes, nose, and mouth. A global GAN ​​generator, an eye GAN generator, a nose GAN generator, and a mouth GAN generator are then constructed. Finally, the original face image and the images of the eyes, nose, and mouth are compared with the facial expression units. The facial expression unit is input into the overall GAN ​​generator, eye GAN generator, nose GAN generator, and mouth GAN generator to obtain generated overall, eye, nose, and mouth images. Generating eye, nose, and mouth images and inputting them into the corresponding generators can better generate detailed image features. Loss adjustment of the global GAN ​​with adversarial examples makes the generated adversarial examples more natural. The original adversarial example image is cropped to obtain adversarial images of the eyes, nose, and mouth. The generated overall, eye, nose, and mouth images are then compared with the original adversarial example images. The adversarial images of the nose and mouth undergo DCT transformation, and high-frequency components are removed. The naturalness loss function of the generated overall, eye, nose, and mouth images after DCT transformation and high-frequency component removal is calculated compared to the original adversarial example image and the adversarial images of the eyes, nose, and mouth. DCT transforms the image into DCT components; adjusting the high and low frequencies of the DCT components can improve the success rate of GAN-generated adversarial examples and make the images more natural. Removing high-frequency components reduces the feature similarity between the adversarial example and the original image, allowing the adversarial example to better learn the features of the target image. The overall image of the target to be tested is input, and the overall image of the target to be tested is cropped into the target images of the eyes, nose, and mouth. The image is processed by DCT transformation on the original adversarial example image, the adversarial images of the eyes, nose and mouth, the overall image of the target to be tested, and the target images of the eyes, nose and mouth. Low-frequency components are removed. The attack success rate loss function is calculated between the original adversarial example image, the adversarial images of the eyes, nose and mouth after DCT transformation and removal of low-frequency components, and the overall image of the target to be tested and the target images of the eyes, nose and mouth. Removing low-frequency components from the adversarial example and the target image can improve the similarity between high-frequency features and the target image while maintaining the visual effect of the adversarial example, thereby improving the accuracy of the attack. Using the MSE loss function can ensure the fine detail of local features.K pre-trained face recognition models are constructed. The original adversarial example image is subjected to probability transformation through an input diversity function Tp, and then attacked to attack the face recognition model. A diversity attack loss function is obtained for the adversarial example processed by the input diversity function Tp and the feature image obtained from the overall image of the target object through the face recognition model. This is used to attack the k pre-trained face recognition models, thereby improving the transferability of the adversarial attack and the success rate of the black-box attack. Adding noise and random cropping to the adversarial example through the input diversity function Tp can improve the transferability of the adversarial example, ensuring that the adversarial example maintains good adversarial performance across different face recognition systems. The parameters of the global GAN ​​generator and the discriminator are updated using the three loss functions. The original face image is then input into the updated global GAN ​​generator and discriminator to generate the final adversarial example.

[0059] In a specific embodiment, the scheme of inputting the original face image and facial expression units into the global GAN ​​generator, and using a discriminator for adversarial training to obtain the original adversarial example image that matches the facial expression units is as follows:

[0060] S11. Input the original face image and facial expression unit into the global GAN ​​generator to obtain the original adversarial example image;

[0061] S12. The original face image and the original adversarial example image are evaluated using a discriminator, and adversarial training is performed against the global GAN ​​generator. The adversarial training process is shown in formulas (6) and (7).

[0062]

[0063] in, Indicates discriminator loss. Let λ represent the loss of the global GAN ​​generator, D represent the discriminator, G represent the global GAN ​​generator, and λ represent the loss of the global GAN ​​generator. c λ represents the discriminative loss weight. gp Indicates the gradient penalty loss weights. a0 represents the original face image, and a0 represents the facial expression unit of the original face image itself. i The input facial expression unit, x * This represents the random interpolation distribution between the original face image and the generated image.

[0064] Original image Based on the expression a0, adversarial examples matching the a0 facial features are generated. A discriminator distinguishes between real and generated images. The loss function makes the features of real images approach 1 and the features of generated images approach 0. A random interpolation distribution loss is added to encourage the generator to produce more realistic adversarial examples. The global GAN ​​generator and discriminator are based on an existing deep learning model architecture. Training the GAN allows for continuous parameter optimization to improve its generative capabilities. The GAN generator is a component of the GAN.

[0065] In a specific embodiment, the original face image is cropped to obtain images of the eyes, nose, and mouth; an overall GAN ​​generator, an eye GAN generator, a nose GAN generator, and a mouth GAN generator are constructed; and the original face image and the images of the eyes, nose, and mouth, along with the facial expression unit, are input into the overall GAN ​​generator, the eye GAN generator, the nose GAN generator, and the mouth GAN generator to obtain the generated overall, eye, nose, and mouth images.

[0066] The generator is a pre-trained generator derived from the Adv-Makeup model. It crops the original face image to obtain images of the eyes, nose, and mouth, and inputs these images into the corresponding generator to obtain the corresponding generated image.

[0067] This invention generates images of eyes, nose, and mouth, and inputs them into the corresponding generator to generate more detailed image features. It also adjusts the loss of the global GAN ​​based on adversarial examples to generate more natural adversarial examples.

[0068] In a specific embodiment, the original adversarial example image is cropped to obtain adversarial images of the eyes, nose, and mouth; the generated overall, eye, nose, and mouth images, along with the original adversarial example image and the adversarial images of the eyes, nose, and mouth, undergo DCT transformation, and high-frequency components are removed; the scheme for calculating the naturalness loss function of the generated overall, eye, nose, and mouth images after DCT transformation and high-frequency component removal compared to the original adversarial example image and the adversarial images of the eyes, nose, and mouth is as follows:

[0069] S31. Crop the original adversarial example image to obtain adversarial images of the eyes, nose and mouth;

[0070] S32. Perform DCT transformation on the generated overall, eye, nose and mouth images and the adversarial images of the eyes, nose and mouth to obtain DCT components;

[0071] S33. Set the value greater than the DCT component to be a high-frequency component, reset the high-frequency component to 0, and remove the high-frequency component.

[0072] S34. Calculate the naturalness loss function of the overall image, the eye, nose, and mouth images after removing high-frequency components, and the adversarial image of the eye, nose, and mouth. The naturalness loss function includes SSIM loss and MSE loss. The overall image is subjected to SSIM loss, and the local features are subjected to MSE loss. The naturalness loss function is shown in formula (8).

[0073]

[0074] in, λ represents the DCT component obtained after the image undergoes DCT transformation and high-frequency components are removed. g This represents the global natural loss weight, set to 20, G p Denotes local GAN ​​generators, specifically eye GAN generator, nose GAN generator, and mouth GAN generator, λ l The local natural loss weight is set to 20, Crop p The adversarial images of the eyes, nose, and mouth are obtained by cropping the adversarial examples.

[0075] This scheme employs DCT transformation, a signal processing technique and a variant of Fourier transform. DCT can decompose a signal into a combination of cosine waves of different frequencies. Its main advantage is that it provides an effective way to represent the energy distribution of a signal. DCT transforms an image into DCT components. By adjusting the high and low frequencies of the DCT components, GANs can generate more successful attack examples with higher success rates and more natural images.

[0076] The human eye primarily relies on low-frequency recognition. Therefore, it's sufficient to maintain low-frequency similarity. By removing high-frequency components, the feature similarity between the adversarial example and the original image can be reduced, allowing the adversarial example to better learn the features of the target image. The generated image and the adversarial example are weighted together to calculate a loss, which is then passed to the discriminator. The discriminator learns to distinguish between the real and generated images, ensuring the overall consistency and naturalness of local feature details in the adversarial example. The overall image generated by G and G... global The generated image is subjected to SSIM loss to ensure the consistency of overall perception, while local features are subjected to MSE loss to ensure the fineness of local features.

[0077] In a specific embodiment, the input is an overall image of the target to be tested, which is then cropped into target images of the eyes, nose, and mouth. The original adversarial example image, the adversarial images of the eyes, nose, and mouth, the overall image of the target to be tested, and the target images of the eyes, nose, and mouth are all subjected to DCT transformation, and low-frequency components are removed. The scheme for calculating the attack success rate loss function of the original adversarial example image, the adversarial images of the eyes, nose, and mouth after DCT transformation and low-frequency component removal, compared with the overall image of the target to be tested and the target images of the eyes, nose, and mouth, is as follows:

[0078] S41. Input the overall image of the target to be tested and crop the overall image of the target to be tested into target images of eyes, nose and mouth;

[0079] S42. The original adversarial example image, the adversarial images of the eyes, nose and mouth, the overall image of the target to be tested and the target images of the eyes, nose and mouth are respectively subjected to DCT transformation to obtain DCT components;

[0080] S43. Set the value less than the DCT component to a low-frequency component, reset the low-frequency component to 0, and remove the low-frequency component.

[0081] S44. Calculate the attack success rate loss function of the original adversarial example image after removing the low-frequency components, the adversarial image of the eyes, nose and mouth, and the overall image of the target and the target images of the eyes, nose and mouth. The attack success rate loss function is calculated using MSE loss, as shown in formula (9).

[0082]

[0083] in, The x represents the DCT component obtained after the image undergoes DCT transformation and low-frequency components are removed. t Let λ represent the target image, λ1 represent the global attack loss weight (set to 20), λ2 represent the local attack loss weight (set to 20), and m represent the median of the image after DCT transformation.

[0084] In this approach, low-frequency components of both the adversarial example and the target image are removed as a loss function. This improves the similarity between high-frequency features and the target image while maintaining the visual quality of the adversarial example, thereby increasing the accuracy of the attack. The use of the MSE loss function ensures the refinement of local features.

[0085] In a specific embodiment, k pre-trained face recognition models are constructed. The original adversarial example image is subjected to probability transformation through an input diversity function Tp to attack the face recognition model. The scheme for obtaining the diversity attack loss function of the adversarial example processed by the input diversity function Tp and the feature image obtained by the face recognition model of the overall image of the target under test is as follows:

[0086] S51. Construct k pre-trained face recognition models;

[0087] S52. The original adversarial example image is subjected to probability transformation through the input diversity function Tp, and the adversarial example and target image processed by the input diversity function Tp are input into the k pre-trained face recognition models to obtain k feature images of the adversarial example and target image;

[0088] S53. Calculate the diversity attack loss function for the k feature images of the adversarial example and the k feature images of the target image, as shown in formula (10).

[0089]

[0090] Where, λ trans The weights represent the transferable augmentation loss, set to 25, where K represents k face recognition models, and C... k (.) represents the feature image obtained by the face recognition model k.

[0091] This scheme attacks k pre-trained face recognition models to improve the transferability of adversarial attacks and the success rate of black-box attacks. After input diversity function Tp probability transformation, which is a transformation function that adds noise to the adversarial examples, the adversarial examples are randomly pruned with probability p. The input diversity function Tp adds noise to the adversarial examples and randomly prunes them, which can improve the transferability of the adversarial examples, that is, the ability of the adversarial examples to maintain good adversarial performance in different face recognition systems.

[0092] In a specific embodiment, the parameters of the global GAN ​​generator and the discriminator are updated using the three loss functions. The original face image is then input into the updated global GAN ​​generator and discriminator to generate the final adversarial example.

[0093] S61. Based on the three loss functions, the gradient values ​​of the generation parameters of the global GAN ​​generator and the discrimination parameters of the discriminator are calculated using the backpropagation algorithm. New generation parameters and discrimination parameters are calculated based on the gradient values ​​and the learning rate in the GMAA method. In this embodiment, the initial learning rate is 0.0001.

[0094] S62. Update the global GAN ​​generator generation parameters and the discriminator discrimination parameters using the new generation parameters and discrimination parameters, generate and save the GAN generator weight file, configure the new global GAN ​​generator according to the GAN generator weight file, and obtain the updated global GAN ​​generator and discriminator.

[0095] S63. Input the original face image into the updated global GAN ​​generator to generate adversarial examples, and use the updated discriminator to evaluate the original face image and the adversarial examples to obtain the evaluated loss function;

[0096] S64. Based on the loss function, perform adversarial training on the updated global GAN ​​generator and the updated discriminator to obtain the minimum loss function and generate the final adversarial example.

[0097] like Figure 2 As shown, a clean facial image (real) and a facial expression unit (AU) are input. The AU is simultaneously fed into the overall GAN ​​generator, eye GAN generator, nose GAN generator, and mouth GAN generator to ensure the naturalness training module generates natural, detailed, and expressive overall and partial facial images. The overall GAN ​​generator, eye GAN generator, nose GAN generator, and mouth GAN generator are subjected to mean squared error loss along with the generated adversarial example (fake). The eyes, nose, and mouth of the fake and target images are cropped and subjected to DCT transformation. Since face recognition systems identify faces based on low and high frequencies, and the human eye primarily perceives faces through low frequencies, components less than half the average of the DCT components are considered low-frequency components and removed. The high-frequency components of the fake and target images are then used as the loss. The fake image is cropped with probability p. The transformed fake and target images are input into k pre-trained face recognition models, and the outputs are used as the loss. The above losses are then processed using the backpropagation algorithm to calculate the gradient values. The parameters of the global GAN ​​generator are updated based on the gradient values ​​and the learning rate. The discriminator evaluates the original face image and the generated adversarial examples, and calculates the loss function and gradient values. The discriminator's parameters are updated based on the gradient values ​​and the learning rate. The discriminator and generator engage in adversarial competition to obtain the minimum loss function, prompting the generator to produce more natural adversarial examples with higher attack accuracy and greater diversity of attackable models.

[0098] The table below shows the attack success rates of GMAA and the face recognition model FR@0.01 of this method under different input Au expressions. The pre-trained models are FaceNet, IR152, and IRSE50. The black-box model used for the attack is MobileFace. The dataset is 128x128 CelebA-HQ. The training environment is RTX2060 and Ubuntu 22.04.4LTS.

[0099] Au0 Au1 Au2 Au3 Au4 Au5 GMAA 59.49 59.23 61.19 76.66 57.39 57.69 This method 63.82 62.61 66.57 81.98 61.50 71.45

[0100] As can be seen from the table, the larger values ​​obtained by this invention indicate that the success rate of the attack is higher and the privacy protection effect is better. Figure 3 This is an example of an input image and a result of an adversarial example in one embodiment of the present invention.

[0101] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for defending against adversarial attacks on face recognition based on GAN, characterized in that, include: S1: Input the original face image and the facial expression unit into the global GAN ​​generator, and use the discriminator to perform adversarial training to obtain the original adversarial example image that matches the facial expression unit; S2: Crop the original face image to obtain images of the eyes, nose, and mouth; construct an overall GAN ​​generator, an eye GAN generator, a nose GAN generator, and a mouth GAN generator; and input the original face image and the images of the eyes, nose, and mouth, along with the facial expression unit, into the overall GAN ​​generator, the eye GAN generator, the nose GAN generator, and the mouth GAN generator to obtain the generated overall, eye, nose, and mouth images; S3: Crop the original adversarial example image to obtain adversarial images of the eyes, nose, and mouth; perform DCT transformation on the generated overall, eye, nose, and mouth images and the original adversarial example, eye, nose, and mouth images, and remove high-frequency components; calculate the naturalness loss function of the generated overall, eye, nose, and mouth images after DCT transformation and high-frequency component removal compared to the original adversarial example, eye, nose, and mouth images; specific steps include: S31. Crop the original adversarial example image to obtain adversarial images of the eyes, nose and mouth; S32. Perform DCT transformation on the generated overall, eye, nose and mouth images and the adversarial images of the eyes, nose and mouth to obtain DCT components; S33. Set the value greater than the DCT component to be a high-frequency component, reset the high-frequency component to 0, and remove the high-frequency component. S34. Calculate the naturalness loss function of the overall image, the eye, nose, and mouth images after removing high-frequency components, and the adversarial image of the eye, nose, and mouth. The naturalness loss function includes SSIM loss and MSE loss. The overall image is subjected to SSIM loss, and the local features are subjected to MSE loss. The naturalness loss function is shown in formula (3). （3） in, (.) represents the DCT component obtained after the image has undergone DCT transformation and high-frequency components have been removed. Represents the global natural loss weight. This represents a local GAN ​​generator, specifically an eye GAN generator, a nose GAN generator, and a mouth GAN generator. Indicates the local natural loss weight. The adversarial images of the eyes, nose, and mouth are obtained by cropping the adversarial examples; S4: Input the overall image of the target to be tested and crop it into target images of the eyes, nose, and mouth. Perform DCT transformation on the original adversarial example image, the adversarial images of the eyes, nose, and mouth, the overall image of the target to be tested, and the target images of the eyes, nose, and mouth, respectively, and remove low-frequency components. Calculate the attack success rate loss function between the original adversarial example image, the adversarial images of the eyes, nose, and mouth after DCT transformation and low-frequency component removal, and between the overall image of the target to be tested and the target images of the eyes, nose, and mouth. Specific steps include: S41. Input the overall image of the target to be tested and crop the overall image of the target to be tested into target images of eyes, nose and mouth; S42. The original adversarial example image, the adversarial images of the eyes, nose and mouth, the overall image of the target to be tested and the target images of the eyes, nose and mouth are respectively subjected to DCT transformation to obtain DCT components; S43. Set the value less than the DCT component to a low-frequency component, reset the low-frequency component to 0, and remove the low-frequency component. S44. Calculate the attack success rate loss function of the original adversarial example image after removing the low-frequency components, the adversarial image of the eyes, nose and mouth, and the overall image of the target and the target images of the eyes, nose and mouth. The attack success rate loss function is calculated using MSE loss, as shown in formula (4). = （4） in, (.) indicates the DCT component obtained after the image has undergone DCT transformation and low-frequency components have been removed. Represents the target image. Indicates the global attack loss weight. This indicates the weight of the loss due to a local attack. m This represents the median of the image after DCT transformation; S5: Construct k pre-trained face recognition models, perform probability transformation on the original adversarial example image through the input diversity function Tp, attack the face recognition model, and obtain the diversity attack loss function of the adversarial example processed by the input diversity function Tp and the feature image obtained by the face recognition model of the overall image of the target under test; S6: Update the parameters of the global GAN ​​generator and the discriminator using three loss functions, input the original face image into the updated global GAN ​​generator and discriminator, and generate the final adversarial example.

2. The GAN-based face recognition adversarial attack defense method according to claim 1, characterized in that, S1 inputs the original face image and facial expression units into the global GAN ​​generator, and uses a discriminator for adversarial training to obtain original adversarial example images that match the facial expression units, including: S11. Input the original face image and facial expression unit into the global GAN ​​generator to obtain the original adversarial example image; S12. The original face image and the original adversarial example image are evaluated using a discriminator, and adversarial training is performed against the global GAN ​​generator. The adversarial training process is shown in formulas (1) and (2). + + （1） （2） in, Indicates discriminator loss. This represents the global GAN ​​generator loss. Indicates the discriminator, This represents the global GAN ​​generator. Indicates the weight of the discriminant loss. Indicates the gradient penalty loss weights. Represents the original human face image. Facial expression units representing the original human face image itself. The facial expression unit represents the input. This represents the random interpolation distribution between the original face image and the generated image.

3. The GAN-based face recognition adversarial attack defense method according to claim 1, characterized in that, S5 constructs k pre-trained face recognition models, performs probability transformation on the original adversarial example image through the input diversity function Tp, attacks the face recognition model, and obtains a diversity attack loss function for the adversarial example processed by the input diversity function Tp and the feature image obtained by the face recognition model from the overall image of the target under test, including: S51. Construct k pre-trained face recognition models; S52. The original adversarial example image is subjected to probability transformation through the input diversity function Tp, and the adversarial example and target image processed by the input diversity function Tp are input into the k pre-trained face recognition models to obtain k feature images of the adversarial example and target image; S53. Calculate the diversity attack loss function for the k feature images of the adversarial example and the k feature images of the target image, as shown in formula (5). （5） in, This indicates the transferable enhanced loss weight. Let k be face recognition models. This represents the feature image obtained through the face recognition model k.

4. The GAN-based face recognition adversarial attack defense method according to claim 1, characterized in that, S6 updates the parameters of the global GAN ​​generator and the discriminator using the three loss functions, inputs the original face image into the updated global GAN ​​generator and discriminator, and generates the final adversarial example, including: S61. Based on the three loss functions, use the backpropagation algorithm to calculate the gradient values ​​of the generation parameters of the global GAN ​​generator and the discrimination parameters of the discriminator, and calculate new generation parameters and discrimination parameters based on the gradient values ​​and the learning rate in the GMAA method. S62. Update the global GAN ​​generator generation parameters and the discriminator discrimination parameters using the new generation parameters and discrimination parameters, generate and save the GAN generator weight file, configure the new global GAN ​​generator according to the GAN generator weight file, and obtain the updated global GAN ​​generator and discriminator. S63. Input the original face image into the updated global GAN ​​generator to generate adversarial examples, and use the updated discriminator to evaluate the original face image and the adversarial examples to obtain the evaluated loss function; S64. Based on the loss function, perform adversarial training on the updated global GAN ​​generator and the updated discriminator to obtain the minimum loss function and generate the final adversarial example.

Images