Natural face confrontation sample generation method and system based on face mask

Abstract

The invention relates to a natural face confrontation sample generation method and system based on a face mask, and belongs to the technical field of confrontation attack. According to the method, feature point information in a face image is extracted, a mask image is generated, and disturbance is limited only in a mask area. A gradient-based method is adopted to generate adversarial disturbance, and a disturbance smooth loss function based on a Gaussian function is designed, so that the disturbance intensity can be effectively controlled, and the visual quality of the image is optimized. The system generates an adversarial sample according to the target image features, attacks the target model, and improves the attack effect. By accurately controlling the disturbance area and intensity, the attack success rate, mobility and visual quality of the adversarial sample can be remarkably improved, and the method is widely applied to computer vision systems such as face recognition and the like and has high safety and application prospects.

Description

Technical Field

[0001] This invention relates to the field of adversarial attack technology, and in particular to a method and system for generating natural adversarial samples of faces based on face masking. Background Technology

[0002] With the rapid development of deep learning technology, facial recognition technology has made significant progress in fields such as security and finance, and is widely used in scenarios with high security requirements, such as airport security checks and financial payments. However, with the popularization of the technology, facial recognition models face increasing security threats, especially adversarial attacks. Attackers can bypass facial recognition systems and cause incorrect identity judgments by making subtle perturbations to the input image. These perturbations are usually very small and difficult to detect by the human eye, but they can significantly affect the accuracy of the recognition system and create potential security vulnerabilities.

[0003] Existing adversarial attack methods primarily focus on disrupting facial recognition systems by adding patches or forged patterns. For example, AdvHat attacks by attaching specially processed adversarial patterns to hats or by adding makeup to the face. However, these methods have limited effectiveness in practical applications. First, they are limited by the attack scenarios and cannot be universally applied to all situations. Second, the added perturbations (such as stickers or makeup) differ significantly from the original facial image, making them easily detectable by the human eye and resulting in unsatisfactory attack effects. Furthermore, existing methods often involve excessively large perturbation areas and significant visual differences between perturbation and non-perturbation areas, causing the generated adversarial examples to appear unnatural and affecting the visual quality of the images.

[0004] Therefore, there is an urgent need for a technical solution that can precisely control the perturbation area and improve the visual quality and naturalness of adversarial samples, which can both enhance the attack effect and ensure the concealment and high visual quality of the image, thereby improving the security of the face recognition system. Summary of the Invention

[0005] The purpose of this invention is to provide a method and system for generating natural adversarial examples based on face masks, solving the problems of excessively large perturbation regions and unnatural visual effects in existing adversarial example generation technologies. This invention effectively reduces the visual difference between perturbation and undisturbed regions by precisely controlling the perturbation region and combining it with Gaussian function smoothing of the perturbation intensity, thereby improving the visual quality and concealment of adversarial examples.

[0006] To achieve the above objectives, the present invention employs the following technical solution:

[0007] A method for generating natural adversarial examples of faces based on face masking includes the following steps:

[0008] Feature point information is extracted from a face image, and a corresponding face mask image is generated by connecting the feature point information. The face mask image covers a specific area of ​​the face, and perturbation is only performed on the mask area.

[0009] Based on the face mask image, an adversarial perturbation is generated in the mask region using a gradient-based method. The adversarial perturbation affects the target image by adjusting the pixel values ​​of the mask region.

[0010] Design a weighted perturbation smoothing loss function based on Gaussian function to control the perturbation intensity in the mask region during backpropagation, so that the perturbation at the edge of the mask region gradually decreases.

[0011] Extract feature information from the target image, generate adversarial examples, and attack the target model.

[0012] As a preferred embodiment of the present invention, generating the corresponding face mask image includes the following steps:

[0013] A facial feature point extraction tool was used to extract 68 key points from the facial image. The key points include the eyes, nose, and mouth. The extraction tool was used to locate these key points using a deep learning-based algorithm.

[0014] Based on the extracted key points, the area below the eyebrow line and above the lower lip line is selected as the core area of ​​the face. A mask image is generated by connecting the key points of the area. The mask image is a binary image in which the pixel value is in the range of [0,1] and the value of each pixel is 0 or 1. The perturbation is only performed inside the mask area.

[0015] As a preferred embodiment of the present invention, the step of generating adversarial perturbations within the masked region using a gradient-based method includes the following steps:

[0016] Using a gradient-based method, adversarial perturbations are generated within the masked region by combining a [0,1] binary mask of the face image. The formula is as follows:

[0017] x adv =x+ò×sign(▽ x L(x,x t ));

[0018] Where: x adv This is an adversarial example; x is the input image; x t t represents the target image; ò represents the step size of the perturbation; ▽ x L(x,x t ) represents the gradient of the loss function with respect to the input image x;

[0019] Combining the masking mechanism, the perturbation area is limited to the masked area. A dynamic perturbation range method is used to gradually reduce the perturbation from the center of the masked area towards the edge, as shown in the formula:

[0020]

[0021] in: The adversarial example generated for the next iteration; α is the step size for each iteration; clip x,ò () is used to constrain adversarial examples to satisfy l ∞ Norm; The adversarial sample generated in the t-th iteration;

[0022] The Momentum Iterative FGSM (MIM) method is introduced to add a momentum term to the gradient-based perturbation generation method, optimizing the gradient update process. The formula is as follows:

[0023]

[0024] Wherein: g t The current gradient; g t+1 The gradient is the updated gradient; μ is the momentum factor. It is the L1 norm of the gradient;

[0025] The Diverse Input Iterative FGSM (DIM) method is employed to enhance the transferability of adversarial examples by randomly transforming the input. The formula is as follows:

[0026]

[0027] Where: T() represents the random transformation function; p represents the probability of performing the transformation.

[0028] As a preferred embodiment of the present invention, the design is based on a Gaussian function-based weighted perturbation smoothing loss function, which controls the perturbation intensity within the mask region during backpropagation, specifically including the following steps:

[0029] Determine the center point of the face mask region and calculate the distance between each pixel within the mask region and the center point. Calculate the weight of the perturbation based on the distance, and then apply a Gaussian function to weight the perturbation, as shown in the following formula:

[0030]

[0031] Where: i, j are the coordinates of the pixel; c x c y The coordinates of the center point of the mask region;

[0032] The perturbation weights of pixels are calculated using a Gaussian function, with the weights decreasing in regions farther from the mask center, as shown in the following formula:

[0033]

[0034] Where: σ is the standard deviation of the Gaussian function; W max This represents the maximum weight value of the Gaussian function;

[0035] Weighted perturbation is applied to each region of the image, preserving the perturbation intensity in the central region while reducing the perturbation intensity in the edge region, thereby optimizing image quality;

[0036] By introducing dynamically adjusted perturbation intensity, combined with Gaussian function and weight calculation, the adversarial example generation process of images is improved. The formula is as follows:

[0037]

[0038] Where: P′ i,j The perturbation is weighted; P i,j M represents the initial perturbation of pixel x(i,j); i,j A face [0,1] mask; P′ i+1,j To be related to the current pixel P′ i,j The perturbation value of adjacent vertical pixels; P′ i,j+1 To be related to the current pixel P′ i,j The perturbation value of adjacent horizontal pixels.

[0039] As a preferred embodiment of the present invention, the generation of adversarial examples and the attack on the target model specifically includes the following steps:

[0040] The cosine similarity loss is combined to measure the similarity between the target image and the adversarial example. The loss function is constructed by calculating the similarity between the two.

[0041] Based on backpropagation of loss, adversarial examples are generated through multiple iterations to attack the target model. In each iteration, the generated adversarial examples are optimized using the loss function to adjust the perturbation to a predetermined position.

[0042] By combining cosine similarity loss and perturbation loss, a loss function is constructed, and the generation process of adversarial examples is optimized by adjusting the weights of the two.

[0043] A system for generating natural adversarial examples of faces based on face masking, comprising:

[0044] The feature point extraction module is used to extract feature point information from a face image and generate a corresponding face mask image by connecting the feature point information. The face mask image covers a specific area of ​​the face and only perturbs within the mask area.

[0045] The adversarial perturbation generation module is used to generate adversarial perturbations in the masked area based on the face mask image using a gradient-based method. The adversarial perturbations affect the target image by adjusting the pixel values ​​of the masked area.

[0046] The perturbation smoothing control module is used to design a weighted perturbation smoothing loss function based on a Gaussian function, and to control the perturbation intensity in the mask region during backpropagation so that the perturbation at the edge of the mask region gradually decreases.

[0047] The target image feature extraction module is used to extract feature information from the target image and generate adversarial examples to attack the target model.

[0048] As a preferred embodiment of the present invention, the feature point extraction module includes:

[0049] The feature point extraction unit is used to extract feature point information in a face image and extract 68 key points in the face image using a deep learning-based algorithm. The key points include the eyes, nose, and mouth.

[0050] The mask generation unit is used to select the area below the eyebrow line and above the lower lip line as the core area of ​​the face based on the extracted key points, and generate a mask image by connecting the key points of the area. The mask image is a binary image.

[0051] As a preferred embodiment of the present invention, the anti-disturbance generation module includes:

[0052] The gradient calculation unit is used to generate adversarial perturbations in the mask region based on the face mask image using a gradient-based method. The adversarial perturbations affect the target image by adjusting the pixel values ​​of the mask region.

[0053] A dynamic perturbation adjustment unit is used to gradually reduce the perturbation from the center of the mask area to the edge;

[0054] The momentum iterative update unit is used to introduce the MIM method, adding a momentum term during the perturbation generation process to optimize the gradient update process;

[0055] The input transformation unit is used to perform random transformations on the input using the DIM method to enhance the transferability of generated adversarial examples.

[0056] As a preferred embodiment of the present invention, the disturbance smoothing control module includes:

[0057] A Gaussian weighted unit is used to calculate the distance of each pixel based on the center point of the face mask region and to calculate the weight of the perturbation using a Gaussian function.

[0058] The perturbation distribution adjustment unit is used to perform weighted perturbation on each region of the image, retaining the perturbation intensity of the central region while reducing the perturbation intensity of the edge region;

[0059] The dynamic adjustment unit is used to dynamically adjust the perturbation intensity based on the characteristics of the image and the result of Gaussian weighting.

[0060] Compared with existing technologies, the beneficial effects of this invention are as follows: This invention, through a natural face adversarial example generation method based on face masks, effectively solves the problem of unnatural visual effects in adversarial examples generated in existing technologies, particularly the problem of excessive differences between the edges of perturbed regions and undisturbed regions, leading to poor attack effectiveness. This invention extracts feature point information from face images and generates mask images, restricting perturbation to specific areas and avoiding influence on other irrelevant areas of the image, thereby ensuring the stealth of the attack. By designing a Gaussian-based weighted perturbation smoothing loss function, the intensity of the perturbation is controlled, gradually reducing the perturbation at the edges of the masked region, reducing the obvious visual difference between the perturbed and undisturbed regions, optimizing the visual quality of the image, and maintaining a high degree of naturalness. Furthermore, this invention combines gradient-based methods to generate adversarial perturbations, which can more accurately affect the target image, improving the attack effect of the adversarial examples. In terms of visual effects, the adversarial examples generated by this invention not only have high offensiveness but also maintain a high degree of visual naturalness, reducing the probability of being detected by the human eye. Overall, this invention enhances the attack effectiveness and stealth of adversarial examples, while optimizing the visual quality of images, improving the usability of images and the security of face recognition systems. Attached Figure Description

[0061] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0062] in:

[0063] Figure 1 This is a flowchart of a method according to an embodiment of the present invention;

[0064] Figure 2 This is a flowchart illustrating an embodiment of the present invention.

[0065] Figure 3 This is a schematic diagram of the modular structure of the system according to an embodiment of the present invention. Detailed Implementation

[0066] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the described embodiments of the present invention are within the scope of protection of the present invention.

[0067] Example 1

[0068] like Figure 1 , Figure 2 As shown, this is an embodiment of the present invention, which provides a method for generating natural adversarial examples of faces based on face masks, including:

[0069] (1) Extract feature point information from the face image, and generate a corresponding face mask image by connecting the feature point information. The face mask image covers a specific area of ​​the face, and only the mask area is disturbed.

[0070] Specifically, generating the corresponding face mask image includes the following steps:

[0071] A facial feature point extraction tool was used to extract 68 key points from the facial image. The key points include the eyes, nose, and mouth. The extraction tool was used to locate these key points using a deep learning-based algorithm.

[0072] Based on the extracted key points, the area below the eyebrow line and above the lower lip line is selected as the core area of ​​the face. A mask image is generated by connecting the key points of the area. The mask image is a binary image in which the pixel value is in the range of [0,1] and the value of each pixel is 0 or 1. The perturbation is only performed inside the mask area.

[0073] In this embodiment, facial landmark extraction begins with the dlib tool. The dlib tool uses a facial landmark generation algorithm to extract 68 landmarks from the face image, covering areas such as the eyes, nose, and mouth. To ensure the accuracy of landmark extraction, images with poor visual quality or those where the dlib tool cannot correctly identify the face region are removed. These images typically contain significant noise or have occlusions that prevent effective facial recognition, thus affecting the accuracy of feature extraction.

[0074] In the data preprocessing stage, the MTCNN (Multi-task Cascaded Convolutional Networks) model is used for face detection and alignment. The MTCNN model first detects facial regions in each input image and then aligns these regions to ensure the standardization of faces within the image. The aligned images are then cropped and resized to 112×112 pixels to meet the input requirements of subsequent models.

[0075] Generation of the mask image and limitation of the perturbation region: For the 68 extracted key points, this method selects the area below the eyebrow line and above the lower lip line in the face image as the core facial region for perturbation. By connecting the key points within these regions, a corresponding mask image is generated. This mask image is a binary image in the range [0,1], with pixel values ​​of 0 or 1, where 1 represents the perturbated area and 0 represents the unperturbed area. It is important to note that this method only applies perturbation within the mask region to ensure the integrity and naturalness of the non-masked areas in the image.

[0076] To ensure the diversity and challenge of the data, the LFW (Labeled Faces in the Wild) dataset was used. The LFW dataset contains approximately 13,000 face images of different individuals, covering 5,749 different individuals. The face images in the dataset exhibit strong diversity, involving complex transformations such as different poses, expressions, and lighting, presenting a certain challenge. The images in the LFW dataset also contain variations such as facial rotation and occlusion; these features make this dataset ideal for evaluating the robustness of face recognition systems.

[0077] (2) Based on the face mask image, an adversarial perturbation is generated in the mask area using a gradient-based method. The adversarial perturbation affects the target image by adjusting the pixel values ​​of the mask area.

[0078] Specifically, the method of generating adversarial perturbations within the masked region using a gradient-based approach includes the following steps:

[0079] Using a gradient-based method, adversarial perturbations are generated within the masked region by combining a [0,1] binary mask of the face image. The formula is as follows:

[0080] x adv =x+ò×sign(▽ x L(x,x t ));

[0081] Where: x adv This is an adversarial example; x is the input image; x t t represents the target image; ò represents the step size of the perturbation; ▽ x L(x,xt ) represents the gradient of the loss function with respect to the input image x;

[0082] Combining the masking mechanism, the perturbation area is limited to the masked area. A dynamic perturbation range method is used to gradually reduce the perturbation from the center of the masked area towards the edge, as shown in the formula:

[0083]

[0084] in: The adversarial example generated for the next iteration; α is the step size for each iteration; clip x,ò () is used to constrain adversarial examples to satisfy l ∞ Norm; The adversarial sample generated in the t-th iteration;

[0085] The Momentum Iterative FGSM (MIM) method is introduced to add a momentum term to the gradient-based perturbation generation method, optimizing the gradient update process. The formula is as follows:

[0086]

[0087] Wherein: g t The current gradient; g t+1 The gradient is the updated gradient; μ is the momentum factor. It is the L1 norm of the gradient;

[0088] The Diverse Input Iterative FGSM (DIM) method is employed to enhance the transferability of adversarial examples by randomly transforming the input. The formula is as follows:

[0089]

[0090] Where: T() represents the random transformation function; p represents the probability of performing the transformation.

[0091] In this example, BIM, MIM, and DIM methods are used to generate adversarial examples. Based on the masked image, adversarial perturbations are generated using existing gradient-based methods. Since the objectives and loss functions of face recognition tasks differ from those of general image classification tasks, the algorithms used to generate adversarial examples also differ. Three common methods are briefly introduced below:

[0092] Fast Gradient Sign Method (FGSM) generates adversarial examples from the original image; Basic Iterative Method (BIM) is an iterative version of FGSM, updating the perturbation through multiple iterations during adversarial example generation; Momentum Iterative FGSM (MIM) adds a momentum term to BIM, improving the stability and effectiveness of the attack by introducing local optima prevention during the optimization process; Diverse Input Iterative FGSM (DIM) enhances the transferability of adversarial examples by randomly transforming the input; (The last sentence appears to be a separate, unrelated statement about enhancing the transferability of adversarial examples through input transformation.)

[0093] Several methods were used to combine a face image mask [0,1] to generate suitable regions and image perturbations. First, a perturbation mechanism was introduced to control the size of the perturbation region, ensuring that the image is perturbed only in the core area while the background area remains unchanged. To further reduce the impact of the perturbation on the visual effect, a dynamic perturbation range method was adopted, gradually decreasing the perturbation from the center of the mask region towards the edge, avoiding unnatural artifacts.

[0094] During processing, the MTCNN model was used to detect and align the input image, ensuring the standardization of face regions. By dynamically adjusting the perturbation region, it can be ensured that the generated adversarial examples are not only highly aggressive but also maintain a high degree of visual naturalness.

[0095] This optimized description clearly demonstrates how to combine existing adversarial example generation methods to generate adversarial perturbations within the masked region using methods such as BIM, MIM, and DIM, and further enhance the transferability of adversarial examples through input transformation.

[0096] (3) Design a weighted perturbation smoothing loss function based on Gaussian function to control the perturbation intensity in the mask region during backpropagation, so that the perturbation at the edge of the mask region gradually decreases.

[0097] Specifically, the design is based on a Gaussian function-based weighted perturbation smoothing loss function, which controls the perturbation intensity within the mask region during backpropagation. This includes the following steps:

[0098] Determine the center point of the face mask region and calculate the distance between each pixel within the mask region and the center point. Calculate the weight of the perturbation based on the distance, and then apply a Gaussian function to weight the perturbation, as shown in the following formula:

[0099]

[0100] Where: i, j are the coordinates of the pixel; c x cy The coordinates of the center point of the mask region;

[0101] The perturbation weights of pixels are calculated using a Gaussian function, with the weights decreasing in regions farther from the mask center, as shown in the following formula:

[0102]

[0103] in: σ W is the standard deviation of the Gaussian function. max This represents the maximum weight value of the Gaussian function;

[0104] Weighted perturbation is applied to each region of the image, preserving the perturbation intensity in the central region while reducing the perturbation intensity in the edge region, thereby optimizing image quality;

[0105] By introducing dynamically adjusted perturbation intensity, combined with Gaussian function and weight calculation, the adversarial example generation process of images is improved. The formula is as follows:

[0106]

[0107] Where: P′ i,j The perturbation is weighted; P i,j M represents the initial perturbation of pixel x(i,j); i,j A face [0,1] mask; P′ i+1,j To be related to the current pixel P′ i,j The perturbation value of adjacent vertical pixels; P′ i,j+1 To be related to the current pixel P′ i,j The perturbation value of adjacent horizontal pixels.

[0108] (4) Extract feature information from the target image, generate adversarial samples, and attack the target model.

[0109] Specifically, generating adversarial examples and attacking the target model includes the following steps:

[0110] The cosine similarity loss is combined to measure the similarity between the target image and the adversarial example. The loss function is constructed by calculating the similarity between the two.

[0111] Based on backpropagation of loss, adversarial examples are generated through multiple iterations to attack the target model. In each iteration, the generated adversarial examples are optimized using the loss function to adjust the perturbation to a predetermined position.

[0112] By combining cosine similarity loss and perturbation loss, a loss function is constructed, and the generation process of adversarial examples is optimized by adjusting the weights of the two.

[0113] In this embodiment, adversarial examples are generated based on the features of the target image, combined with a loss function and an adversarial example generation strategy. The specific steps are as follows:

[0114] 4.1 Cosine Similarity Loss Function

[0115] First, the cosine similarity loss function is used to measure the similarity between the target image and the adversarial example. The cosine similarity loss function is used to calculate the similarity between the two to ensure that the generated adversarial example maintains visual similarity to the target image as much as possible without affecting the effectiveness of the adversarial attack. The specific formula is:

[0116]

[0117] Where: x adv Indicates the generated adversarial example; x t Represents the original input image; ||x adv || represents the vector norm of the adversarial example; ||x t || represents the vector norm of the input image x.

[0118] The loss function calculates the cosine of the angle between the input image and the adversarial example; the closer the value is to 1, the higher the similarity between the two.

[0119] 4.2 Final Loss Function

[0120] The final form of the loss function combines the cosine similarity loss and the perturbation loss, as shown below:

[0121] L total =L cos +βL pert ;

[0122] Where: L total L represents the cosine similarity loss function, used to measure the similarity between the original image and the adversarial example; pert β represents the perturbation loss function, which measures the perturbation intensity of adversarial examples to ensure that the perturbation is not too large and avoids excessive changes to the visual effect of the image; β is the balance parameter in the loss function, which controls the relative weight between the cosine similarity loss and the perturbation loss, and adjusts the influence of the two.

[0123] 4.3 Attack Success Rate (ASR) and Evaluation Metrics

[0124] To evaluate the effectiveness of the generated adversarial examples, Attack Success Rate (ASR) is used as the evaluation metric. ASR quantifies the attack effectiveness by calculating the proportion of times the generated adversarial examples successfully attack the target model, as shown in the following formula:

[0125]

[0126] Where: ASR represents Attack Success Rate, which measures the proportion of adversarial samples that successfully attack the target model, and indicates the proportion of times the generated adversarial samples can successfully induce the target model to produce incorrect output; N represents Attack Success Rate, which measures the proportion of times the adversarial samples successfully attack the target model, and indicates the proportion of times the generated adversarial samples can successfully induce the target model to produce incorrect output. This indicates that the target model is working on the i-th adversarial example. The predicted output; This represents the cosine similarity between the original target image and the prediction result of the adversarial example. Cosine similarity measures the similarity between two vectors in a direction; the smaller the value, the stronger the attack effect of the adversarial example on the target model. τ represents a preset threshold; when the cosine similarity is below this threshold, the adversarial example is considered to have successfully attacked the target model.

[0127] This formula calculates the Attack Success Rate (ASR), which is the proportion of times a generated adversarial example can successfully induce the target model to make incorrect predictions. For each adversarial example, the cosine similarity between it and the target image is calculated. If the cosine similarity is less than a preset threshold τ, the adversarial example is considered to have successfully attacked the target model. The final attack success rate is calculated from the results of all samples and expressed as a percentage.

[0128] 4.4 Image Quality Assessment Indicators

[0129] In addition to attack success rate, metrics such as SSIM (structural similarity), PSNR (peak signal-to-noise ratio), and LPIPS (perceptual similarity) were used to evaluate the image quality of generated adversarial examples.

[0130] SSIM (Structural Similarity): Used to measure the similarity between two images in terms of structure and brightness. The formula is as follows:

[0131]

[0132] Where: μ x and μ y These are the means of the x and y values ​​of the image, respectively. and Its variance; σ xy Its covariance;

[0133] μ x μ y C1 is the product of the means of the input image and the reference image, representing the brightness similarity between the two; C2 and C1 are two constants used to prevent the denominator from being zero, and are usually set to small constant values.

[0134] The value range of SSIM is [-1, 1], where 1 indicates that the two images are completely identical, 0 indicates that the two images have no similarity, and negative values ​​indicate inverse correlation.

[0135] PSNR (Peak Signal-to-Noise Ratio) is a metric that quantifies the difference between a reconstructed image and the original benign image. It is frequently used to evaluate the effectiveness of image processing techniques, including image compression, transmission, and denoising. PSNR is quantified in decibels (dB); a higher value indicates better image quality and a high degree of similarity between the reconstructed and original images.

[0136] LPIPS (Learned Perceptual Image Patch Similarity) is based on a deep learning model that learns human perceptual characteristics to calculate differences between images. It extracts image features using pre-trained convolutional neural networks (e.g., AlexNet, VGGNet, SqueezeNet). The extracted features are normalized, and then the normalized features are compared layer by layer to determine the differences between two images at each feature layer. The L2 distance between feature vectors is calculated to complete this stage. Finally, a weighted average of the estimated feature distances at each layer is used to generate the LPIPS score. By leveraging the feature extraction capabilities of deep neural networks and taking into account the complexity of the human visual system, LPIPS outperforms current image similarity metrics such as PSNR and SSIM. This allows LPIPS to more accurately capture subtle perceptual details in images. LPIPS scores typically range from 0 to 1, with scores closer to 0 indicating higher perceptual similarity between the two images.

[0137] To evaluate the effectiveness of the proposed method, comparative analyses were conducted using benchmark methods BIM, MIM, and DIM, focusing on attack success rate and multiple visual quality standards. ArcFace was used as a white-box model to generate adversarial examples for attacking other models. The relevant experimental results are shown below. BIM-MP represents a combination of the proposed improved method and traditional BIM methods.

[0138] Table 1. Attack success rate (%) for different face recognition models

[0139]

[0140] When using the method proposed in this invention, the attack success rate (ASR) of adversarial examples generated by the ArcFace model is improved. For example, adversarial instances generated using the DIM-MP method of the ArcFace model successfully attacked the FaceNet model with a success rate of 75.02%. When FaceNet is used as an alternative model and MIM-MP and DIM-MP are implemented to generate adversarial examples for attacking MobileFace and SphereFace, the attack success rate decreases slightly, but by a very small margin.

[0141] The table below provides a quantitative evaluation of image quality, showing the values ​​of SSIM, PSNR, and LPIPS for adversarial examples generated based on the ArcFace model and different methods. SSIM and PSNR increase with improving image quality, while LPIPS decreases with improving visual quality.

[0142] Table 2. SSIM, PSNR, and LPIPS of adversarial examples generated by different methods.

[0143]

[0144] As shown in Table 2, the method proposed in this invention combines these three methods, significantly improving SSIM and PSNR while reducing the LPIPS index, indicating an improvement in the visual quality of all methods. SSIM is an index that measures the visual similarity between two images; the closer its value is to 1, the more similar the images are and the smaller the visual difference. The visual quality of the image improves as the LPIPS index decreases. Specifically, using MIM-MP, the SSIM index of the adversarial examples generated by FaceNet increased from 0.8424 to 0.9477, the PSNR increased from 32.4327 to 36.8329, and the LPIPS decreased from 0.1872 to 0.0889. This demonstrates that by combining gradient-based methods, the method of this invention can effectively improve the visual quality of adversarial examples. This improvement in visual quality is achieved without sacrificing the transferability of adversarial examples.

[0145] Example 2

[0146] like Figure 3 As shown, another embodiment of the present invention provides a natural face adversarial example generation system based on face masking, characterized in that it includes:

[0147] The feature point extraction module is used to extract feature point information from a face image and generate a corresponding face mask image by connecting the feature point information. The face mask image covers a specific area of ​​the face and only perturbs within the mask area. The adversarial perturbation generation module is used to generate adversarial perturbations within the mask area based on the face mask image using a gradient-based method. The adversarial perturbations affect the target image by adjusting the pixel values ​​of the mask area.

[0148] The perturbation smoothing control module is used to design a weighted perturbation smoothing loss function based on a Gaussian function, and to control the perturbation intensity in the mask region during backpropagation so that the perturbation at the edge of the mask region gradually decreases.

[0149] The target image feature extraction module is used to extract feature information from the target image and generate adversarial examples to attack the target model.

[0150] Specifically, the feature point extraction module includes:

[0151] The feature point extraction unit is used to extract feature point information in a face image and extract 68 key points in the face image using a deep learning-based algorithm. The key points include the eyes, nose, and mouth.

[0152] The mask generation unit is used to select the area below the eyebrow line and above the lower lip line as the core area of ​​the face based on the extracted key points, and generate a mask image by connecting the key points of the area. The mask image is a binary image.

[0153] Specifically, the anti-disturbance generation module includes:

[0154] The gradient calculation unit is used to generate adversarial perturbations in the mask region based on the face mask image using a gradient-based method. The adversarial perturbations affect the target image by adjusting the pixel values ​​of the mask region.

[0155] A dynamic perturbation adjustment unit is used to gradually reduce the perturbation from the center of the mask area to the edge;

[0156] The momentum iterative update unit is used to introduce the MIM method, adding a momentum term during the perturbation generation process to optimize the gradient update process;

[0157] The input transformation unit is used to perform random transformations on the input using the DIM method to enhance the transferability of generated adversarial examples.

[0158] Specifically, the disturbance smoothing control module includes:

[0159] A Gaussian weighted unit is used to calculate the distance of each pixel based on the center point of the face mask region and to calculate the weight of the perturbation using a Gaussian function.

[0160] The perturbation distribution adjustment unit is used to perform weighted perturbation on each region of the image, retaining the perturbation intensity of the central region while reducing the perturbation intensity of the edge region;

[0161] The dynamic adjustment unit is used to dynamically adjust the perturbation intensity based on the characteristics of the image and the result of Gaussian weighting.

[0162] In summary, this invention effectively solves the problems of excessively large perturbation regions and unnatural visual effects in adversarial example generation in existing technologies by precisely controlling the perturbation region and introducing a Gaussian-based perturbation smoothing loss function. This method not only optimizes the visual quality of the image when generating adversarial perturbations but also improves the concealment of the perturbations, ensuring a balance between attack effectiveness and image naturalness. Furthermore, by combining gradient updates, momentum methods, and input transformation techniques, this invention enhances the attack success rate and transferability of the generated adversarial examples, improving the robustness and security of face recognition systems against adversarial attacks. This invention has broad application prospects; it can not only improve the security of existing face recognition systems but also provide an effective defense mechanism for other image recognition-based systems, ensuring the stability and security of the system in practical applications. Through these innovative technologies, this invention will provide a more feasible and efficient solution for adversarial attack defense in the field of face recognition, possessing high market value and application potential.

[0163] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this application. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of those different embodiments or examples.

[0164] Any process or method description in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or more executable instructions for implementing a particular logical function or process. Furthermore, the scope of the preferred embodiments of this application includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the functionality involved.

[0165] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any person skilled in the art can easily conceive of various variations or substitutions within the technical scope disclosed in this application, and these should all be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for generating natural adversarial examples of faces based on face masks, characterized in that, Includes the following steps: Feature point information is extracted from a face image, and a corresponding face mask image is generated by connecting the feature point information. The face mask image covers a specific area of ​​the face, and perturbation is only performed on the mask area. Based on the face mask image, an adversarial perturbation is generated in the mask region using a gradient-based method. The adversarial perturbation affects the target image by adjusting the pixel values ​​of the mask region. Design a weighted perturbation smoothing loss function based on Gaussian function to control the perturbation intensity in the mask region during backpropagation, so that the perturbation at the edge of the mask region gradually decreases. Extract feature information from the target image, generate adversarial examples, and attack the target model.

2. The method for generating natural adversarial examples of faces based on face masking according to claim 1, characterized in that, The process of generating the corresponding face mask image includes the following steps: A facial feature point extraction tool was used to extract 68 key points from the facial image. The key points include the eyes, nose, and mouth. The extraction tool was used to locate these key points using a deep learning-based algorithm. Based on the extracted key points, the area below the eyebrow line and above the lower lip line is selected as the core area of ​​the face. A mask image is generated by connecting the key points of the area. The mask image is a binary image in which the pixel value is in the range of [0,1] and the value of each pixel is 0 or 1. The perturbation is only performed inside the mask area.

3. The method for generating natural adversarial examples of faces based on face masking according to claim 1, characterized in that, The method of generating adversarial perturbations within the masked region using a gradient-based approach includes the following steps: Using a gradient-based method, adversarial perturbations are generated within the masked region by combining a [0,1] binary mask of the face image. The formula is as follows: Where: x adv This is an adversarial example; x is the input image; x t For the target image; The step size of the perturbation; The gradient of the loss function with respect to the input image x; Combining the masking mechanism, the perturbation area is limited to the masked area. A dynamic perturbation range method is used to gradually reduce the perturbation from the center of the masked area towards the edge, as shown in the formula: in: The adversarial example generated for the next iteration; α is the step size for each iteration; Used to constrain adversarial examples to satisfy l ∞ Norm; The adversarial sample generated in the t-th iteration; The MIM method is introduced, which adds a momentum term to the gradient-based perturbation generation method to optimize the gradient update process. The formula is as follows: Wherein: g t The current gradient; g t+1 The gradient is the updated gradient; μ is the momentum factor. It is the L1 norm of the gradient; The DIM method enhances the transferability of adversarial examples by randomly transforming the input. The formula is as follows: Where: T() represents the random transformation function; p represents the probability of performing the transformation.

4. The method for generating natural adversarial examples of faces based on face masking according to claim 1, characterized in that, The design is based on a Gaussian-based weighted perturbation smoothing loss function, which controls the perturbation intensity within the masked region during backpropagation. Specifically, it includes the following steps: Determine the center point of the face mask region and calculate the distance between each pixel within the mask region and the center point. Calculate the weight of the perturbation based on the distance, and then apply a Gaussian function to weight the perturbation, as shown in the following formula: Where: i, j are the coordinates of the pixel; c x c y The coordinates of the center point of the mask region; The perturbation weights of pixels are calculated using a Gaussian function, with the weights decreasing in regions farther from the mask center, as shown in the following formula: Where: σ is the standard deviation of the Gaussian function; W max The maximum weight value; Weighted perturbation is applied to each region of the image, preserving the perturbation intensity in the central region while reducing the perturbation intensity in the edge region, thereby optimizing image quality; By introducing dynamically adjusted perturbation intensity, combined with Gaussian function and weight calculation, the adversarial example generation process of images is improved. The formula is as follows: P′ i,j =W i,j ×P i,j ×M i,j ; Where: P′ i,j The perturbation is weighted; P i,j M represents the initial perturbation of pixel x(i,j); i,j A face [0,1] mask; P′ i+1,j To be related to the current pixel P′ i,j The perturbation value of adjacent vertical pixels; P′ i,j+1 To be related to the current pixel P′ i,j The perturbation value of adjacent horizontal pixels.

5. The method for generating natural adversarial examples of faces based on face masking according to claim 1, characterized in that, The process of generating adversarial examples and attacking the target model specifically includes the following steps: The cosine similarity loss is combined to measure the similarity between the target image and the adversarial example. The loss function is constructed by calculating the similarity between the two. Based on backpropagation of loss, adversarial examples are generated through multiple iterations to attack the target model. In each iteration, the generated adversarial examples are optimized using the loss function to adjust the perturbation to a predetermined position. By combining cosine similarity loss and perturbation loss, a loss function is constructed, and the generation process of adversarial examples is optimized by adjusting the weights of the two.

6. A system for generating natural adversarial examples of faces based on face masking as described in claim 1, characterized in that, include: The feature point extraction module is used to extract feature point information from a face image and generate a corresponding face mask image by connecting the feature point information. The face mask image covers a specific area of ​​the face and only perturbs within the mask area. The adversarial perturbation generation module is used to generate adversarial perturbations within the mask area based on the face mask image using a gradient-based method. The adversarial perturbations affect the target image by adjusting the pixel values ​​of the mask area. The perturbation smoothing control module is used to design a weighted perturbation smoothing loss function based on a Gaussian function, and to control the perturbation intensity in the mask region during backpropagation so that the perturbation at the edge of the mask region gradually decreases. The target image feature extraction module is used to extract feature information from the target image and generate adversarial examples to attack the target model.

7. The natural face adversarial example generation system based on face masking according to claim 6, wherein, The feature point extraction module includes: The feature point extraction unit is used to extract feature point information in a face image and extract 68 key points in the face image using a deep learning-based algorithm. The key points include the eyes, nose, and mouth. The mask generation unit is used to select the area below the eyebrow line and above the lower lip line as the core area of ​​the face based on the extracted key points, and generate a mask image by connecting the key points of the area. The mask image is a binary image.

8. The natural face adversarial example generation system based on face masking according to claim 6, wherein, The anti-disturbance generation module includes: The gradient calculation unit is used to generate adversarial perturbations in the mask region based on the face mask image using a gradient-based method. The adversarial perturbations affect the target image by adjusting the pixel values ​​of the mask region. A dynamic perturbation adjustment unit is used to gradually reduce the perturbation from the center of the mask area to the edge; The momentum iterative update unit is used to introduce the MIM method, adding a momentum term during the perturbation generation process to optimize the gradient update process; The input transformation unit is used to perform random transformations on the input using the DIM method to enhance the transferability of generated adversarial examples.

9. The natural face adversarial example generation system based on face masking according to claim 6, wherein, The disturbance smoothing control module includes: A Gaussian weighted unit is used to calculate the distance of each pixel based on the center point of the face mask region and to calculate the weight of the perturbation using a Gaussian function. The perturbation distribution adjustment unit is used to perform weighted perturbation on each region of the image, retaining the perturbation intensity of the central region while reducing the perturbation intensity of the edge region; The dynamic adjustment unit is used to dynamically adjust the perturbation intensity based on the characteristics of the image and the result of Gaussian weighting.

Images