Face adversarial sample generation method, device, system and storage medium
By using a gradient iteration method and a preset area mask to generate perturbation images in face recognition technology, the problem of high complexity in black-box attacks is solved, achieving efficient face adversarial attacks and reducing computational complexity.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- CHINA MERCHANTS BANK
- Filing Date
- 2022-10-20
- Publication Date
- 2026-06-05
AI Technical Summary
Existing adversarial face attack methods generally perform poorly in black-box attack scenarios and have high computational complexity. Traditional methods usually require adding perturbations to the entire image, which increases computational complexity.
By acquiring original face images, fusion feature learning is performed based on a pre-trained face recognition model. Combined with a mask of a preset area size, perturbation images are generated using a gradient iteration method, adding perturbations only to small regions to generate face adversarial attack samples.
It reduces the implementation complexity of adversarial face attack methods, is applicable to both white-box and black-box attacks, requires no querying or model migration, and improves attack efficiency.
Smart Images
Figure CN115798056B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of facial recognition technology, and in particular to a method, apparatus, system, and storage medium for generating adversarial examples of faces. Background Technology
[0002] Facial recognition technology is now widely used across various industries, and there are multiple recognition methods. However, various attack methods targeting facial recognition technology are constantly emerging, jeopardizing the security of facial recognition computing. Therefore, researchers aim to improve the security of facial recognition algorithms by studying attack methods, and adversarial attacks on faces are one of the most popular topics in this field.
[0003] While many adversarial face attack methods exist in the industry, most have achieved good results in white-box attacks (where all information about the model is known), but their performance is generally poor in black-box attacks (where the model's structure and specific parameters are unknown). Furthermore, most face adversarial attack methods use queries to add perturbations to the entire image to increase the success rate, but this introduces higher computational complexity. Summary of the Invention
[0004] The main objective of this invention is to provide a method, apparatus, system, and storage medium for generating adversarial samples of faces, aiming to reduce the implementation complexity of adversarial face attack methods.
[0005] To achieve the above objectives, the present invention provides a method for generating adversarial examples of faces, the method comprising the following steps:
[0006] Obtain the original face image, which includes: the original attacked face image and the original attacking face image;
[0007] Based on a pre-trained face recognition model, the original attacked face image and the original attacking face image are fused feature learning, and a perturbation image is generated by combining a pre-generated mask of a preset area size with a gradient iteration method.
[0008] The perturbation image is added to the original attack face image to generate a face adversarial attack sample.
[0009] Optionally, the step of generating a perturbation image using a gradient iteration method based on a pre-trained face recognition model, which involves learning fusion features between the original attacked face image and the original attacking face image, and combining this with a pre-generated mask of a preset area size, includes:
[0010] The original attacked face image is input into a pre-trained multi-face recognition model for feature extraction. The extracted multiple attacked face features are fused to obtain the first fused feature of the original attacked face image.
[0011] The original attack face image is input into a pre-trained multi-face recognition model for feature extraction, and the gradient in the model parameters of the original attack face image and the extracted multiple attack face features are obtained. The extracted multiple attack face features are fused to obtain the second fused feature of the original attack face image.
[0012] Calculate the loss function based on the first fusion feature and the second fusion feature;
[0013] Based on the loss function, gradient, and a pre-generated mask of a preset area size, a perturbation image is generated using a gradient iteration method.
[0014] Optionally, the step of generating a perturbation image using a gradient iteration method based on the loss function, gradient, and a pre-generated mask of a preset area size includes:
[0015] Calculate the mean of the absolute values of the gradient of the original attack face image in each RGB channel;
[0016] Calculate the average gradient based on the mean;
[0017] Based on the average gradient and combined with a mask of a pre-generated area of a preset size, the original attack face image is updated to obtain the updated attack face image.
[0018] The updated attack face images are normalized.
[0019] The gradient of the updated attack face image is updated according to the loss function; the next iteration begins, and the iteration count is repeated until a preset number of iterations is reached. The final updated attack face image is then used as the perturbation image.
[0020] Optionally, before the step of inputting the original attack face image into a pre-trained multi-face recognition model for feature extraction, the method further includes:
[0021] The original attack face image is preprocessed using at least one of the following methods:
[0022] The original attack face image is randomly transformed using input diversity;
[0023] Randomly generated fine noise is added to the original attack face image;
[0024] An affine transformation is performed on the original attack face image.
[0025] Optionally, the step of obtaining the original face image includes:
[0026] Obtain the original images of the people being attacked and the images of the people attacking the attacking person.
[0027] The original human image is subjected to face detection, alignment, and normalization to obtain the original human face image.
[0028] Optionally, before the step of generating a perturbation image using a gradient iteration method based on a pre-trained face recognition model, which performs fusion feature learning on the original attacked face image and the original attacking face image and combines it with a pre-generated mask of a preset area size, the method further includes:
[0029] The feature heatmaps of multiple face recognition models are obtained by performing feature recognition on face sample images using multiple face recognition models.
[0030] The feature heatmaps of multiple face recognition models are analyzed to determine the corresponding sensitive areas of face features.
[0031] Based on the facial feature sensitive area and the preset attack scenario, a mask of a preset area size is generated.
[0032] Optionally, the method further includes:
[0033] The face adversarial attack sample was tested and verified using a test model, and the test results were obtained.
[0034] Furthermore, embodiments of the present invention also propose a face adversarial sample generation device, the device comprising:
[0035] The acquisition module is used to acquire the original face image, which includes: the original attacked face image and the original attacking face image;
[0036] The perturbation generation module is used to perform fusion feature learning on the original attacked face image and the original attacking face image based on the pre-trained face recognition model, and generate perturbation images by combining a pre-generated mask of a preset area size and using a gradient iteration method.
[0037] An add module is used to add the perturbation image to the original attack face image to generate a face adversarial attack sample.
[0038] Furthermore, this invention also proposes a face adversarial sample generation system, which includes: a memory, a processor, and a face adversarial sample generation program stored in the memory and executable on the processor. When the face adversarial sample generation program is executed by the processor, it implements the steps of the face adversarial sample generation method described above.
[0039] Furthermore, embodiments of the present invention also propose a computer-readable storage medium storing a face adversarial example generation program, which, when executed by a processor, implements the steps of the face adversarial example generation method described above.
[0040] This invention proposes a method, apparatus, system, and storage medium for generating adversarial examples of faces. The method involves acquiring an original face image, including an original target face image and an original attacker face image. Based on a pre-trained face recognition model, the method performs fusion feature learning on the target and attacker face images and, combined with a pre-generated mask of a preset area size, generates a perturbation image using a gradient iteration method. The perturbation image is then added to the attacker face image to generate an adversarial attack example. This solution utilizes a pre-generated mask of a preset area size, achieving adversarial attacks by adding perturbation to a small region, avoiding the need to perturb the entire image. This solution is applicable to both white-box and black-box attacks, eliminating the need for querying and model transfer, thus reducing the implementation complexity of adversarial attack methods. Compared to traditional methods that often require perturbation of the entire image, resulting in high time complexity, this method can achieve attacks using only small-region perturbation. Attached Figure Description
[0041] Figure 1 This is a schematic diagram of the functional modules of the terminal device to which the face adversarial sample generation device of the present invention belongs;
[0042] Figure 2 This is a flowchart illustrating the first embodiment of the face adversarial sample generation method of the present invention;
[0043] Figure 3 This is a flowchart illustrating the second embodiment of the face adversarial sample generation method of the present invention;
[0044] Figure 4 This is a schematic diagram of the mask involved in an embodiment of the face adversarial sample generation method of the present invention;
[0045] Figure 5 This is a flowchart illustrating the third embodiment of the face adversarial sample generation method of the present invention;
[0046] Figure 6 This is a schematic diagram of the overall process of an embodiment of the face adversarial sample generation method of the present invention.
[0047] The realization of the objective, functional features and advantages of the present invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation
[0048] It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.
[0049] The main solution of this invention is as follows: First, obtain original face images, including original attacked face images and original attacking face images. Second, based on a pre-trained face recognition model, perform fusion feature learning on the original attacked face images and the original attacking face images, and combine this with a pre-generated mask of a preset area size to generate a perturbation image using a gradient iteration method. Third, add the perturbation image to the original attacking face image to generate a face adversarial attack sample. This solution, by using a pre-generated mask of a preset area size, achieves face adversarial attacks by adding perturbation to a small region, avoiding the need to perturb the entire image. This solution is applicable to both white-box and black-box attacks, requiring no querying or model transfer, thus reducing the implementation complexity of face adversarial attack methods. Compared to traditional solutions that often require perturbation of the entire image, resulting in high time complexity, this face adversarial sample generation method can achieve the attack by perturbating only a small region.
[0050] This invention takes into account that while many adversarial face attack methods exist in the industry, most have achieved good results in white-box attacks (where all information about the model is known), but their performance is generally poor in black-box attacks (where the model's structure and specific parameters are unknown). Furthermore, most adversarial face attack methods use queries to add perturbations to the entire image to improve the success rate, but this introduces higher computational complexity.
[0051] Based on this, embodiments of the present invention provide a solution that can reduce the implementation complexity of face adversarial attack methods.
[0052] Specifically, refer to Figure 1 , Figure 1 This is a schematic diagram of the functional modules of the terminal device to which the face adversarial example generation device of the present invention belongs. The face adversarial example generation device can be a device independent of the terminal device, and it can be implemented on the terminal device or system in the form of hardware or software. The terminal device can be a smart mobile terminal such as a mobile phone or tablet computer, or a network device such as a server.
[0053] In this embodiment, the terminal device to which the face adversarial sample generation device belongs includes at least an output module 110, a processor 120, a memory 130, and a communication module 140.
[0054] The memory 130 stores the operating system and the face adversarial sample generation program; the output module 110 can be a display screen, speaker, etc. The communication module 140 can include a WIFI module, a mobile communication module, and a Bluetooth module, etc., and communicates with external devices or servers through the communication module 140.
[0055] In one embodiment, when the face adversarial example generation program in memory 130 is executed by the processor, it performs the following steps:
[0056] Obtain the original face image, which includes: the original attacked face image and the original attacking face image;
[0057] Based on a pre-trained face recognition model, the original attacked face image and the original attacking face image are fused feature learning, and a perturbation image is generated by combining a pre-generated mask of a preset area size and using a gradient iteration method.
[0058] The perturbation image is added to the original attack face image to generate a face adversarial attack sample.
[0059] Furthermore, when the face adversarial example generation program in memory 130 is executed by the processor, it also performs the following steps:
[0060] The original attacked face image is input into a pre-trained multi-face recognition model for feature extraction. The extracted multiple attacked face features are fused to obtain the first fused feature of the original attacked face image.
[0061] The original attack face image is input into a pre-trained multi-face recognition model for feature extraction, and the gradient in the model parameters of the original attack face image and the extracted multiple attack face features are obtained. The extracted multiple attack face features are fused to obtain the second fused feature of the original attack face image.
[0062] Calculate the loss function based on the first fusion feature and the second fusion feature;
[0063] Based on the loss function, gradient, and a mask of a pre-generated area, a perturbation image is generated using a gradient iteration method.
[0064] Furthermore, when the face adversarial example generation program in memory 130 is executed by the processor, it also performs the following steps:
[0065] Calculate the mean of the absolute values of the gradient of the original attack face image in each RGB channel;
[0066] Calculate the average gradient based on the mean;
[0067] Based on the average gradient and combined with a mask of a pre-generated area of a preset size, the original attack face image is updated to obtain the updated attack face image.
[0068] The updated attack face images are normalized.
[0069] The gradient of the updated attack face image is updated according to the loss function; the next iteration begins, and the iteration count continues until the preset number of iterations is reached.
[0070] Furthermore, when the face adversarial example generation program in memory 130 is executed by the processor, it also performs the following steps:
[0071] The original attack face image is preprocessed using at least one of the following methods:
[0072] The original attack face image is randomly transformed using input diversity;
[0073] Randomly generated fine noise is added to the original attack face image;
[0074] An affine transformation is performed on the original attack face image.
[0075] Furthermore, when the face adversarial example generation program in memory 130 is executed by the processor, it also performs the following steps:
[0076] Obtain the original images of the people being attacked and the images of the people attacking the attacking person.
[0077] The original human image is subjected to face detection, alignment, and normalization to obtain the original human face image.
[0078] Furthermore, when the face adversarial example generation program in memory 130 is executed by the processor, it also performs the following steps:
[0079] The feature heatmaps of multiple face recognition models are obtained by performing feature recognition on face sample images using multiple face recognition models.
[0080] The feature heatmaps of multiple face recognition models are analyzed to determine the corresponding sensitive areas of face features.
[0081] Based on the facial feature sensitive area and the preset attack scenario, a mask of a preset area size is generated.
[0082] Furthermore, when the face adversarial example generation program in memory 130 is executed by the processor, it also performs the following steps:
[0083] The face adversarial attack sample was tested and verified using a test model, and the test results were obtained.
[0084] This embodiment, through the above-described scheme, specifically obtains original face images, including an original target face image and an original attacker face image. Based on a pre-trained face recognition model, it performs fusion feature learning on the original target face image and the original attacker face image, and combines this with a pre-generated mask of a preset area size, using a gradient iteration method to generate a perturbation image. The perturbation image is then added to the original attacker face image to generate a face adversarial attack sample. This scheme, by using a pre-generated mask of a preset area size, achieves face adversarial attacks by adding perturbation to a small region, avoiding the need to perturb the entire image. This scheme is applicable to both white-box and black-box attacks, requiring no querying or model transfer, thus reducing the implementation complexity of the face adversarial attack method. Compared to traditional schemes that often require perturbation of the entire image, resulting in high time complexity, this scheme's face adversarial sample generation method can achieve the attack by perturbating only a small region.
[0085] Based on, but not limited to, the terminal device architecture described above, embodiments of the method of the present invention are proposed.
[0086] The execution subject of the method in this embodiment can be a face adversarial example generation device. This face adversarial example generation device can be a device independent of the terminal device, and it can be carried on the terminal device or system in the form of hardware or software. The terminal device can be a smart mobile terminal such as a mobile phone or tablet computer, or it can be a network device such as a server.
[0087] The face adversarial sample generation method of the present invention can be applied to face adversarial attack methods in various face recognition scenarios to reduce the implementation complexity of face adversarial attack methods and improve the security of face recognition algorithms through face adversarial attacks.
[0088] Reference Figure 2 , Figure 2 This is a flowchart illustrating the first embodiment of the face adversarial sample generation method of the present invention.
[0089] like Figure 2 As shown in the embodiment of the present invention, a method for generating adversarial examples of faces includes the following steps:
[0090] Step S101: Obtain the original face image, which includes: the original attacked face image and the original attacking face image;
[0091] The original face images are pre-collected and processed face images, which can be extracted from images of people, or obtained from various online platforms or various face recognition scenarios. Images of people can also be obtained from various face recognition scenarios.
[0092] The original face images include: the original attacked face image and the original attacking face image. Adversarial face samples are generated using the original attacked face image and the original attacking face image, employing a pre-defined adversarial generation strategy.
[0093] Specifically, as one implementation, the step of obtaining the original face image may include:
[0094] Obtain the original images of the people being attacked and the images of the people attacking the attacking person.
[0095] The original human image is subjected to face detection, alignment, and normalization to obtain the original human face image.
[0096] Specifically, after obtaining the original images of people, data preprocessing can be performed on the original images.
[0097] Data preprocessing is an indispensable step in improving network training performance and further enhancing the accuracy of face adversarial example processing.
[0098] In this embodiment, the input is a complete image of a person, therefore face detection and face alignment are typically required. This embodiment can use Retinaface for face detection. Simultaneously, the image data is normalized to improve network convergence speed.
[0099] Furthermore, for black-box attack scenarios, this embodiment can employ input diversity, randomly transforming the input image in each iteration to mitigate overfitting and improve the generalization of the attack. In addition, this embodiment also employs the following processing schemes: adding randomly generated minor noise to the input image under a certain probability, and performing affine transformations such as translation and rotation on the input image.
[0100] It should be noted that the above preprocessing scheme mainly targets images of attackers, employing input diversity. This involves randomly transforming the input image in each iteration, adding randomly generated minor noise to the input image, and performing affine transformations such as translation and rotation on the input image.
[0101] Step S102: Based on the pre-trained face recognition model, perform fusion feature learning on the original attacked face image and the original attacking face image, and combine it with a pre-generated mask of a preset area size to generate a perturbation image using a gradient iteration method.
[0102] This embodiment mainly utilizes artificial intelligence deep learning technology to perform fusion feature learning through a pre-trained face recognition model. By continuously updating the gradient, a perturbation image of the same size as the mask is obtained. This perturbation image can then be added to the attacker's image to generate an adversarial attack image, i.e., a face adversarial sample.
[0103] In this embodiment, the perturbation is added using gradient iteration and fusion attack. The fusion attack mainly involves fusing the features of face sample images recognized by multiple face recognition models (such as FaceNet, MobileFace, etc.) to improve the generalization of black-box attacks.
[0104] Specifically, as one implementation method, firstly, the original attacked face image is input into a pre-trained multi-face recognition model for feature extraction, and the extracted multiple attacked face features are fused to obtain the first fused feature of the original attacked face image.
[0105] The original attack face image is input into a pre-trained multi-face recognition model for feature extraction, and the gradient in the model parameters of the original attack face image and the extracted multiple attack face features are obtained. The extracted multiple attack face features are fused to obtain the second fused feature of the original attack face image.
[0106] Calculate the loss function based on the first fusion feature and the second fusion feature;
[0107] Based on the loss function, gradient, and a mask of a pre-generated area, a perturbation image is generated using a gradient iteration method.
[0108] The step of generating a perturbation image using a gradient iteration method based on the loss function, gradient, and a pre-generated mask of a preset area size may include:
[0109] Calculate the mean of the absolute values of the gradient of the original attack face image in each RGB channel;
[0110] Calculate the average gradient based on the mean;
[0111] Based on the average gradient and combined with a mask of a pre-generated area of a preset size, the original attack face image is updated to obtain the updated attack face image.
[0112] The updated attack face images are normalized.
[0113] The gradient of the updated attack face image is updated according to the loss function; the next iteration begins, and this process continues until the preset number of iterations is reached. Based on the final updated attack face image, a perturbation image is obtained.
[0114] More specifically, as one implementation method, the gradient iteration steps for perturbation generation can be as follows:
[0115] Image input: Image of the attacked face vic Attacking facial images I att Mask I mask , Fusion model M.
[0116] Step 1: Initialize the input image, including the attacked face image I. vic Input the fusion model M to obtain the fusion feature f1, where the fusion model M includes multiple pre-trained face recognition models. The input image is processed by multiple face recognition models to extract features, and the features extracted by the multiple face recognition models are fused to obtain the fused feature.
[0117] Step 2: Start the loop;
[0118] Step 3, attack the face image I att Add input diversity and random transformations;
[0119] Step 4: Process the attacked face image I att Input into the fusion model M to obtain I att The gradient g and the fused feature f2;
[0120] Step 5: Calculate the loss function L = 1 - F cos (f1,f2), where F cos (f1,f2) represents the cosine similarity between the two.
[0121] Step 6: Calculate the absolute value of each RGB channel of g and its mean value g′, then calculate the average gradient.
[0122] Step 7: Update the perturbation attack map Where F(*) represents the user-defined sign function, and λ = 1.0 / 255;
[0123] Step 8, I att The RGB values are mapped to [-1, 1], which is to perform image normalization processing;
[0124] Step 9: Update I according to the loss function att The gradient is used to proceed to the next iteration, until the preset number of iterations is reached.
[0125] The updated image of the attacked face is then obtained, from which a perturbation image is generated.
[0126] Among them, the pre-trained multiple face recognition models can adopt different commonly used face recognition models, referring to... Figure 6 As shown, for example, Model 1, Model 2, Model 3, and Model 4 can be IRSE50, IR101, Facenet, and Mobileface, respectively: commonly used face recognition models.
[0127] Step S103: Add the perturbation image to the original attack face image to generate a face adversarial attack sample.
[0128] After obtaining the perturbation image, the perturbation image is added to the original attack face image to generate a face adversarial attack sample.
[0129] This embodiment, through the above-described scheme, specifically obtains original face images, including an original target face image and an original attacker face image. Based on a pre-trained face recognition model, it performs fusion feature learning on the original target face image and the original attacker face image, and combines this with a pre-generated mask of a preset area size, using a gradient iteration method to generate a perturbation image. The perturbation image is then added to the original attacker face image to generate a face adversarial attack sample. This scheme, by using a pre-generated mask of a preset area size, achieves face adversarial attacks by adding perturbation to a small region, avoiding the need to perturb the entire image. This scheme is applicable to both white-box and black-box attacks, requiring no querying or model transfer, thus reducing the implementation complexity of the face adversarial attack method. Compared to traditional schemes that often require perturbation of the entire image, resulting in high time complexity, this scheme's face adversarial sample generation method can achieve the attack by perturbating only a small region.
[0130] Reference Figure 3 , Figure 3 This is a flowchart illustrating the second embodiment of the face adversarial sample generation method of the present invention.
[0131] like Figure 3 As shown, based on the above Figure 2 In the embodiment shown, before step S102 above, where the pre-trained face recognition model performs fusion feature learning on the original attacked face image and the original attacking face image, and generates the perturbation image using a gradient iteration method in conjunction with a pre-generated mask of a preset area size, the invention further includes:
[0132] Step S80: Obtain feature heatmaps of multiple face recognition models obtained by performing feature recognition on face sample images using multiple face recognition models;
[0133] Step S90: Analyze the feature heatmaps of multiple face recognition models to determine the corresponding face feature sensitive areas;
[0134] Step S100: Based on the facial feature sensitive area and the preset attack scenario, generate a mask of a preset area size.
[0135] Compared to the above Figure 2 The embodiment shown also includes a scheme for generating a mask of a preset area size.
[0136] Specifically, the mask, as an important basis for perturbation attacks, is mainly obtained by analyzing the sensitive areas of the face, that is, by analyzing the feature heatmaps of multiple face recognition models to obtain the mask.
[0137] Among these, the most sensitive areas for facial features are the eyes, nose, and mouth. Figure 4 As shown in the diagram. Through extensive experimental comparisons, the weighting of regions influencing facial feature information is as follows: eyes > nose > mouth. That is, for less perturbation, using only the eye region as a mask is slightly more effective than using only the nose region. Figure 4 Figures a, b, c, and d illustrate the permissible effects of the disturbance area proportions for four different regional combinations.
[0138] Furthermore, even by retaining only the upper and lower areas of the eye region and removing the middle portion, a similar perturbation effect can still be preserved. For example... Figure 4 The perturbation effects in graphs a and d are basically the same, but the perturbation area in graph d is significantly smaller.
[0139] The size of the mask area can be determined based on the actual attack scenario. For example, in a white-box attack scenario, since the model parameters are known, the mask can be appropriately reduced in size. However, in a black-box attack scenario, which is more difficult, the mask area needs to be appropriately increased.
[0140] This embodiment can generate adversarial examples of faces based on a small-region perturbation patching method (i.e., a small-region mask perturbation patching method). Compared with traditional methods that often require adding perturbations to the entire image, which has high time complexity, this embodiment proposes a method that can achieve the attack by only perturbing a small region.
[0141] The overall process of this embodiment can be referred to Figure 6 As shown. This embodiment utilizes artificial intelligence deep learning technology to learn fusion features through a pre-trained face recognition model. By continuously updating the gradient, a perturbation image with the same size as the small region mask is obtained. This perturbation is added to the attacker's image, thus generating a perturbation attack image sample. Figure 6 As shown, the core of this embodiment can be divided into three parts: data preprocessing, mask generation, and perturbation generation.
[0142] Therefore, this scheme, by employing a mask with a preset area size, achieves adversarial attacks on faces by adding perturbations to a small region, avoiding the need to perturb the entire image. This scheme is applicable to both white-box and black-box attacks, eliminating the need for querying and model transfer, thus reducing the implementation complexity of adversarial face attack methods. Compared to traditional schemes that often require perturbations to the entire image, resulting in high time complexity, this scheme's adversarial face sample generation method can achieve attacks using only small-region perturbations.
[0143] Reference Figure 5 , Figure 5 This is a flowchart illustrating the second embodiment of the face adversarial sample generation method of the present invention.
[0144] like Figure 5 As shown, based on the above Figure 3 In the illustrated embodiment, the method further includes:
[0145] Step S104: The face adversarial attack sample is tested and verified using a test model to obtain test results.
[0146] Compared to the above Figure 3 The embodiment shown also includes a scheme for testing and verifying the generated adversarial face attack samples.
[0147] Specifically, refer to Figure 6 As shown, a corresponding test model can be configured, such as model 5, to test and verify the face adversarial attack sample. Specifically, the test model can be IR50.
[0148] When the test model verifies the face adversarial attack sample, it can be compared with the original attacked image.
[0149] Specifically, adversarial attack samples and the original attacked image are input into the test model. The test model verifies the adversarial attack samples and compares them with the output of the original attacked image to obtain the loss value (Lcos). This loss value can be obtained by the cosine similarity between the two. Thus, by verifying the adversarial attack samples, the generalization and robustness of the adversarial attack samples can be detected. Based on the test results, the relevant parameters in the adversarial attack sample generation algorithm can be updated to improve the accuracy, generalization and robustness of the adversarial attack samples.
[0150] It should be noted that the above embodiments can be reasonably combined according to the actual situation, which will not be described in detail here.
[0151] Furthermore, embodiments of the present invention also propose a face adversarial sample generation device, the device comprising:
[0152] The acquisition module is used to acquire the original face image, which includes: the original attacked face image and the original attacking face image;
[0153] The perturbation generation module is used to perform fusion feature learning on the original attacked face image and the original attacking face image based on the pre-trained face recognition model, and generate perturbation images by combining a pre-generated mask of a preset area size and using a gradient iteration method.
[0154] An add module is used to add the perturbation image to the original attack face image to generate a face adversarial attack sample.
[0155] The principle of generating adversarial examples for faces in this embodiment of the invention can be referred to in the above embodiments, and will not be repeated here.
[0156] Furthermore, this invention also proposes a face adversarial sample generation system, characterized in that the face adversarial sample generation system includes: a memory, a processor, and a face adversarial sample generation program stored in the memory and executable on the processor. When the face adversarial sample generation program is executed by the processor, it implements the face adversarial sample generation method as described in the above embodiments.
[0157] The principle of generating adversarial examples for faces in this embodiment of the invention can be referred to in the above embodiments, and will not be repeated here.
[0158] Furthermore, this embodiment of the invention also proposes a computer-readable storage medium, characterized in that the computer-readable storage medium stores a face adversarial example generation program, which, when executed by a processor, implements the face adversarial example generation method as described in the above embodiments.
[0159] The principle of generating adversarial examples for faces in this embodiment of the invention can be referred to in the above embodiments, and will not be repeated here.
[0160] Compared to existing technologies, the face adversarial sample generation method, apparatus, system, and storage medium proposed in this invention acquire an original face image, which includes an original target face image and an original attacker face image. Based on a pre-trained face recognition model, the original target face image and the original attacker face image are fused with features. A perturbation image is generated using a gradient iteration method, combined with a pre-generated mask of a preset area size. The perturbation image is then added to the original attacker face image to generate a face adversarial attack sample. This solution, by using a pre-generated mask of a preset area size, achieves face adversarial attacks by adding perturbation to a small region, avoiding the need to perturb the entire image. This solution is applicable to both white-box and black-box attacks, eliminating the need for querying and model transfer, thus reducing the implementation complexity of face adversarial attack methods. Compared to traditional solutions that often require perturbation of the entire image, resulting in high time complexity, this face adversarial sample generation method can achieve attacks using only small-region perturbation.
[0161] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or system that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or system. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or system that includes that element.
[0162] The sequence numbers of the above embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.
[0163] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) as described above, and includes several instructions to cause a recommendation effect evaluation system (which may be a mobile phone, computer, server, controlled terminal, or network device, etc.) to execute the methods of each embodiment of the present invention.
[0164] The above are merely preferred embodiments of the present invention and do not limit the scope of the patent. Any equivalent structural or procedural transformations made based on the description and drawings of the present invention, or direct or indirect applications in other related technical fields, are similarly included within the scope of patent protection of the present invention.
Claims
1. A method for generating adversarial examples of faces, characterized in that, The method includes the following steps: Obtain the original face image, which includes: the original attacked face image and the original attacking face image; The feature heatmaps of multiple face recognition models are obtained by performing feature recognition on face sample images using multiple face recognition models. The feature heatmaps of multiple face recognition models are analyzed to determine the corresponding sensitive areas of face features. Based on the facial feature sensitive area and the preset attack scenario, a mask of a preset area size is generated; The original attacked face image is input into a pre-trained multi-face recognition model for feature extraction. The extracted multiple attacked face features are fused to obtain the first fused feature of the original attacked face image. The original attack face image is input into a pre-trained multi-face recognition model for feature extraction, and the gradient in the model parameters of the original attack face image and the extracted multiple attack face features are obtained. The extracted multiple attack face features are fused to obtain the second fused feature of the original attack face image. Calculate the loss function based on the first fusion feature and the second fusion feature; Based on the loss function, gradient, and a mask of a pre-generated area, a perturbation image is generated using a gradient iteration method. The gradient iteration method includes the following steps: adding input diversity and random variations to the original attack face image; inputting the processed original attack face image into a fusion model to obtain the gradient and fusion features of the original attack face image, wherein the fusion model includes pre-trained multiple face recognition models; calculating the mean of the absolute values of each RGB channel to obtain the average gradient; updating the original attack face image according to the average gradient and a mask of a preset area size; normalizing the updated original attack face image and updating the gradient of the original attack face image according to the loss function, and proceeding to the next iteration until the preset number of iterations is reached; The perturbation image is added to the original attack face image to generate a face adversarial attack sample.
2. The method according to claim 1, characterized in that, Before the step of inputting the original attack face image into a pre-trained multi-face recognition model for feature extraction, the following steps are also included: The original attack face image is preprocessed using at least one of the following methods: The original attack face image is randomly transformed using input diversity; Randomly generated fine noise is added to the original attack face image; An affine transformation is performed on the original attack face image.
3. The method according to claim 1, characterized in that, The steps for obtaining the original face image include: Obtain the original images of the people being attacked and the images of the people attacking the attacking person. The original human image is subjected to face detection, alignment, and normalization to obtain the original human face image.
4. The method according to claim 1, characterized in that, The method further includes: The face adversarial attack sample was tested and verified using a test model, and the test results were obtained.
5. A face adversarial sample generation device, characterized in that, The device includes: The acquisition module is used to acquire the original face image, which includes: the original attacked face image and the original attacking face image; The acquisition module is further configured to acquire feature heatmaps of multiple face recognition models obtained by performing feature recognition on face sample images by multiple face recognition models; analyze the feature heatmaps of multiple face recognition models to determine the corresponding face feature sensitive regions; and generate a mask of a preset area size based on the face feature sensitive regions and a preset attack scenario. The perturbation generation module is used to input the original attacked face image into a pre-trained multi-face recognition model for feature extraction, and to fuse the extracted multiple attacked face features to obtain the first fused feature of the original attacked face image. The original attack face image is input into a pre-trained multi-face recognition model for feature extraction, and the gradient in the model parameters of the original attack face image and the extracted multiple attack face features are obtained. The extracted multiple attack face features are fused to obtain the second fused feature of the original attack face image. Calculate the loss function based on the first fusion feature and the second fusion feature; Based on the loss function, gradient, and a mask of a pre-generated area, a perturbation image is generated using a gradient iteration method. The gradient iteration method includes the following steps: adding input diversity and random variations to the original attack face image; inputting the processed original attack face image into a fusion model to obtain the gradient and fusion features of the original attack face image, wherein the fusion model includes pre-trained multiple face recognition models; calculating the mean of the absolute values of each RGB channel to obtain the average gradient; updating the original attack face image according to the average gradient and a mask of a preset area size; normalizing the updated original attack face image and updating the gradient of the original attack face image according to the loss function, and proceeding to the next iteration until the preset number of iterations is reached; An add module is used to add the perturbation image to the original attack face image to generate a face adversarial attack sample.
6. A face adversarial example generation system, characterized in that, The face adversarial sample generation system includes: a memory, a processor, and a face adversarial sample generation program stored in the memory and executable on the processor. When the face adversarial sample generation program is executed by the processor, it implements the steps of the face adversarial sample generation method as described in any one of claims 1 to 4.
7. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a face adversarial example generation program, which, when executed by a processor, implements the steps of the face adversarial example generation method as described in any one of claims 1 to 4.