Face recognition attack antagonism patch generation method and device
By generating and optimizing face recognition adversarial patches, using reinforcement learning agents and policy gradient methods, the problem of difficult attacking black box face recognition systems in the existing technology is solved, and efficient and highly adaptable adversariality is achieved Patch generation enhances the migration and robustness of the patch.
Patent Information
- Application Number
- CN202410849487.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-06-27
- Publication Date
- 2024-11-12
AI Technical Summary
The existing face recognition adversarial patches are difficult to successfully attack the black box face recognition system on the white box model, making it difficult to detect vulnerabilities in the face recognition system.
By generating perturbing patch face feature maps, using reinforcement learning agents to generate adversarial patches on standard face maps, and adjust the loss weight of the white box model until the patch can successfully attack multiple white box models, and then use the policy gradient method Iteratively updates parameters so that the patch can successfully attack the black box model.
It improves the attack success rate and adaptability of the anti-patch patch, solves the problem of low attack success rate due to fixed patch location, and enhances the migration and robustness of the patch.
Smart Images

Figure CN118942131A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of computer vision technology, and in particular to a method and device for generating face recognition attack adversarial patches. Background Art
[0002] Deep neural networks have demonstrated excellent performance in tasks such as image classification, text recognition, object detection, and face recognition. However, even face recognition technology based on deep neural networks may still have some security risks that are difficult to detect through manual detection. The latest research shows that deep neural networks are prone to making incorrect judgments about adversarial examples. Unlike other attacks, adversarial example attacks mainly occur during model testing and deployment. They can be input into the model like normal data and produce deceptive results. In recent years, they have been widely used to attack deep neural networks and are an important research direction in the field of computer vision and artificial intelligence security.
[0003] With the rapid development and integration of artificial intelligence technology, concepts such as smart cities, smart e-commerce, smart healthcare, and smart security are attracting more and more attention. The rapid development of artificial intelligence is inseparable from the emergence of deep neural networks. Although deep neural networks have surpassed human accuracy in various recognition tasks, they are inherently fragile and vulnerable to adversarial attacks. In the field of face recognition, adversarial attacks include identity impersonation and identity hiding.
[0004] Identity impersonation attack refers to a targeted attack in which the attacker wears physical attack media such as adversarial glasses and adversarial stickers to impersonate a specified identity. The main application scenarios include cracking the facial unlocking of mobile phones, impersonating others to pay with face, and impersonating others to enter high-security places.
[0005] Identity hiding attacks refer to untargeted attacks in which the attacker wears adversarial masks, adversarial hats, etc., making it impossible for the model to correctly identify the attacker's identity, thereby avoiding Skynet tracking.
[0006] Adversarial samples were first proposed to deceive classification models. Adversarial samples are generated by adding perturbations that are imperceptible to the human eye, thereby achieving the purpose of preventing the model from correctly outputting classification results. Specifically, the adversarial perturbation is changed in the direction opposite to the model gradient, and the perturbation is defined as ε, so that the clean sample with the perturbation deviates from the original classification area and enters the target category area, thereby achieving the purpose of the model giving an erroneous output with high confidence. This method is difficult to distinguish for the human eye and has a very high concealment when applied in digital space. However, because the perturbation is too small, and physical adversarial attacks require the adversarial sample to be printed and then recaptured by a sensor (such as a camera), the tiny perturbation will be masked by factors such as noise in the physical world, so it cannot be applied to physical attacks in the real world.
[0007] Taking into account the various limitations of physical space, physical adversarial attacks no longer limit the perturbation to noise that is difficult for the human eye to detect, but instead limit the perturbation to a small patch generated in a smaller local area without perturbation constraints. Adversarial patches are robust, universal, and portable. They can not only effectively implement attacks in digital space, but are also very suitable for physical attacks in the real world. Therefore, they have attracted the attention of many researchers.
[0008] Compared with small perturbations, adversarial patches for face recognition are suitable for attacks in physical space. However, existing adversarial patches for faces often only consider attacking a single model and then migrating the attack to other models, and the location of the patch is often artificially set, such as glasses, noses, masks, etc., without considering the impact of the location of the patch on the effect of the attack, nor the fact that the face recognition system may use multiple models for decision-making. In practical applications, a successful attack on a white-box model does not necessarily lead to a successful attack on a black-box face recognition system. For a black-box face recognition system with an unknown model, the model's architecture, parameters, and training data set are all unknown. Therefore, in most cases, the attack will fail, making it difficult to detect vulnerabilities in the face recognition system. Summary of the invention
[0009] In view of the defects existing in the prior art, the purpose of the present invention is to provide a method and device for generating adversarial patches for face recognition attacks, which can solve the problem in the prior art that the adversarial patches may successfully attack a white box model but may not successfully attack a black box face recognition system, making it difficult to detect vulnerabilities in the face recognition system.
[0010] In order to achieve the above purpose, the technical solution adopted by the present invention is:
[0011] In one aspect, the present invention provides a method for generating a face recognition attack adversarial patch, comprising the following steps:
[0012] Perform face cropping on each image in the sample set to obtain a standard face image sample set;
[0013] The constructed intelligent agent generates a perturbation patch on a standard face image to obtain a perturbation patch face feature map;
[0014] Using the perturbed patch face feature map to attack a centralized model including multiple white-box face recognition models, and adjusting the loss weights of each white-box face recognition model until the perturbed patch face feature map can successfully attack the centralized model;
[0015] The perturbation patch face feature map that can successfully attack the centralized model is used to continue attacking the black-box face recognition model, and the policy gradient method is used to iteratively update the parameters of the agent in the direction of increasing the reward for the agent, until the optimized perturbation patch face feature map can successfully attack the black-box face recognition model, and the perturbation patch face feature map is output.
[0016] In some optional solutions, the step of cropping faces from each image in the sample set to obtain a sample set of facial feature images includes:
[0017] Use the face detection model to detect and crop the faces in each photo of different people;
[0018] The cropped face images are normalized into face images of a set size to obtain a standard face image sample set.
[0019] In some optional solutions, the constructed intelligent agent generates a perturbation patch on a standard face image to obtain a perturbation patch face feature map, including:
[0020] Based on the U-Net architecture, a reinforcement learning agent is designed, which is used to learn to update patch position parameters and adjust attack step size;
[0021] The standard face image is input into the reinforcement learning agent, and a perturbation patch is generated on the face feature image to obtain a perturbation patch face feature image.
[0022] In some optional schemes, using the perturbation patch face feature map to attack a centralized model including multiple white-box face recognition models, and adjusting the loss weights of each white-box face recognition model until the perturbation patch face feature map can successfully attack the centralized model, includes:
[0023] Using the perturbed patch face feature map to attack each recognition model in the integrated model including several white-box face recognition models;
[0024] The loss of the perturbed patch face feature map attacking the white-box face recognition model is obtained. When the collective loss of the integrated model is greater than the set value, the loss weights of each white-box face recognition model are updated based on the Gaussian distribution until the perturbed patch face feature map can successfully attack the centralized model.
[0025] In some optional schemes, the loss weight ρ of each white-box face recognition model is i According to the formula Determine, where L i is the loss calculation function for the i-th face similarity model, f i is the i-th white-box face recognition model, n is the number of white-box face recognition models, x is the sample, y is * Represents the target sample label.
[0026] In some optional schemes, the perturbation patch face feature map that can successfully attack the centralized model is used to continue attacking the black box face recognition model, and the policy gradient method is used to iteratively update the parameters of the agent in a direction that increases the reward of the agent, until the optimized perturbation patch face feature map can successfully attack the black box face recognition model, and the perturbation patch face feature map is output, including:
[0027] Continue to attack the black-box face recognition model using the perturbed patched face feature map that can successfully attack the centralized model;
[0028] Based on the result of the perturbation patch face feature map attacking the black-box face recognition model, the policy gradient method is used to iteratively update the parameters of the reinforcement learning agent in the direction of increasing the maximum reward for the reinforcement learning agent, thereby optimizing the attack step size and patch position parameters to adjust the parameters of the perturbation patch face feature map until the optimized perturbation patch face feature map can successfully attack the black-box face recognition model, and the similarity with any other standard face image in the standard face image sample set exceeds the threshold, then the perturbation patch face feature map is output.
[0029] In some optional schemes, the updating strategy of the patch position parameters and the attack step size follows a classification distribution.
[0030] In some optional schemes, when an intelligent agent generates perturbation patches on a standard face image, an image smoothness loss function is introduced to generate patches.
[0031] In some optional schemes, after outputting the perturbation patch face feature map, the perturbation patch face feature map is printed, and the adversarial patch in the perturbation patch face feature map is cut out for use as a physical adversarial attack.
[0032] On the other hand, the present invention provides a face recognition attack adversarial patch generation device, comprising:
[0033] A sample cropping module is used to crop faces from each image in the sample set to obtain a standard face image sample set;
[0034] A patch generation module, which is used to generate a perturbation patch on a standard face image through the constructed intelligent agent to obtain a perturbation patch face feature image;
[0035] A white-box attack module, which is used to attack a centralized model including multiple white-box face recognition models using a perturbation patch face feature map, and adjust the loss weight of each white-box face recognition model until the perturbation patch face feature map can successfully attack the centralized model;
[0036] The black-box training module is used to continue attacking the black-box face recognition model with the perturbation patch face feature map that can successfully attack the centralized model, and use the policy gradient method to iteratively update the parameters of the agent in the direction of increasing the reward for the agent until the optimized perturbation patch face feature map can successfully attack the black-box face recognition model, and then output the perturbation patch face feature map.
[0037] Compared with the prior art, the advantages of the present invention are: the present invention uses a perturbation patch face feature map to attack a centralized model including multiple white-box face recognition models, and adjusts the loss weights of each white-box face recognition model until the perturbation patch face feature map can successfully attack the centralized model; then the perturbation patch face feature map that can successfully attack the centralized model is used to continue to attack the black-box face recognition model, and a policy gradient method is used to iteratively update the parameters of the intelligent agent in order to increase the direction in which the intelligent agent obtains rewards, until the optimized perturbation patch face feature map can successfully attack the black-box face recognition model, and then the perturbation patch face feature map is output. The position of the face recognition adversarial patch is optimized, and an adversarial patch with a high attack success rate and strong adaptability can be generated. In the training stage, the intelligent agent modeled by the present invention is very efficient in finding the most suitable patch position, which solves the problem of low attack success rate caused by the fixed position of the adversarial patch. Starting from the fact that current face recognition attacks are mainly in a complex black-box physical environment, model integration is introduced into the generation process of adversarial patches, and it is proposed to dynamically adjust the weight coefficient of the model according to the loss size of the sub-model. Compared with the previous fixed model weight coefficient, the present invention fully improves the influence of the key alternative model, making the adversarial patch more transferable. BRIEF DESCRIPTION OF THE DRAWINGS
[0038] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.
[0039] Figure 1 Schematic diagram of the process of generating a face recognition attack adversarial patch in an embodiment of the present invention.
[0040] Figure 2 This is a technical solution design diagram for generating a face recognition anti-attack patch of the present invention. DETAILED DESCRIPTION
[0041] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.
[0042] The embodiments of the present invention are further described in detail below with reference to the accompanying drawings.
[0043] like Figure 1 and Figure 2 As shown, on the one hand, the present invention provides a method for generating a face recognition attack adversarial patch, comprising the following steps:
[0044] S1: Perform face cropping on each image in the sample set to obtain a standard face image sample set.
[0045] In this example, the prepared face recognition dataset contains a total of 13,233 face images. The images are all from natural scenes in life. Each image is given a corresponding name. There are a total of 5,749 people. The size of each image is 250×250. Most of them are color images, and a few are black and white images.
[0046] In some optional embodiments, step S1 includes:
[0047] S11: Use the face detection model to detect and crop the faces in each photo of multiple people.
[0048] In this example, the face detection MTCNN algorithm is used to detect the face position in the face image and crop it out.
[0049] S12: Normalize the cropped face image into a face image of a set size to obtain a standard face image sample set.
[0050] In this example, the face is cropped, aligned and normalized to a size of 112×112 for training and testing of the present invention.
[0051] S2: Generate a perturbation patch on a standard face image through the constructed intelligent agent to obtain a perturbation patch face feature map.
[0052] In some optional embodiments, step S2 includes:
[0053] S21: Based on the U-Net architecture, a reinforcement learning agent is designed, which is used to learn to update patch position parameters and adjust the attack step size.
[0054] A network based on the U-Net architecture is designed as an agent in reinforcement learning. The agent is designed to learn to update the patch position parameters and adjust the attack step size. Specifically, the U-Net network can ensure the correspondence between the position and the image pixel, and the output feature map is the same size as the input image, which helps to map the position and perturbation to the feature map output by the agent.
[0055] S22: Input the standard face image into the reinforcement learning agent, generate a perturbation patch on the face feature image, and obtain a perturbation patch face feature image.
[0056] Constructing a U-Net structured agent, when constructing a U-Net structured agent, a fully connected layer is used at the top layer of the network to map the feature map M to vectors V of different attack steps. Specifically, for the position selection action, the optional position range is discrete, so the position update strategy is designed to follow a categorical distribution. Given each selected position P position The probability of calculating the location parameter (c x ,c y )~Cat(P position ): Where Cat represents the categorical distribution.
[0057] The attack step size is set to 20 discrete values in the range of 0.01 to 0.2. Due to the discreteness of the step size parameter value, the classification distribution is used as the attack step size update strategy. Attack step parameter ε~Cat(P step ), where Cat represents the classification distribution. The probability of each candidate value is P step =softmax(FC(P position )), FC represents the fully connected layer. By sampling from the corresponding distribution, we can obtain the location parameter (c x ,c y ) and the attack step parameter ε. The attack step is used to adjust the image of the generated patch.
[0058] S3: Use the perturbation patch face feature map to attack the centralized model including multiple white-box face recognition models, and adjust the loss weights of each white-box face recognition model until the perturbation patch face feature map can successfully attack the centralized model.
[0059] In this example, n white-box face recognition models are given as substitute models, and the weight initialization of each model for the loss function follows a Gaussian distribution. The agent is designed to output a feature map with n channels, where n represents the number of substitute models. The substitute models we use include FaceNet proposed by Google, ArcFace proposed by InsightFace, and CosFace proposed by Tencent. The accuracy of these face recognition models tested on the LFW dataset exceeds 99%. Since the difference between ArcFace and CosFace lies in the different loss functions during training, the backbone networks used are Resnet, while the backbone network used by FaceNet is Inception-ResNetV1. In order to fully consider the impact of different backbone networks and loss functions on face recognition models, the substitute models we use for model integration include FaceNet, ArcFace34, ArcFace50, and Cosface50, where the numbers represent the number of layers of the model structure. These models are used for model integration attacks, that is, updating the strategy according to the integrated losses of these substitute models. Each channel in the feature map corresponds to a substitution model. The relative value of each pixel on the feature map indicates the importance of the substitution model corresponding to that position. The softmax operation reflects the proportion of the corresponding substitution model in the integrated model.
[0060] Step S3 specifically includes:
[0061] S31: Use the perturbation patch face feature map to attack each recognition model in the integrated model including several white box face recognition models.
[0062] S32: Obtain the loss of the perturbed patch face feature map to attack the white-box face recognition model. When the collective loss of the integrated model is greater than the set value, update the loss weights of each white-box face recognition model based on the Gaussian distribution until the perturbed patch face feature map can successfully attack the centralized model.
[0063] In this example, the weight ratio of the loss of each white-box face recognition model to the ensemble loss is a continuous value, and the weight update strategy is Set to follow the Gaussian distribution, so the i-th weight parameter ρ i Then you can use To calculate, where L i is the loss function for calculating the face similarity model, f i is the i-th white-box face recognition model, n is the number of white-box face recognition models, x is the sample, y is * represents the target sample label. The purpose of weight balancing is to ensure that all alternative models can be successfully attacked, making the generated examples more adversarial to the black-box victim model.
[0064] S4: Use the perturbation patch face feature map that can successfully attack the centralized model to continue attacking the black-box face recognition model, and use the policy gradient method to iteratively update the parameters of the agent in the direction of increasing the reward for the agent, until the optimized perturbation patch face feature map can successfully attack the black-box face recognition model, then output the perturbation patch face feature map.
[0065] In this example, a policy function is constructed to guide the generation of actions according to the actions predicted by the policy π. These actions guide the generation of adversarial patches, and then the face adversarial samples with adversarial patches are input into the target model. The target model is regarded as the environment in reinforcement learning, and whether to obtain a reward depends on whether the attack is successful. The policy gradient method is used to iteratively update the parameters of the agent in the direction of increasing the reward for the agent, resulting in actions that perform better with high probability on the target model, optimizing the attack step size and patch position parameters.
[0066] Specifically, step S4 includes:
[0067] S41: Continue to attack the black-box face recognition model using the perturbed patch face feature map that can successfully attack the centralized model.
[0068] S42: Based on the result of the perturbation patch face feature map attacking the black box face recognition model, the policy gradient method is used to iteratively update the parameters of the reinforcement learning agent in the direction of increasing the maximum reward for the reinforcement learning agent, thereby optimizing the attack step size and patch position parameters to adjust the parameters of the perturbation patch face feature map until the optimized perturbation patch face feature map can successfully attack the black box face recognition model, and the similarity with any other standard face image in the standard face image sample set exceeds a threshold, then the perturbation patch face feature map is output.
[0069] The updating strategy of the patch position parameters and the attack step length follows the classification distribution introduced in step S22.
[0070] In addition, the image smoothing loss function L is introduced tv Adversarial patches are mostly pixel-level perturbations in the digital domain, and this perturbation can cause large differences between adjacent pixels and cause visual differences. In order to ensure that the adversarial patches generated by the agent have better spatial smoothness, this paper introduces a minimization Function, where P represents the perturbed image, pi,j represents the pixel value of a point on the perturbed image, i+1 represents a pixel to the right, and i,j are pixel coordinates, to generate a more natural perturbation adversarial patch.
[0071] In order to enhance the attack capability in physical space. The target of the attack of the present invention is the face recognition model in the black box state. We use an integrated substitution model for training, and the output is the similarity between two face pictures. In order to present the adversarial patch in the physical space, the patch needs to be printed out by a printer. However, since modern printing technology can only reproduce a limited part of the RGB color space, the present invention uses the non-printable score (NPS) to quantify whether the color in the adversarial patch can be accurately printed by the printer. Specifically, the loss function is minimized. To encourage the generated adversarial patches to remain printable. In this example, in order to make the generated perturbation patch face feature map printable, the agent introduces a minimization loss function when generating patches. To encourage the generation of better printable adversarial patches, where P represents the perturbation image, Cpatch represents a pixel in the image, and Cprint represents a pixel value in the printable color dataset, that is, the color patch existing in the printer is selected to generate.
[0072] The real space environment is also simulated. When the adversarial patch is presented in the real world, many external environmental factors will affect its attack effect, such as rotation, jitter, noise, enlargement or reduction, etc. Since the face recognition model is more sensitive to face images, the expected transformation of the face image with the adversarial patch only includes adding noise and adjusting brightness.
[0073] In this example, in order to enhance the offensiveness of the adversarial patch proposed in this invention, we adopted large-scale training and used 13,233 faces for training. After selecting a face image for attack, another face image of another person is randomly selected as the attack target, and the process is repeated until the similarity between the two exceeds a predetermined threshold, and the attack is considered successful.
[0074] S5: After outputting the perturbation patch face feature map, print the perturbation patch face feature map, cut out the adversarial patch in the perturbation patch face feature map, and use it as a physical adversarial attack.
[0075] We collected multiple sets of experimental data and summarized them in the following table. The attack success rate represents the proportion of successfully attacked adversarial samples to the original data samples. The mathematical formula is: It reflects the attack success probability that the face adversarial patches obtained after training of the present invention make different black box victim models regard the attacker as the target.
[0076] Table 1 The success rate of adversarial patch attack after training on the LFW face dataset
[0077]
[0078] On the other hand, the present invention provides a face recognition attack adversarial patch generation device, including: a sample cropping module, a patch generation module, a white box attack module and a black box training module.
[0079] The sample cropping module is used to crop the faces of each image in the sample set to obtain a standard face image sample set; the patch generation module is used to generate perturbation patches on a standard face image through the constructed intelligent agent to obtain a perturbation patch face feature map; the white box attack module is used to use the perturbation patch face feature map to attack a centralized model including multiple white box face recognition models, and adjust the loss weights of each white box face recognition model until the perturbation patch face feature map can successfully attack the centralized model; the black box training module is used to continue to attack the black box face recognition model using the perturbation patch face feature map that can successfully attack the centralized model, and use the policy gradient method to iteratively update the parameters of the intelligent agent in the direction of increasing the reward for the intelligent agent, until the optimized perturbation patch face feature map can successfully attack the black box face recognition model, and then output the perturbation patch face feature map.
[0080] Among them, the functional implementation of each module in the above-mentioned face recognition attack adversarial patch generation device corresponds to the various steps in the above-mentioned face recognition attack adversarial patch generation method embodiment, and its functions and implementation processes will not be repeated here one by one.
[0081] In a third aspect, the present invention also provides a face recognition attack adversarial patch generation device, the face recognition attack adversarial patch generation device comprising a processor, a memory, and a program stored on the memory and executable by the processor, wherein when the AAAA program is executed by the processor, the steps of the face recognition attack adversarial patch generation method described in any one of the above items are implemented.
[0082] In summary, the present invention uses a perturbation patch face feature map to attack a centralized model including multiple white-box face recognition models, and adjusts the loss weights of each white-box face recognition model until the perturbation patch face feature map can successfully attack the centralized model; then the perturbation patch face feature map that can successfully attack the centralized model is used to continue attacking the black-box face recognition model, and the policy gradient method is used to iteratively update the parameters of the agent in order to increase the direction in which the agent obtains rewards, until the optimized perturbation patch face feature map can successfully attack the black-box face recognition model, and then the perturbation patch face feature map is output. The position of the face recognition adversarial patch is optimized, and an adversarial patch with a high attack success rate and strong adaptability can be generated. In the training stage, the agent modeled by the present invention is very efficient in finding the most suitable patch position, which solves the problem of low attack success rate caused by the fixed position of the adversarial patch. Starting from the fact that current face recognition attacks are mainly in a complex black-box physical environment, model integration is introduced into the generation process of adversarial patches, and it is proposed to dynamically adjust the weight coefficient of the model according to the loss size of the sub-model. Compared with the previous fixed model weight coefficient, the present invention fully improves the influence of the key alternative model, making the adversarial patch more transferable.
[0083] In addition, a series of physical environment simulation enhancement methods are used to enhance the robustness of the adversarial patch, so that it can still maintain good offensiveness in the real world and reduce the impact of various real-world factors on the adversarial patch.
[0084] In the description of the present application, it should be noted that the terms "upper", "lower", etc. indicate orientations or positional relationships based on the orientations or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present application and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the present application. Unless otherwise clearly specified and limited, the terms "installed", "connected", and "connected" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be an indirect connection through an intermediate medium, or it can be a connection between the two elements. For those of ordinary skill in the art, the specific meanings of the above terms in the present application can be understood according to the specific circumstances.
[0085] It should be noted that, in this application, relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.
[0086] The above description is only a specific implementation of the present application, so that those skilled in the art can understand or implement the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest range consistent with the principles and novel features applied for herein.
Claims
1. A method for generating adversarial patches for face recognition attacks, It is characterized in that The following steps are involved: Perform face cropping on each image in the sample set to obtain a standard face image sample set; The constructed intelligent agent generates a perturbation patch on a standard face image to obtain a perturbation patch face feature map; Using the perturbed patch face feature map to attack a centralized model including multiple white-box face recognition models, and adjusting the loss weights of each white-box face recognition model until the perturbed patch face feature map can successfully attack the centralized model; The perturbation patch face feature map that can successfully attack the centralized model is used to continue attacking the black-box face recognition model, and the policy gradient method is used to iteratively update the parameters of the agent in the direction of increasing the reward for the agent, until the optimized perturbation patch face feature map can successfully attack the black-box face recognition model, and the perturbation patch face feature map is output.
2. The face recognition attack adversarial patch generation method according to claim 1, It is characterized in that The method of performing face cropping on each image in the sample set to obtain a face feature map sample set includes: Use the face detection model to detect and crop the faces in each photo of different people; The cropped face images are normalized into face images of a set size to obtain a standard face image sample set.
3. The face recognition attack adversarial patch generation method as claimed in claim 1, It is characterized in that The constructed intelligent agent generates a disturbance patch on a standard face image to obtain a disturbance patch face feature image, including: Based on the U-Net architecture, a reinforcement learning agent is designed, which is used to learn to update patch position parameters and adjust attack step size; The standard face image is input into the reinforcement learning agent, and a perturbation patch is generated on the face feature image to obtain a perturbation patch face feature image.
4. The method for generating face recognition attack adversarial patches as claimed in claim 3, It is characterized in that The method of using the perturbation patch face feature map to attack a centralized model including multiple white-box face recognition models, and adjusting the loss weights of each white-box face recognition model until the perturbation patch face feature map can successfully attack the centralized model, includes: Using the perturbed patch face feature map to attack each recognition model in the integrated model including several white-box face recognition models; The loss of the perturbed patch face feature map attacking the white-box face recognition model is obtained. When the collective loss of the integrated model is greater than the set value, the loss weights of each white-box face recognition model are updated based on the Gaussian distribution until the perturbed patch face feature map can successfully attack the centralized model.
5. The method for generating face recognition attack adversarial patches as claimed in claim 4, It is characterized in that Loss weight ρ of each white-box face recognition model i According to the formula Determine, where L i is the loss calculation function for the i-th face similarity model, f i is the i-th white-box face recognition model, n is the number of white-box face recognition models, x is the sample, y is * Represents the target sample label.
6. The method for generating face recognition attack adversarial patches as claimed in claim 1, It is characterized in that The perturbation patch face feature map that can successfully attack the centralized model is used to continue attacking the black box face recognition model, and the policy gradient method is used to iteratively update the parameters of the agent in order to increase the direction in which the agent obtains the reward, until the optimized perturbation patch face feature map can successfully attack the black box face recognition model, and then the perturbation patch face feature map is output, including: Continue to attack the black-box face recognition model using the perturbed patched face feature map that can successfully attack the centralized model; Based on the result of the perturbation patch face feature map attacking the black-box face recognition model, the policy gradient method is used to iteratively update the parameters of the reinforcement learning agent in the direction of increasing the maximum reward for the reinforcement learning agent, thereby optimizing the attack step size and patch position parameters to adjust the parameters of the perturbation patch face feature map until the optimized perturbation patch face feature map can successfully attack the black-box face recognition model, and the similarity with any other standard face image in the standard face image sample set exceeds the threshold, then the perturbation patch face feature map is output.
7. The method for generating face recognition attack adversarial patches as claimed in claim 6, It is characterized in that The update strategy of the patch position parameters and the attack step size follows the classification distribution.
8. The method for generating face recognition attack adversarial patches as claimed in claim 1, It is characterized in that When using an intelligent agent to generate perturbation patches on a standard face image, an image smoothness loss function is introduced to generate patches.
9. The method for generating face recognition attack adversarial patches as claimed in claim 1, It is characterized in that After outputting the perturbation patch face feature map, the perturbation patch face feature map is printed, and the adversarial patch in the perturbation patch face feature map is cut out for use as a physical adversarial attack.
10. A face recognition attack adversarial patch generation device, It is characterized in that include: A sample cropping module is used to crop faces from each image in the sample set to obtain a standard face image sample set; A patch generation module, which is used to generate a perturbation patch on a standard face image through the constructed intelligent agent to obtain a perturbation patch face feature map; A white-box attack module, which is used to attack a centralized model including multiple white-box face recognition models using a perturbation patch face feature map, and adjust the loss weight of each white-box face recognition model until the perturbation patch face feature map can successfully attack the centralized model; The black-box training module is used to continue attacking the black-box face recognition model with the perturbation patch face feature map that can successfully attack the centralized model, and use the policy gradient method to iteratively update the parameters of the agent in the direction of increasing the reward for the agent until the optimized perturbation patch face feature map can successfully attack the black-box face recognition model, and then output the perturbation patch face feature map.
Citation Information
Patent Citations
Directional attack adversarial patch generation method and device
CN113255816A
