Face adversarial attack sample generation method and system based on attribute editing

Abstract

This invention proposes a method and system for generating adversarial attack samples for faces based on attribute editing. It maps the latent space of the input real face image and the target image to be identified into a low-dimensional manifold representation. Simultaneously, it improves the similarity of facial identity information through feature layer fusion. The attribute editing step optimizes the editing intensity of the optimal edited attributes obtained in the attribute selection step by designing a set of attribute editing parameters. It also introduces various controls to optimize image generation effects, achieving the best model attack success rate while maintaining the edited image quality and the visually recognizable identity information. This is used to test the robustness of the face recognition model and fine-tune the model to improve its recognition accuracy.

Description

Technical Field

[0001] This method belongs to the field of robustness, security, and robustness evaluation technology of artificial intelligence algorithms, and specifically relates to a method and system for generating adversarial attack samples for face recognition based on attribute editing. Background Technology

[0002] Facial recognition, as a biometric identification technology, uses the visual features of a face for identity verification, offering advantages such as non-intrusiveness, non-contact nature, user-friendliness, and convenience. With the development of artificial intelligence, the ability of facial recognition technology has been further enhanced by extracting facial features based on convolutional neural networks (CNNs), leading to its widespread application in daily life, such as facial recognition access control, face unlocking, and payment. Adversarial attacks involve adding noise to the input data that is imperceptible to humans, causing the model to make incorrect judgments, thereby confusing or fooling the intelligent model. The added noise is called adversarial perturbation, and the resulting sample is called an adversarial example. Facial recognition systems are also highly vulnerable to adversarial attacks. Depending on the purpose, facial recognition adversarial attacks can be divided into two categories:

[0003] ● Impersonation attack: After adding adversarial perturbations, the face recognition model will identify the input face image as the specified person.

[0004] ●Attacking evasion: After adding adversarial perturbations, the face recognition model will input face images that are incorrectly recognized (identified as any person other than a real person).

[0005] We can use adversarial attacks to evaluate the adversarial robustness of face recognition systems in practical applications. The adversarial robustness of a model can be measured by the success rate of adversarial attacks. For example, using 100 adversarial samples as input to the target model, the model's output is used, and the "number of successfully attacked samples / total sample size" is used as the evaluation result for adversarial robustness. A lower success rate indicates stronger robustness against such attack methods, and vice versa.

[0006] Based on the adversarial examples and the aforementioned robustness evaluation results, the robustness and security of the model can be further improved. For example, the generated adversarial examples can be used as training data to fine-tune the model, or they can be used to train a binary classifier to distinguish whether an input sample is an adversarial example.

[0007] Most adversarial attack methods for face recognition in the digital world generate adversarial examples through an optimization process. Based on the space added for the adversarial perturbation, they can be divided into image-domain-based and latent-space-based adversarial attack methods. Image-domain-based adversarial attack methods typically perturb the pixel value of each pixel in the image, limiting the magnitude of the pixel value change to constrain the visual effect of the generated adversarial examples. This includes using the fast gradient sign method (FGSM), which approximates the non-convex optimization problem of constructing adversarial images into a linear form. However, this algorithm cannot guarantee the success rate of the attack, especially in targeted attacks. Subsequently, researchers proposed improving the fast gradient sign method through multiple iterations, but at the cost of increased computation. Later, the randomized fast gradient sign method was proposed. The main idea is to add random perturbations to the input samples before gradient calculation, escaping large curvatures near data points, making the adversarial examples generated by the fast gradient sign method more generalizable. Another adversarial example generation technique based on the decision hyperplane was proposed, utilizing the iterative linear representation of the target model to generate adversarial examples. Because this method deals with a hyperplane problem under a constrained neural network, its application in non-neural network models is quite limited. The algorithm has low versatility and requires significant time to analyze model characteristics. While these methods offer fast iteration speeds and are simple to implement, the perturbations added to the pixel domain can be defended against using simple image transformations such as filtering, and transferability and visual quality are difficult to guarantee. Adversarial attack methods based on latent space add adversarial perturbations to the image's latent space (feature layer). An encoder transforms the image from the spatial domain to the latent space. Adversarial perturbations are added to the latent codes in the latent space to obtain adversarial latent codes. A generator, such as a generative adversarial network (GAN), then remaps the image from the latent space back to the spatial domain, yielding the final adversarial sample. The transferability of adversarial examples is highly susceptible to initialization issues, easily falling into local optima. Furthermore, the transferability is related to the search space; as the search space increases, the transferability initially increases and then decreases. When the search space becomes extremely large, the adversarial examples overfit to the substitute model, leading to a significant decline in transferability. Optimizing adversarial perturbations on the manifold learned by the generative model based on face data provides ample search space. The manifold learned by the generative model offers better regularization and generates features (faces) semantically relevant to the adversarial task, effectively linking the substitute and target models and significantly improving the transferability of adversarial examples.

[0008] In recent years, deep learning technology has continued to develop and has achieved widespread application in the field of computer vision. On the one hand, deep learning technology has led a new wave of artificial intelligence; on the other hand, a series of security issues caused by deep learning have also attracted increasing attention. Currently, image and video recognition technologies based on deep learning are widely used in all aspects of people's lives, such as intelligent supervision of network content, automatic video surveillance and analysis, access control systems based on facial recognition, and facial recognition payment. In these key application areas, the reliability and security of information and data should be valued and guaranteed. Since 2017, some fake images and videos generated based on deepfake technology have attracted widespread attention on the Internet, especially when deepfakes are used on influential figures, often leveraging that person's influence to generate even greater impact. In addition, a large number of "one-click" face-swapping software programs have made it increasingly easy to obtain fake videos. Fake images and videos have become one of the most significant information and data security threats, and their detection and supervision face enormous challenges.

[0009] AI-generated fake faces pose a significant threat, capable of creating videos that mimic real facial expressions and body movements to mimic a target person's actions or words, thus overturning the "seeing is believing" mentality. The industry urgently needs an effective technology to detect fake face images or videos in the online environment, but this is extremely difficult, primarily because the forged areas in fake face images are weak and localized. Detection is highly susceptible to image noise. Furthermore, the forged areas are often unpredictable, and different forgery methods target different areas, making accurate detection and classification of forged regions very challenging.

[0010] Deepfake detection methods can be broadly categorized into: methods based on image forgery defects, improved network structures, multi-feature fusion, and methods leveraging other tasks.

[0011] Methods based on image forgery defects are mainstream approaches that attempt to detect inconsistencies in facial images caused by scaling, rotation, and distortion during forgery. One CNN-based detection method detects low facial resolution due to tampering by detecting the facial region and surrounding areas. Another method, using facial X-rays, designs a facial contour mask to guide the model to focus on facial contour areas prone to forgery, achieving good generalization results.

[0012] Methods that improve network structure aim to enhance the model's performance in classifying real and fake images by modifying or improving the classification network. Based on mesoscopic and steganalysis features, two different networks are proposed to focus on the mesoscopic properties of images: the Meso-4 network and a variant of the Meso-4 network incorporating an Inception module. A forgery detection system based on Capsule Networks is also presented, which requires fewer parameters compared to traditional CNNs.

[0013] Multi-feature fusion methods attempt to extract more features from images for forgery detection. Using frequency domain features for forgery detection reveals traces of forgery in the frequency domain. Subsequently, based on classical frequency domain analysis, classifiers such as SVM are used for classification, achieving good detection results with a small number of labeled training samples. Addressing the compression problem of forged face images, it was found that mining forgery patterns in the frequency domain signal can solve the compression issue. A face forgery network (F3-Net) was proposed, using a two-stage collaborative learning framework to deeply mine forgery patterns. Finally, it significantly outperformed other methods on the compressed FaceForensics++ dataset. Since the swapped face and its context differ, a two-branch network was constructed: one for facial semantic segmentation as input, and the other for facial context (e.g., hair, ears, neck) as input. This method utilizes the features of both branches for forgery detection.

[0014] This type of method leverages other tasks to improve detection performance or is directly used for forgery detection. It employs a multi-task learning approach to simultaneously classify videos as real or fake while locating forgery regions at the pixel level. The authors use a Y-shaped decoder and three loss functions to constrain the network, aiming to share valuable features across multiple tasks. The forgery detection problem is defined as a pixel-level image segmentation task. A fully convolutional network is used for feature extraction, and the segmentation results are binarized to label forgery regions in the image.

[0015] The advantage of intra-frame tampering detection methods is that training and detection speeds are often very fast, and they are particularly effective when forgery traces are obvious in a single frame. However, the disadvantage is that few single-frame methods can adaptively focus on local forged regions and detect suspicious areas.

[0016] In summary, various adversarial attack algorithms for face recognition have been proposed, but existing methods suffer from the following two technical problems: (1) The success rate of adversarial attack algorithms is low when attacking black-box models (i.e., the attacker does not know the network structure, parameters, and other information of the face recognition model), and the attack algorithms trained using alternative models have poor attack performance when attacking other models, i.e., the transferability of the algorithms is weak. (2) Existing methods often result in poor visual effect of the generated adversarial attack images, specifically manifested as poor realism of the images perceived by the naked eye and alteration of the identity of the observed face. Summary of the Invention

[0017] This invention provides a face recognition adversarial attack method based on attribute editing, and proposes three attribute attack modules, which are named the latent space mapping module, the two-stream attribute editing module, and the gradient integration manifold in-attack module.

[0018] The face adversarial attack method based on attribute editing proposed in this invention is mainly used to attack face recognition algorithms based on deep learning technology. By making minor attribute edits to the original face image (e.g., skin color, expression, etc.), the face recognition algorithm can mistakenly identify it as another specified face identity, while keeping the identity of the edited image unchanged to the naked eye.

[0019] The latent space mapping module proposed in this invention performs latent space mapping on the input real face image and the target image to be identified to a low-dimensional manifold representation, and improves the similarity of face identity information through feature layer fusion.

[0020] The dual-stream attribute editing module proposed in this invention consists of two steps: attribute selection and attribute editing. The attribute selection step, through the design of a set of attribute selection parameters, automatically selects different attack attributes based on different original and target images to improve model attack efficiency. The attribute editing step, through the design of a set of attribute editing parameters, optimizes the editing intensity for the best editing attribute obtained in the attribute selection step. It also introduces various controls to optimize image generation effects, achieving the best model attack success rate while maintaining the edited image quality and the visually recognizable identity information unchanged.

[0021] This invention also proposes a method for generating adversarial face attack samples based on attribute editing, including:

[0022] Step 1: Extract the original face image x src and target identity image x tgt The latent space vectors of each vector are mixed to obtain the latent space vector z. mix ;

[0023] Step 2, in the latent space vector z mix Overlay edit attribute vector ωs The product of the attribute editing direction B and the latent space vector after attribute selection is obtained. The selection image x is then generated based on this latent space vector. selected ;

[0024] Step 3: In the latent space vector z mix Overlay edit attribute vector ω s The weight matrix ω used to control the magnitude of attribute editing e The product of these terms yields the attribute-edited latent space vector, and the edited image x is generated based on this attribute-edited latent space vector. edited ;

[0025] Step 4: Edit image x respectively edited and select image x selected Both and the target identity image x tgt The first and second loss functions are constructed based on the similarity between the generated face image and the target identity image x. tgt The similarity is used to construct a third loss function; and the weight matrix ω is constrained by a fourth loss function. e Standard deviation;

[0026] Step 5: Using the total loss function formed by the first to the fourth loss functions, train and adjust the edit attribute vector ω. s and weight matrix ω e Continue until the total loss function converges, then save the currently edited image x. edited This is an adversarial example, and the person identification label of this adversarial example is the person identification of the original face image.

[0027] The aforementioned method for generating adversarial face attack samples based on attribute editing, wherein the first loss function Second loss function They are respectively:

[0028]

[0029]

[0030] Where f(.) represents the feature extractor of the image, D(.) represents the cosine similarity between two features, and the feature extractor is a pre-trained face recognition model.

[0031] The method for generating adversarial attack samples based on attribute editing, wherein step 5 includes: using the recognition accuracy of the adversarial sample by the face recognition model to be robustly evaluated as the adversarial robustness evaluation result of the face recognition model.

[0032] The method for generating adversarial attack samples for faces based on attribute editing, wherein step 5 includes: fine-tuning the target face recognition model with the adversarial sample to improve the recognition accuracy of the target face recognition model.

[0033] This invention also proposes a face adversarial attack sample generation system based on attribute editing, which includes:

[0034] The latent space mapping module is used to map the original face image x src and target identity image x tgt The latent space vectors of each vector are mixed to obtain the latent space vector z. mix ;

[0035] The dual-stream attribute editing module is used to edit the latent space vector z. mix Overlay edit attribute vector ω s The product of the attribute editing direction B and the latent space vector after attribute selection is obtained. The selection image x is then generated based on this latent space vector. selected ; used in the latent space vector z mix Overlay edit attribute vector ω s The weight matrix ω used to control the magnitude of attribute editing e The product of these terms yields the attribute-edited latent space vector, and the edited image x is generated based on this attribute-edited latent space vector. edited ;

[0036] The manifold-in-the-loop attack module is used to attack images x based on the edited image x. edited and select image x selected Both and the target identity image x tgt The first and second loss functions are constructed based on the similarity between the generated face image and the target identity image x. tgt The similarity is used to construct a third loss function; and the weight matrix ω is constrained by a fourth loss function. e The standard deviation; the total loss function composed of the first to the fourth loss functions; and the training and adjustment of the edit attribute vector ω using the total loss function. s and weight matrix ω e Continue until the total loss function converges, then save the currently edited image x. edited This is an adversarial example, and the person identification label of this adversarial example is the person identification of the original face image.

[0037] The aforementioned face adversarial attack sample generation system based on attribute editing, wherein the first loss function Second loss function They are respectively:

[0038]

[0039]

[0040] Where f(.) represents the feature extractor of the image, D(.) represents the cosine similarity between two features, and the feature extractor is a pre-trained face recognition model.

[0041] The aforementioned attribute-edit-based face adversarial attack sample generation system, wherein the manifold attack module is used to: use the recognition accuracy of the adversarial sample by the face recognition model to be robustly evaluated as the adversarial robustness evaluation result of the face recognition model.

[0042] The aforementioned face adversarial attack sample generation system based on attribute editing, wherein the manifold attack module is used to: fine-tune the target face recognition model with the adversarial sample to improve the recognition accuracy of the target face recognition model.

[0043] The present invention also proposes a storage medium for storing a program that executes any of the attribute-edit-based face adversarial attack sample generation methods.

[0044] This invention also proposes a client for any of the aforementioned attribute-edit-based face adversarial attack sample generation systems.

[0045] The gradient ensemble manifold attack module proposed in this invention utilizes multiple face recognition models as discriminant models. By learning the attribute selection parameters and attribute editing parameters within the manifold, it obtains the optimal adversarial face samples, thereby improving the transfer effect of adversarial face attacks. In other words, it can achieve good attack results even against unknown black-box models. Attached Figure Description

[0046] Figure 1 This is a schematic diagram of an adversarial attack process based on attribute editing.

[0047] Figure 2 The image shows the results of a qualitative test of the visual quality of the image. Detailed Implementation

[0048] This invention proposes a face adversarial attack method based on attribute editing. It utilizes facial attribute information for adversarial attacks. Based on the original image and the target image to be identified as a specific person by the face recognition system, it automatically learns the optimal attribute category and editing strength, thereby maintaining the quality of the original image and the visually identifiable identity information while successfully launching the attack. More specifically, the method includes the following parts:

[0049] (1) The method of latent space mapping and latent vector fusion proposed in this invention can map real images to low-dimensional representations for editing, while improving the similarity between the original face and the target face information.

[0050] (2) The face attribute editing method proposed in this invention can automatically learn the optimal attribute type and attribute editing intensity by introducing attribute selection parameters and attribute editing parameters. At the same time, the introduced standard deviation loss and LPIPS loss can ensure that the generated image quality and the identity information observed by the naked eye remain unchanged.

[0051] (2) The gradient integration manifold attack proposed in this invention integrates the gradients of multiple face recognition models during training to optimize the representation of the image within the manifold, thereby improving the transferability of adversarial attacks and the image generation effect.

[0052] Experimental results demonstrate that the attribute editing adversarial attack method proposed in this invention achieves a high attack success rate on multiple black-box models and commercial APIs, and also achieves optimal results in the test metrics of image generation effect, verifying the effectiveness of the method.

[0053] To make the above features and effects of the present invention clearer and easier to understand, specific embodiments are described below, and detailed descriptions are provided in conjunction with the accompanying drawings.

[0054] To address the technical problems existing in current technologies, we propose a face recognition adversarial attack method based on attribute editing, such as... Figure 1 As shown, a two-stream attribute editing method is used to maintain high-quality face image generation while preventing the face recognition system from misidentifying the face as a specified identity. The following describes the various modules of the network.

[0055] (1) Implicit space mapping module

[0056] x src and target identity image x tgt Mapping to the image's latent space yields the original face image x. src and target identity image x tgt After obtaining the respective latent space vectors, they are mixed to obtain a new latent space vector z. mix The mixed vector z mix It is closer to the target identity in terms of identity information, and closer to the original human face in terms of visual effect. Among them, z mix and x src and x tgt Both are 18-layer vectors, z mix The first n layers come from x src The first n layers, z mix The later (18-n) layers come from x tgt The subsequent (18-n) layers.

[0057] (2) Dual-stream attribute editing module

[0058] By analyzing z mix By editing, we can obtain the corresponding attribute editing effects of the image. Specifically, based on the attribute editing direction B, two steps are designed: attribute selection and attribute editing. The attribute selection step utilizes the N-dimensional vector ω representing the edited attribute. s Learn discrete attribute editing directions. Editing attributes means editing objects, such as eyes, glasses, hair, expressions, etc., through z... mix Superimposed ω s The product of B and the vector of attribute selection yields the latent space vector. Finally, the attribute-selected image x is generated using a pre-trained generator. selected The attribute selection is to prepare for subsequent counter-attacks; combined with these counter-attacks, the generated x... selected The image's appearance has changed attributes, and it also contains adversarial information.

[0059] The attribute editing steps are similar to the attribute selection steps described above. The editing range is based on the editing direction; only editing along a specific direction can achieve the desired editing effect. A weight matrix ω with the same dimensions as B is used. e Controlling the extent of attribute editing, the generator then produces the attribute-edited image x. edited The two steps above can be summarized as follows: the purpose of attribute selection is to learn the most effective attribute editing direction B for the target face recognition algorithm, and the purpose of attribute editing is to learn the magnitude of the editing against adversarial attacks.

[0060] (3) Intramandibular adversarial attacks based on gradient integration

[0061] For ω s and ω e The parameters are optimized using gradient descent, and the adversarial losses for attribute selection and attribute editing are expressed by Equations 1 and 2, respectively:

[0062]

[0063]

[0064] Where f(.) represents the feature extractor of the image, and D(.) represents the cosine similarity between two features. The gradient descent algorithm is used to optimize the parameter update and reduce the adversarial loss of Equations 1 and 2.

[0065] Simultaneously, we utilize an ensemble training strategy to enhance the transferability against adversarial attacks. Specifically, we employ multiple pre-trained face recognition models as white-box models (…). Figure 1 The rightmost FR1, FR2, ... FR n (Having mastered all the information about the model's network structure, parameters, etc.) to extract and fuse facial image features.

[0066] To ensure the generated face image x edited Quality, using the image similarity metric LPIPS to construct the loss function L LPIPS To constrain x edited Approaching x src Equation 3 represents:

[0067] L LPIPS =LPIPS(x edited ,x src Formula 3

[0068] In addition, to constrain the orientation during the face attribute editing process, we designed a standard deviation loss to limit ω. e The standard deviation is expressed by Equation 4.

[0069] L std =STD(ω e Equation 4

[0070] Finally, the overall adversarial attack training loss function is expressed by Equation 5:

[0071]

[0072] Among them Λ T These represent the hyperparameters between these losses.

[0073] In the actual experiments, the model used a pre-trained generator such as Karres to generate images. IRSE50, FaceNet, IR152, and Mobileface were selected as the face recognition models for training and testing (three were used as training models, and the remaining one as the testing model). Additionally, we tested the adversarial attack performance on two open commercial face recognition APIs, Aliyun and Face++. For testing metrics, we primarily used Attack Success Rate (ASR) and Face Similarity Confidence (Conf.) to measure the algorithm's attack effectiveness. Pixel-level L2 distance, LPIPS distance, and MS-SSIM distance were used to measure the quality of the generated images. For comparison, we tested the best face recognition adversarial attack methods of recent years, including PGD, MI-FGSM, TIP-IM, AMT-GAN, and Semantic-adv.

[0074] Experimental results

[0075] To verify the effectiveness of the method of this invention, we chose to test it on the CelebA-HQ dataset, which is a widely used face dataset that provides a large amount of real high-definition face image data.

[0076] • Attack success rate test

[0077] First, the supply success rate under different black-box models was tested, and the results are shown in Table 1:

[0078]

[0079] Table 1 Comparison of attack success rates under different models

[0080] For different black-box models, this method outperforms the current best model in terms of attack success rate for IRSE50, IR152, MobileFace models, and the average attack success rate of the four models.

[0081] The attack success rate was also tested on Aliyun and Face++. Two similar attack methods based on altering facial attributes, SemanticADV and AMT-GAN, were tested, with a false acceptance rate (FAR) of 10. -5 10 -4 10 -3 As the threshold for judging the success of the attack, the results are shown in Table 2:

[0082]

[0083] Table 2 Comparison of attack success rates under different commercial APIs

[0084] Test results on black-box models and APIs show that this method achieves the best attack success rate compared to other advanced face adversarial attack methods. The success rate of black-box attacks is 24.54% higher than the best method, and a significant improvement is achieved in API attacks.

[0085] Image quality test

[0086] Secondly, quantitative and qualitative tests were conducted on the image quality of similar face attribute editing methods, SemanticADV and AMT-GAN. For quantitative experiments, we used three metrics: L2, LPIPS, and MS-SSIM. Lower L2 and LPIPS scores indicate better image quality, while higher MS-SSIM scores also indicate better image quality. The quantitative test results are shown in Table 3, where Ours-w / o attributes represents a comparison experiment with our method without the attribute editing module, and Ours-w / omixing represents a comparison experiment with our method without the latent vector fusion module.

[0087]

[0088] Table 3 Comparison results of different visual quality indicators

[0089] Experiments show that our method outperforms AMT-GAN in quantitative image quality metrics. It's worth noting that while SemanticAdv outperforms our method in metrics, as shown in Tables 1 and 2, its attack success rate is extremely low, lacking practical usability. Furthermore, comparative experiments with our own method demonstrate that our attribute editing module effectively improves image quality, while the latent vector mixing module, although slightly reducing image quality, significantly increases the attack success rate.

[0090] Qualitative test results as follows Figure 2 ,in Figure 2 The first column shows the original image, the second column shows the effect of the SemanticAdv algorithm, the third column shows the effect of the AMT-GAN algorithm, and the fourth column shows the corresponding adversarial example image. The values ​​below the images represent the confidence level in determining that the two images belong to the same person under the Face++ test. The higher the confidence level, the higher the probability that the face recognition algorithm considers the two faces to belong to the same person.

[0091] observe Figure 2 This demonstrates that our method outperforms AMT-GAN in terms of image generation realism and is far superior to SemantivAdv in terms of adversarial attack effectiveness, achieving a balance between attack success rate and image quality.

[0092] The following are system embodiments corresponding to the above method embodiments. This embodiment can be implemented in conjunction with the above embodiments. The relevant technical details mentioned in the above embodiments are still valid in this embodiment, and will not be repeated here to reduce repetition. Accordingly, the relevant technical details mentioned in this embodiment can also be applied to the above embodiments.

[0093] This invention also proposes a face adversarial attack sample generation system based on attribute editing, which includes:

[0094] The latent space mapping module is used to map the original face image x src and target identity image x tgt The latent space vectors of each vector are mixed to obtain the latent space vector z. mix ;

[0095] The dual-stream attribute editing module is used to edit the latent space vector z. mix Overlay edit attribute vector ω s The product of the attribute editing direction B and the latent space vector after attribute selection is obtained. The selection image x is then generated based on this latent space vector. selected ; used in the latent space vector z mix Overlay edit attribute vector ω s The weight matrix ω used to control the magnitude of attribute editing eThe product of these terms yields the attribute-edited latent space vector, and the edited image x is generated based on this attribute-edited latent space vector. edited ;

[0096] The manifold-in-the-loop attack module is used to attack images x based on the edited image x. edited and select image x selected Both and the target identity image x tgt The first and second loss functions are constructed based on the similarity between the generated face image and the target identity image x. tgt The similarity is used to construct a third loss function; and the weight matrix ω is constrained by a fourth loss function. e The standard deviation; the total loss function composed of the first to the fourth loss functions; and the training and adjustment of the edit attribute vector ω using the total loss function. s and weight matrix ω e Continue until the total loss function converges, then save the currently edited image x. edited This is an adversarial example, and the person identification label of this adversarial example is the person identification of the original face image.

[0097] The aforementioned face adversarial attack sample generation system based on attribute editing, wherein the first loss function Second loss function They are respectively:

[0098]

[0099]

[0100] Where f(.) represents the feature extractor of the image, D(.) represents the cosine similarity between two features, and the feature extractor is a pre-trained face recognition model.

[0101] The aforementioned attribute-edit-based face adversarial attack sample generation system, wherein the manifold attack module is used to: use the recognition accuracy of the adversarial sample by the face recognition model to be robustly evaluated as the adversarial robustness evaluation result of the face recognition model.

[0102] The aforementioned face adversarial attack sample generation system based on attribute editing, wherein the manifold attack module is used to: fine-tune the target face recognition model with the adversarial sample to improve the recognition accuracy of the target face recognition model.

[0103] The present invention also proposes a storage medium for storing a program that executes any of the attribute-edit-based face adversarial attack sample generation methods.

[0104] This invention also proposes a client for any of the aforementioned attribute-edit-based face adversarial attack sample generation systems.

Claims

1. A method for generating adversarial face attack samples based on attribute editing, characterized in that, include: Step 1: Extract the original face image x src and target identity image x tgt The latent space vectors of each vector are mixed to obtain the latent space vector z. mix ; Step 2, in the latent space vector z mix Overlay edit attribute vector ω s The product of the attribute editing direction B and the latent space vector after attribute selection is obtained. The selection image x is then generated based on this latent space vector. selected ; Step 3: In the latent space vector z mix Overlay edit attribute vector ω s The weight matrix ω used to control the magnitude of attribute editing e The product of these terms yields the attribute-edited latent space vector, and the edited image x is generated based on this attribute-edited latent space vector. edited ; Step 4: Edit image x respectively edited and select image x selected Both and the target identity image x tgt The first and second loss functions are constructed based on the similarity between the generated face image and the target identity image x. tgt The similarity is used to construct a third loss function; and the weight matrix ω is constrained by a fourth loss function. e Standard deviation; Step 5: Using the total loss function formed by the first to the fourth loss functions, train and adjust the edit attribute vector ω. s and weight matrix ω e Continue until the total loss function converges, then save the currently edited image x. edited This is an adversarial example, and the person identification label of this adversarial example is the person identification of the original face image.

2. The method for generating adversarial face attack samples based on attribute editing as described in claim 1, characterized in that, The first loss function Second loss function They are respectively: Where f(.) represents the feature extractor of the image, D(.) represents the cosine similarity between two features, and the feature extractor is a pre-trained face recognition model.

3. The method for generating adversarial face attack samples based on attribute editing as described in claim 1, characterized in that, Step 5 includes: using the recognition accuracy of the face recognition model to be robustly evaluated for the adversarial example as the adversarial robustness evaluation result of the face recognition model.

4. The method for generating adversarial face attack samples based on attribute editing as described in claim 1, characterized in that, Step 5 includes: fine-tuning the target face recognition model with the adversarial example to improve the recognition accuracy of the target face recognition model.

5. A face adversarial attack sample generation system based on attribute editing, characterized in that, include: The latent space mapping module is used to map the original face image x src and target identity image x tgt The latent space vectors of each vector are mixed to obtain the latent space vector z. mix ; The dual-stream attribute editing module is used to edit the latent space vector z. mix Overlay edit attribute vector ω s The product of the attribute editing direction B and the latent space vector after attribute selection is obtained. The selection image x is then generated based on this latent space vector. selected ; Used in the latent space vector z mix Overlay edit attribute vector ω s The weight matrix ω used to control the magnitude of attribute editing e The product of these terms yields the attribute-edited latent space vector, and the edited image x is generated based on this attribute-edited latent space vector. edited ; The manifold-in-the-loop attack module is used to attack images x based on the edited image x. edited and select image x selected Both and the target identity image x tgt The first and second loss functions are constructed based on the similarity between the generated face image and the target identity image x. tgt The similarity is used to construct a third loss function; and the weight matrix ω is constrained by a fourth loss function. e The standard deviation; used to train and adjust the edit attribute vector ω using the total loss function constructed from the first to the fourth loss functions. s and weight matrix ω e Continue until the total loss function converges, then save the currently edited image x. edited This is an adversarial example, and the person identification label of this adversarial example is the person identification of the original face image.

6. The face adversarial attack sample generation system based on attribute editing as described in claim 5, characterized in that, The first loss function Second loss function They are respectively: Where f(.) represents the feature extractor of the image, D(.) represents the cosine similarity between two features, and the feature extractor is a pre-trained face recognition model.

7. The face adversarial attack sample generation system based on attribute editing as described in claim 5, characterized in that, The manifold-in-attack module is used to: use the recognition accuracy of the face recognition model to be robustly evaluated against the adversarial example as the adversarial robustness evaluation result of the face recognition model.

8. The face adversarial attack sample generation system based on attribute editing as described in claim 5, characterized in that, The manifold-in-the-loop attack module is used to fine-tune the target face recognition model with the adversarial sample in order to improve the recognition accuracy of the target face recognition model.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the computer program implements the steps of the method for generating adversarial face attack samples based on attribute editing as described in any of claims 1-4.

10. A client, characterized in that, The face adversarial attack sample generation system based on attribute editing as described in any of claims 5-8.

Images