Black box adversarial sample attack method, device, equipment and medium for face recognition

Abstract

The present invention discloses a black-box adversarial sample attack method for face recognition, which includes S1. preparing and preprocessing a data set. S2. using a DIP (Deep Image Prior) network to convert parameters to be optimized into generator network parameters, and introducing multiple feature extraction networks and directional features into the training process to improve attack accuracy and transferability. S3. using an image generation model to generate a latent code z randomly selected at the beginning of the training to obtain a final attack image, and then mixing the attack image with a clean image in the form of an image pair to input into a face recognition model to obtain face verification and recognition results. In the attack scenario, the DIP network is used to expand the search space of the attack optimization algorithm, thereby solving the problem of low attack accuracy and transferability of the existing model in black-box attacks.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence, and in particular to a black-box adversarial sample attack method, device, equipment and medium for face recognition. Background Art

[0002] With the widespread adoption of information technology, facial recognition technology, powered by deep neural networks, has gained widespread application. However, the vulnerabilities and insecurity exposed by deep learning have become a hidden danger. The ability to mislead facial recognition systems by launching adversarial attacks on input facial samples has become a key component in revealing the vulnerabilities of deep learning networks and a crucial prerequisite for defense. Facial features hold a crucial position in biometrics due to their unique advantages. Compared to other biometric features, facial features are more intuitive and convenient for identity verification, require no contact during collection, and can be collected covertly in public safety scenarios without the need for cooperation. Precisely because of these advantageous characteristics, facial recognition technology has found widespread application. For example, facial recognition technology is used in traffic control, enabling comprehensive, timely, and effective control of traffic violations such as traffic violations, drunk driving, overloading, and running red lights. It is also used in video surveillance, enabling rapid identification and real-time surveillance of criminal suspects. Compared with traditional manual inspections and supervision, automatic facial recognition can bring improvements in efficiency, safety, cost, and quality. Therefore, the facial recognition security issues involved in adversarial sample attacks are crucial. How to improve the efficiency of facial recognition adversarial sample attacks has attracted increasing attention from academia and the business community.

[0003] Current mainstream adversarial attack methods can be roughly divided into two categories: white-box and black-box. White-box attacks involve attacks that exploit a thorough understanding of the architecture or parameters of the model being attacked. Derivative attack methods primarily rely on gradient optimization, employing backpropagation for continuous optimization to generate adversarial examples, with excellent results. However, their limitations stem from the difficulty in obtaining the architecture and parameters in actual production environments, limiting their widespread application. Black-box attacks, on the other hand, offer a wide range of applications but lack transparency into the model's internal architecture and parameters. When query conditions are demanding, the mainstream approach is to substitute a model, using an approximate white-box model to replace the unknown model. This approach continuously enhances the transferability of adversarial examples, ensuring effective attacks across diverse facial recognition systems.

[0004] Feature vectors, as unique representations of facial images, are widely used in current deep learning black-box attack schemes. Accurately extracting feature vectors has become a major topic of current black-box attack research. Currently, the vast majority of feature vectors are obtained within a search space defined by a small number of parameters. This limited search space restricts the attack capabilities of adversarial examples. Summary of the Invention

[0005] In order to improve the attack accuracy and sample migration problem of current adversarial samples under face recognition models, the present invention provides a black box adversarial sample attack method, device, equipment and medium for face recognition, and proposes an attack scheme based on DIP (Deep Image Prior). The main feature is to utilize the powerful image modeling ability of the DIP network to greatly expand the search space of the optimization algorithm and meet the solution conditions of the optimal solution. At the same time, by introducing a directional feature distance constraint in the loss function, the feature direction selection problem of non-directional attack is strengthened, thereby improving the attack capability and migration capability of the adversarial sample. The introduction of directional features ensures that the feature selection in the parameter optimization process is more inclined to the identity feature of the directional image, which reduces unnecessary search processes to a certain extent.

[0006] According to a first aspect of an embodiment of the present application, a black-box adversarial sample attack method for face recognition includes:

[0007] Get labeled face image data;

[0008] Preprocessing the acquired facial image data to obtain preprocessed facial image data;

[0009] The pre-processed face image data is input into the DIP (Deep Image Prior) network. Combined with the set constraint function, the DIP network parameters are iterated until the preset iteration stop condition is reached to obtain the optimized DIP (Deep Image Prior) network.

[0010] Based on the optimized DIP (Deep Image Prior) network and the set latent code z, an attack image is obtained;

[0011] The attack image is mixed with the clean image in the form of an image pair and input into the multiple face recognition model to obtain face verification and recognition results.

[0012] In some embodiments, the acquired facial image data is preprocessed, including: transforming the facial image data into a size that meets the input and output requirements of the training model, and / or using a face alignment algorithm to extract the face.

[0013] Furthermore, the integrated expression formula after the feature extraction of the multiple face recognition model is as follows:

[0014]

[0015] Among them, image input is the original input image; P(·) is the preprocessing function for the image; F i (·) is the i-th feature extractor; λi is the weight factor corresponding to the i-th eigenvector; embeddings represents the input image image input Feature integration expression after feature extraction from k face recognition networks;

[0016] In some embodiments, the constraint function includes: taking the difference characterization of the adversarial image and the input clean image as the first loss function; taking the difference characterization of the integrated feature vector of the adversarial image and the integrated feature vector of the input clean image as the second loss function; taking the difference characterization of the integrated feature vector of the directional face and the integrated feature vector of the adversarial image as the third loss function.

[0017] Furthermore, the first loss function is specifically:

[0018] L space =Dist space (Image adv ,Image input ) (2)

[0019] Among them, Image adv To fight against the image; Image input Dist is the input clean image; space (·) is the distance measurement function of the visual pixel space; L space is the distance between the adversarial image and the input clean image in the visual pixel space.

[0020] Furthermore, the second loss function is specifically:

[0021] L feature =Dist feature (embeddings adv ,embeddings input ) (3)

[0022] Among them, embeddings adv It is an integrated expression of the adversarial image after multiple face recognition network feature extraction; embeddings input Dist is the integrated expression of the input clean image after multiple face recognition network feature extraction; feature (·) is the distance measurement function in the face feature space; L feature is the distance between the adversarial image and the input clean image in the face feature space.

[0023] Furthermore, the third loss function is specifically:

[0024] L target =Dist feature (embeddingsadv ,embeddings target ) (4)

[0025] Among them, embeddings adv It is an integrated expression of the adversarial image after multiple face recognition network feature extraction; embeddings target It is the integrated expression of feature-guided images after multiple face recognition network feature extraction; Dist feature (·) is the distance measurement function in the face feature space; L target The distance between the adversarial image and the feature guidance image in the face feature space.

[0026] Furthermore, the attack image is mixed with the clean image as an image pair and input into the face recognition model to obtain face verification and recognition results, including:

[0027] During face verification, the clean, unprocessed face image data and the corresponding adversarial processed face image data are first paired one by one, and then jointly input into a uniformly selected face recognition model as a test set, ultimately obtaining the classification probability of each adversarial face image data. The accuracy rate ACC and verification rate Val are used to evaluate the verification performance of the face recognition black box adversarial sample attack method. The specific process is as follows: each clean, unprocessed face image data and the adversarial processed face image data are paired in pairs to form positive and negative test samples, where the positive test samples are samples formed by pairing each clean, unprocessed face image data with the adversarial processed face image data with the same label, and the negative samples are samples formed by pairing each clean, unprocessed face image data with clean, unprocessed face image data with different labels. Therefore, the accuracy rate ACC indicator can be expressed as:

[0028]

[0029] In formula (6), TP refers to the number of positive test samples judged as positive test samples by the feature classification network according to the classification probability, TN refers to the number of negative test samples judged as negative test samples by the feature classification network according to the classification probability, FP refers to the number of negative test samples judged as positive test samples by the feature classification network according to the classification probability, and FN refers to the number of positive test samples judged as negative test samples by the feature classification network according to the classification probability;

[0030] The verification rate Val indicator can be expressed as:

[0031]

[0032] in, The feature classification network determines the positive test samples as the proportion of all true positive test samples according to the classification probability; It is the ratio of negative test samples that are judged as positive test samples by the feature classification network according to the classification probability to all true negative test samples.

[0033] The steps to verify the attack are as follows:

[0034] Step 1: Randomly select 3,000 pairs of positive and negative samples, for a total of 6,000 pairs of samples. Positive samples consist of two different face images of the same identity, and negative samples consist of face images of different identities;

[0035] Step 2: Replace a face image in the positive sample with the corresponding adversarial sample;

[0036] Step 3: Select a face recognition model and input 6,000 pairs of samples into the model for verification;

[0037] Step 4: Get ACC and Val.

[0038] The present invention is an attack method, so it is hoped that the test result ACC after the attack is as small as possible. According to the above steps, the minimum value that ACC can reach is 50%, so the closer it is to this value, the better the attack effect. The Val indicator needs to ensure the same It is worthwhile to analyze The minimum value can reach 0%, so the better the attack effect, The closer it is to 0%.

[0039] Furthermore, the preset iteration stopping condition includes that the training reaches a preset number of training times, or the total loss function reaches a preset threshold, wherein the total loss function is as follows:

[0040] L total =α space L space -α feature L feature +α target L target (7)

[0041] Among them, α space , α feature , α target is the weight factor of each loss; L total is the total loss function.

[0042] According to a second aspect of an embodiment of the present application, a black-box adversarial sample attack device for face recognition includes:

[0043] at least one processor;

[0044] at least one memory for storing at least one program;

[0045] When the at least one program is executed by at least one processor, the at least one processor implements the black-box adversarial sample attack method for face recognition described in the first aspect.

[0046] According to a third aspect of an embodiment of the present application, a computer device includes: the black-box adversarial sample attack device for face recognition described in the second aspect.

[0047] According to a fourth aspect of an embodiment of the present application, a computer-readable storage medium stores a program executable by a processor, and when the program executable by the processor is executed by the processor, it is used to implement the black-box adversarial sample attack method for face recognition described in any one of the first aspects.

[0048] The beneficial effects of the present invention are:

[0049] 1. Leveraging the powerful modeling capabilities of DIP, the original image optimization search range in the pixel domain was shifted to the parameter space of the DIP self-supervised network. Essentially, this approach utilizes the latent code z and the DIP network to represent images. This expanded search space facilitates optimal solution selection.

[0050] 2. The introduction of the third loss function helps in selecting the search path, provides a more specific direction for the entire search process, and makes it move closer to the directional feature space, which helps to save unnecessary search attempts. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] Figure 1 A schematic diagram of an implementation environment for a black-box adversarial sample attack method for face recognition provided in an embodiment of the present application;

[0052] Figure 2 This is an algorithm flow chart of a black-box adversarial sample attack method for face recognition provided in an embodiment of the present application;

[0053] Figure 3 This is a schematic diagram of the structure of a black-box adversarial sample attack device for face recognition provided in an embodiment of the present application;

[0054] Figure 4 A schematic diagram of the structure of a computer device provided in an embodiment of the present application;

[0055] Figure 5 A schematic diagram of the input format of FaceNet positive and negative sample pairs provided in an embodiment of the present application;

[0056] Figure 6 This is a black box adversarial sample effect diagram for face recognition provided in an embodiment of the present application. DETAILED DESCRIPTION

[0057] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. The temporary and first in the present invention are for illustrating different stages in the algorithm training and have no limiting meaning. With respect to the embodiments of the present invention, all other embodiments obtained by those skilled in the art without making creative work are within the scope of protection of the present invention.

[0058] The black box adversarial attack method for face recognition provided by the present invention can be applied to Figure 1 In the application environment diagram shown. The application environment includes a terminal 120 and a server 140. The terminal 120, in the first application environment, is connected to the server 140 via a network. There is no limit on the number of terminals 120 and servers 140. The network includes but is not limited to: a wide area network, a metropolitan area network or a local area network. At this time, the terminal 120 can collect pictures and send them to the server for black-box adversarial sample attack or verification for face recognition. In the second application environment, the terminal 120 includes a mobile phone, a tablet, etc. The terminal 120 has the functions of image collection and black-box adversarial sample attack or verification for face recognition. Therefore, the black-box adversarial sample attack method for face recognition provided by the present invention does not limit the application environment.

[0059] like Figure 2 As shown, according to a first aspect of an embodiment of the present application, a black box adversarial sample attack for face recognition includes the following steps:

[0060] S1. Prepare and preprocess the dataset. A labeled face dataset is used. Label-free image generation is performed, while labeled data is used for face recognition. The preprocessing process includes: 1. Resizing. To meet the input and output requirements of the training model, the input image size must be strictly controlled to ensure it meets the model input requirements. Common sizes include 124×124 and 160×160. 2. Face alignment. To more accurately extract the face model, a suitable face alignment algorithm, such as MTCNN, is used for face extraction to filter out non-face background areas.

[0061] S2. Use the DIP network to convert the parameters to be optimized into the generator network parameters. The training process introduces multiple feature extraction networks and directional features to improve attack accuracy and transferability.

[0062] The final effect of image training needs to be considered based on two aspects. First, there is no significant difference between the adversarial image and the original clean input image in the visual pixel domain. The difference between the adversarial image and the input clean image is characterized as the first loss function, ensuring that its value continues to decrease with iteration. The smaller the value, the better the visual effect and the stronger the image's authenticity. Second, the adversarial image and the original clean image need to show significant differences in the expression of the feature domain to distinguish the difference in facial identity between the adversarial image and the input clean image. Therefore, the difference between the integrated feature vector of the adversarial image and the integrated feature vector of the input clean image is characterized as the second loss function, ensuring that its value continues to increase with iteration. The larger the value, the more drastic the identity change and the stronger the attack effect.

[0063] The multi-extraction network comprises multiple face recognition-related networks, each using its own feature extractor to extract facial features, which serve as the feature representation of the same face in different models. For transferability, the feature vectors extracted by these different face recognition models need to be integrated and ultimately incorporated into the loss function. To improve attack accuracy, additional face images with significantly different identities from the original image are introduced during the attack process. The feature representations of these images also require an integrated representation using the aforementioned different feature extraction networks, thereby guiding the original image's facial identity features to change in the desired direction. Therefore, the third loss function is constructed to characterize the difference between the integrated feature vectors of the directional face and the integrated feature vectors of the adversarial image. Its value must decrease with each iteration; a smaller value indicates that the adversarial image's identity features are closer to the directional features, thereby improving the attack's effectiveness.

[0064] S3. The latent code z randomly selected at the beginning of the training is used to generate the final attack image through the image generation model. The attack image is mixed with the clean image and input into the face recognition model in the form of an image pair to obtain face verification and recognition results.

[0065] Furthermore, the integrated expression formula after the feature extraction of the multiple face recognition model is as follows:

[0066]

[0067] Among them, image input is the original input image; P(·) is the preprocessing function for the image, and the MTCNN algorithm can generally be used to achieve size and face alignment; F i (·) is the i-th feature extractor. The feature extractors that can be selected are generally FaceNet, ArcFace, CosFace, etc., and appropriate feature extractors can be selected and combined according to different task scenarios; λ i is the weight factor corresponding to the i-th eigenvector; embeddings represents the input image imageinput Feature integration expression after feature extraction from k face recognition networks;

[0068] Furthermore, the first loss function is specifically:

[0069] L space =Dist space (Image adv ,Image input ) (2)

[0070] Among them, Image adv To fight against the image; Image input Dist is the input clean image; space (·) is the distance measurement function in the visual pixel space. The smaller the value of this function, the smaller the difference between the adversarial sample and the input clean image pixels, and the higher the quality of the adversarial image. space is the distance between the adversarial image and the input clean image in the visual pixel space.

[0071] Furthermore, the second loss function is specifically:

[0072] L feature =Dist feature (embeddings adv ,embeddings input ) (3)

[0073] Among them, embeddings adv It is an integrated expression of the adversarial image after multiple face recognition network feature extraction; embeddings input Dist is the integrated expression of the input clean image after multiple face recognition network feature extraction; feature (·) is the distance measurement function of the face feature space. The larger the value of this function, the greater the feature distance between the adversarial sample and the input clean image, and the greater the identity feature, which is one of the most important manifestations of attack accuracy; L feature is the distance between the adversarial image and the input clean image in the face feature space.

[0074] Furthermore, the third loss function is specifically:

[0075] L target =Dist feature (embeddings adv ,embeddings target ) (4)

[0076] Among them, embeddings advIt is an integrated expression of the adversarial image after multiple face recognition network feature extraction; embeddings target It is the integrated expression of feature-guided images after multiple face recognition network feature extraction; Dist feature (·) is the distance measurement function of the face feature space. The smaller the value of this function, the higher it should be. In order to guide the adversarial sample to develop in the direction we expect, the guided image method is used to make the identity characteristics of the adversarial sample close to the directional image characteristics, which helps to further improve the attack accuracy; L target The distance between the adversarial image and the feature guidance image in the face feature space.

[0077] Furthermore, in step S3, the attack image is mixed with the clean image in the form of an image pair and inputted into the face recognition model to obtain face verification and recognition results, which includes the following steps:

[0078] During face verification, the clean, unprocessed face image data and the corresponding adversarial processed face image data are first paired one by one, and then jointly input into a uniformly selected face recognition model as a test set, ultimately obtaining the classification probability of each adversarial face image data. The accuracy rate ACC and verification rate Val are used to evaluate the verification performance of the face recognition black box adversarial sample attack method. The specific process is as follows: each clean, unprocessed face image data and the adversarial processed face image data are paired in pairs to form positive and negative test samples, where the positive test samples are samples formed by pairing each clean, unprocessed face image data with the adversarial processed face image data with the same label, and the negative samples are samples formed by pairing each clean, unprocessed face image data with clean, unprocessed face image data with different labels. Therefore, the accuracy rate ACC indicator can be expressed as:

[0079]

[0080] In formula (6), TP refers to the number of positive test samples judged as positive test samples by the feature classification network according to the classification probability, TN refers to the number of negative test samples judged as negative test samples by the feature classification network according to the classification probability, FP refers to the number of negative test samples judged as positive test samples by the feature classification network according to the classification probability, and FN refers to the number of positive test samples judged as negative test samples by the feature classification network according to the classification probability;

[0081] The verification rate Val indicator can be expressed as:

[0082]

[0083] in, The feature classification network determines the positive test samples as the proportion of all true positive test samples according to the classification probability; It is the ratio of negative test samples that are judged as positive test samples by the feature classification network according to the classification probability to all true negative test samples.

[0084] The steps to verify the attack are as follows:

[0085] Step 1: Randomly select 3,000 pairs of positive and negative samples, for a total of 6,000 pairs of samples. Positive samples consist of two different face images of the same identity, and negative samples consist of face images of different identities;

[0086] Step 2: Replace a face image in the positive sample with the corresponding adversarial sample;

[0087] Step 3: Use the FaceNet face recognition model and input 6000 pairs of samples into the model for verification;

[0088] Step 4: Get ACC and Val.

[0089] The present invention is an attack method, so it is hoped that the test result ACC after the attack is as small as possible. According to the above steps, the minimum value that ACC can reach is 50%, so the closer it is to this value, the better the attack effect. In this embodiment, ACC is about 86%. The Val indicator needs to ensure the same It is worthwhile to analyze The minimum value can reach 0%. The closer it gets to 28%.

[0090] Furthermore, the preset iteration stopping condition in step S3 includes that the training reaches a preset number of training times, or the total loss function reaches a preset threshold, wherein the total loss function is as follows:

[0091] L total =α space L space -α feature L feature +α target L target (7)

[0092] Among them, α space , α feature , α target is the weight factor of each loss; L total is the total loss function.

[0093] Acquisition and Preprocessing Module 201 is responsible for preparing and preprocessing a dataset, using a labeled face dataset. Labels are not involved in the image generation process, while labeled data is used for face recognition. The preprocessing process includes: 1. Resizing: To meet the input and output requirements of the training model, the input image size must be strictly controlled to ensure it meets the model input requirements. 2. Face alignment: To more accurately extract the face model, a suitable face alignment algorithm is used for face extraction, filtering out non-face background areas.

[0094] The attack image generation module 202 uses the DIP network to convert the parameters to be optimized into the generator network parameters. The training process introduces multiple feature extraction networks and directional features to improve the attack accuracy and transferability.

[0095] The final effect of image training needs to be considered based on two aspects. First, there is no significant difference between the adversarial image and the original clean input image in the visual pixel domain. The difference between the adversarial image and the input clean image is characterized as the first loss function, ensuring that its value continues to decrease with iteration. The smaller the value, the better the visual effect and the stronger the image's authenticity. Second, the adversarial image and the original clean image need to show significant differences in the expression of the feature domain to distinguish the difference in facial identity between the adversarial image and the input clean image. Therefore, the difference between the integrated feature vector of the adversarial image and the integrated feature vector of the input clean image is characterized as the second loss function, ensuring that its value continues to increase with iteration. The larger the value, the more drastic the identity change and the stronger the attack effect.

[0096] The multi-extraction network comprises multiple face recognition-related networks, each using its own feature extractor to extract facial features, which serve as the feature representation of the same face in different models. For transferability, the feature vectors extracted by these different face recognition models need to be integrated and ultimately incorporated into the loss function. To improve attack accuracy, additional face images with significantly different identities from the original image are introduced during the attack process. The feature representations of these images also require an integrated representation using the aforementioned different feature extraction networks, thereby guiding the original image's facial identity features to change in the desired direction. Therefore, the third loss function is constructed to characterize the difference between the integrated feature vectors of the directional face and the integrated feature vectors of the adversarial image. Its value must decrease with each iteration; a smaller value indicates that the adversarial image's identity features are closer to the directional features, thereby improving the attack's effectiveness.

[0097] The verification and recognition module 203 is used to verify and recognize the face of the input image in the face recognition model and the recognition result.

[0098] According to the third aspect of the embodiment of the present application, Figure 4 As shown, a computer device includes:

[0099] at least one processor 301;

[0100] At least one memory 302, configured to store at least one program;

[0101] The processor 301 is used to provide computing and control capabilities to support the operation of the entire server. The memory 302 may include a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The computer program can be executed by the processor. When the at least one program is executed by at least one processor, the at least one processor implements the black box adversarial sample attack method for face recognition described in the first aspect. The above-mentioned computer device includes a mobile phone, a tablet computer, a personal digital assistant, a wearable device, or a server, etc. This embodiment does not impose specific restrictions on the computer.

[0102] According to a fourth aspect of an embodiment of the present application, a computer-readable storage medium stores a program executable by a processor, and when the program executable by the processor is executed by the processor, it is used to implement the black-box adversarial sample attack method for face recognition described in any one of the first aspects.

[0103] In order to illustrate the black-box adversarial sample attack method for face recognition provided by the present invention, the clean image data input in this embodiment are all derived from the LFW dataset, and the image data to be attacked include 2551 images of 1342 people. The MTCNN face detection and alignment network is used to detect and align the face areas of the above-mentioned LFW partial data images. According to the coordinates of the five facial feature points, namely the two eye centers, the nose tip and the two mouth corners, the resolution of the high-definition face images of the LFW partial data is normalized to 160×160 through affine transformation. The 160×160 size adjustment fits the input size of the face recognition network in the subsequent verification and recognition modules. The purpose of MTCNN face detection alignment is to better extract the face image identity feature vector for each image.

[0104] In this embodiment, the total loss function

[0105] L total =α space L space -α feature L feature +α target L target In the first loss function L space The distance between the pixels in the space is measured by the structural similarity of DSSIM; the second loss function L feature And the third loss function L targetThe distances in the feature space are all characterized by cosine similarity. The weight factor is initially set to: space =0.1,α feature =0.05,α target =0.03. The DIP self-supervised network architecture is based on the article “Deep Image Prior” published by Dmitry Ulyanov et al.

[0106] This example trains the above model in the PyTorch deep learning framework. The Adam optimizer is used, the initial learning rate is set to 0.1, and the weight decay is set to 1e -5 , the set iteration stop condition ends after 3000 iterations of training, and the total loss function is around -0.005;

[0107] In this embodiment, the backbone network used in the verification and recognition process is Inception ResNet v1, the overall architecture adopts the FaceNet model, and the network model parameters are obtained by training the VGGFace2 dataset.

[0108] In this embodiment, a total of 6000 pairs of samples are input to FaceNet, with 3000 pairs of positive and negative samples. Random pairing method is used to select them and generate LFW_Validation.txt text file. This file is used as input in the verification process. The format of LFW_Validation.txt is as follows: Figure 5 shown.

[0109] In this embodiment, the experimental operating system is the Ubuntu 18.04 operating system, the server GPU is 1080ti, the programming software used is Pycharm 2020, and the deep learning framework is Pytorch 1.6.

[0110] The LFW dataset contains 13,233 images of 5,749 subjects. Faces in the LFW dataset are first detected using MTCNN and aligned to a 160×160 resolution to create clean face image data. To simulate the setup of an adversarial attack scenario, the attack images after the attack are replaced one by one according to the image file name. The clean input face image data and the attacked face image data are paired to form positive and negative test samples for face verification. The LFW test set is used for face verification testing, and the evaluation metrics used are accuracy (ACC) and verification rate (Val).

[0111] To verify the performance of this black-box adversarial attack method for face recognition, we compared it with the paper "Fawkes: Protecting Privacy against Unauthorized Deep Learning Models" by Shawn Shan, Emily Wenger, et al. The results of attacking the FaceNet face recognition model on the LFW dataset are shown in Table 1.

[0112] In Table 1, the ACC of the unattacked original clean positive and negative sample pairs is about 99.5%. When FAR = 0.00067, Val is 97.4%, indicating that FaceNet has a high face recognition rate and good recognition effect. After the Fawkes attack, ACC drops to 88.2%, FAR remains unchanged, and Val drops to 34.9%, indicating that the Fawkes attack is effective. Under the DIP method, ACC further drops to 86.8%, FAR remains unchanged, and Val further drops to 28.3%. Undoubtedly, the attack effect of the DIP method has been further improved, indicating that the expansion of the search space is indeed reasonable and effective.

[0113] We can see that the black-box adversarial attack method for face recognition proposed in this paper performs significantly better on the FaceNet face recognition model than Fawkes on FaceNet. This demonstrates that the present invention can improve the effectiveness of black-box adversarial attack methods for face recognition.

[0114] Some adversarial sample images based on the DIP method are shown in Figure 6 In Fwakes, the visual experience of adversarial facial images differs slightly from the original clean images, and the sensory stimulation is not obvious. The adversarial facial images generated by the DIP method have slightly worse sensory effects than the corresponding images under Fawkes, but the sensory stimulation is still within the acceptable range.

[0115] Table 1. Brief table of attack methods against FaceNet

[0116] Accuracy Validation rate Unattacked 0.99517±0.00361 0.97467±0.01454@FAR=0.00067 Fawkes 0.88217±0.01406 0.34967±0.05738@FAR=0.00067 DIP(ours) 0.86883±0.03407 0.28367±0.10972@FAR=0.00067

[0117] Those skilled in the art will appreciate that the embodiments of the present application can be provided as methods, systems, or computer program products. Therefore, the present application can adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application can adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0118] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the steps in the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0119] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0120] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0121] The above embodiments merely illustrate several implementations of the present invention. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present invention. Those skilled in the art will readily appreciate that the present invention is susceptible to various modifications and variations. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present invention are intended to be within the scope of protection of the present invention.

Claims

1. A black box adversarial attack method for face recognition, characterized by: include: Get labeled face image data; Preprocessing the acquired facial image data to obtain preprocessed facial image data; The pre-processed face image data is input into the DIP network. Combined with the set constraint function, the DIP network parameters are iterated until the preset iteration stop condition is reached to obtain the optimized DIP network. Based on the optimized DIP network and the set latent code z, an attack image is obtained; Mix the attack image with the clean image and input them into the multi-face recognition model in the form of image pairs to obtain face verification and recognition results; The constraint function includes: taking the difference between the adversarial image and the input clean image as the first loss function; taking the difference between the integrated feature vector of the adversarial image and the integrated feature vector of the input clean image as the second loss function; taking the difference between the integrated feature vector of the directional face and the integrated feature vector of the adversarial image as the third loss function; the total loss function is as follows: L total =a space L space -a feature L feature +a target L target Among them, L space L is the distance between the adversarial image and the input clean image in the visual pixel space. feature is the distance between the adversarial image and the input clean image in the face feature space, L target is the distance between the adversarial image and the feature-guided image in the face feature space, α space , α feature , α target is the weight factor of each loss; L total is the total loss function.

2. The black-box adversarial attack method for face recognition according to claim 1, characterized in that: The acquired facial image data is preprocessed, including: transforming the facial image data into a size that meets the input and output requirements of the training model, and / or using a face alignment algorithm to extract the face.

3. The black-box adversarial attack method for face recognition according to claim 1, characterized in that: The integrated expression formula after feature extraction of multiple face recognition models is as follows: Among them, image input is the original input image; P(·) is the preprocessing function for the image; F i (·) is the i-th feature extractor; λ i is the weight factor corresponding to the i-th eigenvector; embeddings represents the input image image input Feature integration expression after feature extraction from k face recognition networks.

4. The black-box adversarial attack method for face recognition according to claim 1, characterized in that: The first loss function is specifically: L space =Dist space (Image adv ,Image input ) Among them, Image adv To fight against the image; Image input Dist is the input clean image; space (·) is the distance measurement function of the visual pixel space; L space is the distance between the adversarial image and the input clean image in the visual pixel space; The second loss function is specifically: L feature =Dist feature (embeddings adv ,embeddings input ) Among them, embeddings adv It is an integrated expression of the adversarial image after multiple face recognition network feature extraction; embeddings input Dist is the integrated expression of the input clean image after multiple face recognition network feature extraction; feature (·) is the distance measurement function in the face feature space; L feature is the distance between the adversarial image and the input clean image in the facial feature space; The third loss function is specifically: L target =Dist feature (embeddings adv ,embeddings target ) Among them, embeddings adv It is an integrated expression of the adversarial image after multiple face recognition network feature extraction; embeddings target It is the integrated expression of feature-guided images after multiple face recognition network feature extraction; Dist feature (·) is the distance measurement function in the face feature space; L target The distance between the adversarial image and the feature guidance image in the face feature space.

5. The black-box adversarial attack method for face recognition according to claim 1, characterized in that: The attack image is mixed with the clean image as an image pair and input into the multi-face recognition model to obtain face verification and recognition results, including: During face verification, clean, unprocessed face image data and corresponding adversarial processed face image data are paired one by one and jointly input into a uniformly selected face recognition model as a test set to obtain the classification probability of each adversarial face image data; The accuracy rate ACC and verification rate Val are used to evaluate the verification performance of the face recognition black box adversarial sample attack method, including: pairing each clean unprocessed face image data with the adversarially processed face image data to form positive and negative test samples, where the positive test samples are samples formed by pairing each clean unprocessed face image data with the adversarially processed face image data with the same label, and the negative samples are samples formed by pairing each clean unprocessed face image data with clean unprocessed face image data with different labels. The accuracy rate ACC indicator is expressed as: Among them, TP refers to the number of positive test samples judged as positive test samples by the feature classification network according to the classification probability, TN refers to the number of negative test samples judged as negative test samples by the feature classification network according to the classification probability, FP refers to the number of negative test samples judged as positive test samples by the feature classification network according to the classification probability, and FN refers to the number of positive test samples judged as negative test samples by the feature classification network according to the classification probability; The verification rate Val indicator can be expressed as: in, The feature classification network determines the positive test samples as the proportion of all true positive test samples according to the classification probability; It is the ratio of negative test samples that are judged as positive test samples by the feature classification network according to the classification probability to all true negative test samples.

6. The black-box adversarial attack method for face recognition according to claim 1, characterized in that: The preset iteration stopping condition is that the training reaches a preset number of training times and / or the total loss function reaches a preset threshold.

7. A black box adversarial sample attack device for face recognition, characterized in that: include: at least one processor; at least one memory for storing at least one program; When the at least one program is executed by at least one processor, the at least one processor implements the black-box adversarial sample attack method for face recognition according to any one of claims 1 to 6.

8. A computer device, characterized in that: Including the black box adversarial sample attack device for face recognition as described in claim 7.

9. A computer-readable storage medium storing a program executable by a processor, characterized in that: When the processor-executable program is executed by the processor, it is used to implement the black-box adversarial sample attack method for face recognition as described in any one of claims 1-6.

Images