Method, device, equipment and storage medium for acquiring anti-attack sample generation model

Abstract

The present application proposes a method, device, equipment and storage medium for acquiring an adversarial attack sample generation model. The specific implementation scheme is: acquiring D images of a first object and L images of a second object, and generating a regional image and K images containing at least one facial component of the first object based on the D images of the first object; inputting the K images into the generator to obtain K output images, and calculating the total variation value of the K output images; inputting the K output images and the regional images into the discriminator to obtain the first loss value and the second loss value output by the discriminator; generating K*L query images based on the K output images and the L images of the second object, and calculating the similarity between the first object and the object contained in the query image; generating the loss value of the generator, and training the generator and the discriminator based on the loss value and the second loss value of the generator, and determining the trained generator as the adversarial attack sample generation model.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence, specifically to the field of computer vision and deep learning technology, and can be used in scenarios such as face recognition, and in particular to a method, device, equipment and storage medium for acquiring a generation model for adversarial attack samples. Background Art

[0002] In recent years, with the large-scale implementation of artificial intelligence technology, face recognition technology has developed rapidly and has also received a lot of attention in related fields. With the complete collection of face data and the update and optimization of algorithms, face recognition systems have been implemented in large numbers in recent years and play an important role in public security monitoring, airport security, mobile phone unlocking, face payment, etc. However, due to the rise of neural network counterattacks, the security of face recognition systems has encountered a huge test. Summary of the invention

[0003] The present application provides a method, apparatus, device and storage medium for acquiring an anti-attack sample generation model.

[0004] According to a first aspect of the present application, a counter-attack sample generation model is provided, comprising:

[0005] Acquire D images of a first object and L images of a second object, and generate a regional image and K images containing at least one facial component of the first object based on the D images of the first object; wherein D and L are positive integers respectively, and K is a positive integer less than or equal to D;

[0006] Input the K images into a pre-trained generator to obtain K output images, and calculate the total variation value of the K output images;

[0007] Inputting the K output images and the region image into a discriminator to obtain a first loss value and a second loss value output by the discriminator;

[0008] Generate K*L query images according to the K output images and the L images of the second object, and calculate the similarity between the first object and the object included in the query image according to the D images of the first object and the K*L query images;

[0009] The loss value of the generator is generated according to the total variation value, the first loss value and the similarity, and the generator and the discriminator are trained according to the loss value of the generator and the second loss value, and the trained generator is determined as the adversarial attack sample generation model.

[0010] In some embodiments of the present application, generating the loss value of the generator according to the total variation value, the first loss value and the similarity includes:

[0011] The total variation value, the first loss value and the similarity are weightedly summed to obtain the loss value of the generator.

[0012] In some embodiments of the present application, the step of training the generator and the discriminator according to the loss value of the generator and the second loss value includes:

[0013] During the training process, the loss value of the generator and the second loss value are minimized by gradient descent, and the model parameters of the generator and the model parameters of the discriminator are continuously updated.

[0014] Wherein, updating the model parameters of the generator and the model parameters of the discriminator includes:

[0015] In each update iteration, the model parameters of the discriminator are first updated according to the second loss value, and then the model parameters of the generator are updated according to the loss value of the generator.

[0016] In some embodiments of the present application, generating K*L query images according to the K output images and the L images of the second object includes:

[0017] Mask processing is performed on the K output images, and K*L query images are generated according to the K output images after the mask processing and the L images of the second object.

[0018] Further, the calculating, based on the D images of the first object and the K*L query images, the similarity between the first object and the object included in the query image includes:

[0019] Input the D images of the first object and the K*L query images into M face recognition models respectively, and obtain the average similarity output by each face recognition model; wherein M is a positive integer;

[0020] The similarity between the first object and the second object is calculated according to the M average similarities.

[0021] The step of inputting the D images of the first object and the K*L query images into M face recognition models respectively to obtain the average similarity output by each face recognition model includes:

[0022] For each face recognition model, the D images of the first object and the K*L query images are respectively input into the face recognition model to obtain D first features and K*L second features extracted by the face recognition model;

[0023] The average cosine similarity of the D first features and the K*L second features is calculated, and the average cosine similarity is used as the average similarity output by the face recognition model.

[0024] According to a second aspect of the present application, a device for acquiring an adversarial attack sample generation model is provided, comprising:

[0025] A first generating module is used to obtain D images of a first object and L images of a second object, and generate a regional image and K images containing at least one facial component of the first object according to the D images of the first object; wherein D and L are positive integers respectively, and K is a positive integer less than or equal to D;

[0026] A first computing module, used for inputting the K images into a pre-trained generator to obtain K output images, and calculating total variation values ​​of the K output images;

[0027] An acquisition module, used for inputting the K output images and the region image into a discriminator to obtain a first loss value and a second loss value output by the discriminator;

[0028] A second generating module, configured to generate K*L query images according to the K output images and the L images of the second object;

[0029] A second calculation module, configured to calculate the similarity between the first object and the object included in the query image based on the D images of the first object and the K*L query images;

[0030] A third generating module, used to generate a loss value of the generator according to the total variation value, the first loss value and the similarity;

[0031] A training module is used to train the generator and the discriminator according to the loss value of the generator and the second loss value, and determine the trained generator as the adversarial attack sample generation model.

[0032] In some embodiments of the present application, the third generation module is specifically used to:

[0033] The total variation value, the first loss value and the similarity are weightedly summed to obtain the loss value of the generator.

[0034] In some embodiments of the present application, the training module is specifically used for:

[0035] During the training process, the loss value of the generator and the second loss value are minimized by gradient descent, and the model parameters of the generator and the model parameters of the discriminator are continuously updated.

[0036] Furthermore, the training module is specifically used for:

[0037] In each update iteration, the model parameters of the discriminator are first updated according to the second loss value, and then the model parameters of the generator are updated according to the loss value of the generator.

[0038] In some embodiments of the present application, the second generating module is specifically used to:

[0039] Mask processing is performed on the K output images, and K*L query images are generated according to the K output images after the mask processing and the L images of the second object.

[0040] In addition, in the embodiment of the present application, the second calculation module includes:

[0041] an acquisition unit, configured to input the D images of the first object and the K*L query images into M face recognition models respectively, to obtain an average similarity output by each face recognition model; wherein M is a positive integer;

[0042] A calculation unit is used to calculate the similarity between the first object and the object included in the query image according to the M average similarities.

[0043] The acquisition unit is specifically used for:

[0044] For each face recognition model, the D images of the first object and the K*L query images are respectively input into the face recognition model to obtain D first features and K*L second features extracted by the face recognition model;

[0045] The average cosine similarity of the D first features and the K*L second features is calculated, and the average cosine similarity is used as the average similarity output by the face recognition model.

[0046] According to a third aspect of an embodiment of the present application, there is provided an electronic device, including:

[0047] at least one processor; and

[0048] a memory communicatively connected to the at least one processor; wherein,

[0049] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the method described in the first aspect above.

[0050] According to a fourth aspect of an embodiment of the present application, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to enable the computer to execute the method described in the first aspect above.

[0051] According to the technical solution of the present application, for real face recognition scenarios, a generator and a discriminator are used to form an adversarial network, and the discriminator performs authenticity discrimination on the image output by the generator. After training, the image output by the generator can be made to conform to the real appearance of the corresponding area of ​​the face as much as possible, so that the obtained generator can output adversarial attack samples. At the same time, the image output by the generator is combined with the attacker image to obtain a query image, and the generator is trained by calculating the similarity between the query image and the target person image, so that the image output by the generator is more in line with the real scene, so that the generated adversarial samples can be used as a supplement to the existing face data used to train the face recognition model, assist in the testing and training of the recognition model and the defense model, and help promote the security research of artificial intelligence. At the same time, this solution also has the advantages of strong operability and low training cost.

[0052] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present application, nor is it intended to limit the scope of the present application. Other features of the present application will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] The accompanying drawings are used to better understand the present solution and do not constitute a limitation of the present application.

[0054] Figure 1 It is a flowchart of a method for obtaining an anti-attack sample generation model proposed in an embodiment of the present application;

[0055] Figure 2 A schematic diagram of a network structure for obtaining a counter-attack sample generation model in an embodiment of the present application;

[0056] Figure 3 This is a flowchart of the generator pre-training in an embodiment of the present application;

[0057] Figure 4 A flowchart of another method for acquiring an anti-attack sample generation model proposed in an embodiment of the present application;

[0058] Figure 5 A schematic diagram of a network structure of another model for generating adversarial attack samples in an embodiment of the present application;

[0059] Figure 6 A structural block diagram of a device for acquiring a counter-attack sample generation model proposed in an embodiment of the present application;

[0060] Figure 7 A schematic block diagram of an example electronic device that can be used to implement embodiments of the present application is shown. DETAILED DESCRIPTION

[0061] The following is a description of exemplary embodiments of the present application in conjunction with the accompanying drawings, including various details of the embodiments of the present application to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the present application. Similarly, for the sake of clarity and conciseness, the description of well-known functions and structures is omitted in the following description.

[0062] It should be noted that with the development of artificial intelligence technology, face recognition systems have been applied to public security monitoring, airport security, mobile phone unlocking and other scenarios. However, due to the rise of neural network counterattacks, the security of face recognition systems has encountered a huge test. For example, an attacker can impersonate another person by wearing a sticker with a characteristic pattern printed on it, causing the face recognition system to fail and make a wrong judgment.

[0063] It should be pointed out that in the technical solution of the present invention, the collection, acquisition, storage, processing, transmission, provision, disclosure and application of user personal information (such as facial images, etc.) are all carried out with the user's knowledge and explicit authorization, comply with the relevant laws and regulations, and do not violate public order and good morals.

[0064] In order to improve the security of face recognition systems, this application proposes a method, device, equipment and storage medium for obtaining an adversarial attack sample generation model that can be used in real scenarios, so that the generated adversarial attack samples can be used as a supplement to the existing face data used to train face recognition models, help test and train recognition models and defense models, improve the face recognition system's defense against attacks, and also help promote the security research of artificial intelligence.

[0065] Based on the above problems, the present application provides a method, device, equipment and storage medium for obtaining an adversarial attack sample generation model that can be used in real scenarios.

[0066] It should be noted that the user data involved in the embodiments of the present application have been authorized, acquired, processed, transmitted, etc. in compliance with the requirements of laws and regulations.

[0067] Figure 1This is a flow chart of a method for acquiring an adversarial attack sample generation model proposed in an embodiment of the present application. It should be noted that the method for acquiring an adversarial attack sample generation model proposed in an embodiment of the present application can be used in the device for acquiring an adversarial attack sample generation module proposed in an embodiment of the present application, and the device can be configured in an electronic device. Figure 2 This is a schematic diagram of the network structure of the anti-attack sample generation model obtained in the embodiment of the present application. Figure 1 As shown, the method includes:

[0068] Step 101, obtaining D images of a first object and L images of a second object, and generating a regional image and K images containing at least one facial component of the first object based on the D images of the first object; wherein D and L are positive integers respectively, and K is a positive integer less than or equal to D.

[0069] In an embodiment of the present application, the image of the first object refers to a photo of the target person, that is, a photo of the victim whose identity is impersonated, and the image of the second object refers to a photo of the attacker. The image of the first object and the image of the second object can be collected by taking photos or searching on the Internet. It can be understood that generating a regional image containing at least one facial component of the first object based on D images of the first object means extracting a regional image containing at least one of the facial components of the first object, such as the nose, eyes, and eyebrows, from the image of the first object. K frontal face photos are randomly selected from the D images of the first object, and K images including at least one of the eyes, eyebrows, and nose are extracted from the selected K photos, and used as the input of the pre-trained generator. As shown in FIG. Figure 2 As shown, in the embodiment of the present application, the extracted facial components including eyes, nose and eyebrows are taken as an example for explanation, that is, the obtained regional image and K photos are both taken as an example including the eyes, nose and eyebrows of the first object.

[0070] Step 102: input K images into a pre-trained generator, obtain K output images, and calculate the total variation value of the K output images.

[0071] In the embodiments of the present application, Figure 2 As shown, the pre-trained generator can output an image that is approximately the same as the input image, so K images are input into the pre-trained generator, and each input image can obtain an output image, that is, K output images. As an example, calculating the total variation value of K output images can be used to calculate, which is a commonly used regularization term, used to make the generated sample pattern as smooth and continuous as possible in color and shape, so that the sample image can be printed. Among them, the calculation formula of the total variation value of each image is shown in formula (1):

[0072] Among them, refers to the value of the th image in the K images, and is the value of the pixel at the th position on the th image. By averaging the values ​​of each image, or performing weighted operations, etc., the total variation value of the K output images can be obtained.

[0073] like Figure 3 As shown, the pre-training of the generator can be performed in the following way: input the obtained K images into the generator for forward propagation, compare the output of the generator with the input, calculate the two loss values ​​and respectively, the calculation formulas are shown in formula (2) and formula (3), and calculate the total loss function through formula (3):

[0074] Among them, is the input of the th generator, and is the output of the generator for the th input. This method uses gradient descent to minimize the overall loss function, thereby optimizing the parameters of the generator. When the number of update iterations reaches a certain threshold, the update is stopped, and the generator parameters at this time are retained to complete the pre-training.

[0075] Step 103: Input K output images and the region image into the discriminator to obtain a first loss value and a second loss value output by the discriminator.

[0076] It can be understood that after the K output images and the region images are input into the discriminator, the discriminator will calculate the similarity between the K output images and the region images based on the input image information. That is to say, the discriminator needs to calculate the scores of the K output images based on the input image information, and judge whether the K output images are real face regions based on the scores.

[0077] Since the ideal state of the discriminator is to calculate as high a score as possible for the real photo and as low a score as possible for the output image of the generator, the image generated by the generator can be discriminated as a non-real face area image. In the embodiment of the present application, the second loss value is used to represent the difference between the output result of the discriminator and the real situation, and its specific calculation method can be shown as formula (5):

[0078] Among them, is the second loss value, is the i-th region image, is the i-th input of the generator, and is the output corresponding to the i-th input.

[0079] On the other hand, the ideal state of the generator is to generate an image that is as close to the real appearance of the corresponding area of ​​the face as possible for the input image, so that the discriminator can mistake its output image for a real face area image, thereby achieving the purpose of adversarial training. In the embodiment of the present application, in order to form an adversarial network between the generator and the discriminator, a first loss value is used to represent the difference between the output result of the discriminator and the ideal state of the generator, wherein the calculation method of the first loss value is shown in formula (6):

[0080] Among them, is the second loss value, is the i-th input of the generator, and is the output corresponding to the i-th input.

[0081] Step 104 , generating K*L query images based on the K output images and the L images of the second object, and calculating the similarity between the first object and the object included in the query image based on the D images of the first object and the K*L query images.

[0082] In the embodiments of the present application, Figure 2 As shown, the query image refers to a new image formed by combining K output images with the image of the second object, that is, replacing the facial components in the image of the second object with the facial components in the output image. Wherein, L query images can be obtained by combining each output image with L images of the second object, so K output images can be combined with L images of the second object, respectively, to obtain K*L query images.

[0083] As an example, the implementation method of combining K output images with the image of the second object can be: converting the K output images into images of the same size as the facial area in the image of the second object; taking out one of the converted images to cover the facial areas in the L images of the second object, to obtain L query images; and so on, until all the K converted images are taken out, to obtain K*L query images.

[0084] As another example, K output images may be masked, and the K masked output images may be projected onto L images of the second object through a neural network of geometric transformation to generate K*L query images.

[0085] In an embodiment of the present application, based on D images of the first object and K*L query images, the similarity between the object contained in the query image and the first object in the image of the first object is calculated to indicate whether the query image can achieve an adversarial attack on the first object. For example, if the object contained in the query image has a large similarity to the target person, the query image can play an adversarial attack role, which means that the output image of the generator is not only approximately consistent with the input image of the target person, but also can achieve an adversarial attack on the target person. The similarity can be calculated by the pixel value of each pixel in the image, or by the feature vector of the image, or by other possible implementation methods.

[0086] Step 105, generating a loss value of the generator according to the total variation value, the first loss value and the similarity, and training the generator and the discriminator according to the loss value and the second loss value of the generator, and determining the trained generator as the adversarial attack sample generation model.

[0087] It can be understood that the purpose of training the generator is, on the one hand, to make the image it generates consistent with the appearance of the real face area, and on the other hand, to make the image it generates be able to output adversarial attack samples by combining it with the attacker's image. Therefore, in the embodiment of the present application, the loss value of the generator is calculated based on the total variation value, the first loss value and the similarity.

[0088] As an example, the total variation value, the first loss value, and the similarity may be weightedly summed to obtain the loss value of the generator.

[0089] In addition, since the purpose of discriminator training is to determine the authenticity of the image, and the second loss value is used to represent the difference between the output result of the discriminator and the actual situation, the second loss value can be used as the loss value of the discriminator.

[0090] like Figure 2 As shown, in order to enable the generator and the discriminator to achieve the desired purpose, in the embodiment of the present application, the generator and the discriminator are trained according to the loss value of the generator and the loss value of the discriminator. As a possible implementation method, during the training process, the loss value of the generator and the second loss value can be minimized by gradient descent, and the model parameters of the generator and the model parameters of the discriminator are continuously updated until the number of updates is greater than a certain threshold.

[0091] Specifically, during the training process, at each update iteration, the model parameters of the discriminator are first updated according to the second loss value, and then the model parameters of the generator are updated according to the loss value of the generator. After each update, K images are input to the generator again, and the above steps are repeated, and so on, until the number of updates exceeds a certain threshold, and the trained generator is determined as the adversarial attack sample generation model.

[0092] In addition, the adversarial attack samples generated by the adversarial attack sample generation model can serve as a supplement to the existing face recognition data used to train face recognition models, which helps in the testing and training of recognition models and defense models, so that face recognition technology can defend against the above-mentioned attacks, and further promote the security research of artificial intelligence.

[0093] According to the method for obtaining the adversarial attack sample generation model proposed in the embodiment of the present application, for real face recognition scenarios, a generator and a discriminator are used to form an adversarial network, and the discriminator performs authenticity discrimination on the image output by the generator. After training, the image output by the generator can be made to conform to the real appearance of the corresponding area of ​​the face as much as possible, so that the obtained generator can output adversarial attack samples. At the same time, the image output by the generator is combined with the attacker image to obtain a query image, and the generator is trained by calculating the similarity between the object contained in the query image and the target person, so that the adversarial attack sample output by the generator can meet the applicability of the real scene, so that the generated adversarial sample can be used as a supplement to the existing face data used to train the face recognition model, help the testing and training of the recognition model and the defense model, and help promote the security research of artificial intelligence. At the same time, this solution also has the advantages of strong operability and low training cost.

[0094] Next, we will take the use of mask processing to generate query images as an example to introduce the entire training process in detail for similarity calculation.

[0095] Figure 4 A flowchart of another method for acquiring an anti-attack sample generation model proposed in an embodiment of the present application. Figure 5 This is another schematic diagram of a network structure for obtaining a counter-attack sample generation model in an embodiment of the present application. Based on the above embodiment, the method includes:

[0096] Step 401, obtaining D images of a first object and L images of a second object, and generating a regional image and K images containing at least one facial component of the first object based on the D images of the first object; wherein D and L are positive integers respectively, and K is a positive integer less than or equal to D.

[0097] Step 402: input K images into a pre-trained generator, obtain K output images, and calculate the total variation value of the K output images.

[0098] Step 403: Input the K output images and the region images into the discriminator to obtain the first loss value and the second loss value output by the discriminator.

[0099] in, Figure 4 Steps 401-403 in Figure 1 The implementation method of steps 101-103 in is the same and will not be repeated here.

[0100] Step 404 , performing mask processing on the K output images, and generating K*L query images based on the K output images that have undergone mask processing and the L images of the second object.

[0101] It should be noted that in the embodiment of the present application, the specifications of the mask can be as shown in the following table (1). The function of the mask is to block the output image obtained by the generator so that it becomes a T-shape including eyes, eyebrows and nose, which is used to be combined with the image of the second object to form a query image.

[0102] Table 1

[0103] Template Name Template Shape Template Description

[0104] The size of the mask is consistent with the output pattern of the generator, where the value of the white part is 1 and the value of the black part is 0. The attack sample will be multiplied by the mask when it is fitted with the output image of the generator, and the pattern of the white part will be retained.

[0105] In the embodiment of the present application, the specific implementation method of generating K*L query images may be as follows: Figure 5 As shown, the K output images of the generator are masked, and the K masked output images are projected onto L images of the second object using a neural network with geometric transformation to obtain K*L query images. Among them, the neural network responsible for geometric transformation does not need to be trained, and is only initialized based on the parameters received according to the set fitting position. During fitting, in order to enhance the robustness of the training, on the basis of satisfying the pre-set position and size of each output image, each masked output image will undergo a random left-right flip during fitting, and will also include slight random perturbations in position and size. Finally, as shown in Figure 5 As shown, in each image of the second object, the output image processed by the mask will cover the eyes, eyebrows, nose and other areas of the second object in the image of the second object.

[0106] Step 405 , inputting D images of the first object and K*L query images into M face recognition models respectively, and obtaining the average similarity output by each face recognition model; wherein M is a positive integer.

[0107] In order to ensure that the adversarial attack samples generated by the final adversarial attack sample generation model can still maintain the corresponding attack effect when the face recognition model to be attacked is unknown, M face recognition models are used in the embodiment of the present application, so that the output adversarial attack samples of the generator can be suitable for different face recognition models.

[0108] like Figure 5As shown, in an embodiment of the present application, D images of the first object and K*L query images are respectively input into M face recognition models for feature extraction. It should be noted that the face recognition model used in the embodiment of the present application can be an existing model and does not need to be trained again. Among them, the implementation method of obtaining the average similarity output by each face recognition model can be: for each face recognition model, D images of the first object and K*L query images are respectively input into the face recognition model to obtain D first features and K*L second features extracted by the face recognition model; the average cosine similarity of the D first features and K*L second features is calculated, and the average cosine similarity is used as the average similarity output by the face recognition model. Among them, the first feature refers to the feature of the first object extracted by each face recognition model for the D images of the first object. The second feature refers to the feature of the object contained in the query image extracted by each face recognition model for the K*L query images.

[0109] As an example, the average similarity of each face recognition model output can be calculated as shown in formula (7):

[0110] Among them, is the average similarity output by the ith face recognition model, is the feature extracted by the ith face recognition model for the ith query image, is the feature extracted by the ith face recognition model for the th image of the first object, is the inner product of the vector, and is the 2-norm of the vector.

[0111] Step 406: Calculate the similarity between the first object and the object included in the query image based on the M average similarities.

[0112] As an example, the M average similarities obtained may be averaged to calculate the similarity between the first object and the object contained in the query image, as shown in formula (8):

[0113] Among them, is the similarity between the first object and the object contained in the query image, and is the average similarity output by the m-th face recognition model.

[0114] Step 407, generates a loss value of the generator according to the total variation value, the first loss value and the similarity, and trains the generator and the discriminator according to the loss value and the second loss value of the generator, and determines the trained generator as the adversarial attack sample generation model.

[0115] In the embodiment of the present application, the loss value of the generator can be calculated by weighted average, as shown in formula (9):

[0116] Among them, is the loss value of the generator, is the similarity between the first object and the object contained in the query image, is the first loss value, is the total variation value, and is the preset weight value.

[0117] As a possible implementation method, during the training process, the gradient descent method can be used to minimize the loss value and the second loss value, and the model parameters of the generator and the model parameters of the discriminator are continuously updated until the number of updates is greater than a certain threshold.

[0118] Specifically, at each update iteration, the model parameters of the discriminator are first updated according to the second loss value, and then the model parameters of the generator are updated according to the loss value of the generator. After each update, K images are obtained and input to the generator again, and the above steps are repeated, and so on, until the number of updates exceeds a certain threshold, and the trained generator is determined as the adversarial attack sample generation model. It can be found that the adversarial attack samples obtained by the adversarial attack sample generation model can maintain the corresponding attack effect even in a black box scenario (the face recognition model to be attacked is unknown).

[0119] According to the method for acquiring the adversarial attack sample generation model proposed in the embodiment of the present application, the output image of the generator is masked and fitted with the attacker's image, so that the resulting image combination effect is better, and random slight perturbations are performed during the fitting process, which can enhance the robustness of the training, thereby improving the model training effect. In addition, multiple face recognition models are used to extract features from the query image and the target person image respectively, and the average cosine similarity is used to calculate the similarity between the object contained in the query image and the target person, so that the output of the generator is not restricted by the face recognition model, and then the adversarial attack sample output by the adversarial attack sample generation model can still maintain the corresponding attack effect in a black box scenario (the face recognition model to be attacked is unknown), effectively improving the applicability of the adversarial attack sample in real scenarios, so that the generated adversarial sample can be used as a supplement to the existing face data used to train the face recognition model, and help the testing and training of the recognition model and the defense model.

[0120] In order to implement the above embodiment, the present application proposes a device for acquiring an anti-attack sample generation model.

[0121] Figure 6 This is a structural block diagram of a device for acquiring a counter-attack sample generation model proposed in an embodiment of the present application. Figure 6 As shown, the device comprises:

[0122] A first generating module 610 is configured to obtain D images of a first object and L images of a second object, and generate a regional image and K images containing at least one facial component of the first object according to the D images of the first object; wherein D and L are positive integers respectively, and K is a positive integer less than or equal to D;

[0123] A first calculation module 620 is used to input K images into a pre-trained generator, obtain K output images, and calculate the total variation value of the K output images;

[0124] An acquisition module 630 is used to input the K output images and the region image into the discriminator to obtain a first loss value and a second loss value output by the discriminator;

[0125] A second generating module 640, configured to generate K*L query images according to the K output images and the L images of the second object;

[0126] A second calculation module 650, configured to calculate the similarity between the first object and the object included in the query image based on the D images of the first object and the K*L query images;

[0127] A third generating module 660, configured to generate a loss value of a generator according to the total variation value, the first loss value and the similarity;

[0128] The training module 670 is used to train the generator and the discriminator according to the loss value and the second loss value of the generator, and determine the trained generator as the adversarial attack sample generation model.

[0129] In some embodiments of the present application, the third generation module 660 is specifically used to:

[0130] The total variation value, the first loss value, and the similarity are weighted summed to obtain the loss value of the generator.

[0131] In some embodiments of the present application, the training module 670 is specifically used to:

[0132] During the training process, the gradient descent method is used to minimize the generator's loss value and the second loss value, and the model parameters of the generator and the discriminator are continuously updated.

[0133] Furthermore, the training module 670 is specifically used for:

[0134] In each update iteration, the model parameters of the discriminator are first updated according to the second loss value, and then the model parameters of the generator are updated according to the loss value of the generator.

[0135] In some embodiments of the present application, the second generating module 640 is specifically used to:

[0136] Mask processing is performed on the K output images, and K*L query images are generated according to the K output images after the mask processing and the L images of the second object.

[0137] In addition, in the embodiment of the present application, the second calculation module 650 includes:

[0138] An acquisition unit 651 is used to input the D images of the first object and the K*L query images into M face recognition models respectively to obtain an average similarity output by each face recognition model; wherein M is a positive integer;

[0139] The calculation unit 652 is used to calculate the similarity between the first object and the object included in the query image according to the M average similarities.

[0140] The acquisition unit 651 is specifically used for:

[0141] For each face recognition model, D images of the first object and K*L query images are respectively input into the face recognition model to obtain D first features and K*L second features extracted by the face recognition model;

[0142] Calculate the average cosine similarity of the D first features and the K*L second features, and use the average cosine similarity as the average similarity output by the face recognition model.

[0143] According to the device for acquiring the adversarial attack sample generation model proposed in the embodiment of the present application, for real face recognition scenarios, a generator and a discriminator are used to form an adversarial network, and the discriminator discriminates the authenticity of the image output by the generator. After training, the image output by the generator can be made to conform to the real appearance of the corresponding area of ​​the face as much as possible, so that the obtained generator can output adversarial attack samples. At the same time, the image output by the generator is combined with the attacker image to obtain a query image, and the generator is trained by calculating the similarity between the object contained in the query image and the target person, thereby improving the applicability of the adversarial attack samples output by the generator in real scenarios, so that the generated adversarial samples can be used as a supplement to the existing face data used to train the face recognition model, assist in the testing and training of the recognition model and the defense model, and help promote the security research of artificial intelligence. At the same time, this solution also has the advantages of strong operability and low training cost.

[0144] Based on the embodiments of the present application, the present application also provides an electronic device, at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute any of the aforementioned methods for acquiring a counter-attack sample generation model.

[0145] Based on the embodiments of the present application, the present application also provides a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to enable a computer to execute any of the aforementioned methods for obtaining an adversarial attack sample generation model provided in the embodiments of the present application.

[0146] Figure 7A schematic block diagram of an example electronic device 700 that can be used to implement an embodiment of the present application is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workbenches, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present application described and / or required herein.

[0147] like Figure 7 As shown, the device 700 includes a computing unit 701, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 702 or a computer program loaded from a storage unit 708 into a random access memory (RAM) 703. In the RAM 703, various programs and data required for the operation of the device 700 can also be stored. The computing unit 701, the ROM 702, and the RAM 703 are connected to each other via a bus 704. An input / output (I / O) interface 705 is also connected to the bus 704.

[0148] A number of components in the device 700 are connected to the I / O interface 705, including: an input unit 706, such as a keyboard, a mouse, etc.; an output unit 707, such as various types of displays, speakers, etc.; a storage unit 708, such as a disk, an optical disk, etc.; and a communication unit 709, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 709 allows the device 700 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0149] The computing unit 701 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 701 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 701 performs the various methods and processes described above, such as the adversarial attack sample generation model acquisition method. For example, in some embodiments, the adversarial attack sample generation model acquisition method may be implemented as a computer software program, which is tangibly included in a machine-readable medium, such as a storage unit 708. In some embodiments, part or all of the computer program may be loaded and / or installed on the device 700 via the ROM 702 and / or the communication unit 709. When the computer program is loaded into the RAM 703 and executed by the computing unit 701, one or more steps of the adversarial attack sample generation model acquisition method described above may be performed. Alternatively, in other embodiments, the computing unit 701 may be configured to execute the method for acquiring the adversarial attack sample generation model in any other appropriate manner (for example, by means of firmware).

[0150] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0151] The program code for implementing the method of the present application can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, implements the functions / operations specified in the flow chart and / or block diagram. The program code can be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0152] In the context of the present application, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or device, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0153] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0154] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), the Internet, and a blockchain network.

[0155] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship between the client and the server is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services ("Virtual Private Server", or "VPS" for short). The server may also be a server of a distributed system, or a server combined with a blockchain.

[0156] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this disclosure can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution disclosed in this application can be achieved, and this document is not limited here.

[0157] The above specific implementations do not constitute a limitation on the protection scope of this application. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions and improvements made within the spirit and principles of this application should be included in the protection scope of this application.

Claims

1. A method for acquiring an adversarial attack sample generation model, characterized in that: include: Acquire D images of a first object and L images of a second object, and generate a regional image and K images containing at least one facial component of the first object based on the D images of the first object; wherein D and L are positive integers, K is a positive integer less than or equal to D, and the K images are K frontal face images randomly selected from the D images of the first object; Input the K images into a pre-trained generator to obtain K output images, and calculate the total variation value of the K output images; Input the K output images and the region image into a discriminator to obtain a first loss value and a second loss value output by the discriminator, wherein the first loss value represents a difference between an output result of the discriminator and an ideal state of the generator, and the second loss value represents a difference between an output result of the discriminator and a real situation; Generate K*L query images according to the K output images and the L images of the second object, and calculate the similarity between the first object and the object included in the query image according to the D images of the first object and the K*L query images; The loss value of the generator is generated according to the total variation value, the first loss value and the similarity, and the generator and the discriminator are trained according to the loss value of the generator and the second loss value, and the trained generator is determined as the adversarial attack sample generation model.

2. The method according to claim 1, characterized in that Generating the loss value of the generator according to the total variation value, the first loss value and the similarity comprises: The total variation value, the first loss value and the similarity are weightedly summed to obtain the loss value of the generator.

3. The method according to claim 1, characterized in that The training of the generator and the discriminator according to the loss value of the generator and the second loss value comprises: During the training process, the loss value of the generator and the second loss value are minimized by gradient descent, and the model parameters of the generator and the model parameters of the discriminator are continuously updated.

4. The method according to claim 3, characterized in that: The updating of the model parameters of the generator and the model parameters of the discriminator includes: In each update iteration, the model parameters of the discriminator are first updated according to the second loss value, and then the model parameters of the generator are updated according to the loss value of the generator.

5. The method according to claim 1, characterized in that The step of generating K*L query images according to the K output images and the L images of the second object comprises: Mask processing is performed on the K output images, and K*L query images are generated according to the K output images after the mask processing and the L images of the second object.

6. The method according to claim 1, characterized in that The calculating, based on the D images of the first object and the K*L query images, the similarity between the first object and the object included in the query image comprises: Input the D images of the first object and the K*L query images into M face recognition models respectively, and obtain the average similarity output by each face recognition model; wherein M is a positive integer; The similarity between the first object and the object included in the query image is calculated according to the M average similarities.

7. The method according to claim 6, characterized in that The step of inputting the D images of the first object and the K*L query images into M face recognition models respectively to obtain an average similarity output by each of the face recognition models includes: For each face recognition model, the D images of the first object and the K*L query images are respectively input into the face recognition model to obtain D first features and K*L second features extracted by the face recognition model; The average cosine similarity of the D first features and the K*L second features is calculated, and the average cosine similarity is used as the average similarity output by the face recognition model.

8. A device for acquiring a model for generating an attack sample, characterized in that: include: A first generating module is used to obtain D images of a first object and L images of a second object, and generate a regional image and K images containing at least one facial component of the first object according to the D images of the first object; wherein D and L are positive integers, K is a positive integer less than or equal to D, and the K images are K frontal face images randomly selected from the D images of the first object; A first computing module, used for inputting the K images into a pre-trained generator to obtain K output images, and calculating total variation values ​​of the K output images; an acquisition module, configured to input the K output images and the region image into a discriminator, and obtain a first loss value and a second loss value output by the discriminator, wherein the first loss value indicates a difference between an output result of the discriminator and an ideal state of the generator, and the second loss value indicates a difference between an output result of the discriminator and a real situation; A second generating module, configured to generate K*L query images according to the K output images and the L images of the second object; A second calculation module, configured to calculate the similarity between the first object and the object included in the query image based on the D images of the first object and the K*L query images; A third generating module, used to generate a loss value of the generator according to the total variation value, the first loss value and the similarity; A training module is used to train the generator and the discriminator according to the loss value of the generator and the second loss value, and determine the trained generator as the adversarial attack sample generation model.

9. The device according to claim 8, characterized in that The third generation module is specifically used for: The total variation value, the first loss value and the similarity are weightedly summed to obtain the loss value of the generator.

10. The device according to claim 8, characterized in that The training module is specifically used for: During the training process, the loss value of the generator and the second loss value are minimized by gradient descent, and the model parameters of the generator and the model parameters of the discriminator are continuously updated.

11. The device according to claim 10, characterized in that The training module is specifically used for: In each update iteration, the model parameters of the discriminator are first updated according to the second loss value, and then the model parameters of the generator are updated according to the loss value of the generator.

12. The device according to claim 8, characterized in that The second generating module is specifically used for: Mask processing is performed on the K output images, and K*L query images are generated according to the K output images after the mask processing and the L images of the second object.

13. The device according to claim 8, characterized in that The second calculation module includes: an acquisition unit, configured to input the D images of the first object and the K*L query images into M face recognition models respectively, to obtain an average similarity output by each face recognition model; wherein M is a positive integer; A calculation unit is used to calculate the similarity between the first object and the object included in the query image according to the M average similarities.

14. The device according to claim 13, characterized in that The acquisition unit is specifically used for: For each face recognition model, the D images of the first object and the K*L query images are respectively input into the face recognition model to obtain D first features and K*L second features extracted by the face recognition model; The average cosine similarity of the D first features and the K*L second features is calculated, and the average cosine similarity is used as the average similarity output by the face recognition model.

15. An electronic device, characterized in that: include: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 7.

16. A non-transitory computer-readable storage medium storing computer instructions, characterized in that: The computer instructions are used to cause the computer to execute the method according to any one of claims 1 to 7.

Images