Adversarial intensity-guided multi-scale attention face adversarial defense method and system

Abstract

The application discloses a kind of multi-scale attention face confrontation defense methods and systems to confrontation strength guide, increase a multi-scale attention module in existing confrontation purification model, multi-scale attention module is converted into the feature of multiple scales to the image input, simultaneously using attention mechanism and confrontation strength value is weighted for the feature of multiple scales, the multi-scale feature after weighting is input into confrontation purification model, generates the face image after purification.The application can carry out purification pretreatment before face image input face recognition system, effectively reduce the possible containing confrontation noise in face image, simultaneously according to the estimated confrontation intensity adjusts confrontation purification strategy, under the attack of various intensity, can effectively improve the defense performance of confrontation purification model, to protect face recognition system from the influence of confrontation attack, enhance its security under various potential confrontation attacks.

Description

Technical Field

[0001] This invention belongs to the field of face recognition technology, specifically relating to a multi-scale attention-based face adversarial defense method and system guided by adversarial strength. Background Technology

[0002] With the rapid development of computer vision in recent years, deep neural network-based facial recognition technology has achieved extremely high accuracy in recognition tasks and is widely used in various real-world scenarios such as access control systems and electronic device unlocking. However, in recent years, researchers have proposed adversarial attack techniques. These techniques can implant imperceptible adversarial perturbations into images through specific algorithms, significantly affecting the recognition results of neural networks. In certain facial recognition scenarios, attackers can easily impersonate others to pass facial verification or prevent the facial recognition system from recognizing their own identity, seriously affecting personal property security and disrupting social order. To address this issue, researchers have proposed adversarial defense techniques aimed at mitigating or even eliminating the impact of adversarial attacks on classification results. Applying adversarial defense techniques to facial recognition systems can ensure their security and stability, enabling them to complete facial recognition tasks with high accuracy in both unattacked and attacked scenarios.

[0003] Existing adversarial defense techniques are mainly divided into two categories: 1) Adversarial training, which uses images subjected to adversarial attacks as training samples to train the neural network. This method is simple to implement, but requires retraining the protected model, which consumes a lot of time and computing power for large-scale models; at the same time, it will cause a significant decrease in the accuracy of the original neural network's classification or recognition without attacks. 2) Adversarial preprocessing, which processes the images input to the neural network to reduce the adversarial noise contained therein. Simple image preprocessing techniques, such as JPEG image compression and simple noise reduction, can effectively preserve the features of the original image while removing some adversarial noise, but the defense effect is often poor; preprocessing based on generative networks (i.e., adversarial sanitization), although it can remove strong adversarial noise, will also cause visually noticeable changes to the appearance of the original image, which will have a certain negative impact on the accuracy of the protected network.

[0004] For the specific defense category of adversarial sanitization, it usually involves two steps: destruction and restoration of the original image information. There is a dilemma in this process: if the destruction is too strong, the restored image will be too different from the original image, while if the destruction is too weak, the adversarial perturbation information will not be effectively removed. Existing methods often choose a compromise destruction level, but this makes it impossible for adversarial sanitization models to achieve the best defense effect under attacks of various intensities. Summary of the Invention

[0005] To address the shortcomings of existing adversarial cleanup methods in defense strategies, this invention provides an adversarial defense method and system based on intensity-guided multi-scale attention (IMA) for faces. This method fully considers various attack scenarios of different intensities that a face recognition system may suffer. By estimating the attack intensity, the model adaptively adjusts the combined weights of the multi-scale systems to better cope with attacks.

[0006] To achieve the above objectives, this invention provides a multi-scale attention-based face adversarial defense method guided by adversarial strength, comprising the following steps:

[0007] Step 1: Prepare a face image dataset for training and testing;

[0008] Step 2: Construct a multi-scale purification model;

[0009] Step 2.1, Select an anti-purification model As a basic adversarial purification model, the first layer of the network is a convolutional layer structure. The input size of the model is an RGB image of size 3×H1×W1, and the output size is an RGB image of size 3×H1×W1, where H1 and W1 are the height and width of the image, respectively.

[0010] Step 2.2, the basic adversarial purification model The input size of the first convolutional layer is changed from 3×H1×W1 to C1×H1×W1. The modified adversarial purification model is denoted as...

[0011] Step 2.3, Construct a multi-scale attention module The input to this module is an image. and the scalar of resistance strength The output is IMA image features. H0 and W0 are the height and width of the image, respectively;

[0012] Step 2.4, add the multi-scale attention module Compared to the modified anti-purification model Combined into a multi-scale purification model in

[0013] Step 3: Train the multi-scale purification model;

[0014] Step 4: Use the trained multi-scale purification model to output the purified face image;

[0015] Step 4.1: Input a face image and estimate the adversarial strength contained in the face image;

[0016] Step 4.2: Input the face image and the estimated adversarial strength into the trained multi-scale de-sanitizing model to obtain the de-sanitized face image.

[0017] Furthermore, in step 1, each image in the face dataset that has not been subjected to adversarial attacks is aligned and scaled to a size of 3×H0×W0, where H0 and W0 are the height and width of the image, respectively. The image pixels are then normalized, and the normalized images are divided into training and testing sets.

[0018] Furthermore, step 2.3 includes the following steps:

[0019] Step 2.3.1: Downsample the input image x using bilinear interpolation to N different resolutions. The 0th resolution is the original image size. Starting from the 1st resolution, the resolution of each level is reduced to half the length and width of the previous level, represented as follows:

[0020] Step 2.3.2, process the multi-scale image x () Input parallel network ,∈{0,1,…,N-1} Preliminary feature extraction at various resolutions

[0021] Step 2.3.3, transfer feature φ () Compression is performed on the length and width dimensions of the channel ∈ {0,1,…,N-1} by averaging the values ​​of all pixels in each channel. The formula for calculating the c-th channel value is as follows:

[0022]

[0023] In the formula, φ (i) [,j,k] represents φ (i) The value of the c-th channel and the j-th row and k-th column of the tensor, where C1, W1, and H1 are the model values. The input RGB image contains the number of channels, width, and height, and N is the resolution of the image obtained after downsampling.

[0024] Step 2.3.4, will Flattened to a length of The vector is input into the fully connected layer FC(·) to obtain the attention weights. The input length of the fully connected layer is Given a vector, output a vector of length N;

[0025] Step 2.3.5: Calculate the weight bias from the adversarial strength scalar θ. The calculation formula is as follows:

[0026]

[0027]

[0028] In the formula, b low b is the preset lower limit of the bias. up The upper limit of the preset bias is T, which is the reference value for the resistance strength, and θ is the upper limit of the bias. t The threshold for adversarial strength is set, and N is the number of resolutions of the image obtained after downsampling.

[0029] Step 2.3.6, Calculate the IMA weights The calculation formula is:

[0030]

[0031]

[0032] In the formula, w attn This represents the attention weight, α is the adversarial strength coefficient, and w bias This represents the weight bias calculated from the adversarial strength scalar θ;

[0033] Step 2.3.7, calculate the IMA image feature φ IMA The specific calculation formula for the output of the multi-scale attention module is as follows:

[0034]

[0035] In the formula, w () φ represents the IMA weight. () This represents the extracted features at various resolutions. This indicates that tensors are concatenated along the channel dimension.

[0036] Furthermore, in step 2.4, the face image x undergoes a multi-scale attention module. Then the IMA image features φ were obtained. IMM , will φ IMA Input into the modified anti-purification model In the process, the final purified image of the face is obtained.

[0037] Furthermore, in step 3, the face training dataset that has not been subjected to adversarial attacks is divided into batches containing s images, and then input into the multi-scale cleanup model. In this context, the adversarial strength scalar is set to 0 during training; training the network can be transformed into optimizing the following problem:

[0038]

[0039] in:

[0040]

[0041]

[0042] In the formula, To combat the losses; For KL divergence loss; To train the auxiliary discriminator network, its input is a 3×H1×W1 RGB image, where H1 and W1 are the height and width of the RGB image, and the output is a probability value representing that the image comes from the real training data; D LL (·) represents the function for calculating the KL divergence; q(z|x) represents the input image x after passing through... The encoder obtains the distribution of the latent variable z; N(0,I) represents the standard normal distribution; λ is a manually set hyperparameter.

[0043] After every n iterations, the test set data is used to... Conduct tests; if the generated image can be recognized in a face recognition system with an accuracy exceeding 'a', then stop training and output the result. The model parameters are determined; otherwise, training iteration continues. In the above process, n and a are manually set hyperparameters.

[0044] Furthermore, in step 4.1, the face image x is input into the multi-scale purification model. Obtain a roughly purified image x rough The roughly cleaned image x rough The mean squared error between the original image x and the mean squared error is used to obtain the estimated adversarial strength θ, as shown in the following formula:

[0045]

[0046] In the formula, x[,j,k] represents the value of the c-th channel, j-th row and k-th column of image x, and x rough [c,j,k] represents the image x rough The value of the c-th channel, the j-th row and the k-th column, where W0 and H0 are the width and height of image x.

[0047] Furthermore, in step 4.2, the face image x and the estimated adversarial strength θ are input into the multi-scale purification model. In the process, the final purified image result is calculated. and x prf It is used as input for facial recognition systems.

[0048] The present invention also provides an adversarial strength-guided multi-scale attention face adversarial defense system, used to implement the adversarial strength-guided multi-scale attention face adversarial defense method described above.

[0049] Furthermore, it includes a processor and a memory, the memory being used to store program instructions, and the processor being used to invoke the program instructions in the memory to execute an adversarial strength-guided multi-scale attention-based face adversarial defense method as described above.

[0050] Alternatively, it may include a readable storage medium storing a computer program that, when executed, implements an adversarial strength-guided multi-scale attention-based face adversarial defense method as described above.

[0051] Compared with the prior art, the present invention has the following advantages:

[0052] 1) By organically fusing features from input images at multiple scale resolutions through an attention mechanism, and using a simple adversarial strength estimator to optimize the fusion weights, the defense model can significantly improve its defense performance under both weak and strong attacks, and effectively protect the face recognition system under various adversarial attack scenarios.

[0053] 2) By simply adding a lightweight multi-scale attention module to the basic adversarial purification model, the defense performance is improved while the computational complexity and the number of model parameters are increased only to a negligible extent. Attached Figure Description

[0054] Figure 1 This is a flowchart of the multi-scale attention-based face adversarial defense method guided by adversarial strength according to an embodiment of the present invention.

[0055] Figure 2 This is a flowchart of the training process for the multi-scale purification model according to an embodiment of the present invention.

[0056] Figure 3 This is a flowchart of the multi-scale purification model test according to an embodiment of the present invention. Detailed Implementation

[0057] This invention provides a multi-scale attention-based face adversarial defense method and system guided by adversarial strength. The technical solution of this invention will be further described below with reference to the accompanying drawings and embodiments.

[0058] Example 1

[0059] like Figure 1 As shown, this embodiment of the invention provides a multi-scale attention-based face adversarial defense method guided by adversarial strength, comprising the following steps:

[0060] Step 1: Prepare a face image dataset for training and testing.

[0061] Each image in the face dataset that has not been subjected to adversarial attacks is aligned, scaled to a size of 3×H0×W0, and the image pixels are normalized. The normalized images are then divided into training and test sets.

[0062] Step 2: Construct a multi-scale purification model.

[0063] Step 2.1, Select an anti-purification model As a basic adversarial desiccation model, the first layer of this model is a convolutional layer structure. The input size of the model is an RGB image of size 3×H1×W1, and the output size is an RGB image of size 3×H1×W1. In specific implementation, Generative networks are typically "encoder-decoder" structures, such as variational autoencoders and generative adversarial networks.

[0064] Step 2.2, the basic adversarial purification model The input size of the first convolutional layer is changed from 3×H1×W1 to C1×H1×W1. The modified adversarial purification model is denoted as...

[0065] Step 2.3, Construct a multi-scale attention module The input to this module is an image. and the scalar of resistance strength The output is IMA image features. Specifically, the following processing steps are included:

[0066] Step 2.3.1: Downsample the input image x using bilinear interpolation to N different resolutions. The 0th resolution is the original image size. Starting from the 1st resolution, the resolution of each level is reduced to half the length and width of the previous level, represented as follows:

[0067] Step 2.3.2, process the multi-scale image x () Input parallel network ,∈{0,1,…,N-1} Preliminary feature extraction at various resolutions

[0068] Step 2.3.3, transfer feature φ () Compression is performed on the length and width dimensions of the channel ∈ {0,1,…,N-1} by averaging the values ​​of all pixels in each channel. The formula for calculating the c-th channel value is as follows:

[0069]

[0070] In the formula, φ (i) [,j,k] represents φ(i) The value of the c-th channel and the j-th row and k-th column of the tensor, where C1, W1, and H1 are the model values. The input RGB image contains the number of channels, width, and height, and N is the resolution of the image obtained after downsampling.

[0071] Step 2.3.4, will Flattened to a length of The vector is input into the fully connected layer FC(·) to obtain the attention weights. The input length of the fully connected layer is Given a vector, output a vector of length N.

[0072] Step 2.3.5: Calculate the weight bias from the adversarial strength scalar θ. The calculation formula is as follows:

[0073]

[0074]

[0075] In the formula, b low b is the preset lower limit of the bias. up The upper limit of the preset bias is T, which is the reference value for the resistance strength, and θ is the upper limit of the bias. t The threshold for adversarial strength is set, and N is the number of resolutions of the image obtained after downsampling.

[0076] Step 2.3.6, Calculate the IMA weights The calculation formula is:

[0077]

[0078]

[0079] In the formula, w attn This represents the attention weight, α is the adversarial strength coefficient, and w bias This represents the weight bias calculated from the adversarial strength scalar θ.

[0080] Step 2.3.7, calculate the IMA image feature φ IMA The specific calculation formula for the output of the multi-scale attention module is as follows:

[0081]

[0082] In the formula, w () φ represents the IMA weight. () This represents the extracted features at various resolutions. This indicates that tensors are concatenated along the channel dimension.

[0083] Step 2.4, add the multi-scale attention module Compared to the modified anti-purification model Combined into a multi-scale purification model in

[0084] Image x passes through a multi-scale attention module Then the IMA image features φ were obtained. IMA , will φ IMA Input into the modified anti-purification model In this process, the final purified image is obtained.

[0085] Step 3: Train the multi-scale purification model.

[0086] The unadversarial face training dataset is divided into batches of s images and fed into a multi-scale cleansing model. In this model, the adversarial strength scalar is set to 0 during training. Training the model can be transformed into optimizing the following problem:

[0087]

[0088] in:

[0089]

[0090]

[0091] In the formula, To combat the losses; For KL divergence loss; To train the auxiliary discriminator network, its input is a 3×H1×W1 RGB image, and its output is a probability value representing that the image comes from the real training data; D KL (·) represents the function for calculating the KL divergence; q(z|x) represents the input image x after passing through... The encoder obtains the distribution of the latent variable z; N(0,I) represents the standard normal distribution; λ is a manually set hyperparameter.

[0092] After every n iterations, the test set data is used to... Conduct tests; if the generated image can be recognized in a face recognition system with an accuracy exceeding 'a', then stop training and output the result. The model parameters are determined; otherwise, training iterations continue. In the above process, n and a are manually set hyperparameters.

[0093] Step 4: Use the trained multi-scale purification model to output the purified face image.

[0094] Step 4.1: Input a face image and estimate the adversarial strength contained in the face image.

[0095] Input the face image x into the multi-scale purification model Output a roughly cleaned image x rough The roughly cleaned image x rough The mean squared error between the original image x and the mean squared error is used to obtain the estimated adversarial strength θ, as shown in the following formula:

[0096]

[0097] In the formula, x[,j,k] represents the value of the c-th channel, j-th row and k-th column of image x, and x rough [c,j,k] represents the image x rough The value of the c-th channel, the j-th row and the k-th column, where W0 and H0 are the width and height of image x.

[0098] Step 4.2: Input the face image and the estimated adversarial strength into the multi-scale de-sanitizing model to obtain the de-sanitized face image.

[0099] The face image x and the estimated adversarial strength θ are input into the multi-scale decontamination model. In the process, the final purified image result is calculated. and x prf It is used as input for facial recognition systems.

[0100] Example 2

[0101] Based on the same inventive concept, the present invention also provides an adversarial strength-guided multi-scale attention face adversarial defense system, including a processor and a memory, wherein the memory is used to store program instructions, and the processor is used to call the program instructions in the memory to execute the adversarial strength-guided multi-scale attention face adversarial defense method as described above.

[0102] Example 3

[0103] Based on the same inventive concept, the present invention also provides an adversarial strength-guided multi-scale attention face adversarial defense system, including a readable storage medium on which a computer program is stored. When the computer program is executed, it implements the adversarial strength-guided multi-scale attention face adversarial defense method as described above.

[0104] In specific implementation, the method proposed in the technical solution of this invention can be automatically executed by those skilled in the art using computer software technology. System devices for implementing the method, such as computer-readable storage media storing the corresponding computer program of the technical solution of this invention and computer equipment including the computer program running the corresponding computer program, should also be within the protection scope of this invention.

[0105] The specific embodiments described herein are merely illustrative of the spirit of the invention. Those skilled in the art to which this invention pertains may make various modifications or additions to the described specific embodiments or use similar methods to replace them, without departing from the spirit of the invention or exceeding the scope defined by the appended claims.

Claims

1. A multi-scale attention-based adversarial face defense method guided by adversarial strength, characterized in that, Includes the following steps: Step 1: Prepare a face image dataset for training and testing; Step 2: Construct a multi-scale purification model; The multi-scale descaling model adds a multi-scale attention module to the existing adversarial descaling model. The input face image first passes through the multi-scale attention module, which transforms the face image into multi-scale features. Simultaneously, it uses an attention mechanism and adversarial strength values ​​to weight the multi-scale features. Finally, the weighted multi-scale features are input into the adversarial descaling model to generate a descaling face image as the model's output. Specifically, it includes the following steps: Step 2.1, Select an anti-purification model As a basic adversarial purification model, the first layer of this model is a convolutional layer structure, and the input of the model is a data structure of size [size missing]. The RGB image is output with a size of RGB image, These are the height and width of the image, respectively; Step 2.2, the basic adversarial purification model The input size of the first convolutional layer is determined by Modified to The modified anti-purification model is denoted as ; Step 2.3, Construct a multi-scale attention module The input to this module is an image. and the scalar of resistance strength The output is IMA image features. , These are the height and width of the image, respectively; Step 2.3.1, input image Using bilinear interpolation downsampling Images of different resolutions are represented as follows: Level 0 resolution is the original image size; starting from Level 1, the resolution of each subsequent level is halved in both length and width compared to the previous level. ; Step 2.3.2, convert the multi-scale image Input parallel network Preliminary feature extraction at various resolutions , ; Step 2.3.3, feature Compression is performed along the length and width dimensions, and the average value of all pixels in each channel is calculated. , The The formula for calculating the channel value is as follows: （1） In the formula, express Tensor First The first channel, the first Line number The value of the column, , For the model The number of channels, width, and height of the input RGB image. This represents the resolution of the image obtained after downsampling. Step 2.3.4, will Flattened to a length of The vector is input into the fully connected layer FC. In this process, attention weights are obtained. The input length of the fully connected layer is The vector, the output length is ; Step 2.3.5, from the scalar of resistance strength Calculate the weight bias The calculation formula is as follows: （2） （3） In the formula, This is the preset lower limit of the bias. This is the preset upper limit of the bias. As a reference value for combat strength, The threshold for the set confrontation strength, This represents the resolution of the image obtained after downsampling. Step 2.3.6, Calculate the IMA weights The calculation formula is: （4） （5） In the formula, Indicates attention weights. To counteract the strength coefficient, Represented by the scalar of adversarial strength The calculated weight bias; Step 2.3.7, Calculate IMA image features The specific calculation formula for the output of the multi-scale attention module is as follows: （6） In the formula, Indicates the IMA weight. This represents the extracted features at various resolutions. This indicates that tensors are concatenated along the channel dimension; Step 2.4, add the multi-scale attention module Compared to the modified anti-purification model Combined into a multi-scale purification model ,in ; Step 3: Train the multi-scale purification model; Step 4: Use the trained multi-scale purification model to output the purified face image; Step 4.1: Input a face image and estimate the adversarial strength contained in the face image; Step 4.2: Input the face image and the estimated adversarial strength into the trained multi-scale de-sanitizing model to obtain the de-sanitized face image.

2. The adversarial strength-guided multi-scale attention-based face adversarial defense method as described in claim 1, characterized in that: In step 1, each image in the face dataset that has not been subjected to adversarial attacks is aligned and scaled to the specified size. , The height and width of the image are defined as the image height and width, respectively. The image pixels are then normalized, and the normalized images are divided into training and testing sets.

3. The adversarial strength-guided multi-scale attention-based face adversarial defense method as described in claim 1, characterized in that: Input image in step 2.4 After multi-scale attention module IMA image features ,Will Input into the modified anti-purification model In this process, the final purified image is obtained.

4. The adversarial strength-guided multi-scale attention-based face adversarial defense method as described in claim 1, characterized in that: In step 3, the face training dataset that has not been subjected to adversarial attacks is divided into groups containing... A batch of images is generated and input into a multi-scale cleanup model. In this context, the adversarial strength scalar is set to 0 during training; training the network can be transformed into optimizing the following problem: （7） in: （8） （9） In the formula, ; For KL divergence loss; To train the auxiliary discriminator network, its input is RGB image, Given the height and width of the RGB image, the output is a probability value indicating that the image comes from the real training data; The function that calculates the KL divergence; Indicates the input image go through The encoder obtains the hidden variables. Distribution; Represents a standard normal distribution; Hyperparameters set manually; Every time The next iteration uses the test set data to... Test it; if the generated image can achieve a score exceeding [a certain threshold] in a face recognition system... Once the accuracy is identified, training stops and outputs the result. The model parameters are determined; otherwise, training iterations continue. During the above process... These are hyperparameters that are manually set.

5. The adversarial strength-guided multi-scale attention-based face adversarial defense method as described in claim 1, characterized in that: In step 4.1, the face image Input multi-scale purification model Output a roughly cleaned image. The roughly purified image and the original image Calculate the mean square error to obtain the estimated strength of the confrontation. The calculation formula is as follows: （10） In the formula, Representing an image No. The first channel, the first Line number The value of the column, Representing an image No. The first channel, the first Line number The value of the column, , For image Width and height.

6. The adversarial strength-guided multi-scale attention-based face adversarial defense method as described in claim 1, characterized in that: In step 4.2, the face image and estimated intensity of confrontation Input into the multi-scale purification model In the process, the final purified image result is calculated. and will It is used as input for facial recognition systems.

7. A multi-scale attention-based face adversarial defense system guided by adversarial strength, characterized in that, It includes a processor and a memory, the memory being used to store program instructions, and the processor being used to invoke the program instructions in the memory to execute the adversarial strength-guided multi-scale attention face adversarial defense method as described in any one of claims 1-6.

8. A multi-scale attention-based face adversarial defense system guided by adversarial strength, characterized in that, It includes a readable storage medium on which a computer program is stored, and when the computer program is executed, it implements the adversarial strength-guided multi-scale attention-based face adversarial defense method as described in any one of claims 1-6.

Images