Face forgery recognition method and device for unknown adversarial attacks

Abstract

Embodiments of the present disclosure provide a face forgery recognition method and device facing unknown adversarial attacks, electronic equipment and storage medium, comprising: acquiring an actual image containing a face; performing face detection processing on the actual image to obtain an actual face region image, and performing reconstruction processing on the actual face region image to obtain an actual face reconstruction image; inputting the actual face reconstruction image into a pre-trained adversarial face discrimination network to predict the face authenticity of the actual image; wherein the adversarial face discrimination network is trained using a clean face dataset and an adversarial face image set with added disturbance noise. Embodiments of the present disclosure reconstruct the actual face region image data, change the distribution pattern of adversarial disturbance, suppress adversarial noise, and at the same time retain face forgery traces; the adversarial face discrimination network is trained using the adversarial face image set with added disturbance noise, thereby improving the robustness of the discrimination network model.

Description

Technical Field

[0001] This disclosure belongs to the field of image processing technology, specifically relating to a method, apparatus, electronic device, and storage medium for face forgery recognition against unknown adversarial attacks. Background Technology

[0002] With the development of deepfake technology, applications of face forgery are becoming increasingly widespread, leading to the emergence of technologies for identifying genuine and fake faces. Face forgery image identification technologies mainly include methods based on traditional image forensics, biometric detection methods, and spatial feature detection methods. Traditional image forensics and biometric methods require manually defined or targeted forgery features and regions for identification. Spatial feature detection methods, on the other hand, extract and identify forgery features from a single image. Currently, mainstream methods are based on deep neural networks for feature extraction and identification, and they all face a major common problem: vulnerability to adversarial attacks. Adversarial attacks are a type of interference against machine learning algorithms that adds subtle, imperceptible interference to cause automatic recognition models to misidentify targets. Adversarial attacks mainly include two types: white-box attacks and black-box attacks. [1] White-box attacks exploit the structure and parameters of a specific model, while black-box attacks typically lack detailed model parameters and rely on simple input-output mapping information to attack unknown models. Black-box attacks are more commonly used in practical applications.

[0003] Currently, defense algorithms against black-box attacks mainly include data-level defense and model-level defense. Data-level defense involves modifying model parameters during training and input data during testing. Methods for modifying model parameters during training primarily include adversarial training, while methods for modifying input data during testing mainly include data compression and denoising. Model-level defense methods mainly include two types: modifying the network and adding an adversarial network. Common methods for modifying the network include changing the loss function or activation function, defensive distillation, and regularization. Defense methods using an adversarial network refer to jointly training a deep discriminative network and an adversarial example generation network to improve the model's robustness. Adding an adversarial network is a commonly used defense method. Its advantage is that it does not rely excessively on dataset samples, such as in "Adversarial Examples Improvement".

[0004] The AdvProp method proposed in the paper "Image Recognition" achieves robustness of the network model's recognition by adding a generative network using AuxiliaryBN to generate adversarial noise (using PGD) and training with joint defense.

[0005] Existing technologies suffer from at least the following problems: Most mainstream deepfake face detection algorithms are based on deep neural networks for feature extraction and identification. While these network models can achieve ideal results with clean images, they are susceptible to adversarial attacks designed to target these neural network models, leading to identification failures. How to defend against adversarial attacks is a key issue in improving the robustness of detection. Since practical applications typically face unknown attacks, current defense methods mainly rely on modifying data and models. However, adversarial attack defense techniques have the following problems: 1) Most adversarial attack defenses only achieve good results against a few fixed known attack methods and cannot address the risks posed by known attacks; 2) They are dependent on the parameters of the target model.

[0006] Reference 1: Yi Ping, Wang Kedi, Huang Cheng, et al. A review of research on adversarial attacks against artificial intelligence [J]. Journal of Shanghai Jiao Tong University, 2018(10):1298-1306. Summary of the Invention

[0007] The embodiments disclosed herein aim to at least solve one of the technical problems existing in the prior art, and provide a method, apparatus, electronic device and storage medium for face spoofing recognition against unknown adversarial attacks.

[0008] One aspect of this disclosure provides a method for face spoofing recognition against unknown adversarial attacks. The method includes:

[0009] Obtain the actual image containing the human face;

[0010] The actual image is subjected to face detection processing to obtain an actual face region image, and the actual face region image is subjected to reconstruction processing to obtain an actual face reconstructed image;

[0011] The actual face reconstruction image is input into a pre-trained adversarial face discrimination network to predict the authenticity of the face in the actual image; wherein, the adversarial face discrimination network is trained using a clean face dataset and an adversarial face image set with added perturbation noise.

[0012] Optionally, the adversarial face discrimination network is trained using the following steps:

[0013] The process involves acquiring real face images without interference and fake face images, performing face detection processing on the real face images and fake face images to obtain a clean face region image dataset, and reconstructing a portion of the face region images to obtain a face reconstruction image dataset; wherein, the clean face region image dataset and the face reconstruction image dataset together constitute the clean face dataset.

[0014] Select a portion of target clean face images from the clean face dataset, and input the target clean face images into a pre-defined general perturbation noise generation network to generate the adversarial face images;

[0015] The clean face image and the adversarial face image are input into the adversarial face discrimination network to be trained, and the trained adversarial face discrimination network is obtained.

[0016] Optionally, the adversarial face discrimination network is an EfficientNet network model, and the step of inputting the clean face image and the adversarial face image into the adversarial face discrimination network to be trained includes:

[0017] The clean face image and the adversarial face image are respectively input into the EfficientNet network model for forward propagation to calculate the loss. The EfficientNet network model is configured with a first loss function and a second loss function.

[0018] The clean face image is input into the first loss function L. c (θ,x c ,) Calculate its loss value; θ is the parameter of the EfficientNet network model, x c For a clean face image, y represents the real / fake label;

[0019] The adversarial face image is input into the second loss function L. a (,x a ,) Calculate its loss value; where x a To combat facial images.

[0020] The gradient backpropagation of the clean face image and the adversarial face image loss is calculated using the min-max optimization algorithm, and the parameters of the EfficientNet network model are updated until the final loss function converges, resulting in the trained EfficientNet network model. The final loss function is:

[0021]

[0022] Where, x c Let y be a clean face image, y be the true / false label, ∈ represent the added perturbation noise, and x be the true / false label. c +∈ represents adversarial face images, x c +∈=x a S represents the permissible perturbation range for noise, and θ is a parameter of the EfficientNet network model. For data distribution, This indicates the difference between the training samples and the labels.

[0023] Optionally, the general perturbation noise generation network includes: a convolutional layer and three deconvolutional layers. The convolutional layer uses the DenseNet network to perform convolution processing on the input clean face image to obtain a feature map. The output feature map has a size of 7×7×256. After passing through a Baztch Normalization (BN) process and then through the ReLU activation function, a feature map is obtained, which is used as the input of the next deconvolutional layer.

[0024] The kernel size of each deconvolutional layer is 5×5. Each deconvolutional layer undergoes BaztchNormalization (BN) and is then processed by a corresponding activation function to obtain a feature map, which serves as the input to the next deconvolutional layer.

[0025] Optionally, the step of performing face detection processing on the actual image to obtain an actual face region image includes:

[0026] Adjust the actual image to a preset size;

[0027] The resized actual image is input into the MobileNet neural network model for feature extraction, and a feature map is output.

[0028] An anchor box mechanism is used to anchor the feature map, and the anchor boxes are filtered using the Soft-NMS algorithm to obtain face detection boxes. The face detection boxes are then segmented to obtain the actual face region image.

[0029] Optionally, the step of reconstructing the actual face region image to obtain an actual face reconstruction image includes:

[0030] The face region image is blurred using at least one of Gaussian blur, box blur, Kawase blur, double blur, bokeh blur, and directional blur.

[0031] The blurred face region image is deblurred using an SDWNet network based on wavelet transform and linear expansion to obtain the actual face reconstruction image.

[0032] One aspect of this disclosure provides a face spoofing recognition device resistant to unknown adversarial attacks. The device includes:

[0033] The acquisition module is used to acquire actual images containing human faces;

[0034] The reconstruction module is used to perform face detection processing on the actual image to obtain an actual face region image, and to perform reconstruction processing on the actual face region image to obtain an actual face reconstruction image.

[0035] The recognition module is used to input the actual face reconstruction image into a pre-trained adversarial face discrimination network to predict the authenticity of the face in the actual image.

[0036] Optionally, the device further includes a training module, the training module being used for:

[0037] The process involves acquiring real face images without interference and fake face images, performing face detection processing on the real face images and fake face images to obtain a clean face region image dataset, and reconstructing a portion of the face region images to obtain a face reconstruction image dataset; wherein, the clean face region image dataset and the face reconstruction image dataset together constitute the clean face dataset.

[0038] Select a portion of target clean face images from the clean face dataset, and input the target clean face images into a pre-defined general perturbation noise generation network to generate the adversarial face images;

[0039] The clean face image and the adversarial face image are input into the adversarial face discrimination network to be trained, and the trained adversarial face discrimination network is obtained.

[0040] One aspect of this disclosure provides an electronic device, including: one or more processors;

[0041] A storage unit is used to store one or more programs that, when executed by one or more processors, enable the one or more processors to implement the method described above.

[0042] One aspect of this disclosure provides a computer-readable storage medium having a computer program stored thereon that, when executed by a processor, can implement the method as described above.

[0043] In the face forgery recognition method against unknown adversarial attacks disclosed in this embodiment, a method combining face image data reconstruction and adversarial training with the addition of an additional network is adopted. First, the actual face region image data is reconstructed with high quality, which can change the distribution pattern of adversarial perturbations, suppress adversarial noise, and retain face forgery traces. Then, the adversarial face discrimination network is trained with an adversarial face image set with added perturbation noise, which improves the robustness of the discrimination network model. This embodiment of the disclosure defends against adversarial attacks from both data and network levels, which can make up for the deficiencies of defending only from a single aspect of data or network from multiple dimensions. Attached Figure Description

[0044] Figure 1This is a flowchart of a face spoofing recognition method for unknown adversarial attacks according to an embodiment of the present disclosure;

[0045] Figure 2 This is a schematic diagram of the structure of a face spoofing recognition device for unknown adversarial attacks according to an embodiment of this disclosure. Detailed Implementation

[0046] To enable those skilled in the art to better understand the technical solutions of the embodiments of this disclosure, the embodiments of this disclosure will be further described in detail below with reference to the accompanying drawings and specific implementation methods.

[0047] like Figure 1 As shown, a face forgery recognition method S100 for unknown adversarial attacks includes:

[0048] S110. Obtain the actual image containing the human face.

[0049] This disclosure embodiment can use existing image acquisition equipment to acquire actual images containing human faces in videos or pictures, etc. The actual image containing human faces is the actual image containing human faces to be identified, and the actual image may include fake face images.

[0050] S120. Perform face detection processing on the actual image to obtain an actual face region image, and perform reconstruction processing on the actual face region image to obtain an actual face reconstruction image.

[0051] This embodiment employs an improved MobileNet network, combined with a novel anchor-box mechanism suitable for efficient operation on mobile GPUs and the Soft-NMS algorithm, to detect unrestricted face images in real-world images and obtain actual face region images. Unrestricted face detection refers to the detection of faces without occlusion, primarily used in applications such as forged video detection. The MobileNet network is a lightweight convolutional neural network that uses a more compact and lightweight feature extraction method, enabling real-time face detection in videos. The novel anchor-box mechanism reduces the number of anchor boxes, lowers computational complexity, and accelerates the final execution speed, allowing deployment on mobile devices. Other anchor-box filtering methods can also be used; this embodiment employs the Soft-NMS algorithm for higher accuracy.

[0052] For example, performing face detection processing on the actual image to obtain an actual face region image includes:

[0053] S121. Adjust the actual image to a preset size.

[0054] As an example, this step normalizes the actual image by scaling it to a uniform size of 128×128.

[0055] S122. Input the adjusted actual image into the MobileNet neural network model for feature extraction and output the feature map.

[0056] MobileNet is a lightweight convolutional neural network that uses far fewer parameters than standard convolutions, enabling it to extract feature maps from real-time images. The feature maps are available in 8×8 and 16×16 sizes.

[0057] For example, the MobileNet neural network model includes: one 2D convolutional layer and five single BlazeBlocks and six double BlazeBlocks; the 2D convolutional layer adopts a 5×5×3×24 deformable convolutional network layer.

[0058] The 2D convolutional layer described in this embodiment can increase the receptive field, and the MobileNet neural network model can preserve features as much as possible.

[0059] S123. Anchor boxes are generated on the feature map using a preset anchor box mechanism. The anchor boxes are then filtered using the Soft-NMS algorithm to obtain face detection boxes. The face detection boxes are then segmented to obtain the actual face region image.

[0060] As an example, the pre-defined anchor box mechanism reduces the number of anchor boxes, lowers computational complexity, and speeds up the final execution, making it suitable for deployment on mobile devices. This embodiment uses the Soft-NMS algorithm for anchor box selection, which offers higher accuracy. Other methods can also be used for anchor box selection; this is not limited to these specific methods. Given the potential for detecting varying lengths of face regions in a real image, it is necessary to segment out these different face regions to obtain the corresponding actual face region images.

[0061] For example, the anchor frame mechanism sets six 8×8 and two 16×16 anchor frames at each pixel position on the feature map, with the aspect ratio of the anchor frames being 1:1.

[0062] As an example, the anchor frame mechanism sets six 8×8 and two 16×16 anchor frames at each pixel position on the feature map to reduce the number of anchor frames. Since the aspect ratio of the face has limited variation, the aspect ratio of the anchor frames can be set to 1:1.

[0063] For example, the step of reconstructing the actual face region image to obtain an actual face reconstruction image includes:

[0064] S124. The face region image is blurred using at least one of Gaussian blur, box blur, Kawase blur, double blur, bokeh blur and directional blur.

[0065] As an example, the blur radius can be set to 5, or adjusted according to the actual situation; this is not limited. This embodiment can use the six blurring methods described above sequentially to blur the face image; the blurring order is not limited. Those skilled in the art can perform blurring according to the actual situation. Face region image sizes may vary. Before blurring the face region image, its size is normalized to a uniform 416×416. For face images smaller than 416×416, pixel grayscale values ​​of 0 are used to supplement the image from both rows and columns outwards.

[0066] S124. The blurred face region image is deblurred using the SDWNet network based on wavelet transform and linear expansion to obtain the actual face reconstruction image.

[0067] As an example, the face region image processed in step S123 is deblurred to restore the face region image. If the face region image has been subjected to adversarial perturbations, deblurring the face region image to restore it can change the perturbation and suppress adversarial noise. This embodiment of the disclosure achieves face region image reconstruction by blurring and deblurring the face region image, thereby limiting or changing the perturbation of the face region image by adversarial attacks and improving recognition accuracy.

[0068] For example, the SDWNet network model training process includes:

[0069] Set the batch size to 8 samples in the training set; the learning rate is adaptively adjusted starting from 4×10⁻⁴ using cosine annealing.

[0070] The parameters of the model trained on the GoPro dataset are used as the initial training parameters for the SDWNet network model. The parameters are then fine-tuned and trained for several epochs until the SDWNet network model reaches its optimal state.

[0071] As an example, approximately 100 epochs of training can be performed. This embodiment employs the SDWNet network model to train and deblur the face region image, which aids in the reconstruction and restoration of high-frequency texture details, better preserves subtle traces of forgery in the face region image, and the SDWNet network model has a simple structure, is easy to train, and converges quickly. Setting the batch size to 8 training samples ensures rapid convergence of the SDWNet network model.

[0072] S130. Input the actual face reconstruction image into a pre-trained adversarial face discrimination network to predict the authenticity of the face in the actual image; wherein, the adversarial face discrimination network is trained using a clean face dataset and an adversarial face image set with added perturbation noise.

[0073] It is easy to understand that the predicted result is "true" when the actual face reconstruction image is a real face image, and "false" when it is a fake face image. This embodiment of the disclosure uses a clean face dataset and an adversarial face image set with added perturbation noise to train the adversarial face discrimination network, thereby enhancing the robustness of the adversarial face discrimination network.

[0074] For example, the adversarial face discrimination network is trained using the following steps:

[0075] S131. Obtain real face images and fake face images without interference, and perform face detection processing on the real face images and fake face images to obtain a clean face region image dataset, and perform reconstruction processing on some face region images to obtain a face reconstruction image dataset; wherein, the clean face region image dataset and the face reconstruction image dataset together constitute the clean face dataset.

[0076] As an example, the face detection and reconstruction processing method of step S120 above can be used to obtain a clean face region image dataset and a face reconstruction image dataset. Here, reconstructing the clean face region images can expand the dataset and increase the generalization of the adversarial face discrimination network.

[0077] S132. Select a portion of target clean face images from the clean face dataset, and input the target clean face images into a pre-set general perturbation noise generation network to generate the adversarial face image.

[0078] As an example, the general perturbation noise generation network is an additional network in the adversarial face discrimination network, designed to add perturbation noise to the target clean face image during the training of the adversarial face discrimination network to generate the adversarial face image.

[0079] S133. Input the clean face image and the adversarial face image into the adversarial face discrimination network to be trained for training, and obtain the trained adversarial face discrimination network.

[0080] Understandably, the true / false labels corresponding to the target clean face image and the adversarial face image are used as the output of the adversarial face discrimination network to train the adversarial face discrimination network, thus obtaining the trained adversarial face discrimination network.

[0081] For example, the adversarial face discrimination network is an EfficientNet network model, and the AdvProp training method is used to perform adversarial training on the EfficientNet network model. The step of inputting the clean face image and the adversarial face image into the adversarial face discrimination network to be trained includes:

[0082] The clean face image and the adversarial face image are respectively input into the EfficientNet network model for forward propagation to calculate the loss. The EfficientNet network model is configured with a first loss function and a second loss function.

[0083] The clean face image is input into the first loss function L. c (θ,x c Calculate its loss value using θ, y); where θ is the parameter of the EfficientNet network model, x c For a clean face image, y represents the real / fake label;

[0084] The adversarial face image is input into the second loss function L. a (,x a Calculate its loss value using x,y; where x a To combat facial images.

[0085] The gradient backpropagation of the clean face image and the adversarial face image loss is calculated using the min-max optimization algorithm, and the parameters of the EfficientNet network model are updated until the final loss function converges, resulting in the trained EfficientNet network model. The final loss function is:

[0086]

[0087] Where, x c Let y be a clean face image, y be the true / false label, ∈ represent the added perturbation noise, and x be the true / false label. c +∈ represents adversarial face images, x c +∈=x aS represents the permissible perturbation range for noise, and θ is a parameter of the EfficientNet network model. For data distribution, This indicates the difference between the training samples and the labels.

[0088] This embodiment employs the AdvProp method to train the adversarial face discrimination network based on the EfficientNet network. Adversarial face images generated by a general perturbation noise generation network attached to the EfficientNet network model are used to train the EfficientNet network model, improving its robustness. Furthermore, the EfficientNet network model has a simple structure, is easy to train, can be lightweightly deployed, and has a wide range of applications, including mobile terminals. Specifically, the AdvProp training method uses primary Batch Normalization (BN) to calculate the loss value of clean face images and auxiliary BN to calculate the loss value of adversarial face images. When using the trained adversarial face discrimination network to recognize actual images, the auxiliary BN is removed; details are omitted here.

[0089] For example, the general perturbation noise generation network includes: a convolutional layer and three deconvolutional layers. The convolutional layer uses the DenseNet network to perform convolution processing on the input clean face image to obtain a feature map. The output feature map has a size of 7×7×256. After passing through a Batch Normalization (BN) process and then through the ReLU activation function, a feature map is obtained, which is used as the input of the next deconvolutional layer.

[0090] The kernel size of each deconvolution layer is 5×5. Each deconvolution layer undergoes BatchNormalization (BN) and is then processed by a corresponding activation function to obtain a feature map, which serves as the input to the next deconvolution layer.

[0091] Specifically, as shown in Table 1, the convolutional layers use the DenseNet network to downsample the input clean face image. The DenseNet network has fewer parameters and higher feature connectivity.

[0092] The first Conv2DTranspose deconvolutional layer includes 128 5×5 convolutional kernels. The output of the convolutional layer is used as the input of the first Conv2DTranspose deconvolutional layer to perform upsampling deconvolution operation, which amplifies the features. The convolution stride s is set to 1, and the padding value is set to same. After deconvolution processing, a feature map is obtained. After BatchNormalization (BN) processing, it is then processed by the ReLU activation function to obtain a feature map, which is used as the input of the second Conv2DTranspose deconvolutional layer.

[0093] The second Conv2DTranspose deconvolution layer includes 64 5×5 convolutional kernels. The output of the first Conv2DTranspose deconvolution layer is used as the input of the second Conv2DTranspose deconvolution layer to perform upsampling deconvolution operation, which amplifies the features. The convolution stride s is set to 1, and the padding value is set to same. After deconvolution processing, a feature map is obtained. After Batch Normalization (BN) processing, the feature map is obtained by passing it through the ReLU activation function, which is used as the input of the third Conv2DTranspose deconvolution layer.

[0094] The third Conv2DTranspose deconvolution layer includes a 5×5 convolution kernel. The output of the second Conv2DTranspose deconvolution layer is used as the input of the third Conv2DTranspose deconvolution layer to perform upsampling deconvolution operation, which amplifies the features. The convolution stride s is set to 2, and the padding value is set to same. After deconvolution processing, a feature map is obtained. After Batch Normalization (BN) processing, the feature map is obtained after Tanh activation function. Dropout processing is performed on each deconvolution layer, and the final output is an adversarial face image.

[0095] Table 1 Disturbance Noise Generation Network

[0096]

[0097] In each iteration of the general perturbation noise generation network, a small number of clean face images are sampled from the clean face dataset in step S132 as target clean face images and input into the general perturbation noise generation network to generate adversarial face images. The number of iterations can be set according to the actual situation.

[0098] As an example, the DenseNet network is designed for generating this generalized perturbation noise. Each layer of the generalized perturbation noise generation network uses Batch Normalization (BN) as a normalization method. This network adds noise to face images to generate adversarial face images. The robustness of the adversarial face discrimination network, namely the EfficientNet network model, is enhanced by training the adversarial face discrimination network using these adversarial face images. While common methods like Free Gaussian Dispersion (FGD) might be used for adding generalized perturbation noise, the generalized perturbation noise in this embodiment is more suitable for unknown adversarial attacks. Dropout is performed on each layer after upsampling; Dropout helps prevent overfitting.

[0099] The embodiments of this disclosure employ neural networks and other methods with fewer parameters and lower computational load in each step, making them suitable for devices with limited storage space, such as mobile terminals. This disclosure proposes a face spoofing recognition method resistant to unknown adversarial attacks. In face spoofing recognition, to counter unknown attacks, the face image is transformed and reconstructed to alter the distribution pattern of adversarial perturbations. Simultaneously, image features are preserved to the greatest extent possible to avoid destroying spoofing traces. Finally, the image is input into an adversarial face discrimination network for recognition. This adversarial face discrimination network generates adversarial face image samples and clean face image samples through an additional general perturbation noise generation network, which are then used to train the adversarial face discrimination network, enhancing the accuracy of face spoofing recognition. This disclosure addresses face spoofing recognition against adversarial attacks at both the data and network levels, compensating for the shortcomings of individual methods from multiple dimensions.

[0100] One aspect of this disclosure provides a face spoofing recognition device resistant to unknown adversarial attacks. For example... Figure 2 As shown, the device 100 includes:

[0101] The acquisition module 110 is used to acquire the actual image containing the human face;

[0102] The reconstruction module 120 is used to perform face detection processing on the actual image to obtain an actual face region image, and to perform reconstruction processing on the actual face region image to obtain an actual face reconstruction image.

[0103] The recognition module 130 is used to input the actual face reconstruction image into a pre-trained adversarial face discrimination network to predict the authenticity of the face in the actual image.

[0104] Specifically, an actual image containing a human face is acquired using existing image acquisition equipment. The reconstruction module 120 performs face detection processing on the actual image using the method described in step S120 above, and reconstructs the detected actual face region image to obtain an actual face reconstruction image. The actual face reconstruction image is then input into the recognition module 130 of the adversarial face discrimination network trained in step 130 for recognition, and the recognition result of the authenticity of the face in the actual image is output.

[0105] This disclosure discloses a face spoofing recognition device for unknown adversarial attacks. It reconstructs the detected actual face region image, alters the distribution pattern of adversarial perturbations, and retains traces of face spoofing. The adversarial face image is used to train the adversarial face discrimination network, improving its robustness. The device in this disclosure involves minimal computational load, occupies little space, operates quickly, is suitable for various scenarios, and features wide application range and low cost.

[0106] For example, the device further includes a training module 140, the training module being used for:

[0107] The process involves acquiring real face images without interference and fake face images, performing face detection processing on the real face images and fake face images to obtain a clean face region image dataset, and reconstructing a portion of the face region images to obtain a face reconstruction image dataset; wherein, the clean face region image dataset and the face reconstruction image dataset together constitute the clean face dataset.

[0108] Select a portion of target clean face images from the clean face dataset, and input the target clean face images into a pre-defined general perturbation noise generation network to generate the adversarial face images;

[0109] The clean face image and the adversarial face image are input into the adversarial face discrimination network to be trained, and the trained adversarial face discrimination network is obtained.

[0110] Specifically, in the training module, perturbation noise is added to the input clean face image by an additional general perturbation noise generation network to generate adversarial face images. The adversarial face discrimination network is then trained in combination with the clean face images to improve the robustness of the adversarial face discrimination network and enhance the recognition effect in multiple dimensions.

[0111] One aspect of this disclosure provides an electronic device, including one or more processors.

[0112] A storage unit is used to store one or more programs that, when executed by one or more processors, enable the one or more processors to implement the method described above.

[0113] The memory and processor are connected via a bus, which can include any number of interconnecting buses and bridges, connecting various circuits of one or more processors and memories. The bus can also connect various other circuits, such as peripheral devices, voltage regulators, and power management circuits, which are well known in the art and will not be elaborated upon here. The bus interface provides an interface between the bus and the transceiver. The transceiver can be a single element or multiple elements, such as multiple receivers and transmitters, providing a unit for communicating with various other devices over a transmission medium. Data processed by the processor is transmitted over the wireless medium via an antenna, which further receives data and transmits it to the processor.

[0114] The processor manages the bus and general processing, and also provides various functions, including timing, peripheral interfaces, voltage regulation, power management, and other control functions. Memory is used to store data used by the processor during operation.

[0115] An electronic device in this embodiment implements the face spoofing recognition method for unknown adversarial attacks as described above. It performs face detection processing and reconstruction on the actual image of the acquired face, achieving high recognition accuracy and better adapting to various recognition conditions.

[0116] One aspect of this disclosure provides a computer-readable storage medium having a computer program stored thereon that, when executed by a processor, can implement the method as described above.

[0117] The computer-readable medium may be included in the systems or electronic devices disclosed herein, or it may exist independently.

[0118] Computer-readable storage media can be any tangible medium that contains or stores a program, and can be an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device. More specific examples include, but are not limited to: electrical connections having one or more wires, portable computer disks, hard disks, optical fibers, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0119] Computer-readable storage media may also include data signals propagated in baseband or as part of a carrier wave, carrying computer-readable program code, specific examples of which include, but are not limited to, electromagnetic signals, optical signals, or any suitable combination thereof.

[0120] The embodiments of this disclosure use additional networks to generate adversarial face images as training samples to train the adversarial face discrimination network, thereby improving the robustness of the adversarial face discrimination network, achieving high recognition accuracy, and better adapting to various recognition conditions.

[0121] It is understood that the above embodiments are merely exemplary implementations used to illustrate the principles of the embodiments of this disclosure, and the embodiments of this disclosure are not limited thereto. For those skilled in the art, various modifications and improvements can be made without departing from the spirit and essence of the embodiments of this disclosure, and these modifications and improvements are also considered to be within the protection scope of the embodiments of this disclosure.

Claims

1. A face spoofing recognition method for unknown adversarial attacks, characterized in that, The method includes: Obtain the actual image containing the human face; The actual image is subjected to face detection processing to obtain an actual face region image, and the actual face region image is subjected to reconstruction processing to obtain an actual face reconstructed image; The actual face reconstruction image is input into a pre-trained adversarial face discrimination network to predict the authenticity of the face in the actual image; wherein, the adversarial face discrimination network is trained using a clean face dataset and an adversarial face image set with added perturbation noise; The process of reconstructing the actual face region image to obtain the actual face reconstruction image includes: blurring the face region image using at least one of Gaussian blur, box blur, Kawase blur, double blur, bokeh blur, and directional blur; and deblurring the blurred face region image using an SDWNet network based on wavelet transform and linear expansion to obtain the actual face reconstruction image. The adversarial face discrimination network is trained using the following steps: The process involves acquiring real face images without interference and fake face images, performing face detection processing on the real face images and fake face images to obtain a clean face region image dataset, and reconstructing a portion of the face region images to obtain a face reconstruction image dataset; wherein, the clean face region image dataset and the face reconstruction image dataset together constitute the clean face dataset. Select a portion of target clean face images from the clean face dataset, and input the target clean face images into a pre-defined general perturbation noise generation network to generate the adversarial face images; The clean face image and the adversarial face image are input into the adversarial face discrimination network to be trained, and the trained adversarial face discrimination network is obtained. The adversarial face discrimination network is an EfficientNet network model. The step of inputting the clean face image and the adversarial face image into the adversarial face discrimination network to be trained includes: The clean face image and the adversarial face image are respectively input into the EfficientNet network model for forward propagation to calculate the loss. The EfficientNet network model is configured with a first loss function and a second loss function. The clean face image is input into the first loss function. Calculate its loss value; where, θ These are the parameters of the EfficientNet network model. x c For clean facial images, y True / False tags; The adversarial face image is input into the second loss function. Calculate its loss value; where, x a To combat facial images; The gradient backpropagation of the clean face image and the adversarial face image loss is calculated using the min-max optimization algorithm, and the parameters of the EfficientNet network model are updated until the final loss function converges, resulting in the trained EfficientNet network model. The final loss function is: in, x c For clean facial images, y For authenticity labels, This represents the added disturbance noise. To combat facial images, = x a ; S This represents the permissible range of disturbance noise. θ These are the parameters of the EfficientNet network model. For data distribution, This indicates the difference between the training samples and the labels.

2. The method according to claim 1, characterized in that, The general perturbation noise generation network includes a convolutional layer and three deconvolutional layers. The convolutional layer uses the DenseNet network to perform convolution processing on the input clean face image to obtain a feature map. The output feature map has a size of 7×7×256. After passing through a Batch Normalization (BN) process and then through the ReLU activation function, a feature map is obtained, which is used as the input of the next deconvolutional layer. The kernel size of each deconvolution layer is 5×5. Each deconvolution layer undergoes BatchNormalization (BN) and is then processed by a corresponding activation function to obtain a feature map, which serves as the input to the next deconvolution layer.

3. The method according to claim 1 or 2, characterized in that, The step of performing face detection processing on the actual image to obtain an actual face region image includes: Adjust the actual image to a preset size; The resized actual image is input into the MobileNet neural network model for feature extraction, and a feature map is output. An anchor box mechanism is used to anchor the feature map, and the anchor boxes are filtered using the Soft-NMS algorithm to obtain face detection boxes. The face detection boxes are then segmented to obtain the actual face region image.

4. A face spoofing recognition device for unknown adversarial attacks, characterized in that, The device includes: The acquisition module is used to acquire actual images containing human faces; The reconstruction module is used to perform face detection processing on the actual image to obtain an actual face region image, and to perform reconstruction processing on the actual face region image to obtain an actual face reconstruction image. The recognition module is used to input the actual face reconstruction image into a pre-trained adversarial face discrimination network to predict the authenticity of the face in the actual image; wherein, the adversarial face discrimination network is trained using a clean face dataset and an adversarial face image set with added perturbation noise; The reconstruction module is further configured to: perform blurring processing on the face region image using at least one of Gaussian blur, box blur, Kawase blur, double blur, bokeh blur and directional blur; and perform deblurring processing on the blurred face region image using an SDWNet network based on wavelet transform linear expansion to obtain the actual face reconstruction image. The device further includes a training module, the training module being used for: The process involves acquiring real face images without interference and fake face images, performing face detection processing on the real face images and fake face images to obtain a clean face region image dataset, and reconstructing a portion of the face region images to obtain a face reconstruction image dataset; wherein, the clean face region image dataset and the face reconstruction image dataset together constitute the clean face dataset. Select a portion of target clean face images from the clean face dataset, and input the target clean face images into a pre-defined general perturbation noise generation network to generate the adversarial face images; The clean face image and the adversarial face image are input into the adversarial face discrimination network to be trained, and the trained adversarial face discrimination network is obtained. The adversarial face discrimination network is an EfficientNet network model, and the training module is further used for: The clean face image and the adversarial face image are respectively input into the EfficientNet network model for forward propagation to calculate the loss. The EfficientNet network model is configured with a first loss function and a second loss function. The clean face image is input into the first loss function. Calculate its loss value; where, θ These are the parameters of the EfficientNet network model. x c For clean facial images, y True / False tags; The adversarial face image is input into the second loss function. Calculate its loss value; where, x a To combat facial images; The gradient backpropagation of the clean face image and the adversarial face image loss is calculated using the min-max optimization algorithm, and the parameters of the EfficientNet network model are updated until the final loss function converges, resulting in the trained EfficientNet network model. The final loss function is: in, x c For clean facial images, y For authenticity labels, This represents the added disturbance noise. To combat facial images, = x a ; S This represents the permissible range of disturbance noise. θ These are the parameters of the EfficientNet network model. For data distribution, This indicates the difference between the training samples and the labels.

5. An electronic device, characterized in that, include: One or more processors; A storage unit for storing one or more programs that, when executed by one or more processors, enable the one or more processors to implement the method according to any one of claims 1 to 3.

6. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it can implement the method according to any one of claims 1 to 3.

Images