Face forgery detection method based on texture analysis

Abstract

The invention discloses a face forgery detection method based on texture analysis. A backbone network and a texture network are included. The texture network is composed of a local network and a frequency network. According to the method, RGB feature information is extracted through a backbone network, and local feature information and frequency feature information are extracted through a texture network. And then, fusing the local feature information and the frequency feature information by using a channel attention mechanism to obtain texture information features. And finally, inputting the RGB feature information and the texture information features into a classifier for prediction. In combination with texture features, the method can capture subtle differences which are difficult to reveal by high-level semantic information, so that the model is more robust when coping with different counterfeiting methods, and the generalization ability of the model is remarkably improved.

Description

Technical Field

[0001] The invention relates to the technical field of image processing, and in particular to a human face forgery detection method based on texture analysis. Background Art

[0002] Deep fake technology refers to the technology that uses deep learning and artificial intelligence technology to generate fake facial images and videos. With the rapid development of face fake technology based on generative adversarial networks (GAN), fake images tampered with by face fake technology are becoming more and more realistic, and it is no longer possible to distinguish the real from the fake with human eyes alone. These fake technologies are used to create false information and online fraud, posing a serious threat to social and personal security. Therefore, it is crucial to effectively identify false information and online fraud created by deep fake technology.

[0003] Current Deepfake detection methods have achieved some success in dealing with fake images, especially detectors based on convolutional neural networks (CNNs), but they also have obvious limitations. First, most existing methods rely on high-level semantic features, such as facial expressions, structures, and edge features. However, such high-level features are easily imitated by fake generative models, especially when the fake models continue to evolve, the high-level semantic information becomes more and more similar, making it difficult for the model to identify the real and the fake.

[0004] In this context, texture analysis has attracted attention as an effective supplement to high-level semantic features. Unlike high-level features, texture features focus on local details of the image and can reveal the inconsistencies of forged images at the microscopic level. For example, the phase spectrum of a forged image may change during the upsampling process, and detecting these changes can help distinguish between the real and the fake. However, this method based on GAN upsampling features may not be able to effectively identify forged images generated by non-GAN technology, thereby reducing the accuracy and generalization ability of detection. Summary of the invention

[0005] In order to solve the existing technical problems, the present invention provides a face forgery detection method based on texture analysis, comprising the following steps: 1. Obtain a training face dataset, which includes four types of fake face images: DeepFake, Face2Face, FaceSwap, and NeuralTexture; 2. Preprocessing the acquired face data set, wherein the preprocessing method includes processing the face image into a size of 299×299 and performing a random erasing operation on the face image; 3. Input the preprocessed face dataset into the constructed forgery detection model to obtain the detection results; 3.1 Input the preprocessed face image into the backbone network to obtain RGB feature information, wherein the backbone network uses the Xception network; 3.2 Input the preprocessed face image into the texture network to obtain local feature information and frequency feature information, wherein the local feature information is extracted by using the Xception network after filtering with the SRM filter, and the frequency feature information is extracted by using the Xception network after filtering with the Gabor filter; 3.3 The local feature information and frequency feature information are respectively subjected to the channel attention mechanism to obtain the local feature weight and the frequency feature weight, wherein the channel attention performs maximum pooling and average pooling on the feature information, enters the multi-layer perceptron and adds the results, and obtains the channel attention weight after being processed by the Sigmoid activation function; 3.4 Multiplying the local feature weight and the frequency feature information to obtain feature information 1, and multiplying the frequency feature weight and the local feature weight to obtain feature information 2; 3.5 The feature information 1 is fused with the frequency feature information to obtain feature information 3, and the feature information 2 is fused with the local feature information to obtain feature information 4, wherein the fusion performs bilinear pooling on the feature information 1 and the frequency feature information, and the fusion performs bilinear pooling on the feature information 2 and the local feature information; 3.6 Add the feature information 3 and the feature information 4 to obtain texture feature information; 3.7 Input the RGB feature information and texture feature information into the classifier for prediction; BRIEF DESCRIPTION OF THE DRAWINGS

[0006] Figure 1 It is a flow chart of the texture analysis method of the present invention.

[0007] Figure 2 This is a flow chart of the classifier prediction of the present invention. DETAILED DESCRIPTION

[0008] The following is a detailed description with reference to the accompanying drawings.

[0009] like Figure 1 As shown, the face detection method based on texture analysis in an embodiment of the present invention comprises the following steps: 1. Obtain a training face dataset, which includes four types of fake face images: DeepFake, Face2Face, FaceSwap, and NeuralTexture. 720 image files of the dataset are used as training sets, 140 image files are used as validation sets, and 140 datasets are used as test sets; 2. Preprocessing the acquired face data set, wherein the preprocessing method includes adjusting the face image to a size of 299×299, performing a random erasing operation on the image, and converting the image into a Tensor format; 3. Input the preprocessed face data set into the constructed forgery detection model to obtain the detection result. The training process of the forgery detection model includes the following steps: 3.1 Input the preprocessed face image into the backbone network to obtain RGB feature information, wherein the backbone network uses the Xception network; 3.2 Input the preprocessed face image into the texture network to obtain local feature information and frequency feature information, wherein the local feature information is filtered by the SRM filter and then extracted by the Xception network, and the frequency feature information is filtered by the Gabor filter and then extracted by the Xception network; 3.3 The local feature information and frequency feature information are respectively subjected to the channel attention mechanism to obtain the local feature weight and the frequency feature weight. The channel attention performs the maximum pooling and average pooling on the feature information, and then enters the multi-layer perceptron to add the results, and after the Sigmoid activation function is processed, the channel attention weight is obtained. Among them, the multi-layer perceptron includes 3 fully connected layers and ReLU function; 3.4 Multiplying the local feature weight and the frequency feature information to obtain feature information 1, and multiplying the frequency feature weight and the local feature weight to obtain feature information 2; 3.5 Add the feature information 1 to the frequency feature information to obtain feature information 3, and add the feature information 2 to the local feature information to obtain feature information 4, wherein the fusion performs bilinear pooling on the feature information 1 and the frequency feature information, and performs bilinear pooling on the feature information 2 and the local feature information; 3.6 Add the feature information 3 and the feature information 4 to obtain texture feature information; 3.7 The RGB feature information and texture feature information are input into the classifier for prediction. The prediction process of the classifier is as follows: Figure 2 As shown in the figure. First, the RGB features and texture features are convolved to reduce the dimension and normalized using the ReLU activation function. Then, the two types of features are concatenated in dimension and further reduced in dimension through pooling. Finally, the input is classified as true or false through a fully connected layer.

Claims

1. A face forgery detection method based on texture analysis, characterized in that: The steps include: Get the face dataset used for training; Preprocess the acquired face data set; The preprocessed face dataset is input into the constructed forgery detection model to obtain the detection results.

2. The method according to claim 1, characterized in that The forgery detection model includes a backbone network and a texture network, and the texture network includes a local network and a frequency network.

3. The method according to claim 1, characterized in that The training data set includes four types of forged face images, namely: FaceSwap, Face2Face, DeepFake and NeuralTexture.

4. The method according to claim 1, characterized in that The preprocessing method includes processing the face image into a size of 299×299 and performing a random erasing operation on the face image.

5. The method according to claim 4, characterized in that The training process of the forgery detection model includes the following steps: Inputting the preprocessed face image into the backbone network to obtain RGB feature information; Inputting the preprocessed face image into the texture network to obtain local feature information and frequency feature information; The local feature information and the frequency feature information are respectively subjected to a channel attention mechanism to obtain a local feature weight and a frequency feature weight; The local feature weight is multiplied by the frequency feature information to obtain feature information 1, and the frequency feature weight is multiplied by the local feature weight to obtain feature information 2; The feature information 1 is fused with the frequency feature information to obtain feature information 3, and the feature information 2 is fused with the local feature information to obtain feature information 4; Adding the feature information 3 and the feature information 4 to obtain texture feature information; The RGB feature information and texture feature information are input into a classifier for prediction.

Images