Face forgery detection method and system for reducing attribute bias

Abstract

The application discloses a face forgery detection method and system for reducing attribute bias, and the method comprises the following steps: acquiring a video to be detected; pre-processing the video to be detected to obtain an image sequence to be detected; acquiring a pre-trained forgery detection model; inputting the image sequence to be detected into the pre-trained forgery detection model, extracting facial attribute features by using an attribute analysis model, and extracting facial texture features by using a texture analysis model; performing fusion processing on the facial attribute features and the facial texture features by using a space-time feature aggregator to obtain a detection result; and the detection result representing whether the face in the video to be detected is a forgery. The facial attribute features and the facial texture features are extracted by using the attribute analysis model and the texture analysis model, and the facial attribute features and the facial texture features are more distinguishable and robust. The space-time feature aggregator is used for fusing the facial attribute features and the facial texture features, so that the standard deviation between the attributes is reduced, the bias is reduced, and the accuracy, fairness and universality are improved.

Description

Technical Field

[0001] This invention belongs to the field of artificial intelligence technology, specifically relating to a method and system for detecting face forgery that reduces attribute bias. Background Technology

[0002] In recent years, with the rapid development of face editing and face generation, an increasing number of fake videos have circulated on social media, attracting widespread public attention. Existing face forgery detection methods based on spatial texture features have found that forged videos generated by generative adversarial networks exhibit significant inter-frame inconsistencies (such as abrupt texture changes) and unnaturalness in texture details. However, these methods have limitations in handling forged videos with different attributes, often failing to capture the differences in facial attributes displayed between consecutive frames. Facial attributes reveal differences between real and fake faces at a high semantic level. Although facial attributes remain consistent across frames in real videos, forged videos may show significant differences between real and fake faces at a high semantic level.

[0003] Therefore, detection techniques relying solely on spatial texture features cannot comprehensively address the challenges of temporal coherence and facial attribute consistency in forged videos. We need more advanced methods to comprehensively analyze the spatial and temporal dimensions of videos to more accurately identify and distinguish between real and fake faces. Furthermore, we have found that existing face forgery detection methods based on spatial texture features suffer from significant attribute bias. This means they may fail to treat data with different attributes (such as gender, race, and age) fairly, leading to bias against certain groups; and they fail to treat data with equal equality based on attributes such as gender, race, and age, resulting in unfair treatment of certain groups. This bias not only affects the universality and impartiality of the detection technology but also limits its application potential across multiple scenarios and population groups.

[0004] While existing forgery detection techniques based on spatial cues can identify local features and texture anomalies in images, they often overlook the temporal inconsistencies between frames and the high semantic differences in facial attributes in forged videos, leading to insufficient detection accuracy and robustness. Existing forgery detection techniques based on temporal cues suffer from "attribute bias" when processing facial data with different attributes, failing to treat data with equal fairness based on gender, race, age, and other attributes, thus affecting the universality and impartiality of the detection techniques. Furthermore, although temporal cues-based forgery detection techniques can analyze dynamic behavior and physiological features in videos, they also fail to fully capture the consistency of attribute features in videos, limiting the application potential of these detection methods across multiple scenarios and populations. Summary of the Invention

[0005] To address the aforementioned problems in the existing technology, this invention provides a method and system for detecting face forgery that reduces attribute bias. The technical problem to be solved by this invention is achieved through the following technical solution:

[0006] In a first aspect, the present invention provides a method for detecting face forgery with reduced attribute bias, the method comprising:

[0007] Obtain the video to be tested;

[0008] The video to be detected is preprocessed to obtain a sequence of images to be detected;

[0009] Obtain a pre-trained forgery detection model; the pre-trained forgery detection model includes: an attribute analysis model, a texture analysis model, and a spatiotemporal feature aggregator;

[0010] The image sequence to be detected is input into a pre-trained forgery detection model. Facial attribute features are extracted using the attribute analysis model, and facial texture features are extracted using the texture analysis model. The facial attribute features and facial texture features are fused using the spatiotemporal feature aggregator to obtain the detection result of the image sequence to be detected. The detection result indicates whether the face in the video to be detected is forged.

[0011] In one embodiment of the present invention, the video to be detected is preprocessed to obtain a sequence of images to be detected, including:

[0012] For each video to be detected, randomly extract N frames of images from that video;

[0013] A CNN-based face detection model is used to identify face regions in each frame of an image and obtain facial feature points.

[0014] The image sequence to be detected is obtained based on the facial feature points.

[0015] In one embodiment of the present invention, the image sequence to be detected includes several image sequences.

[0016] In one embodiment of the present invention, the training process of the forgery detection model includes:

[0017] Obtain a training dataset; the training dataset includes several face videos and identity information of each face; wherein, each video includes several face attribute labels;

[0018] The face images in the training dataset are segmented into several non-overlapping image windows as an image sequence;

[0019] The image sequence is input into the forgery detection model, and a model loss function is established based on the pre-trained texture analysis model, attribute analysis model, and spatiotemporal feature aggregator.

[0020] The model loss function is processed using the stochastic gradient descent algorithm to minimize the model loss function.

[0021] The gradient of the model loss function with respect to the model parameters is calculated using the backpropagation algorithm. The parameters of the forgery detection model are then updated based on the gradient to obtain the trained forgery detection model.

[0022] In one embodiment of the present invention, the attribute analysis model is obtained by adding a first global average pooling layer and a fully connected layer to a ResNet34 network without pre-trained weights, and then loading the first pre-trained weights.

[0023] In one embodiment of the present invention, the loss function used by the attribute analysis model is expressed as follows:

[0024]

[0025] Where N represents the number of frames in the image sequence to be detected, and F i,a This indicates that F represents the features extracted for the forgery detection task. i, ′ a This represents the features extracted for the facial attribute detection task.

[0026] In one embodiment of the present invention, the texture analysis model is obtained by adding a bottleneck module and a second global average pooling layer to a ResNet101 network without pre-trained weights, and then loading the second pre-trained weights.

[0027] In one embodiment of the present invention, the loss function used in the texture analysis model is expressed as follows:

[0028]

[0029] Where N represents the number of frames in the image sequence to be detected, P i The original label representing a face image, P i, ′ t This represents the prediction result of the texture analysis model for forgery detection.

[0030] In one embodiment of the present invention, the loss function used by the spatiotemporal feature aggregator is expressed as follows:

[0031]

[0032] Where N represents the number of frames in the image sequence to be detected, P iThe original label representing a face image, P i, ′ out This represents the prediction result of the forgery detection model.

[0033] Secondly, the present invention provides a face forgery detection system that reduces attribute bias, the system comprising:

[0034] The video acquisition module is used to acquire the video to be detected.

[0035] The image sequence acquisition module is used to preprocess the video to be detected to obtain the image sequence to be detected;

[0036] The pre-trained forgery detection model includes: an attribute analysis model, a texture analysis model, and a spatiotemporal feature aggregator. The pre-trained forgery detection model is used to extract facial attribute features from the image sequence to be detected using the attribute analysis model and extract facial texture features using the texture analysis model. The spatiotemporal feature aggregator then fuses the facial attribute features and the facial texture features to obtain the detection result of the image sequence to be detected. The detection result indicates whether the face in the video to be detected is forged.

[0037] The beneficial effects of this invention are:

[0038] The solution provided by this invention extracts facial attribute features and facial texture features through attribute analysis model and texture analysis model, which is more discriminative and robust; by using a spatiotemporal feature aggregator to fuse facial attribute features and facial texture features, it can not only identify local texture anomalies in the image, but also detect inconsistencies in facial attributes, thereby reducing the standard deviation between attributes, reducing bias, and improving accuracy, fairness and universality. Attached Figure Description

[0039] Figure 1 This is a schematic diagram illustrating the steps of a face forgery detection method that reduces attribute bias according to an embodiment of the present invention;

[0040] Figure 2 This is a schematic flowchart of a face forgery detection method that reduces attribute bias according to an embodiment of the present invention;

[0041] Figure 3 This is a schematic diagram of the spatiotemporal feature aggregator for a face forgery detection method that reduces attribute bias, as provided in an embodiment of the present invention.

[0042] Figure 4 This is a schematic diagram of a face forgery detection system that reduces attribute bias, as provided in an embodiment of the present invention. Detailed Implementation

[0043] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0044] This invention provides a method and system for detecting face forgery that reduces attribute bias.

[0045] It should be noted that the execution subject of the face forgery detection method with reduced attribute bias provided in the embodiments of the present invention can be a face forgery detection system with reduced attribute bias, but is not limited thereto.

[0046] Below, we will first introduce a face forgery detection method that reduces attribute bias, as provided in the embodiments of the present invention.

[0047] like Figure 1 As shown in the figure, the face forgery detection method with reduced attribute bias provided by the present invention may include the following steps:

[0048] S1, acquire the video to be tested;

[0049] S2, preprocess the video to be detected to obtain the image sequence to be detected;

[0050] S3, Obtain the pre-trained forgery detection model; the pre-trained forgery detection model includes: attribute analysis model, texture analysis model, and spatiotemporal feature aggregator;

[0051] S4. Input the image sequence to be detected into the pre-trained forgery detection model, use the attribute analysis model to extract facial attribute features, and use the texture analysis model to extract facial texture features; use the spatiotemporal feature aggregator to fuse the facial attribute features and facial texture features to obtain the detection result of the image sequence to be detected; the detection result indicates whether the face in the video to be detected is forged.

[0052] To address the shortcomings of spatial cue-based face forgery detection methods, this invention introduces an attribute analysis model to extract facial attribute features, including gender and ethnicity, in addition to texture analysis models. This invention proposes a spatiotemporal feature-based face forgery detection framework based on texture and attributes. The proposed spatiotemporal feature aggregator fuses texture and attribute features, with spatial and temporal attention mechanisms interacting to improve model accuracy and reduce attribute bias across different datasets. The face forgery detection method proposed in this invention integrates texture and attribute features in the spatiotemporal frequency domain for comprehensive analysis. This not only captures the spatial and temporal features of forged videos but also identifies high semantic differences in facial attributes, thus achieving more accurate and fair face forgery detection.

[0053] Please refer to the flowchart of a face forgery detection method that reduces attribute bias according to an embodiment of the present invention. Figure 2 For a schematic diagram of the spatiotemporal feature aggregator, please refer to [link / reference]. Figure 3 To facilitate understanding, the following will combine... Figure 2 and Figure 3 Each step of the face forgery detection method provided in the embodiments of the present invention will be described separately.

[0054] from Figure 2 It can be seen that in the first stage,

[0055] A new face image dataset was constructed from the YFCC-100M Flickr dataset. Specifically, this dataset contains 108,501 face images that are racially balanced and labeled with race, gender, and age group. These images will be used for subsequent training and testing of attribute analysis models. The images in this dataset were obtained through a specified path.

[0056] The dataset images are fed into the attribute analysis model. Specifically, the network uses ResNet34 as the base model and replaces its final fully connected layer with a new layer suitable for attribute prediction.

[0057] The convolutional layers of the model are used to extract feature representations of the input image, which are then fed into a modified fully connected layer for classification.

[0058] This is achieved through the final fully connected layer of the ResNet34 model. This layer transforms the features into predictions for specific attributes.

[0059] The model predicts attributes including gender, age, and race. Gender is predicted using a binary classification problem to determine whether a person is male or female; age is predicted using a multi-class classification problem to determine different age groups; and race is predicted using a multi-class classification problem to determine different race categories.

[0060] In the second phase,

[0061] The specified dataset of fake face videos is processed into a sequence of images to be detected, and then input into the attribute analysis model and the texture analysis model respectively.

[0062] The dataset images are used to extract facial attribute features through an attribute analysis model and facial texture features through a texture analysis model. The attribute and texture features are then input into a spatiotemporal aggregator for feature fusion.

[0063] Temporal attention and spatial attention mechanisms are used to enhance the representation of fused features.

[0064] Determine whether the image is fake by using a binary classification problem.

[0065] For S1, it can include:

[0066] The system iterates through all videos in a specified video folder path, selecting them as the videos to be tested.

[0067] For S2, it can include:

[0068] S21, For each video to be detected, randomly extract N frames of images from the video to be detected.

[0069] S22, using a CNN-based face detection model to identify face regions in each frame of the image and obtain facial feature points.

[0070] Specifically, a shape predictor is used to process the detected faces in each frame of the image to obtain detailed facial feature points.

[0071] S23, obtain the image sequence to be detected based on facial feature points.

[0072] The image sequence to be detected may include several image sequences.

[0073] The image sequence input here is an unprocessed image sequence. At this point, it's unknown how many images to process in a batch, how many frames to extract from a video, and the height and width of the image sequence are also unprocessed. After inputting into the model, the image sequence is loaded and further processed. The following process can be achieved by customizing the `MyDatasetWild` class in the code: for example, customizing the number of images in each batch, the number of video frames in each image sequence, the number of image channels, and the height and width of the images. The number of images can be represented by the batch size, the number of video frames can be set to N frames, and the number of channels can be set to 3 for all color images. In the code, the images are first adjusted to 256x256 pixels, and then cropped to 224x224 for subsequent processing. Of course, these custom parameters can be adjusted according to the user's needs.

[0074] Specifically, based on all facial feature points, the face region is cropped from N frames of images, and the face region is resized and edge-filled to generate N frames of face images as the sequence of images to be detected. Optionally, this embodiment of the invention selects the WildDeepfake face forgery detection dataset, which may include 7314 facial sequences extracted from 707 Deepfake videos. These videos can be collected from Bilibili and YouTube, and they are synthesized using multiple methods, making detection more difficult.

[0075] For S3, a pre-trained forgery detection model can include: an attribute analysis model, a texture analysis model, and a spatiotemporal feature aggregator. For ease of understanding, each module in the pre-trained forgery detection model will be described below.

[0076] Attribute Analysis Model

[0077] Specifically, the attribute analysis model is obtained by adding a first global average pooling layer and a fully connected layer to a ResNet34 network without pre-trained weights, and then loading the first pre-trained weights.

[0078] The face forgery detection method proposed in this invention utilizes face attribute feature detection to train an attribute analysis model, thereby achieving high-level semantic-level attribute feature measurement. To extract face attribute features, face attribute detection is used as a pre-training task to extract attribute information. An attribute analysis model obtained using a face recognition method based on face features is used to extract attribute features such as race, gender, and age. Specifically, this attribute analysis model can adopt the classic ResNet34 architecture, which consists of 34 convolutional layers, including multiple residual blocks. Each residual block consists of two 3×3 convolutional layers, and may also include 1×1 convolutional layers and identity mappings to facilitate cross-layer connections. This residual learning structure helps alleviate the gradient vanishing problem during deep neural network training, making the model easier to train and optimize. This method constructs a new face image dataset from the YFCC-100MFlickr dataset. This dataset contains 108,501 face images that are racially balanced and labeled with race, gender, and age group. N video frames V = I1, I2, I3, ..., I... N Input attribute analysis model, extract attribute features F a ={f a,1 ,f a,2 ,f a,3 ,…,f a,NThe pre-trained model was also fine-tuned for downstream tasks such as face forgery detection. This embodiment of the invention utilizes a loss function to constrain the attribute features before and after the model update, ensuring that the updated attribute features F... a The distance of l1 is closer to the original attribute detection feature F. a ′.

[0079] The expression for the loss function used in the attribute analysis model is as follows:

[0080]

[0081] Where N represents the number of frames in the image sequence to be detected, F i,a This indicates that F represents the features extracted for the forgery detection task. i, ′ a This represents the features extracted for the facial attribute detection task.

[0082] Understandably, the features extracted by a forgery detection task can include those extracted by a forgery detection model, and the features extracted by a facial attribute detection task can include those extracted by an attribute analysis model.

[0083] The process of extracting facial attribute features using an attribute analysis model can include:

[0084] The attribute analysis model is used to process the image sequence to be detected to obtain the corresponding feature vectors;

[0085] The feature vector is mapped and upgraded to an 18-dimensional feature vector, thereby extracting facial attribute features.

[0086] Texture analysis model

[0087] The texture analysis model is obtained by adding a bottleneck module and a second global average pooling layer to a ResNet101 network without loading pre-trained weights, and then loading the second pre-trained weights.

[0088] Considering that forged faces may lack texture details, this embodiment of the invention employs a texture analysis model to extract texture features, thereby improving the performance of forgery detection. A Res2Net-based texture analysis model is applied as the backbone network to obtain texture features F. t ={f t,1 ,f t,2 ,f t,3 ,…,f t,NThis approach can capture detailed spatial forgery clues that complement temporal attribute features. Texture features are extracted using a texture analysis model based on a multi-scale backbone architecture. This model divides input features into several groups. The current group filter first extracts features from one group of input feature maps, then combines the output feature map of the previous group with another group of input feature maps and sends it to the next group filter. This process is repeated until all input feature maps are processed. Finally, the feature maps of all groups are concatenated and sent to another 1×1 filter to integrate the information. Along any possible path from the input feature map to the output feature map, the effective perception increases when passing through a 3×3 filter, and many equivalent feature scales are obtained due to the combination effect. The classification results from the module are used to design a loss function, making the predicted label P... t It is closer to the real label P.

[0089] The expression for the loss function used in the texture analysis model is as follows:

[0090]

[0091] Where N represents the number of frames in the image sequence to be detected, P i The original label representing a face image, P i, ′ t This represents the prediction result of the texture analysis model for forgery detection.

[0092] The process of extracting facial texture features using a texture analysis model can include:

[0093] The bottleneck module and global average pooling layer in the texture analysis model are used to process the image sequence to be detected to obtain the corresponding facial texture features.

[0094] These modules can use depthwise separable convolution and batch normalization techniques to gradually abstract features, ultimately generating high-level feature maps as facial texture features. These facial texture features can capture the texture and shape information of the face.

[0095] Spatiotemporal feature aggregator

[0096] Please refer to the schematic diagram of the spatiotemporal feature aggregator in the face forgery detection method provided in this embodiment of the invention. Figure 3 Using a spatiotemporal feature aggregator to analyze facial attribute features F t and facial texture features F a The process involves fusion processing to achieve forgery detection.

[0097] Specifically, from Figure 3 The working process of the spacetime aggregator can be seen from this, that is... Figure 2 The specific content of the second-stage feature fusion part is as follows:

[0098] (1) In the spatiotemporal aggregator, the input texture features and attribute features are first input into the spatiotemporal attention network. After processing by temporal attention and spatial attention mechanisms, the network selectively focuses on the feature attention region and extracts high-level texture features. and and the attributes and characteristics of high-level personnel and

[0099] (2) Then, the texture features and attribute features obtained through the temporal attention mechanism are further processed by residual blocks to obtain temporally fused features. The texture and attribute features obtained through the spatial attention mechanism are further processed by the residual fast block to obtain spatially fused features.

[0100] (3) Repeat (1) and (2) to obtain more accurate high-level features through multi-layer spatiotemporal feature fusion. and

[0101]

[0102] The obtained features are input into a spatiotemporal feature aggregator, which selectively manages the attention regions of the column features through a temporal and spatial attention mechanism. The expressions for the features input to each layer of the aggregator are as follows:

[0103] F n,1 =Aggre1(f t,n ,f a,n ), n=1,2,…,N;

[0104] F n,m =Aggre m (F 2n-1,m-1 ,F 2n,m-1 ),1,2,…,N / 2 n m = 2, 3, ..., log₂M;

[0105] Among them, F n,1 This represents the output of the first layer of the spatiotemporal feature aggregator, where m represents the m-th layer of the spatiotemporal feature aggregator. Through multi-layer spatiotemporal feature fusion, the final feature F used for forgery detection can be obtained. out The loss function of the last layer of the spatiotemporal feature aggregator is defined as constraining the final prediction P using the true value P. out .

[0106] The expression for the loss function used by the spatiotemporal feature aggregator is as follows:

[0107]

[0108] Where N represents the number of frames in the image sequence to be detected, P i The original label representing a face image, P i, ′ out This represents the prediction result of the forgery detection model.

[0109] Therefore, the loss function in the embodiments of the present invention can be L. sum =L attribute +L texture +L out Understandably, L attribute L represents the loss function used in the attribute analysis model. texture L represents the loss function used in the texture analysis model. out This represents the loss function used by the spatiotemporal feature aggregator.

[0110] Optionally, the network architecture of the pre-trained forgery detection model can be as shown in Table 1.

[0111] Table 1

[0112]

[0113] Specifically, the training process for a forgery detection model includes:

[0114] Obtain the training dataset; the training dataset includes several face videos and the identity information of each face; each video includes several face attribute labels;

[0115] The face images in the training dataset are segmented into several non-overlapping image windows as an image sequence;

[0116] Image sequences are input into a forgery detection model, and a model loss function is established based on a pre-trained texture analysis model, attribute analysis model, and spatiotemporal feature aggregator.

[0117] The stochastic gradient descent algorithm is used to process the model loss function in order to minimize the model loss function.

[0118] The gradient of the model loss function with respect to the model parameters is calculated using the backpropagation algorithm. The parameters of the forgery detection model are then updated based on the gradient, resulting in a successfully trained forgery detection model.

[0119] Understandably, image sequences can be formed by sequentially selecting a certain number of consecutive image frames. Each image includes a face image and its attribute label, which should correspond to the label information of the corresponding video. After the image sequence is input into the forgery detection model, the model converts the image sequence into the format required for model training. When loading the pre-trained texture analysis model and attribute analysis model, the corresponding pre-trained weights also need to be loaded. The capsule network is initialized as a spatiotemporal feature aggregator and used for subsequent classification tasks. In the training loop, when using the stochastic gradient descent algorithm to minimize the model loss function, the attribute analysis model can use the cross-entropy loss function, the texture analysis model can use the L1 loss function, and the spatiotemporal feature aggregator can use the classification loss function. The parameters of the forgery detection model are updated according to the obtained gradients until the performance of the forgery detection model on the training dataset reaches a good level, thus obtaining a well-trained forgery detection model.

[0120] For S4, the sequence of images to be detected is input into a pre-trained forgery detection model. Facial attribute features are extracted using an attribute analysis model, and facial texture features are extracted using a texture analysis model. The facial attribute features and facial texture features are fused using a spatiotemporal feature aggregator to obtain the detection result of the sequence of images to be detected. The detection result indicates whether the face in the video to be detected is forged.

[0121] Specifically, the structural diagram of the spatiotemporal feature aggregator is as follows: Figure 3 As shown, the spatiotemporal feature aggregator inputs facial attribute features and facial texture features into the Capsule Network (CAPNet) for feature fusion. First, the facial attribute features and facial texture features are dimensionality-reduced using a dimensionality reduction layer and adjusted to a shape suitable for subsequent processing. Then, these features are segmented into multiple fragments along the temporal dimension, and each fragment is input into three different spatiotemporal attention mechanism modules. These modules learn the temporal-spatial importance of features in the sequence. Pooling layers and bottleneck layers are then used to further compress the features. Finally, a classifier outputs the detection results of the image sequence to be detected, achieving the detection of facial features after feature fusion.

[0122] The simulation results of this invention are as follows:

[0123] In the simulation experiment of this invention embodiment, the performance of the existing technology based on spatiotemporal texture features and the face forgery detection method based on spatiotemporal texture features and attribute features proposed in this invention embodiment on the gender attribute of the WildDeepfake dataset is compared. The comparison results are shown in Table 2.

[0124] Table 2

[0125]

[0126] As can be seen from Table 2, the method based on spatiotemporal texture features achieves an average accuracy of 82.65%, but there is a huge difference between the male and female attributes, with a standard deviation of 3.38. After using the spatiotemporal fusion features proposed in this embodiment of the invention, the average accuracy not only increases to 82.94%, but the difference between the male and female attributes is also greatly reduced, with a standard deviation of only 0.03, indicating that the bias between different gender attributes is effectively reduced. Based on the same dataset, the average accuracy of the face image forgery detection method using fuzzy networks is 64.47%, the average accuracy of the deep fake video detection method based on face distortion is 55.46%, the average accuracy of the Xception face recognition technology is 77.25%, the average accuracy of the deep fake detection method is 76.25%, the average accuracy of the frequency-aware face forgery detection method is 80.66%, the average accuracy of the face recognition research method based on fake face recognition technology is 77.38%, the average accuracy of the deep fake video detection method based on frame inference is 75.88%, and the average accuracy of the end-to-end deep fake detection method based on visual converter is 81.35%. It can be seen that the average accuracy of the face forgery detection method based on spatiotemporal texture features and attribute features proposed in this embodiment of the invention reaches 82.94%, which demonstrates that the spatiotemporal texture and attribute features extracted by the method provided in this embodiment of the invention are efficient and effective.

[0127] In the simulation experiments of this invention, the performance of existing technologies based on spatiotemporal texture features and the face forgery detection method based on spatiotemporal texture features and attribute features proposed in this invention was compared on the racial attributes of the WildDeepfake dataset. The comparison results are shown in Table 3. Racial attributes can include: European, East Asian, African, and South Asian.

[0128] Table 3

[0129]

[0130] As can be seen from Table 3, the method based on spatiotemporal texture features has an average accuracy of 81.93%, but there are huge attribute differences between different ethnic attributes, with a standard deviation of 6.20. After using the spatiotemporal fusion features proposed in the embodiments of the present invention, the average accuracy has been improved to 84.12%, and the attribute differences between different ethnic attributes have been greatly reduced to 3.75, indicating that the bias between different ethnic attributes has been effectively reduced. Based on the same dataset, the average accuracy rates of the face image forgery detection methods using fuzzy networks were 64.47%, those using deep fake video detection methods based on face distortion were 55.46%, those using Xception face recognition technology were 77.25%, those using deepfake detection methods were 76.25%, those using frequency-aware face forgery detection methods were 80.66%, those using fake face recognition technology were 77.38%, those using frame inference-based deep fake video detection methods were 75.88%, and those using end-to-end deepfake detection methods based on visual converters were 81.35%. It can be seen that the average accuracy of the face forgery detection method based on spatiotemporal texture features and attribute features proposed in this embodiment of the invention reaches 84.12%.

[0131] After conducting 100 rounds of testing using each of the two methods described above, the following results were obtained:

[0132] For gender attributes: using spatial texture methods can lead to gender bias, with the standard deviation of attribute differences ranging from 0.01 to 5.8 and the highest accuracy reaching 82.64%; using spatiotemporal texture + attribute methods can mitigate gender bias, with the standard deviation of attribute differences ranging from 0.01 to 3.155 and the highest accuracy reaching 82.94%.

[0133] Regarding racial attributes: using spatial textures can lead to racial bias, with the standard deviation of attribute differences ranging from 1.158 to 10.645 and a maximum accuracy of 81.93%; using spatiotemporal textures combined with attributes can mitigate racial bias, with the standard deviation of attribute differences ranging from 0.764 to 7.507 and a maximum accuracy of 84.12%.

[0134] In summary, the face forgery detection method proposed in this invention, based on spatiotemporal frequency domain cue enhancement, considers frequency domain forgery cues in both time and space, resulting in more discriminative and robust features. It is the first to employ a fusion technique of texture and attribute features, which can not only identify local texture anomalies in images but also detect inconsistencies in facial attributes. This face forgery detection method not only improves accuracy but also reduces the standard deviation between attributes, minimizing bias and enhancing the fairness and universality of the detection method.

[0135] Secondly, corresponding to the above-described embodiment of a face forgery detection method that reduces attribute bias, this embodiment of the invention also provides a face forgery detection system that reduces attribute bias, such as... Figure 4 As shown, the system may include:

[0136] The video acquisition module is used to acquire the video to be detected.

[0137] The image sequence acquisition module is used to preprocess the video to be detected to obtain the image sequence to be detected;

[0138] The pre-trained forgery detection model includes: an attribute analysis model, a texture analysis model, and a spatiotemporal feature aggregator. The pre-trained forgery detection model is used to extract facial attribute features from the image sequence to be detected using the attribute analysis model and extract facial texture features using the texture analysis model. The spatiotemporal feature aggregator is used to fuse the facial attribute features and facial texture features to obtain the detection result of the image sequence to be detected. The detection result indicates whether the face in the video to be detected is forged.

[0139] It is understood that the working process of the video acquisition module corresponds to step S1 in the method embodiment, the working process of the image sequence acquisition module corresponds to step S2 in the method embodiment, and the working process of the pre-trained forgery detection model corresponds to steps S3-S4 in the method embodiment. For the specific processing process of each module, please refer to the specific method embodiment, which will not be elaborated here.

[0140] The embodiments of the present invention extract facial attribute features and facial texture features through attribute analysis models and texture analysis models, which are more discriminative and robust. By using a spatiotemporal feature aggregator to fuse facial attribute features and facial texture features, it is possible not only to identify local texture anomalies in the image, but also to detect inconsistencies in facial attributes. This reduces the standard deviation between attributes, reduces bias, and improves accuracy, fairness, and universality.

[0141] It should be noted that, in the description of this invention, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this invention, "a plurality of" means two or more, unless otherwise explicitly specified.

[0142] The system embodiments described in this specification are basically similar to the method embodiments, so the description is relatively simple. For relevant details, please refer to the description of the method embodiments.

[0143] The above description is merely a preferred embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention are included within the scope of protection of the present invention.

Claims

1. A face spoofing detection method that reduces attribute bias, characterized by, The method comprises the following steps: acquiring a video to be detected; preprocessing the video to be detected to obtain an image sequence to be detected; acquiring a pre-trained forgery detection model; the pre-trained forgery detection model comprises an attribute analysis model, a texture analysis model and a spatio-temporal feature aggregator; the image sequence to be detected is input into the pre-trained forgery detection model, the attribute analysis model is used to extract facial attribute features, the texture analysis model is used to extract facial texture features, the spatio-temporal feature aggregator is used to fuse the facial attribute features and the facial texture features, and a detection result of the image sequence to be detected is obtained; the detection result represents whether a face in the video to be detected is a fake.

2. The face spoofing detection method of claim 1, wherein, The preprocessing of the video to be detected to obtain the image sequence to be detected comprises the following steps: randomly extracting N pictures in each video to be detected; using a face detection model based on a CNN network to identify a face region in each picture, and acquiring facial feature points; obtaining the image sequence to be detected based on the facial feature points.

3. The face spoofing detection method of claim 1, wherein, The image sequence to be detected comprises a plurality of picture sequences.

4. The face spoofing detection method of claim 1, wherein, The training process of the forgery detection model comprises the following steps: acquiring a training data set; the training data set comprises a plurality of face videos and identity information of each face; each video comprises a plurality of face attribute labels; segmenting face images in the training data set into a plurality of non-overlapping image windows as image sequences; inputting the image sequences into the forgery detection model, loading the pre-trained texture analysis model, attribute analysis model and spatio-temporal feature aggregator to establish a model loss function; processing the model loss function by using a stochastic gradient descent algorithm to minimize the model loss function; calculating the gradient of the model loss function with respect to the model parameters by using a back propagation algorithm, updating the parameters of the forgery detection model according to the gradient, and obtaining a trained forgery detection model.

5. The face spoofing detection method of claim 1, wherein, The attribute analysis model is obtained by adding a first global average pooling layer and a fully connected layer to a ResNet34 network without loading pre-trained weights, and loading first pre-trained weights.

6. The face spoofing detection method of claim 5, wherein, The expression of the loss function used by the attribute analysis model is as follows: wherein N represents the frame number of the image sequence to be detected, F i,a represents the features extracted for the forgery detection task, F i, ′ a represents the features extracted for the face attribute detection task.

7. The face spoofing detection method of claim 1, wherein, The texture analysis model is obtained by adding a bottleneck module and a second global average pooling layer to a ResNet101 network without loading pre-trained weights, and loading second pre-trained weights.

8. The face spoofing detection method of claim 7, wherein, The expression of the loss function used by the texture analysis model is as follows: Wherein, N represents the frame number of the image sequence to be detected, P i The original label of the face image, P i, t The prediction result of the texture analysis model for the forgery detection.​ 9. The face spoofing detection method of claim 1, wherein, The expression of the loss function used by the spatio-temporal feature aggregator is as follows: Wherein, N represents the frame number of the image sequence to be detected, P i represents the original label of the face image, P i, ′ out represents the prediction result of the forgery detection model.

10. A face spoofing detection system that reduces attribute bias, the system comprising: The method comprises the following steps: a video acquisition module is configured to acquire a video to be detected; an image sequence acquisition module is configured to preprocess the video to be detected to obtain an image sequence to be detected; a pre-trained forgery detection model comprises an attribute analysis model, a texture analysis model and a spatio-temporal feature aggregator; The pre-trained forgery detection model is configured to: extract face attribute features from the to-be-detected image sequence by using the attribute analysis model, extract face texture features from the to-be-detected image sequence by using the texture analysis model, and perform fusion processing on the face attribute features and the face texture features by using the spatio-temporal feature aggregator to obtain a detection result of the to-be-detected image sequence, wherein the detection result represents whether a face in the to-be-detected video is a forgery.

Images