Video spoof face detection method and system fusing artifact features and attention guidance
By integrating artifact features with attention-guided video face detection methods, the problems of insufficient artifact processing and limited decision-making regions are solved, achieving efficient and accurate face detection.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- FUJIAN DAZHI NETWORK TECHNOLOGY CO LTD
- Filing Date
- 2026-02-28
- Publication Date
- 2026-06-02
Smart Images
Figure CN122135414A_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of video analysis, and in particular relates to a method and system for detecting fake faces in videos by fusing artifact features and attention guidance. Background Technology
[0002] With the widespread use of social media and online platforms, people increasingly prefer to obtain information through short videos. However, the rapid development of science and technology has also made facial spoofing technology easier to implement, which could be maliciously used for identity theft, fraud, and the spread of false information, posing a greater threat to personal privacy and identity security. Facial spoofing can not only lead to financial losses but also damage personal reputation and even trigger a crisis of social trust on a broader level. For businesses, developing effective methods for detecting facial spoofing in videos can effectively prevent identity theft and fraud, protect sensitive information of the company and its customers, and reduce potential financial losses and legal risks. At the same time, by adopting advanced detection technologies, companies can maintain their brand image, enhance consumer trust in their products and services, and improve market competitiveness. Furthermore, the application of video facial spoofing detection technology can ensure the authenticity of published content, effectively prevent the spread of false information, and promote the steady development of enterprises during digital transformation. In conclusion, video facial spoofing detection not only enhances the security of enterprises but also provides strong support for their sustainable development.
[0003] Most existing methods for spoofing faces in videos involve forging facial images from each frame of a real video. Errors during the forgery process and limitations in the algorithms often result in artifacts in the forged videos. Furthermore, existing models have limitations in their decision region selection, potentially overlooking important features and affecting detection accuracy. To address these issues, we propose a network that integrates artifact features and attention features to extract richer and more effective information from forged videos, thereby optimizing the method for detecting fake faces in videos. Summary of the Invention
[0004] The purpose of this invention is to propose a video fake face detection method and system that integrates artifact features and attention guidance. This method highlights artifact details and key decision areas by integrating artifact features and attention guidance, so as to achieve efficient and accurate video fake face detection.
[0005] To achieve the above objectives, the technical solution of the present invention is: a method for detecting fake faces in videos that integrates artifact features and attention guidance, comprising the following steps:
[0006] Step S1: Construct a face image dataset;
[0007] Step S2: Construct a fake face detection model. The model includes a feature extraction branch, an artifact amplification branch, an attention branch, and a classification network to generate artifact protrusion feature maps and extract attention maps. The fusion features of the artifact protrusion feature maps and attention maps are used as input to the classification network to obtain fake face detection results.
[0008] Step S3: Train a fake face detection model using a face image dataset, and then use the trained fake face detection model for fake face detection.
[0009] Preferably, step S1 specifically includes the following steps:
[0010] Step S11: According to the set target frame rate, read and save the corresponding images frame by frame from the video to obtain a set of continuous video frames;
[0011] Step S12: Use a multi-task cascaded convolutional neural network to identify the face region in each frame of the image, ensuring that only the face in the video frame is considered and irrelevant background information is removed.
[0012] Step S13: Use key point detection method to identify facial feature points and perform face cropping to accurately locate each key position of the face;
[0013] Step S14: By calculating the affine transformation matrix, the cropped face image is transformed to a standard pose to achieve face alignment, thereby reducing the impact of pose changes on subsequent feature extraction and analysis.
[0014] Step S15: The video frames preprocessed by S11-S14 are processed by 1×1 convolution and activation function to obtain tensor X.
[0015] Preferably, the generation of the artifact highlighting feature map specifically includes the following steps:
[0016] Step S21: Input tensor X into the feature extraction branch, pass through dense convolutional blocks and transition modules, and finally output video frame feature maps. ;
[0017] Step S22: Input tensor X into the artifact amplification branch. After applying a Laplace-Gaussian filter to tensor X, the artifact amplification module performs downsampling and upsampling operations to extract new features. Subtract the new features from tensor X to highlight details and artifact effects, and then input the obtained features. Artifact feature maps are obtained through dense convolutional blocks and transition modules. :
[0018]
[0019] in To apply a Laplace-Gaussian filter, down is the downsampling function and up is the upsampling function. Indicates 1 1. Convolution;
[0020] Step S23: Map video frame features With artifact feature mapping The artifact salient map is obtained by fusion. :
[0021]
[0022] in It is a weighting parameter used to control the impact of artifact features on the final output.
[0023] Preferably, the extraction of attention features specifically includes the following steps:
[0024] The tensor X is input into the attention branch, and attention features are obtained through dense convolutional blocks and the attention computation module: temporary features are obtained by fixing the output of the second dense layer of the dense convolutional block. The hyperparameters are fed into the attention calculation module to obtain the hyperparameters. With attention parameters :
[0025]
[0026]
[0027] in and It is a statistic obtained by performing principal component analysis on a pre-defined dataset of facial images;
[0028] Attention parameters Attention features are obtained through a Sigmoid operation, and these features are used to obtain an attention map via a transition module. .
[0029] Preferably, the statistical measure obtained by performing principal component analysis on a preset facial image dataset is specifically acquired as follows:
[0030] Generated by application The face images are stacked column-wise into a data matrix Y, where each column represents a preprocessed face image tensor. The dimension of matrix Y is H. W N; calculate the mean vector of matrix Y. :
[0031]
[0032] in For the sample index, each sample minus We obtain the central matrix y and calculate the covariance matrix C:
[0033]
[0034] Where T represents the transpose operation;
[0035] The covariance matrix C is decomposed into eigenvalues to obtain eigenvalues and corresponding eigenvectors. The eigenvectors are then sorted in descending order according to the magnitude of their corresponding eigenvalues. The first n eigenvectors are selected as basis vectors to obtain the fundamental matrix A.
[0036] .
[0037] Preferably, the fusion feature of the artifact salience feature map and the attention map is specifically calculated as follows:
[0038]
[0039] in Represents element-wise multiplication. Indicates fusion features, This represents a map showing prominent artifact features.
[0040] Preferably, the model training uses a focusing loss function for supervised learning and parameter updates. The formula is as follows:
[0041]
[0042] in It is the model's response to the true category. The predicted probability, if ;if ; It is the model's predicted probability. It is a weighting factor. It is the focusing parameter.
[0043] A video forgery face detection system that integrates artifact features and attention guidance includes a processor, a memory, and a computer program stored in the memory. When the processor executes the computer program, it specifically performs any of the steps in the above-described video forgery face detection method.
[0044] Compared with the prior art, the present invention has the following beneficial effects:
[0045] (1) Artifact detail enhancement: By using the multi-scale Laplacian Gaussian operator, the image content in the low-level feature map is suppressed and the high-frequency features are enhanced. The artifact details are fused with the video frame features, which effectively amplifies the artifact details and makes the detection system more sensitive to artifact details, thereby improving the robustness of detection.
[0046] (2) Attention mechanism guides key area identification: The introduced attention branch enables the network to focus on important areas for decision-making, such as facial feature points (e.g., corners of the eyes, tip of the nose) and textures and details (e.g., uneven skin tone, unnatural lighting, or blurred edges). This mechanism can guide the model to more effectively identify and extract discriminative features, thereby enhancing the overall detection effect and improving the targeting and accuracy of fake face detection. Attached Figure Description
[0047] Figure 1 This is a flowchart of the video fake face detection method that integrates artifact features and attention guidance according to the present invention;
[0048] Figure 2 This is a schematic diagram of the dense convolutional block and transition module of the present invention;
[0049] Figure 3 This is a schematic diagram of the classification network of the present invention. Detailed Implementation
[0050] The following is in conjunction with the appendix Figure 1-3 The technical solution of the present invention will be described in detail below.
[0051] This invention provides a method for detecting fake faces in videos that integrates artifact features and attention guidance, such as... Figure 1 As shown, it includes the following steps:
[0052] Step S1: Dataset preprocessing.
[0053] In this invention, step S1 specifically includes the following steps:
[0054] Step S11: According to the set target frame rate, read and save the corresponding images frame by frame from the video to obtain a set of continuous video frames.
[0055] Step S12: Use a multi-task cascaded convolutional neural network (MTCNN) to identify the face region in each frame of the image, ensuring that only the face in the video frame is considered and irrelevant background information is removed.
[0056] Step S13: Use the Landmark Detection method to identify facial feature points and perform face cropping to accurately locate key positions on the face.
[0057] Step S14: By calculating the affine transformation matrix, the cropped face image is subjected to standard pose transformation to achieve face alignment, thereby reducing the impact of pose changes on subsequent feature extraction and analysis.
[0058] Step S15: The preprocessed video frame is processed by 1×1 convolution and activation function (ReLU) to obtain tensor X.
[0059] Step S2: Generating artifact highlighting feature map.
[0060] In this invention, step S2 specifically includes the following steps:
[0061] Step S21: Input tensor X into the feature extraction branch, pass through dense convolutional blocks and transition modules, and finally output video frame feature maps. .
[0062] Step S22: Input tensor X into the artifact amplification branch. The artifact amplification module applies a Laplace-Gaussian filter to tensor X, followed by downsampling and upsampling operations to extract new features. Next, subtract the new features from tensor X to highlight details and artifact effects, and then process the obtained features. Artifact feature maps are obtained through dense convolutional blocks and transition modules. The calculation formula is:
[0063]
[0064] in To apply a Laplace-Gaussian filter, down is the downsampling function and up is the upsampling function. Indicates 1 1. Convolution.
[0065] Step S23: Map video frame features With artifact feature mapping The artifact salient map is obtained by fusion. The calculation formula is:
[0066]
[0067] in It is a weighting parameter used to control the impact of artifact features on the final output.
[0068] Step S3: Attention feature extraction.
[0069] In this invention, step S3 specifically includes the following steps:
[0070] The tensor X is input into the attention branch, and attention features are obtained through dense convolutional blocks and the attention computation module: temporary features are obtained by fixing the output of the second dense layer of the dense convolutional block. The hyperparameters are fed into the attention calculation module to obtain the hyperparameters. , with attention parameters The calculation formula is as follows:
[0071]
[0072]
[0073] in and The results were obtained by performing principal component analysis on 200 face images generated by the FaceApp application. All face images were stacked column-wise into a data matrix Y, where each column represents a pre-processed face image tensor. The matrix has dimensions H. W N; calculate the mean vector of matrix Y. :
[0074]
[0075] Each sample minus We obtain the central matrix y and calculate the covariance matrix C:
[0076]
[0077] Where T represents the transpose operation
[0078] The covariance matrix C is decomposed into eigenvalues to obtain eigenvalues and their corresponding eigenvectors. The eigenvectors are then sorted in descending order of their corresponding eigenvalues, and the first n eigenvectors are selected as basis vectors to obtain the fundamental matrix A.
[0079]
[0080] Step S4: Feature map fusion.
[0081] In this invention, step S4 specifically includes the following steps:
[0082] Artifact Highlighting Map Attention map obtained after Sigmoid operation and transition module Multiply the results to obtain the fused feature map. The calculation formula is as follows:
[0083]
[0084] in This represents element-wise multiplication.
[0085] Step S5: Feature map classification and model learning and updating.
[0086] In this invention, step S5 specifically includes the following steps:
[0087] Step S51: Input the fused feature map into the subsequent backbone network for classification.
[0088] Step S52: Perform supervised learning and update the parameters using the focusing loss function. The formula for the focusing loss function is as follows:
[0089]
[0090] in It is the model's response to the true category. The predicted probability, if ;if . It is the probability predicted by the model. It is a weighting factor. It is the focusing parameter.
[0091] The above are preferred embodiments of the present invention. Any changes made to the technical solution of the present invention that do not exceed the scope of the technical solution of the present invention shall fall within the protection scope of the present invention.
Claims
1. A method for detecting fake faces in videos that integrates artifact features and attention guidance, characterized in that, Includes the following steps: Step S1: Construct a face image dataset; Step S2: Construct a fake face detection model. The model includes a feature extraction branch, an artifact amplification branch, an attention branch, and a classification network to generate artifact protrusion feature maps and extract attention maps. The fusion features of the artifact protrusion feature maps and attention maps are used as input to the classification network to obtain fake face detection results. Step S3: Train a fake face detection model using a face image dataset, and then use the trained fake face detection model for fake face detection.
2. The video forgery face detection method that integrates artifact features and attention guidance according to claim 1, characterized in that, Step S1 specifically includes the following steps: Step S11: According to the set target frame rate, read and save the corresponding images frame by frame from the video to obtain a set of continuous video frames; Step S12: Use a multi-task cascaded convolutional neural network to identify the face region in each frame of the image, ensuring that only the face in the video frame is considered and irrelevant background information is removed. Step S13: Use key point detection method to identify facial feature points and perform face cropping to accurately locate each key position of the face; Step S14: By calculating the affine transformation matrix, the cropped face image is transformed to a standard pose to achieve face alignment, thereby reducing the impact of pose changes on subsequent feature extraction and analysis. Step S15: The video frames preprocessed by S11-S14 are processed by 1×1 convolution and activation function to obtain tensor X.
3. The video forgery face detection method that integrates artifact features and attention guidance according to claim 2, characterized in that, The generation of the artifact highlighting feature map specifically includes the following steps: Step S21: Input tensor X into the feature extraction branch, pass through dense convolutional blocks and transition modules, and finally output video frame feature maps. ; Step S22: Input tensor X into the artifact amplification branch. After applying a Laplace-Gaussian filter to tensor X, the artifact amplification module performs downsampling and upsampling operations to extract new features. Subtract the new features from tensor X to highlight details and artifact effects, and then input the obtained features. Artifact feature maps are obtained through dense convolutional blocks and transition modules. : in To apply a Laplace-Gaussian filter, down is the downsampling function and up is the upsampling function. Indicates 1 1. Convolution; Step S23: Map video frame features With artifact feature mapping The artifact salient map is obtained by fusion. : in It is a weighting parameter used to control the impact of artifact features on the final output.
4. The video forgery face detection method that integrates artifact features and attention guidance according to claim 2, characterized in that, The extraction of attention features specifically includes the following steps: The tensor X is input into the attention branch, and attention features are obtained through dense convolutional blocks and the attention computation module: temporary features are obtained by fixing the output of the second dense layer of the dense convolutional block. The hyperparameters are fed into the attention calculation module to obtain the hyperparameters. With attention parameters : in and It is a statistic obtained by performing principal component analysis on a pre-defined dataset of facial images; Attention parameters Attention features are obtained through a Sigmoid operation, and these features are used to obtain an attention map via a transition module. .
5. The video forged face detection method that integrates artifact features and attention guidance according to claim 4, characterized in that, The statistical measure obtained by performing principal component analysis on a preset facial image dataset is specifically acquired as follows: Generated by application The face images are stacked column-wise into a data matrix Y, where each column represents a preprocessed face image tensor. The dimension of matrix Y is H. W N; calculate the mean vector of matrix Y. : in For the sample index, each sample minus We obtain the central matrix y and calculate the covariance matrix C: Where T represents the transpose operation; The covariance matrix C is decomposed into eigenvalues to obtain eigenvalues and corresponding eigenvectors. The eigenvectors are then sorted in descending order according to the magnitude of their corresponding eigenvalues. The first n eigenvectors are selected as basis vectors to obtain the fundamental matrix A. 。 6. The video forgery face detection method that integrates artifact features and attention guidance according to claim 4, characterized in that, The fusion feature of the artifact prominence map and the attention map is specifically calculated as follows: in Represents element-wise multiplication. Indicates fusion characteristics, This represents a map showing prominent artifact features.
7. The video forgery face detection method that integrates artifact features and attention guidance according to claim 1, characterized in that, Model training uses a focusing loss function for supervised learning and parameter updates. The formula is as follows: in It is the model's response to the true category. The predicted probability, if ;if ; It is the model's predicted probability. It is a weighting factor. It is the focusing parameter.
8. A video forgery face detection system that integrates artifact features and attention-guided techniques, characterized in that, It includes a processor, a memory, and a computer program stored in the memory. When the processor executes the computer program, it specifically performs the steps in the video fake face detection method as described in any one of claims 1-7.