Face video authentic identification method and system based on time sequence forgery clue analysis, and storage medium

Abstract

The invention discloses a face video authentic identification method and system based on time sequence forgery clue analysis and a storage medium. The method comprises the following steps: acquiring a to-be-detected face video; and inputting a to-be-detected face video into the face counterfeit video detection model to obtain a face authentic identification result of each frame of image in the to-be-detected face video. Wherein the clues of short-term instantaneous anomaly, long-term gradual inconsistency and long-term accumulated distortion in a forged video are finely analyzed through associating an adjacent frame module, a future guidance frame module and a historical frame review module, so that the extraction capability of space general forged features is enhanced, and the accuracy of face authentic identification is improved.

Description

Technical Field

[0001] The present invention belongs to the field of face forgery video detection, and in particular relates to a face video forgery identification method, system and storage medium based on temporal forgery clue analysis. Background Art

[0002] Face forgery video detection (FFVD) is a crucial and challenging task for verifying the authenticity of digital facial videos. The rich dynamic information of facial expressions and motion in forged face videos poses a greater security threat than static forged faces. In particular, the continuous emergence of face forgery methods in practical applications poses a significant challenge to the generalization ability of face forgery video detection models. Therefore, developing practical face forgery video detection methods that can accurately capture universal discriminative features from a limited video training set while avoiding overfitting to specific forgery patterns is crucial and challenging.

[0003] Many approaches to face forgery video detection have achieved significant progress in improving model generalization by extracting spatial information from individual frames, such as data augmentation, attention to forgery features, and pre-training on real faces. However, temporal cues such as artifacts, flicker, and inter-frame discontinuities in forged videos can provide important guidance for extracting spatial forgery features, but these temporal cues have not been effectively exploited. Some face forgery video detection methods fuse spatiotemporal features by adjusting the kernel size of 3DCNNs or combining frames. However, this crude fusion of forged features fails to reflect the natural variations and subtlety of forged features, and thus fails to capture fine-grained forgery temporal cues. In existing face forgery video detection, most previous methods, whether or not they utilize temporal cues, exhibit significant inter-frame fluctuations, indicating that the captured temporal cues are insufficient to stabilize the extraction of general spatial forgery features. Therefore, effectively mining temporal cue features is a key challenge in face forgery video detection. Summary of the Invention

[0004] In view of this, the purpose of the present invention is to provide a facial video authentication method, system and storage medium based on temporal forgery clue analysis, which analyzes the gradual accumulation of changes from historical to future frames over time, thereby carefully analyzing subtle temporal forgery clues.

[0005] The purpose of the present invention is achieved through the following technical solutions:

[0006] The present invention provides a method for detecting fake faces in facial videos based on temporal forgery clue analysis, comprising:

[0007] Get the face video to be detected;

[0008] Inputting a face video to be detected into a face forgery video detection model to obtain a face forgery detection result for each frame in the face video to be detected, wherein the face forgery video detection model includes: a video segmentation module, an image encoder, a module for associating adjacent frames, a module for guiding future frames, a historical review module, and a detection module;

[0009] The video segmentation module is used to segment the acquired face video, and each segment does not overlap with each other and contains the same number of frames;

[0010] The image encoder is used to extract spatial features from each frame of each segment;

[0011] The Correlate Adjacent Frames module captures instantaneous abnormal clues by correlating spatial features between consecutive frames in each segment;

[0012] Guide the future frame module to resolve inconsistent clues by iteratively aggregating historical anomaly clues and gradually propagating them to future frames;

[0013] The historical review frame module resolves the accumulated distortion clues by accumulating inconsistent clues from future frames to historical frames, thereby enhancing the spatial forgery features of each frame image;

[0014] The detection module uses enhanced spatial forgery features to generate face authentication results for each frame of the face video to be detected.

[0015] Furthermore, the module for associating adjacent frames adopts a transformer structure with a cross-attention mechanism, generates spatial features based on the image encoder, and determines the features of image enhancement for each frame in each segment, which are recorded as the first feature Wherein, m∈{1,2,...,M}, k∈{1,2,...,K}, M represents the number of segments, and K represents the number of image frames in each segment; the method also includes training an associated adjacent frame module, and when training the associated adjacent frame module, weighting the cross attention weight using a combination of a random mask and a lower triangular mask matrix.

[0016] Furthermore, the guidance future frame module includes an abnormal clue aggregation unit and an inconsistency clue propagation unit.

[0017] For each segment m, the abnormal clue aggregation unit performs the following operations:

[0018] Take the center frame of segment m as the key frame;

[0019] Generate the first abnormal feature corresponding to segment m using the first feature corresponding to the key frame, the last frame and the first frame of segment m

[0020] Get the inconsistent clue features corresponding to segment m Among them, the inconsistent clue feature f1 of the first segment ip 0, corresponding to inconsistent clue features of segment m The inconsistent clue features corresponding to the key frame, the last frame, and the first frame of the segment m are determined, and the inconsistent clue features of each frame are the output of the inconsistent clue propagation unit;

[0021] The first abnormal feature and inconsistent cue features Perform aggregation to obtain the instantaneous abnormal clue C of segment m m , where the instantaneous abnormal clue C1 of the first segment is the first abnormal feature f1 of the first segment cc ;

[0022] The inconsistency clue propagation unit adopts a transformer structure with a cross attention mechanism.

[0023] For the current segment m, the inconsistency clue propagation unit performs the following steps:

[0024] Determine the T historical segments closest to the current segment m;

[0025] For each historical segment, determine the coordinate offset between the key points in the keyframes of the current segment m and the historical segment R, denoted as the key point offset, where R∈{mT,m-T+1,...,m-1};

[0026] Determine the first abnormal feature of the current segment m The instantaneous abnormal clue C corresponding to the historical segment R R The offset between them is recorded as feature offset;

[0027] According to the key point offset and feature offset, the encoder is prompted by the key points to generate the abnormal clue correction feature C′ corresponding to the historical segment R R ;

[0028] Using C′ R C R Make updates;

[0029] After determining that all T historical segments have been updated, C R Afterwards, the spatial features in the current segment m are As a query, the instantaneous abnormal clues updated with T historical fragments are As the key and value, cross attention calculation is performed to generate the inconsistency clue features of each frame of the current segment m.

[0030] Furthermore, the keypoint prompts the encoder to perform the following operations:

[0031] Linearly encode the keypoint offsets and up-scale the linear encoding results to match the dimension of the feature offsets;

[0032] Concatenate the linear encoding result after dimensionality increase with the feature offset;

[0033] Perform a convolution operation on the concatenated result, where the dimension of the convolution result is the same as the dimension of the feature offset;

[0034] The convolution result is passed through two sequentially connected residual blocks to obtain the abnormal clue correction feature C′ R , where each residual block consists of sequentially connected convolutional layers, normalization layers, and activation layers.

[0035] Furthermore, the method further includes using a bidirectional transformer as a forged feature aggregator to transform spatial features. and inconsistency cue features Enhance and generate enhanced spatial features and enhanced incongruent cue features

[0036] Furthermore, the method includes utilizing the enhanced spatial features and enhanced incongruent cue features Using the cross-entropy loss function, the future guidance frame module is trained while freezing the parameters of the associated adjacent frame module.

[0037] Furthermore, the historical frame review module is a post-processing module, and the historical frame review module performs the following steps:

[0038] Determine all future segments {s} for any segment m, s∈{m+1,m+2,...,M};

[0039] For each future segment s, compute the enhanced inconsistency cue features for all frames in the future segment s The mean of , recorded as the characteristic mean;

[0040] The feature mean corresponding to all future segments s is accumulated into the frame of segment m by setting the attenuation weight, thereby generating the weighted enhanced inconsistency clue feature

[0041] The detection module uses enhanced spatial features and the final enhanced inconsistent cue features Achieve face authentication results for each frame of the face video to be detected.

[0042] The present invention also provides a facial video authentication system based on temporal forgery clue analysis, comprising:

[0043] a memory configured to store a computer program;

[0044] The processor is configured to execute the computer program to implement the above-mentioned method for detecting fake facial videos based on temporal forgery clue analysis.

[0045] The present invention also provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the method for detecting counterfeit facial videos based on temporal forgery clue analysis as described above is implemented.

[0046] The beneficial effects of the present invention are:

[0047] The present invention develops a new face forgery video detection framework, which meticulously analyzes temporal forgery clues at three levels: instantaneous anomalies, gradual inconsistencies, and cumulative distortions. The present invention also designs a forgery clue aggregation and propagation mechanism, which highlights the spatial forgery features of future frames by aggregating historical anomalies and propagating inconsistencies. The present invention also uses a momentum accumulation mechanism to accumulate distortion clues to enhance the spatial forgery features of historical frames, thereby effectively improving the accuracy of face authenticity detection.

[0048] Other advantages, objects, and features of the present invention will be described in part in the following description and, in part, will be apparent to those skilled in the art upon examination of the following description or may be learned from practice of the present invention. The objects and other advantages of the present invention may be realized and obtained through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] In order to make the purpose, technical solutions and advantages of the present invention more clear, the present invention will be further described in detail below with reference to the accompanying drawings, in which:

[0050] Figure 1 It is a temporal forgery clue parsing framework;

[0051] Figure 2 It is the schematic structure of the inconsistent cue propagation unit. DETAILED DESCRIPTION

[0052] The preferred embodiments of the present invention will be described in detail below with reference to the accompanying drawings. It should be understood that the preferred embodiments are only for illustrating the present invention, and are not intended to limit the scope of protection of the present invention.

[0053] The temporal forgery cues required for face forgery video detection are often obscured by abundant forgery-irrelevant information, particularly the natural variations in facial position and expression. Therefore, by analyzing the gradual accumulation of temporal changes from past to future frames, we can meticulously resolve subtle temporal forgery cues. Furthermore, humans often discern discriminative cues over time, typically starting with conscious attention to momentary anomalies, then gradually identifying inconsistencies across longer frames, and ultimately reasoning about the outcome within each frame using accumulated cues. Based on an analysis of the inherent invisibility of temporal cues and inspired by the human discriminative process, this paper abstracts temporal forgery cues into three progressive levels: momentary anomalies, gradual inconsistencies, and cumulative distortion. Momentary anomalies represent instantaneous forgery cues within consecutive frames and are identified by correlating spatial variations in forgery features. Gradual inconsistencies refer to the gradual resolution of temporally coexisting inconsistency cues by propagating momentary anomaly cues from past frames. Cumulative distortion refers to video distortion cues that accumulate over inconsistency cues across all frames. By gradually resolving these cues, we highlight general spatial forgery features.

[0054] Based on this, the present invention proposes a Temporal Forgery Cue Unraveing (TFCU) framework, which includes an associated adjacent frame module, a guided future frame module, and a historical frame review module. The associated adjacent frame module introduces a variable-length frame and random mask strategy to capture abnormal clues through the interaction between spatial features. The associated adjacent frame module implements a clue aggregation and propagation mechanism, iteratively aggregates abnormal clues from historical frames to resolve long-term inconsistent clues and gradually propagate them to future frames. The historical frame review module resolves distortion clues by accumulating inconsistencies and strengthens the spatial forgery features of all frames.

[0055] Figure 1 It is a temporal forgery clue parsing framework, combined with Figure 1 A facial video authentication method based on temporal forgery clue analysis is provided, comprising:

[0056] Get the face video to be detected;

[0057] The face video to be detected is input into the face forgery video detection model to obtain the face forgery identification results for each frame image in the face video to be detected. The face forgery video detection model includes: a video segmentation module, an image encoder, an associated adjacent frame module, a guided future frame module, a historical review module and a detection module; the associated adjacent frame module, the guided future frame module and the historical review module constitute the temporal forgery clue parsing framework.

[0058] The video segmentation module is used to segment the acquired face video, and each segment does not overlap and contains the same number of frames;

[0059] The image encoder is used to extract spatial features from each frame of each segment;

[0060] The Correlate Adjacent Frames module captures instantaneous abnormal clues by correlating spatial features between consecutive frames in each segment;

[0061] Guide the future frame module to resolve inconsistent clues by iteratively aggregating historical anomaly clues and gradually propagating them to future frames;

[0062] The historical review frame module resolves the accumulated distortion clues by accumulating inconsistent clues from future frames to historical frames, thereby enhancing the spatial forgery features of each frame image;

[0063] The detection module uses enhanced spatial forgery features to generate face authentication results for each frame of the face video to be detected.

[0064] The image encoder can use a pre-trained base model, for example, ViT, ResNet, EfficientNet, Xception, etc.

[0065] The associated adjacent frame module uses a transformer structure with a cross-attention mechanism to generate spatial features based on the image encoder and determine the enhanced features of each frame in each segment, which are recorded as the first feature. Where m∈{1,2,...,M}, k∈{1,2,...,K}, M represents the number of segments, and K represents the number of image frames in each segment.

[0066] Specifically, for each position (i.e., image block) in each frame of each segment, the spatial features of that position in the frame can be used as a query, and the spatial features of the corresponding positions in other frames in the segment can be used as keys and values to perform cross-attention calculations, thereby obtaining enhanced features for each frame. The parameters involved in the cross-attention calculation process are trained using training data.

[0067] Among them, when training the module for associating adjacent frames, the cross-attention weights are weighted using a lower triangular mask matrix with a random mask. This strategy stimulates the potential of the transformer to parse instantaneous abnormal clues from the spatial information of variable-length frames. The masking strategy can be expressed as:

[0068]

[0069] in, is the lower triangular mask matrix, is a random mask matrix, Indicates rounding down. Indicates rounding up, p represents the probability of random mask selection, i,j∈{1,2,...,K×N}, and N represents the number of tokens contained in a frame.

[0070] The attention weight w′ obtained by weighting the weight w of the cross attention mechanism based on the lower triangular mask matrix with random mask can be expressed as:

[0071] w′=w⊙(M d +M r ),

[0072] Among them, M d express The matrix composed of r express The matrix is composed of , ⊙ represents the multiplication of the elements at corresponding positions in the matrix.

[0073] This paper selects a base model pre-trained on real faces as the image encoder to extract spatial features. This is then correlated through a transformer with cross-attention to achieve interaction between spatial artifact features between consecutive frames. Furthermore, the Correlate Adjacent Frames module is trained using a cross-entropy loss function, giving it a strong ability to identify and correlate transient anomaly cues.

[0074] Considering the long-term changes in the entire forged video, instantaneous anomalies within a limited number of consecutive frames remain isolated, while parsing deeper temporal inconsistency clues in longer sequences is crucial for reliably capturing general forgery features. Since the token length corresponding to all frame features is quite large, parsing precise inconsistency clues from such information-rich context is challenging and requires a lot of computational resources. Therefore, the present invention designs a guided future frame module to highlight spatial forgery features by iteratively aggregating anomaly clues from consecutive frames of historical segments and then gradually propagating them to future segments.

[0075] The guiding future frame module includes an abnormal clue aggregation unit and an inconsistency clue propagation unit.

[0076] The anomaly clue aggregation unit extracts features from the associated adjacent frame module and takes the output of the inconsistent clue propagation unit as input to aggregate anomaly clues. Specifically, for each segment m, the anomaly clue aggregation unit performs the following operations:

[0077] Take the center frame of segment m as the key frame;

[0078] Generate the first abnormal feature corresponding to segment m using the first feature corresponding to the key frame, the last frame and the first frame of segment m Right now

[0079] Get the inconsistent clue features corresponding to the segment Among them, the inconsistent clue feature f1 of the first segment ip 0, inconsistent clue features is the output of the inconsistent cue propagation unit;

[0080] The first abnormal feature and inconsistent cue features Perform aggregation to obtain the instantaneous abnormal clue C corresponding to the fragment m m ,Right now Among them, E ca Represents a feature aggregation operation.

[0081] Among them, the instantaneous abnormal clue C1 of the first segment is the first abnormal feature f1 of the first segment cc .

[0082] After generating C m Afterwards, these C m It will be stored in the fake clue database for later use in clue dissemination.

[0083] The inconsistency clue propagation unit utilizes iterative anomaly clues to gradually propagate inconsistency clues to future frames, thereby guiding future frames to stably highlight general forgery features.

[0084] In some embodiments, the inconsistency clue propagation unit adopts a transformer structure with a cross-attention mechanism to As a query, the historical clues corresponding to the most recent T historical fragments are used as keys and values. and the interaction between multiple historical clues C, thereby achieving the propagation of forged clues and feature guidance.

[0085] Figure 2 This is the schematic structure of the inconsistent clue propagation unit, refer to Figure 2 , for the current segment m, the inconsistency clue propagation unit performs the following steps:

[0086] Determine the T historical segments closest to the current segment m (if the number of historical segments is less than T, take all historical segments and use them for subsequent calculations);

[0087] For each historical segment, determine the coordinate offset between the key points in the key frame of the current segment m and the historical segment R, which is recorded as the key point offset Where R∈{mT,m-T+1,...,m-1};

[0088] Determine the first abnormal feature of the current segment m The instantaneous abnormal clue C corresponding to the historical segment R R The offset between

[0089] According to the key point offset and feature offset, the facial landmark prompt encoder (Landmarks PromptEncoder) is used to generate the abnormal clue correction feature C′ corresponding to the historical segment R R ,Right now Among them, E lp Indicates the operation of the key point prompt encoder;

[0090] Using C′ R C R Update and get C′ R +C R , which is the enhanced abnormal clue;

[0091] After determining that all T historical segments have been updated, C R (i.e. C′ R +C R ) After that, the spatial features of each image frame in the current segment m are As a query, the instantaneous abnormal clues updated with T historical fragments are As the key and value, cross attention calculation is performed to generate the inconsistency clue features of each image frame in the current segment m. Right now Among them, E cp Represents the cue propagation encoder (Cue Propagation Encoder), which adopts the transformer structure with cross attention mechanism.

[0092] In some embodiments of the present invention, Figure 2 As shown, the keypoint prompts the encoder to perform the following operations:

[0093] The key point offset is linearly encoded by the encoder to obtain the latent space feature representation, and the linear encoding result is dimensionally upgraded to match the dimension of the feature offset;

[0094] The linear encoding result after dimensionality increase and the feature offset are concatenated according to the corresponding dimensions;

[0095] Perform a convolution operation on the concatenated result, where the dimension of the convolution result is the same as the dimension of the feature offset;

[0096] The convolution result is passed through two sequentially connected residual blocks to obtain the abnormal clue correction feature C′ R(or called forgery clue correction features), where each residual block includes a sequentially connected convolution layer, a normalization layer, and an activation layer.

[0097] Correction features for forgery cues are learned through facial key point offset and feature offset, thereby updating and strengthening abnormal cues.

[0098] exist Figure 2 In , D represents the feature dimension of the latent space; H represents the height of each frame image; W represents the width of the image.

[0099] Furthermore, given that the spatial features that capture abnormal clues in the associated adjacent frame module have significant discriminative power, the present invention combines these clues with the propagated inconsistency clues to further enhance the general forgery features. To this end, the present invention uses a bidirectional transformer as a forgery feature aggregator (FFA) to and inconsistency cue features Enhance and generate enhanced spatial features and enhanced incongruent cue features Right now

[0100] The enhanced spatial features and enhanced incongruent cue features Splicing along the corresponding dimension, and then passing through the classification head (H) to generate the frame-level discrimination probability p m,k ,Right now Among them, concat means feature concatenation (i.e. Figure 1 Then, using p m,k And the cross entropy loss function is used to calculate the cross entropy loss, and the future guidance frame module is trained and optimized while freezing the parameters of the associated adjacent frame module.

[0101] Through close collaboration between the Associate Nearby Frames module and the Guide Future Frames module in the TFCU framework, anomalous cues are iteratively aggregated and inconsistent cues are gradually resolved, making temporal forgery cues easier to identify from historical frames to future frames. That is, temporal inconsistency cues are more prominent in subsequent frames, facilitating the identification of spatial forgery features, while earlier frames only obtain weaker cues. To provide more favorable guidance for earlier frames, the present invention introduces a Review Historical Frames module that gradually accumulates forgery cues from later frames (i.e., future frames) to earlier frames (i.e., historical frames), resolving distortion cues to enhance the discriminability of global forgery features.

[0102] In this invention, the Review History Frame module is a post-processing module that accumulates distortion cues from future frames into historical spatial features during the inference phase. Considering that the correlation between frames decreases with increasing intervals, a momentum accumulation operation is introduced to enhance spatial features, with the weight assigned to each frame decreasing with increasing intervals. Each segment output from the Guidance Future Frame module is treated as a review unit, and its feature mean is subsequently calculated and accumulated into frames in the historical segment with decaying weights. Specifically, the Review History Frame module performs the following steps:

[0103] Determine all future segments {s} for any segment m, i.e., s∈{m+1,m+2,...,M}, where future segments are segments after the current segment;

[0104] For each future segment s, compute the enhanced inconsistency cue features for all frames in the future segment s The mean of , recorded as the characteristic mean;

[0105] The feature means corresponding to all future segments s are continuously accumulated into the frame of segment m through the attenuation weight to generate enhanced spatial forged features

[0106] Specifically, for the first future segment (i.e., the future segment closest to segment m, i.e., segment m+1), the following formula is used to generate the enhanced result of the spatial forged feature: Recorded as the first enhancement result,

[0107]

[0108] Among them, the attenuation weight

[0109] Next, for the second future segment, s = m + 2, replace Use the first enhanced result calculated for the first time Perform replacement update to generate a second enhanced result

[0110] For the third future segment, ie, s=m+3, the second enhancement result is used to replace Thus generating the third enhanced result

[0111] Continue to replace and update according to the above method Until the last future segment is calculated, that is, s = M, thus generating the final enhancement result By using the above updating method, all future segments modify segment m (a total of Mm updates), thereby enhancing the identifiability of the global forgery feature.

[0112] After determining the final enhanced inconsistent clue features After that, further and Perform fusion splicing (such as Figure 1 After the “feature fusion” in ), the authentication result of the face in each image frame can be generated.

[0113] The present invention also provides a facial video authentication system based on temporal forgery clue analysis, comprising:

[0114] a memory configured to store a computer program;

[0115] The processor is configured to execute the computer program to implement the above-mentioned method for detecting fake facial videos based on temporal forgery clue analysis.

[0116] The present invention also provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the method for detecting counterfeit facial videos based on temporal forgery clue analysis as described above is implemented.

[0117] This paper focuses on the task of facial video authentication, aiming to improve the generalization and practicality of facial authentication models by deeply mining and analyzing the inherent spatiotemporal variation patterns of videos. A series of innovative methods and techniques are proposed to analyze the spatial internal features, short-term subtle changes, long-term global changes, and long-term dynamic changes of facial videos. Specifically, by associating adjacent frames, guiding future frames, and reviewing historical frames, these methods refine the analysis of short-term transient anomalies, long-term gradual inconsistencies, and long-term cumulative distortion cues in forged videos, thereby enhancing the ability to extract universal spatial forgery features.

[0118] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not limiting. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention can be modified or replaced by equivalents without departing from the purpose and scope of the technical solutions, which should all be included in the scope of the claims of the present invention.

Claims

1. A facial video authentication method based on temporal forgery clue analysis, characterized in that: include: Get the face video to be detected; Inputting a face video to be detected into a face forgery video detection model to obtain a face forgery detection result for each frame in the face video to be detected, wherein the face forgery video detection model includes: a video segmentation module, an image encoder, a module for associating adjacent frames, a module for guiding future frames, a historical review module, and a detection module; The video segmentation module is used to segment the acquired face video, and each segment does not overlap with each other and contains the same number of frames; The image encoder is used to extract spatial features from each frame of each segment; The Correlate Adjacent Frames module captures instantaneous abnormal clues by correlating spatial features between consecutive frames in each segment; Guide the future frame module to resolve inconsistent clues by iteratively aggregating historical anomaly clues and gradually propagating them to future frames; The historical frame review module resolves the accumulated distortion cues by accumulating inconsistent cues from future frames to historical frames, thereby enhancing the spatial forgery features of each frame. The detection module uses enhanced spatial forgery features to generate face authentication results for each frame of the face video to be detected.

2. The method for detecting fake faces in facial videos based on temporal forgery clue analysis according to claim 1, characterized in that: The associated adjacent frame module uses a transformer structure with a cross-attention mechanism to generate spatial features based on the image encoder and determine the enhanced features of each frame in each segment, which are recorded as the first feature. Wherein, m∈{1,2,...,M}, k∈{1,2,...,K}, M represents the number of segments, and K represents the number of image frames in each segment; the method also includes training an associated adjacent frame module, and when training the associated adjacent frame module, weighting the cross attention weight using a lower triangular mask matrix with a random mask.

3. The method for detecting fake faces in facial videos based on temporal forgery clue analysis according to claim 2, characterized in that: The guidance future frame module includes an abnormal clue aggregation unit and an inconsistency clue propagation unit. For each segment m, the abnormal clue aggregation unit performs the following operations: Take the center frame of segment m as the key frame; Generate the first abnormal feature corresponding to segment m using the first feature corresponding to the key frame, the last frame and the first frame of segment m Get the inconsistent clue features corresponding to segment m Among them, the inconsistent clue features of the first segment 0, corresponding to inconsistent clue features of segment m The inconsistent clue features corresponding to the key frame, the last frame, and the first frame of the segment m are determined, and the inconsistent clue features of each frame are the output of the inconsistent clue propagation unit; The first abnormal feature and inconsistent cue features Perform aggregation to obtain the instantaneous abnormal clue C of segment m m , where the instantaneous abnormal clue C1 of the first segment is the first abnormal feature of the first segment The inconsistency clue propagation unit adopts a transformer structure with a cross attention mechanism. For the current segment m, the inconsistency clue propagation unit performs the following steps: Determine the T historical segments closest to the current segment m; For each historical segment, determine the coordinate offset between the key points in the keyframes of the current segment m and the historical segment R, denoted as the key point offset, where R∈{mT,m-T+1,...,m-1}; Determine the first abnormal feature of the current segment m The instantaneous abnormal clue C corresponding to the historical segment R R The offset between them is recorded as the feature offset; According to the key point offset and feature offset, the encoder is prompted by the key points to generate the abnormal clue correction feature C′ corresponding to the historical segment R R ; Using C′ R C R Make updates; After determining that all T historical segments have been updated, C R Afterwards, the spatial features in the current segment m are As a query, the instantaneous abnormal clues updated with T historical fragments are As the key and value, cross attention calculation is performed to generate the inconsistency clue features of each frame of the current segment m.

4. The method for detecting fake faces in facial videos based on temporal forgery clue analysis according to claim 3, characterized in that: The keypoint prompt encoder performs the following operations: Linearly encode the keypoint offsets and up-scale the linear encoding results to match the dimension of the feature offsets; Concatenate the linear encoding result after dimensionality increase with the feature offset; Perform a convolution operation on the concatenated result, where the dimension of the convolution result is the same as the dimension of the feature offset; The convolution result is passed through two sequentially connected residual blocks to obtain the abnormal clue correction feature C′ R , where each residual block consists of sequentially connected convolutional layers, normalization layers, and activation layers.

5. The method for detecting fake faces in facial videos based on temporal forgery clue analysis according to claim 3, characterized in that: The method also includes using a bidirectional transformer as a fake feature aggregator for spatial features and inconsistency cue features Enhance and generate enhanced spatial features and enhanced incongruent cue features 6. The method for detecting fake faces in facial videos based on temporal forgery clue analysis according to claim 5, characterized in that: The method includes utilizing enhanced spatial features and enhanced incongruent cue features Using the cross-entropy loss function, the future guidance frame module is trained while freezing the parameters of the associated adjacent frame module.

7. The method for detecting fake faces in facial videos based on temporal forgery clue analysis according to claim 5, characterized in that: The historical frame review module is a post-processing module. The historical frame review module performs the following steps: Determine all future segments {s} for any segment m, s∈{m+1,m+2,...,M}; For each future segment s, compute the enhanced inconsistency cue features for all frames in the future segment s The mean of , recorded as the characteristic mean; The feature mean corresponding to all future segments s is accumulated into the frame of segment m by setting the attenuation weight to generate the final inconsistent clue feature 8. The method for detecting fake faces in facial videos based on temporal forgery clue analysis according to claim 7, characterized in that: The detection module uses enhanced spatial features and the final enhanced inconsistent cue features Achieve face authentication results for each frame of the face video to be detected.

9. A facial video authentication system based on temporal forgery clue analysis, characterized in that: include: a memory configured to store a computer program; The processor is configured to execute the computer program to implement the facial video authentication method based on temporal forgery clue analysis according to any one of claims 1 to 8.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method for identifying facial videos based on temporal forgery clue analysis according to any one of claims 1 to 8 is implemented.

Images