Generative video information authentic identification method based on cross-modal inconsistency mining

Abstract

The invention relates to a generative video information authentic identification method based on cross-modal inconsistency mining. According to the method, a'frequency domain significance 'and'motion entropy' combined evaluation strategy is adopted to adaptively screen key frames, and inconsistency between appearance and motion is actively diagnosed from physical relevance between modes; a cross attention mechanism is introduced as a diagnosis core, fusion based on causal reasoning is realized, compatibility of expected motion and observation motion is quantified, inconsistency is converted into a core diagnosis signal, inherent defects of universality can be overcome, and high authentic identification precision and prospective generalization ability are kept for unknown generative model video intelligence.

Description

Technical Field

[0001] This invention relates to the fields of artificial intelligence and information security, specifically a generative video intelligence authentication method based on cross-modal inconsistency mining. Background Technology

[0002] In the field of scientific intelligence analysis, video footage, as a crucial intelligence carrier recording event processes, target behavior, and environmental situations, is fundamentally based on its authenticity, ensuring the reliability of intelligence products and even the correctness of subsequent decision-making. In recent years, generative artificial intelligence technology has made groundbreaking progress in video generation. Models such as Sora, Hunyuan, and Veo can synthesize highly realistic video content with resolutions up to 1080p and durations up to minutes. However, this rapid development also brings serious risks of misuse: hostile forces can use it to fabricate inappropriate statements by political figures, generate false images of military deployments or critical infrastructure operations, aiming to manipulate international public opinion and interfere with strategic judgment. Such meticulously crafted generative fake videos targeting intelligence analysis pose a significant challenge to existing intelligence security systems.

[0003] However, current video authentication methods integrated into intelligence analysis processes have significant limitations when dealing with next-generation AIGC videos, mainly in the following aspects: First, the anti-spoofing mechanisms are outdated and lack generalization ability. Existing methods mostly rely on detecting specific artifacts or statistical features left by known forgery models within a single modality (such as texture and edge features extracted by pre-trained CNN models). However, in intelligence warfare, the first to appear are often "zero-day" AIGC fake videos created using entirely new generative models. Because existing methods heavily rely on the distribution of training data, their anti-spoofing performance drops significantly when dealing with such unseen generative models, causing intelligence analysts to remain in a passive response state for a long time during technological iterations.

[0004] Second, there is a lack of in-depth diagnosis of physical causal relationships. High-quality intelligence analysis requires rigorous examination of the physical plausibility of video content. Although some studies have attempted to integrate multiple modal features such as space and motion, their integration strategies mostly remain at a superficial level, such as feature splicing or weighted summation, failing to fundamentally diagnose whether there is a physically plausible relationship between the appearance of an object and its motion pattern. The most common and difficult-to-eradicate defect of generative videos is precisely this kind of cross-modal inconsistency. Existing methods cannot effectively capture and quantify this essential defect, and therefore cannot provide intelligence analysts with physical evidence or reasonable explanations for "why it is judged as forgery." Consequently, their forgery identification conclusions lack a solid physical explanation and are difficult to form strong evidence in key intelligence consultations.

[0005] Third, the high computational complexity makes it difficult to meet the needs of frontline deployment. Frontline nodes in intelligence analysis are often equipped only with ordinary computing devices, which are insufficient to support computationally intensive 3D convolutional networks or large-scale multimodal models. The strong dependence of existing high-performance counterfeit detection solutions on specialized hardware significantly limits the real-time deployment and widespread application of high-precision counterfeit detection technology at the front lines of intelligence work, creating a gap between technological capabilities and practical operational needs.

[0006] In conclusion, the field of scientific intelligence analysis urgently needs a new paradigm for authentication that can directly address the core flaws of AIGC videos, fundamentally diagnose their physical inconsistencies, and reliably operate under resource-constrained environments. This method should proactively identify and utilize inconsistencies between appearance and motion modalities as core authentication criteria, thereby providing intelligence analysts with a precise, interpretable, and forward-looking generalization-capable tool for verifying video authenticity. Summary of the Invention

[0007] To address the shortcomings of existing technologies, this invention aims to provide a generative video intelligence authentication method based on cross-modal inconsistency mining. This method employs a joint evaluation strategy of "frequency domain saliency" and "motion entropy" to adaptively select keyframes. Starting from the physical correlation between modalities, it proactively diagnoses inconsistencies between appearance and motion. A cross-attention mechanism is introduced as the core of the diagnosis, achieving fusion based on causal reasoning, quantifying the compatibility between expected and observed motion, and transforming inconsistencies into core diagnostic signals. This overcomes the inherent limitations of universality, maintains high authentication accuracy even for new models not seen in the training set, and possesses forward-looking generalization capabilities.

[0008] To achieve the above objectives, the technical solution adopted by the present invention is as follows: A generative video intelligence authentication method based on cross-modal inconsistency mining, characterized by the following steps: Step 1: Select keyframes from the input video to be tested and remove duplicates to obtain a keyframe sequence; Step 2: Extract multimodal features from the keyframe sequence obtained in Step 1; the multimodal features include appearance modal features and motion modal features, wherein the appearance modal features include spatial features and frequency domain statistical features; Step 3: Fuse the spatial features and frequency domain statistical features obtained in Step 2 to obtain unified appearance modal features; Step 4: Input the motion modal features obtained in Step 2 and the unified appearance modal features obtained in Step 3 into the cross-modal inconsistency diagnosis module for diagnosis and output confidence scores. Evaluate the authenticity of video intelligence based on the confidence scores.

[0009] Based on the above scheme, the specific steps for keyframe extraction in step 1 are as follows: Step 1-1: Extract single frames from the video, perform a two-dimensional Fourier transform on the extracted single frames, filter the high-frequency components in these single frames using a high-pass mask, and calculate the average energy of the high-frequency components to obtain the frequency domain significance score. S 1; Steps 1-2 involve using the DIS optical flow algorithm to track feature points in these single frames, statistically analyzing the probability distribution of motion directions for each feature point, and calculating the motion entropy score. S 2, as shown in the following formula: ; In the above formula, This represents the probability distribution of the direction of motion. It is a very small constant (it can take the value 10). -8 (), used to prevent numerical calculation errors; Steps 1-3, according to S = S 1+ S 2. Calculate the overall score for each frame. S Sort and select S The first few frames in the value sorting are keyframes. Several frames are kept before and after each keyframe. After deduplication, the keyframe sequence is obtained.

[0010] Based on the above scheme, the spatial feature extraction in step 2 is as follows: The keyframe sequence is pre-trained with ResNet-50 and combined with the CBAM attention module to output 2048-dimensional spatial features.

[0011] Based on the above scheme, the frequency domain statistical feature extraction in step 2 is as follows: A two-dimensional Fourier transform is performed on the keyframe sequence to extract four statistical features: total energy, high-frequency energy ratio, frequency domain entropy, and center low-frequency ratio. These four statistical features are then projected into 128-dimensional frequency domain statistical features. Specifically, the four statistical features are: Total Energy: ,in Indicates the position in the spectrum after two-dimensional Fourier transform. The amplitude value at that point, fft This indicates the size of the Fourier transform, i.e., the size of the spectrum. High-frequency energy percentage: ,in M high For high-frequency mask, To correspond with the position in the spectrum diagram The associated binary mask matrix is ​​used to filter high-frequency regions; Frequency domain entropy: ,in This is a normalized probability distribution, which represents the proportion of each frequency component in the total energy. Low-frequency proportion in the center: ,in M center Center mask, To correspond with the position in the spectrum diagram The associated binary mask matrix is ​​used to filter out the central low-frequency region.

[0012] Based on the above scheme, the motion mode feature extraction in step 2 is the extraction of optical flow features: The keyframe sequence was converted into grayscale images, the optical flow field was calculated using the DIS algorithm, and 128-dimensional optical flow features were extracted using a 4-layer convolutional neural network.

[0013] Based on the above scheme, step 3 specifically includes: The obtained spatial features are dimensionality reduced, and then cross-attention fusion is performed with the frequency domain statistical features to obtain unified appearance modal features.

[0014] Based on the above scheme, the cross-modal inconsistency diagnosis module in step S4 is implemented using a cross-attention mechanism, the specific process of which is as follows: The appearance modal features are used as the query, and the motion modal features are used as the key and value. The obtained Query, Key, and Value are each passed through a linear transformation layer and projected onto the same dimension. The relevance matrix between the Query and Key is then calculated. Attention Scores = Query × Key T The attention weights are generated using the Softmax function based on the aforementioned correlation matrix: ; We use AttentionWeights to perform a weighted summation of Values ​​to obtain the context vector: Context=Attention Weights×Value Analyze the Attention Weights matrix to extract regions where the attention weight values ​​are significantly lower than the threshold, as inconsistency diagnostic signals; perform residual connection or concatenation between the Context vector and the original Query features to form the final fused features; The features processed by the cross-modal inconsistency diagnostic module are input into a classifier, which is a neural network containing at least one fully connected layer (e.g., a 3-layer fully connected network), to output a false detection confidence level between 0 and 1.

[0015] The model is trained using the cross-entropy loss function and the Adam optimizer, with a validation set of videos, following an early stopping strategy. The mathematical formula for the cross-entropy loss function is as follows: ; in Indicates the true label, B represents the positive class probability predicted by the model, and B represents the batch size. If the loss of the validation set videos does not decrease after 10 consecutive rounds, training is stopped to obtain the optimal model, which is the cross-modal inconsistency diagnosis module.

[0016] The generative video intelligence authentication method based on cross-modal inconsistency mining described in this invention has the following advantages: 1. A paradigm shift in artifact detection principles has been achieved, resulting in a breakthrough in generalization capabilities: This invention elevates the traditional approach of "finding artifacts within a single modality" to a new paradigm of "diagnosing physical contradictions between modalities." Since maintaining "cross-modal consistency" is an inherent challenge of generative models, this invention captures a universal defect, thus exhibiting superior generalization capabilities for unknown generative models, improving accuracy by over 10%. This effectively ensures the reliability of artifact detection capabilities against "zero-day" attacks on video intelligence from unknown generative models in intelligence analysis, guaranteeing foresight and initiative in technological countermeasures.

[0017] 2. Simultaneous Improvement in Authentication Accuracy and Decision Interpretability: The inconsistency diagnosis module can proactively locate and quantify physically illogical spatiotemporal regions, making the authentication process more analogous to human reasoning and interpretation, and thus more reliable decision-making. On a self-built test set, this invention achieved an accuracy rate of 94.32% and an F1-score of 93.90%. The inconsistency signals it provides can serve as key physical evidence in intelligence consultations, significantly enhancing the credibility of intelligence products.

[0018] 3. Achieving a balance between high performance and lightweight design, with significant practical value: Through an innovative keyframe filtering strategy and an efficient dual-branch temporal modeling architecture, computational overhead is greatly reduced while maintaining the accuracy of counterfeit detection. The model's single-video inference time on a typical CPU device is ≤8 seconds, and its memory usage is ≤3GB. This enables high-precision counterfeit detection technology to be deployed in real-time at the front lines of intelligence work, breaking the dependence of high-performance counterfeit detection on back-end computing centers and realizing the decentralization of frontline intelligence processing capabilities.

[0019] The main breakthroughs of the generative video authentication method based on cross-modal inconsistency mining described in this invention compared to traditional methods are shown in the table below: Comparison Dimensions Traditional methods This method Keyframe filtering strategy Continuous frame input or uniform sampling is typically used, which results in a large amount of information redundancy, heavy computational load, and low feature density. A joint evaluation strategy of "frequency domain saliency" and "motion entropy" is adopted to adaptively select key frames, focusing on information-dense segments and improving computational efficiency and feature quality from the source. Basic principles of counterfeit detection Finding traces of low fidelity within a single modality is essentially an adversarial exercise against the generative model at its highly optimized pixel or feature level. Starting with the physical causal relationships between modalities, this method actively explores the inconsistencies between appearance and motion, directly addressing the fundamental weakness of generative models in maintaining "physical rationality," and achieving deep and interpretable authentication of generated videos. Multimodal fusion mechanism Non-diagnostic fusion methods such as feature splicing or weighted summation only achieve information overlay and cannot uncover deep causal relationships between modalities. By introducing a cross-attention mechanism as the core of diagnosis, fusion based on causal reasoning is achieved, the compatibility between expected motion and observed motion is quantified, and inconsistency is transformed into core diagnostic signals. Interpretability of the diagnostic process Most of these decisions are made in a "black box" manner, making it difficult to trace the specific basis for the model's judgment of forgery, and the credibility of the conclusions is often questioned. Inconsistency diagnostic signals can accurately pinpoint physically unreasonable spatiotemporal regions. This mechanism clarifies the object that caused the anomaly, the time of occurrence, and its physical and logical inconsistencies, ensuring the traceability and explainability of the entire decision-making process. Generalization ability It heavily relies on the distribution of training data, and its ability to detect fake models generated by unknown sources drops sharply, requiring frequent retraining to catch up with model iterations. It captures the inherent flaws in the universality of generated videos, maintains high accuracy in identifying fake videos from unknown generation models, and possesses forward-looking generalization capabilities. ; Attached Figure Description

[0020] The present invention includes the following figures: Figure 1 This is a flowchart of the present invention; Figure 2 This is a schematic diagram of the overall multimodal framework; Figure 3 Comparison of training metrics for different modality combinations; Figure 4 For comparison of ablation experiment results. Detailed Implementation

[0021] The present invention will be further described in detail below with reference to the accompanying drawings.

[0022] A generative video authentication method based on cross-modal inconsistency mining is proposed. Its core lies in constructing an authentication model capable of performing multimodal feature extraction and inconsistency diagnosis. This model achieves authentication through the following steps: Step 1: Efficient Key Information Condensation: Through an innovative "frequency domain saliency-motion entropy joint evaluation" strategy, the most information-rich key frame sequences are adaptively selected from the input video, effectively eliminating redundant information and reducing costs and increasing efficiency for subsequent in-depth analysis.

[0023] Step 2: Multimodal Feature Extraction for Diagnosis: Parallel extraction of two physically causal modal features: Appearance modal features: These features combine the spatial visual features (describe texture and structure) and frequency domain statistical features (describe spectral distribution patterns) of video frames to form the static context of the scene.

[0024] Motion modal features: Based on optical flow feature extraction, accurately depict the dynamic trajectory of objects between frames.

[0025] Cross-modal inconsistency diagnosis module: As the core of this invention, it employs a cross-attention mechanism, using appearance modal features as the query and motion modal features as the key / value pair. This design simulates the physical diagnosis process of "inferring reasonable motion based on appearance and then comparing it with actual motion." The attention weights output by this module directly quantify the compatibility between "expected motion" and "observed motion," with low-weight regions representing strong inconsistency diagnosis signals.

[0026] Decision-making based on diagnostic signals: The features refined by the inconsistency diagnostic module are input into the classifier, and the final confidence level for true or false is output.

[0027] Specifically: 1. Dataset Construction The training, validation, and test sets are constructed using a combination of self-built datasets and publicly available datasets, with the following specific distribution: Training set: 1100 videos (550 real videos, 550 fake videos) Validation set: 400 videos (200 real videos, 200 fake videos) Test set: 1343 videos (640 real videos, 703 fake videos) The fake videos encompass a variety of generative models, including CogVideo, Mora, Hunyuan, Keling, Tongyi Wanxiang, Step, Veo, and Sora.

[0028] 2. Video preprocessing and keyframe selection Video frame extraction and format conversion: Extract one frame from the input video at intervals of two frames, and convert each frame to RGB format (pixel value range [0,255]). Frequency domain saliency detection: Perform FFT2 (N=512 points) on a single frame, filter high-frequency components using a high-pass mask, calculate the average energy of the high-frequency components, and obtain the frequency domain saliency score. S 1; Motion entropy calculation: The DIS optical flow algorithm is used to track feature points, statistically analyze the probability distribution of motion directions, and calculate the motion entropy score. ; Keyframe selection: By S = S 1+ S 2. Calculate the overall score, select the first 10 key frames, keep 3 frames before and after each key frame, and obtain a 7-frame sequence after deduplication, and unify the resolution to 512×512.

[0029] 3. Multimodal feature extraction Spatial feature extraction (appearance modality features): A pre-trained ResNet-50 (with the classifier head removed) is used in conjunction with the CBAM attention module to output 2048-dimensional spatial features.

[0030] Frequency domain feature extraction (appearance modality features): Perform FFT2 transform on the video frames to extract the following statistical features: Total Energy: ,in To avoid the small constant of division by zero; High-frequency energy percentage: ,in M high For high-frequency masking (the region 70% or more from the center of the spectrum is set to 1, and the rest to 0); Frequency domain entropy: ,in This is a normalized probability distribution; Low-frequency proportion in the center: ,in M centerUse a center mask (set regions less than 40% of the maximum distance from the center of the spectrum to 1, and the rest to 0); The four types of features are projected into 128-dimensional frequency domain features.

[0031] 4. Dual BiLSTM processing Spatial-frequency domain feature fusion: The spatial features are first reduced in dimensionality to output 512-dimensional features, and then cross-attention fusion is performed with the frequency domain features of [B,7,128] to obtain the fused features in the format of [B,7,512], which are then used as the unified appearance modality features. First BiLSTM training: Initialize a 2-layer BiLSTM network with a hidden dimension of 256, dropout of 0.3, input [B,7,512] fusion features, and output spatiotemporal fusion features in [B,512] format after forward propagation (512=2×256, 256 forward hidden states + 256 backward hidden states). Second BiLSTM training: The network structure is exactly the same as that of the first BiLSTM. The input is optical flow features of [B,6,128], where 6 is the inter-frame motion information, that is, the 6 sets of inter-frame optical flow between the frames of the aforementioned "7 frame sequences". The output is motion temporal features in the format of [B,512]. During training, the weights of the two BiLSTMs are updated independently and optimized synchronously.

[0032] 5. Feature fusion and authentication (1) Construction of cross-modal inconsistency diagnosis module: Input features: Query: Appearance modal features (i.e., the [B,512] spatiotemporal fusion features output in step 3.2); Key and Value: Motion modal features (i.e., the [B,512] motion temporal features output in step 3.3).

[0033] Attention calculation: The Query, Key, and Value are projected onto the same dimension (e.g., 512 dimensions) through a linear transformation layer.

[0034] Calculate the relevance matrix between the query and the key: Attention Scores = Query × Key T Where × represents matrix multiplication.

[0035] Scale the Attention Scores (divided by) ,in d k (where the dimension is the key), and the Softmax function is applied to generate attention weights: ; We use Attention Weights to perform a weighted sum of Values ​​to obtain the context vector: Context=Attention Weights×Value Inconsistency signal extraction: By analyzing the Attention Weights matrix, regions with weight values ​​significantly lower than a threshold (e.g., mean minus standard deviation) are identified. These regions represent incompatibility between appearance and motion, serving as diagnostic signals for inconsistency.

[0036] The Context vector is residually concatenated or spliced ​​with the original Query features to form the final fused features ([B,512]).

[0037] Output: The final fused features are used for subsequent classification. (2) Fully connected falsification: The final fused features are input into a 3-layer fully connected network: the first layer (input 512 → output 256, ReLU activation, dropout 0.3), the second layer (input 256 → output 128, ReLU activation, dropout 0.3), and the third layer (input 128 → output 1, Sigmoid activation), and the falsification confidence is output in the format [B,1]. (3) Model training: The cross-entropy loss function and Adam optimizer are used. The training parameters are: batch size 16, initial learning rate 1.6e-5, linear warm-up to 1.1e-6 for the first 10 rounds, and a total of 24 training rounds. An early stopping strategy is adopted: if the loss on the validation set does not decrease for 10 consecutive rounds, training is stopped and the optimal model is saved. (4) Inference deployment: Deployed on CPU (i5-12400) without GPU; input the fake video to be identified, execute steps 1-4, output confidence: greater than 0.5 is judged as fake video, less than or equal to 0.5 is judged as real video, and the inference time for a single video is ≤8 seconds.

[0038] Example 1: Comparison Experiment of Different Data Loading Methods (Verifying Keyframe Selection Strategy) This study verifies whether the proposed "frequency domain saliency-motion entropy joint evaluation" keyframe screening strategy can solve the problems of information redundancy, large computational overhead, and insufficient focus on core features in the traditional "direct input of complete video" approach, and demonstrates the innovative significance of this strategy in "quality improvement and efficiency enhancement".

[0039] The control experiment compared two data loading methods: "direct input of the complete video" and "pre-selected keyframe input." All parameters except for the data input were identical. Through multiple rounds of repeated experiments, the results are shown in Table 1. Figure 3 As shown: Table 1. Comparison of training metrics for two different data loading methods index Filtering keyframes Full video loss function 0.2115±0.0024 0.2244±0.0030 accuracy 93.30%±0.17% 92.50%±0.26% Accuracy 91.28%±0.71% 90.73%±0.66% Recall rate 94.25%±0.58% 93.00%±0.71% F1 - Score 92.74%±0.45% 91.85%±0.32% Video memory usage 9539MB±17MiB 9693MiB±26MB Training round time 4 minutes 25 seconds ± 12 seconds 6 minutes 17 seconds ± 13 seconds ; 1. Improved accuracy in fake detection: After key frame filtering, the accuracy rate is improved by 0.8% and the F1-Score is improved by 0.89%. The essence is that this strategy uses frequency domain (determining details) + motion entropy (quantifying inter-frame complexity) to eliminate redundant information such as static duplicate frames, allowing the model to focus on the core abnormal features of AIGC videos (such as spectral defects and motion anomalies) and avoid interference from invalid information.

[0040] 2. Significantly optimized computational efficiency: The training time per round was reduced from 6 minutes and 17 seconds to 4 minutes and 25 seconds (a reduction of 112 seconds), and the video memory was reduced by 154MB. This directly addresses the pain points of traditional methods, such as "computational complexity and difficulty in deployment," and provides support for real-time applications on resource-constrained devices such as ordinary CPUs.

[0041] 3. Paradigm Breakthrough: Shifting from the traditional "passive reception of the whole frame" to "active condensation of key information" ensures high information density in the input data, improves training efficiency and quality from the source, and reflects the originality of the screening strategy.

[0042] Analysis of this invention suggests that when directly inputting a complete video, randomly selected consecutive frames may contain repeated static scenes or frames without feature changes, diverting the model's attention from the core abnormal features of AIGC videos. Keyframe filtering, by using an inter-frame difference threshold, retains the core discriminative frames, allowing the model to more effectively focus on learning key features, thereby improving its discrimination ability.

[0043] Example 2 Ablation Experiment (Verification of the Cross-modal Inconsistency Diagnostic Module) To quantitatively evaluate the contribution of each modal component in this invention, we designed a rigorous ablation experiment, controlling system variables and changing only the input modal combinations. The experimental results are shown in Table 2 below. Figure 4 As shown: Table 2 Comparison of training metrics for different modality combinations index Spatial modes only Spatial + Frequency Domain Space + Optical Flow Spatial + Frequency Domain + Optical Flow loss function 0.2115±0.0024 0.2027±0.0033 0.1910±0.0022 0.2252±0.0026 accuracy 93.30%±0.17% 93.86%±0.24% 93.98%±0.14% 94.32%±0.10% Accuracy 91.28%±0.71% 90.05%±0.99% 93.27%±0.56% 91.67%±0.45% Recall rate 94.25%±0.58% 97.25%±1.39% 93.50%±0.40% 96.25%±0.57% F1-Score 92.74%±0.45% 93.51%±0.28% 93.38%±0.17% 93.90%±0.10% Video memory usage 9539MB±17MB 9693MB±12MB 13830MB±10MB 13978MB±8MB Time spent on a single round of training 4 minutes 25 seconds ± 12 seconds 5 minutes 07 seconds ± 18 seconds 5 minutes 30 seconds ± 12 seconds 5 minutes 48 seconds ± 14 seconds ; The limitations of single-modality training are highlighted: spatial modality training alone has low precision (91.28%), spatial + frequency domain training has high recall but low precision, and spatial + optical flow training has low loss but insufficient recall. This proves that single or dual-modality training lacks physical causal support and is difficult to balance accuracy and comprehensiveness.

[0044] The contents not described in detail in this specification are existing technologies known to those skilled in the art.

Claims

1. A generative video intelligence authentication method based on cross-modal inconsistency mining, characterized in that, Includes the following steps: Step 1: Select keyframes from the input video to be tested and remove duplicates to obtain a keyframe sequence; Step 2: Extract multimodal features from the keyframe sequence obtained in Step 1; the multimodal features include appearance modality features and motion modality features, wherein the appearance modality features include spatial features and frequency domain statistical features, and the spatial features and frequency domain statistical features together constitute the static context of the scene; Step 3: Fuse the spatial features and frequency domain statistical features obtained in Step 2 to obtain unified appearance modal features; Step 4: Input the motion modal features obtained in Step 2 and the unified appearance modal features obtained in Step 3 into the cross-modal inconsistency diagnosis module for diagnosis and output confidence scores. Evaluate the authenticity of video intelligence based on the confidence scores.

2. The generative video intelligence authentication method based on cross-modal inconsistency mining as described in claim 1, characterized in that, The specific steps for keyframe extraction in step 1 are as follows: Step 1-1: Extract single frames from the video, perform a two-dimensional Fourier transform on the extracted single frames, filter the high-frequency components in these single frames using a high-pass mask, and calculate the average energy of the high-frequency components to obtain the frequency domain significance score. S 1; Steps 1-2 involve using the DIS optical flow algorithm to track feature points in these single frames, statistically analyzing the probability distribution of motion directions for each feature point, and calculating the motion entropy score. S 2, as shown in the following formula: ; In the above formula, This represents the probability distribution of the direction of motion. It is a very small constant used to prevent numerical calculation errors; Steps 1-3, according to S = S 1+ S 2. Calculate the overall score for each frame. S Sort and select S The first few frames in the value sorting are keyframes. Several frames are kept before and after each keyframe. After deduplication, the keyframe sequence is obtained.

3. The generative video intelligence authentication method based on cross-modal inconsistency mining as described in claim 2, characterized in that, The spatial feature extraction described in step 2 is as follows: The keyframe sequence is pre-trained with ResNet-50 and combined with the CBAM attention module to output 2048-dimensional spatial features.

4. The generative video intelligence authentication method based on cross-modal inconsistency mining as described in claim 2, characterized in that, Step 2 summarizes the frequency domain statistical feature extraction as follows: A two-dimensional Fourier transform is performed on the keyframe sequence to extract four statistical features: total energy, high-frequency energy ratio, frequency domain entropy, and center low-frequency ratio. These four statistical features are then projected into 128-dimensional frequency domain statistical features. Specifically, the four statistical features are: Total Energy: ,in Indicates the position in the spectrum after two-dimensional Fourier transform. The amplitude value at that point, fft This indicates the size of the Fourier transform, i.e., the size of the spectrum. High-frequency energy percentage: ,in M high For high-frequency mask, To correspond with the position in the spectrum diagram The associated binary mask matrix is ​​used to filter high-frequency regions; Frequency domain entropy: ,in This is the normalized energy probability distribution, which is the proportion of each frequency component to the total energy. Low-frequency proportion in the center: ,in M center Center mask, To correspond with the position in the spectrum diagram The associated binary mask matrix is ​​used to filter out the central low-frequency region.

5. The generative video intelligence authentication method based on cross-modal inconsistency mining as described in claim 2, characterized in that, The motion mode feature extraction described in step 2 is the extraction of optical flow features: The keyframe sequence was converted into grayscale images, the optical flow field was calculated using the DIS algorithm, and 128-dimensional optical flow features were extracted using a 4-layer convolutional neural network.

6. The generative video intelligence authentication method based on cross-modal inconsistency mining as described in claim 2, characterized in that, Step 3 specifically involves: The spatial features obtained in step 2 are reduced in dimensionality and then fused with frequency domain statistical features through cross-attention to obtain unified appearance modal features.

7. The method as described in claim 1, characterized in that, The cross-modal inconsistency diagnosis module in step S4 is implemented using a cross-attention mechanism, and its specific process is as follows: The appearance modal features are used as the query, and the motion modal features are used as the key and value. The obtained Query, Key, and Value are each passed through a linear transformation layer and projected onto the same dimension. The relevance matrix between the Query and Key is then calculated. Attention Scores=Query×Key T The attention weights are generated using the Softmax function based on the aforementioned correlation matrix: ; We use AttentionWeights to perform a weighted summation of Values ​​to obtain the context vector: Context=Attention Weights×Value Analyze the Attention Weights matrix to extract regions where the attention weight values ​​are significantly lower than the threshold, as inconsistency diagnostic signals; perform residual connection or concatenation between the Context vector and the original Query features to form the final fused features; The features processed by the cross-modal inconsistency diagnostic module are input into a classifier, which is a neural network containing at least one fully connected layer, to output a false detection confidence level between 0 and 1.

Images