AI video detection method and device based on multi-feature branch fusion, and storage medium

Abstract

The invention relates to the technical field of information security, in particular to an AI video detection method based on multi-feature branch fusion, and the method comprises the following steps: extracting a frame extraction color image from a video, and obtaining a standardized frame sequence after preprocessing; calculating an inter-frame differential volume; obtaining a time sequence spectrum volume according to the standardized frame sequence; performing bilateral filtering decomposition on the standardized frame sequence to obtain an illumination consistency volume; respectively inputting the three types of volume features into a deep convolutional neural network, and after feature extraction and fusion, outputting an AI forgery probability through a classifier; and comparing the AI forgery probability with a decision threshold to generate a video category label, and calculating an index. According to the method, the multi-dimensional features of the time-space domain, the frequency domain and the physical illumination domain are fused, the potential traces of the deeply-forged video are effectively captured, the stable detection performance is kept under various video quality conditions, the accuracy is improved, and reliable technical guarantee is provided for media information security.

Description

Technical Field

[0001] This invention relates to the field of information security technology, and in particular to an AI video detection method, device and storage medium based on multi-feature branch fusion. Background Technology

[0002] With the rapid evolution of deepfake technology, its potential for abuse in the field of information security is becoming increasingly significant. Real-world scenarios are characterized by complex target features (such as textures and occlusions), unique environmental constraints (such as complex lighting and multimodal interference), and concealed forgery motives (such as fake targets and partial forgeries), making deepfake video detection for real-world scenarios a key requirement in the field of information security.

[0003] However, existing deepfake video detection methods have the following limitations: Poor scene adaptability: Most studies focus on general scenes (such as faces and daily scenes), and there is insufficient extraction of targeted features for complex scenes, making it difficult to detect unique forgery traces. Domain research is scarce: Scenarios such as remote sensing information and military involve confidentiality and specialization, there are very few publicly available datasets and research results, and there is a lack of labeled data and detection models for specific domains. The technological accumulation is far behind that of general scenarios. Feature singularity problem: Existing methods rely on general features such as spatial texture and single temporal optical flow, ignoring unique forgery traces in real scenes, such as frequency domain anomalies (e.g., abrupt changes in frequency features) and illumination domain constraints (e.g., violations of illumination patterns at different times). Insufficient preprocessing efficiency and input adaptability: Real video data is large in volume, and the frame rate and duration are not uniform. Single-process preprocessing and simple feature cropping / resampling methods cannot meet the real-time requirements of specific tasks.

[0004] In recent years, although numerous patents have been published both domestically and internationally in the field of deepfake detection, the technologies still have significant limitations. For example, patent application number CN202210229571.7, entitled "A Deepfake Detection Method Based on Frequency Domain Filtering Residuals," only proposes a feature scheme based on frequency domain filtering residuals; while patent application number CN202410345964.3, entitled "A Generalizable Deepfake Image Detection Method and System Based on Noise Perception," focuses on noise perception detection strategies. Most of these methods are applied to forged face video scenarios and cannot directly address real-world scenarios.

[0005] Therefore, there is an urgent need for a deepfake video detection technology that can capture unique forgery traces through multi-dimensional feature collaboration, combined with efficient preprocessing and adaptive input adaptation, to fill the technological gap in deepfake detection in real-world scenarios. Summary of the Invention

[0006] The purpose of this invention is to address the shortcomings of existing technologies by providing an AI video detection method based on multi-feature branch fusion, comprising the following steps: S1: Acquire video through video acquisition device, uniformly extract a fixed number of frame-by-frame color images from the video, and preprocess each frame of the frame-by-frame color image to obtain a standardized frame sequence. S2: Based on the standardized frame sequence, perform pixel-by-pixel difference operation on adjacent frames, and take the absolute value of the difference to form the inter-frame difference volume used to characterize the temporal variation features of the video; S3: Based on the standardized frame sequence, perform a fast Fourier transform on each pixel in the time dimension to obtain frequency distribution information and stack it to generate a time-series spectrum volume for identifying frequency domain anomalies. S4: Perform bilateral filtering decomposition on the standardized frame sequence to obtain the illumination component and the reflection component, and use the illumination component to construct the illumination consistency volume for detecting illumination physical consistency defects. S5: The inter-frame difference volume, the temporal spectrum volume, and the illumination consistency volume are respectively input into the three feature branches of the deep convolutional neural network. After feature extraction and fusion, the AI ​​forgery probability is output through the classifier. S6: After comparing the AI ​​forgery probability with the decision threshold, generate video category labels including real and forged, and calculate the detection accuracy, area under the curve, and average precision index to quantify the model's detection performance.

[0007] Preferably, in step S1, the preprocessing of each frame of the extracted color image to obtain a standardized frame sequence further includes: The video stream file is read using a video decoding tool, and the total number of video frames is counted. A target number of frames is preset, and the frame extraction step size is calculated based on the total number of video frames and the target number of frames. A fixed number of frame-extracted color images are extracted evenly according to the frame extraction step size. The extracted color image is adjusted to a fixed number of pixels and converted to a single-channel grayscale image to obtain a grayscale image. The pixel values ​​of the grayscale image are then normalized to obtain the standardized frame sequence.

[0008] Preferably, in step S2, a pixel-by-pixel difference operation is performed on adjacent frames, and the absolute value of the difference is taken to form an inter-frame difference volume used to characterize the temporal variation features of the video, further including: S21: According to the video playback order, pair consecutive frames in the standardized frame sequence to form adjacent frame pairs; S22: Perform pixel-by-pixel difference calculation on the adjacent frame pairs to obtain the difference result. The calculation formula is as follows: , in, The pixels of the current frame, The pixels from the previous frame; S23: Stack the difference results along the time dimension to form an inter-frame difference volume.

[0009] Preferably, in step S3, a fast Fourier transform is performed on each pixel in the time dimension to obtain frequency distribution information, and the information is stacked to generate a time-series spectral volume for identifying frequency domain anomalies, further comprising: S31: Extract the coordinates of each pixel in the standardized frame sequence, and collect the grayscale values ​​of all frames at each pixel coordinate along the time dimension to construct a time series for each pixel. S32: Perform a fast Fourier transform on each time series to obtain the Fourier transform result, and obtain the frequency amplitude spectrum of the pixel time series based on the Fourier transform result; S33: Obtain the time-series spectral volume based on the frequency amplitude spectrum of all pixels.

[0010] Preferably, in step S4, bilateral filtering decomposition is performed on the normalized frame sequence to obtain illumination components and reflection components, and the illumination components are used to construct an illumination consistency volume for detecting illumination physical consistency defects, further comprising: S41: Perform bilateral filtering decomposition on each frame of the normalized frame sequence to obtain the illumination component estimation map. The bilateral filtering calculation formula is as follows: , in, For the spatial domain Gaussian kernel function, Let Gaussian kernel function be the one with the range. Ω is the normalization coefficient, and Ω is the pixel value. neighborhood window, For the first A standardized frame sequence of frames; S42: Obtain the reflection component estimation map based on the normalized frame sequence and the illumination component estimation map, as shown below: , in, It is a constant; S43: Stack the illumination component maps in the time dimension according to their original time order to obtain the illumination consistency volume.

[0011] Preferably, in step S5, the inter-frame difference volume, the temporal spectrum volume, and the illumination consistency volume are respectively input into a deep convolutional neural network, further comprising: S51: The convolutional block processes the inter-frame difference volume, the temporal spectrum volume, and the illumination consistency volume to obtain an inter-frame difference high-level feature map, a temporal spectrum high-level feature map, and an illumination consistency high-level feature map. S52: Perform feature compression and global representation on the inter-frame difference high-level feature map, the temporal spectrum high-level feature map, and the illumination consistency high-level feature map to obtain the inter-frame difference feature vector, the temporal spectrum feature vector, and the illumination consistency feature vector; S53: The inter-frame difference feature vector, the temporal spectrum feature vector, and the illumination consistency feature vector are fused to obtain a fused feature vector. The feature fusion formula is as follows: , in, The inter-frame difference feature vector is... The time-series spectral feature vector, This is the illumination consistency feature vector; S54: Obtain the AI ​​forgery probability based on the fused feature vector.

[0012] Preferably, in step S54, obtaining the AI ​​forgery probability based on the fused feature vector further includes: S541: Map the fused feature vector to a fused feature representation through a fully connected layer; S542: After Dropout regularization, regularized features are obtained. Based on these regularized features, a Sigmoid classifier is used to calculate the forgery probability, using the following formula: , in For the Sigmoid function, and These are the classifier weight parameters.

[0013] Preferably, in step S6, calculating the detection accuracy further includes: The detection accuracy ACC is calculated as follows: , in, Indicates the number of true positive samples, Indicates the number of true negative samples, Indicates the number of false positive samples, This indicates the number of false negative samples.

[0014] Based on the same concept, the present invention also provides a computer device, including a memory and a processor, wherein the memory stores computer-readable instructions, which, when executed by the processor, cause the processor to perform the steps of the AI ​​video detection method based on multi-feature branch fusion as described in any one of the embodiments.

[0015] Based on the same concept, the present invention also provides a storage medium storing computer-readable instructions, which, when executed by one or more processors, cause the one or more processors to perform the steps of the AI ​​video detection method based on multi-feature branch fusion as described in any one embodiment.

[0016] Compared with the prior art, the beneficial effects of the present invention are: This invention is based on a standardized frame sequence. It performs pixel-by-pixel differential operations on adjacent frames and takes the absolute value of the difference to form an inter-frame differential volume. The inter-frame differential enhances the sensitivity to temporal discontinuities and captures frequency-domain periodic abrupt changes through temporal frequency analysis. Based on the standardized frame sequence, a fast Fourier transform is performed on each pixel in the time dimension to obtain frequency distribution information and stack it to generate a temporal spectral volume. This provides a discriminative feature that transcends spatial texture for deep forgery detection. Bilateral filtering decomposition is performed on the standardized frame sequence to obtain illumination and reflection components. The illumination components are used to construct an illumination consistency volume for detecting lighting physical consistency defects. Illumination consistency decomposition reveals physical rendering defects, thereby significantly improving the model's ability to detect diverse forgery traces.

[0017] This invention inputs the inter-frame difference volume, temporal spectrum volume, and illumination consistency volume into the three feature branches of a deep convolutional neural network. After feature extraction and fusion, the AI ​​forgery probability is output through a classifier. The three-branch deep convolutional network performs specific feature extraction on each feature volume. Through feature fusion, it achieves collaborative detection in the spatiotemporal domain, frequency domain, and physical illumination domain, thus completing the task of identifying AI-forged videos in a hierarchical and multi-dimensional manner.

[0018] This invention achieves deep complementarity between the spatiotemporal, frequency, and physical domains at the feature level through multi-branch feature fusion. This enables the detection model to maintain high accuracy while exhibiting strong robustness under different compression conditions, complex lighting environments, and dynamic interference scenarios. It effectively solves the adaptability and reliability problems of deepfake detection and provides a solid technical guarantee for information security. Attached Figure Description

[0019] Various other advantages and benefits will become apparent to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit the invention.

[0020] Figure 1 This is a flowchart of the AI ​​video detection method based on multi-feature branch fusion according to the present invention; Figure 2 This is another flowchart of the AI ​​video detection method based on multi-feature branch fusion of the present invention. Detailed Implementation

[0021] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention. Obviously, the described embodiments are only some, not all, of the embodiments described in this application. All other embodiments obtained by those skilled in the art based on the embodiments in this application without creative effort are within the scope of protection of this application.

[0022] Those skilled in the art will understand that, unless otherwise stated, the singular forms “a” and “an” used herein, and “the”, may also include the plural forms. It should be further understood that the term “comprising” as used in this specification means the presence of the stated features, integers, steps, operations, elements, and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0023] First Embodiment Please see Figure 1 and Figure 2 As shown, the AI ​​video detection method based on multi-feature branch fusion provided in this embodiment specifically includes the following steps: To improve the processing efficiency of large-scale video data, a multiprocessing module (multiprocessing.Pool) can be used to implement parallel preprocessing, forming a standardized grayscale frame sequence as input. S1: Acquire real-scene videos through video acquisition equipment (including drones, monitoring terminals, or reconnaissance devices), uniformly extract a fixed number of frame-by-frame color images from the real-scene videos, and preprocess each frame-by-frame color image to obtain a standardized frame sequence.

[0024] Preferably, in step S1, preprocessing each frame of extracted color image to obtain a standardized frame sequence further includes: Use the video decoding tool OpenCV to read .mp4 format video stream files, count the total number of video frames, preset the target number of frame extractions, and then calculate the total number of video frames. Calculate the frame extraction step size based on the target number of frames extracted. A fixed number of color images are uniformly extracted based on the extraction step size. The calculation formula is as follows: , Specifically, in this embodiment, the target number of frames extracted is 32. To ensure uniform coverage of the video content and fix the processing cost, the target number of frames is preset. Starting from the first frame of the video, one frame is extracted every (Step-1) frames until all 32 frames are extracted. This method avoids focusing only on a certain segment of the video and can better represent the temporal information of the entire video, obtaining a set of 32 extracted color images. The extracted color image is adjusted to a fixed pixel size and converted to single-channel grayscale to obtain a grayscale image. The pixel values ​​of the grayscale image are normalized to obtain a standardized frame sequence. Specifically, in this embodiment, in order to adapt to the standard input requirements of the subsequent deep learning model, the spatial resolution of all frames is uniformly adjusted to a fixed 224 pixels (height) × 224 pixels (width). An image scaling algorithm (such as bilinear interpolation) is used to stretch or compress each frame image to strictly conform to the 224 × 224 pixel size. The color image is converted from the RGB (red, green, blue) color space to the grayscale space. The conversion formula is as follows: , , R, G, and B are the red, green, and blue channel pixel values ​​of the original frame, respectively. The pixel values ​​of the grayscale image are linearly mapped from the original integer range of [0, 255] (where 0 represents pure black and 255 represents pure white) to the floating-point range of [0.0, 1.0]. For each pixel in the image, its original pixel value is divided by 255.0 to obtain the normalized value, which finally generates a standardized grayscale frame sequence (32 grayscale image sequences) with a resolution of 224×224, denoted as Frame[0], Frame[1], ...,Frame

[31] , which provides a unified input for subsequent feature extraction.

[0025] S2: Based on the standardized frame sequence, perform pixel-by-pixel difference operation on adjacent frames, and take the absolute value of the difference to form the inter-frame difference volume used to characterize the temporal variation features of the video.

[0026] Preferably, in step S2, a pixel-by-pixel difference operation is performed on adjacent frames, and the absolute value of the difference is taken to form an inter-frame difference volume used to characterize the temporal variation features of the video, further including: S21: According to the video playback order, pair consecutive frames in the standardized frame sequence to form adjacent frame pairs. Specifically, in this embodiment, the standardized frame sequence includes T frames, frame pair 1: (Frame[0], Frame[1]), frame pair 2: (Frame[1], Frame[2])... frame pair 31: (Frame

[30] , Frame

[31] ). By comparing adjacent frames, the subtle changes that occur in the scene can be perceived most directly, laying the foundation for calculating instantaneous motion or changes. S22: Perform pixel-by-pixel difference calculation on adjacent frame pairs to obtain the difference result. The calculation formula is as follows: , in, The pixels of the current frame, For the pixels of the previous frame, specifically, in this embodiment, in the difference map (i.e., the difference result), the brighter areas correspond to areas where there is significant movement or change between the two frames (such as moving vehicles or soldiers); while the areas with brightness close to black correspond to the background or static, unchanged parts. AI-forged videos may produce discontinuous, blocky, or flickering abnormal patterns in these changing areas; S23: Stack the difference results along the time dimension to form an inter-frame difference volume. Specifically, in this embodiment, the shape of the inter-frame difference volume is as follows: This feature is used to characterize the temporal variation features of videos, enhancing the model's sensitivity to anomalies such as temporal discontinuities and unnatural motion in AI-forged videos. The length of the time dimension is indicated by 31 consecutive time slices, where H and W represent the spatial resolution (height and width) of each time slice.

[0027] S3: Based on the standardized frame sequence, perform a fast Fourier transform on each pixel in the time dimension to obtain frequency distribution information and stack it to generate a time-series spectral volume for identifying frequency domain anomalies.

[0028] Preferably, in step S3, performing a Fast Fourier Transform (FFT) on each pixel in the time dimension to obtain frequency distribution information and stacking it to generate a time-series spectral volume for identifying frequency domain anomalies further includes: S31: Extract the coordinates of each pixel in the normalized frame sequence, and collect the grayscale values ​​of all frames at each pixel coordinate along the time dimension to construct a time series for each pixel. S32: Perform a Fast Fourier Transform on each time series to obtain the Fourier Transform result, and obtain the frequency amplitude spectrum of the pixel time series based on the Fourier Transform result. Specifically, in this embodiment, for the time series of pixel position (h,w)... Its frequency transformation formula is: , , Where |·| represents the modulo operation. The volume of the generated time-series spectrum; S33: Obtain the temporal spectral volume based on the frequency amplitude spectrum of all pixels. By analyzing patterns such as uneven frequency distribution and abrupt periodic changes, identify frequency domain anomalies in AI-generated videos where inter-frame changes do not conform to natural laws. Specifically, in this embodiment, the amplitude values ​​of all pixels at the same frequency index are collected and recombined into a two-dimensional image. Specifically: For frequency index k (k from 0 to 31), take the amplitude value of the spectrum F(h,w) of all pixels (h,w) at the k-th position. These amplitude values ​​are arranged according to their original spatial coordinates (h,w) to form a 224×224 image. This image can be understood as the spatial distribution map of the entire video scene at a specific frequency k. Repeat the above process for 32 frequency indices (k=0 to 31) to generate 32 such frequency spatial maps. Then, stack these images in the frequency dimension in order from low to high frequency.

[0029] S4: Perform bilateral filtering decomposition on the normalized frame sequence to obtain the illumination component and reflection component. Use the illumination component to construct the illumination consistency volume for detecting illumination physical consistency defects. Specifically, in this embodiment, an image can be approximately decomposed into two core components: the illumination component and the reflection component. The illumination component represents the overall lighting conditions of the scene, including the direction and intensity of the light source and the resulting shadows. This is a spatially slowly varying component. The reflection component represents the inherent texture, material, and color properties of the object's surface.

[0030] Preferably, in step S4, bilateral filtering decomposition is performed on the normalized frame sequence to obtain illumination components and reflection components, and the illumination components are used to construct an illumination consistency volume for detecting illumination physical consistency defects, further comprising: S41: Perform bilateral filtering decomposition on each frame of the normalized frame sequence to obtain the illumination component estimation map. The bilateral filtering calculation formula is as follows: , in, For the spatial domain Gaussian kernel function, Let Gaussian kernel function be the one with the range. Ω is the normalization coefficient, and Ω is the pixel value. neighborhood window, For the first A standardized frame sequence of frames; specifically, in this embodiment, to effectively separate them, filter parameters are set. =15 controls the degree of spatial smoothness, determining the smoothing range of the illumination components; r=0.1 controls the sensitivity of the value range, ensuring that only areas with very similar color / brightness are smoothed, thus protecting the edges. The bilateral filter performs a weighted average of the pixel neighborhood, but the weights are determined by two Gaussian functions, and the value range remains the same. The first weight is based on spatial distance—the closer the pixel is to the center pixel, the higher the weight. The second weight is based on pixel value similarity—the more similar the pixel's grayscale value is to the center pixel, the higher the weight. S42: Obtain the reflection component estimation map based on the normalized frame sequence and the illumination component estimation map, as shown below: , in, To prevent division by zero errors, the reflection component estimation map is used as an auxiliary calculation in this embodiment. S43: Stack the illumination component maps in the time dimension according to the original time order to obtain the illumination consistency volume. Finally, the illumination components form the illumination consistency volume (illum volume), which is used to detect areas with abnormal lighting direction, shadow distribution and reflection patterns in fake videos, reflecting their lighting physical consistency defects.

[0031] S5: The inter-frame difference volume, temporal spectrum volume, and illumination consistency volume are input into the three feature branches of the deep convolutional neural network, respectively. After feature extraction and fusion, the AI ​​forgery probability is output through the classifier.

[0032] Preferably, in step S5, the inter-frame difference volume, temporal spectrum volume, and illumination consistency volume are respectively input into the deep convolutional neural network, further including: S51: The convolutional block processes the inter-frame difference volume, temporal spectrum volume, and illumination consistency volume to obtain the inter-frame difference high-level feature map, temporal spectrum high-level feature map, and illumination consistency high-level feature map. Specifically, in this embodiment, the inter-frame difference volume, temporal spectrum volume, and illumination consistency volume are sequentially passed through two identical convolutional blocks. Each convolutional block performs the following operations: multiple 3×3 convolutional kernels slide on the feature map in the convolutional layer to extract local spatial patterns (such as edges, textures, and specific patterns); the data output by the convolution is standardized in the batch normalization layer to stabilize the training process and accelerate model convergence; and ReLU activation is introduced in the ReLU activation layer. Nonlinear transformations enable the model to learn and fit complex feature relationships. In the max-pooling layer, 2×2 downsampling is performed, preserving the most salient features while reducing the feature map size, lowering computational complexity, and increasing the model's robustness to small positional changes. This allows each branch to focus on learning the most discriminative patterns from specific types of features, enabling differential branches to learn and identify temporal anomalies such as disjointed motion and objects appearing / disappearing out of nowhere; spectral branches to learn and identify frequency domain anomalies such as uneven frequency distribution and abnormal high-frequency noise; and illumination branches to learn and identify physical lighting anomalies such as incorrect shadow direction and unreasonable highlights. The convolutional layer structure is as follows: , in, It is a two-dimensional convolutional layer. For two batch normalization layers, It is a two-dimensional max-pooling layer; S52: Perform feature compression and global representation on the inter-frame difference high-level feature map, the temporal spectrum high-level feature map, and the illumination consistency high-level feature map to obtain the inter-frame difference feature vector, the temporal spectrum feature vector, and the illumination consistency feature vector. Specifically, in this embodiment, an adaptive average pooling operation is applied to the feature map output by each branch to compress the feature maps of different spatial sizes into a vector of a fixed size (e.g., 128 dimensions). It forms a scalar that can represent the global information of the channel by calculating the average value of all elements in each feature channel, and outputs three independent 128-dimensional feature vectors, which respectively represent the global features extracted from the time domain, frequency domain, and illumination domain. S53: The inter-frame difference feature vector, temporal spectrum feature vector, and illumination consistency feature vector are fused to obtain a fused feature vector. The feature fusion formula is shown below: , in, This is the inter-frame difference feature vector. This is a time-series spectral feature vector. The feature vector is the illumination consistency feature vector. Specifically, in this embodiment, the fused feature vector is 384-dimensional. S54: Obtain the AI ​​forgery probability based on the fused feature vector to achieve video authenticity determination.

[0033] Preferably, in step S54, obtaining the AI ​​forgery probability based on the fused feature vector further includes: S541: The fused feature vector is mapped to a fused feature representation through a fully connected layer. Specifically, in this embodiment, the fused feature vector is first mapped from 384 dimensions to 128 dimensions through a fully connected layer, and then nonlinearity is introduced through the ReLU activation function. S542: After Dropout regularization following the fully connected layer, regularized features are obtained. Based on these regularized features, a Sigmoid classifier is used to calculate the forgery probability, using the following formula: , in For the Sigmoid function, and For classifier weight parameters, specifically, in this embodiment, The closer the value is to 1, the higher the likelihood that the model determines the video is an AI-generated fake. The closer the value is to 0, the higher the likelihood that the model determines the video to be real.

[0034] S6: After comparing the AI ​​forgery probability with the decision threshold, generate video category labels including real and forged, and calculate the detection accuracy, area under the curve (AUC), and average precision (AP) to quantify the model's detection performance. Specifically, in this embodiment, the decision threshold is 0.5, and the formula is as follows: .

[0035] Preferably, in step S6, calculating the detection accuracy further includes: The detection accuracy (ACC) is calculated as follows: , in, Indicates the number of true positive samples, Indicates the number of true negative samples, Indicates the number of false positive samples, The number of false negative samples is represented by the number of true positives, true negatives, false positives, and false negatives. In this embodiment, true positives are the number of fake videos that are correctly identified as fake, true negatives are the number of real videos that are correctly identified as real, false positives are the number of real videos that are mistakenly identified as fake, and false negatives are the number of fake videos that are mistakenly identified as real.

[0036] To verify the performance of the deepfake video detection method based on multi-feature fusion of the present invention, the following experiment was designed: Experiment 1 Setup: Training set: 640 positive and 640 negative videos (1,280 videos in total) Test set: 160 positive and 160 negative videos (320 videos in total) Experiment 2 Configuration: Training set: 700 positive and 700 negative videos (total 1,400 videos) Test set: 100 positive and 100 negative videos (200 videos in total) The fake videos in the dataset were generated using various deepfake techniques to simulate potential forgery attacks in real-world environments. In this experiment, the SpectrumDiffModel network was used as the deepfake detector, and accuracy (ACC) and average precision (AP) were used as performance metrics. The performance of Scheme 1 and Scheme 2 was tested under different video compression conditions (no processing, QP=23, QP=40), and the results are shown in Tables 1 and 2. Table 1: Performance of Option 1 No processing ACC = 0.9625 AP = 0.9917 QP=23 ACC = 0.975 AP = 0.9872 QP=40 ACC = 0.8219 AP = 0.9832 Table 2: Performance of Option 2 No processing ACC=0.96 AP=0.978 QP=23 ACC=0.96 AP=0.9912 QP=40 ACC=0.96 AP=0.9902 Experiments show that, using the same detector, Experiment 1 exhibits high accuracy (highest ACC = 0.9750) and average precision (highest AP = 0.9917) under no processing and light compression (QP=23) conditions, but the accuracy drops significantly under high-intensity compression (QP=40). Experiment 2 maintains stable accuracy (ACC = 0.9600) under different compression conditions, and the average precision remains at a high level (AP ≥ 0.9780), especially showing stronger robustness under compression. This indicates that the detection method based on multi-feature fusion can effectively improve the reliability of deepfake video detection and adapt to various video quality conditions, providing strong technical protection for information security.

[0037] Second Embodiment In some embodiments of this application, a computer device is also provided, including a memory and a processor, wherein the memory stores computer-readable instructions, which, when executed by the processor, cause the processor to perform the steps of the AI ​​video detection method based on multi-feature branch fusion in the first embodiment of the present invention.

[0038] The present invention also provides a storage medium storing computer-readable instructions, which, when executed by one or more processors, cause the one or more processors to perform the steps of the AI ​​video detection method based on multi-feature branch fusion in the first embodiment of the present invention.

[0039] It is understood that, for the aforementioned AI video detection methods based on multi-feature branch fusion, if all are implemented as software functional modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer server or a network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this invention. The aforementioned storage medium includes: USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, optical disks, and other media capable of storing program code.

[0040] Computer-readable storage media may include data signals propagated in baseband or as part of a carrier wave, carrying readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A readable storage medium may also be any readable medium other than a readable storage medium that can transmit, propagate, or transfer a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the readable storage medium may be transmitted using any suitable medium, including but not limited to wireless, wired, optical fiber, RF, etc., or any suitable combination thereof.

[0041] The above description is merely a preferred embodiment of the present invention. The scope of protection of the present invention is not limited to the above embodiments. All technical solutions falling within the scope of the present invention's concept are within the scope of protection of the present invention. It should be noted that for those skilled in the art, any improvements and modifications made without departing from the principles of the present invention should also be considered within the scope of protection of the present invention.

Claims

1. An AI video detection method based on multi-feature branch fusion, characterized in that, The method comprises the following steps: S1: acquiring a video through a video acquisition device, uniformly extracting a fixed number of frame color images from the video, and pre-processing each frame of the frame color images to obtain a standardized frame sequence; S2: based on the standardized frame sequence, performing a pixel-by-pixel difference operation on adjacent frames, taking the absolute value of the difference to form an inter-frame difference volume for representing the time sequence change characteristics of the video; S3: based on the standardized frame sequence, performing a fast Fourier transform on each pixel in the time dimension to obtain frequency distribution information and stack to generate a time sequence spectrum volume for identifying frequency domain abnormal features; S4: performing bilateral filtering decomposition on the standardized frame sequence to obtain an illumination component and a reflection component, and using the illumination component to form an illumination consistency volume for detecting illumination physical consistency defects; S5: inputting the inter-frame difference volume, the time sequence spectrum volume and the illumination consistency volume into three feature branches of a deep convolutional neural network respectively, after feature extraction and fusion, outputting an AI fake probability through a classifier; S6: comparing the AI fake probability with a decision threshold to generate a video category label including real and fake, and calculating detection accuracy, area under the curve and average precision indicators to quantify the model detection performance.

2. The AI video detection method based on multi-feature branch fusion according to claim 1, characterized in that, In step S1, the pre-processing of each frame of the frame color images to obtain a standardized frame sequence further comprises: reading a video stream file using a video decoding tool, counting the total number of video frames, presetting a target frame extraction number, calculating a frame extraction step based on the total number of video frames and the target frame extraction number, and uniformly extracting a fixed number of frame color images according to the frame extraction step; adjusting the frame color images to a fixed number of pixels and converting them to a single channel gray scale to obtain a gray scale image, normalizing the pixel values of the gray scale image to obtain the standardized frame sequence.

3. The AI video detection method based on multi-feature branch fusion according to claim 1, characterized in that, In step S2, the pixel-by-pixel difference operation on adjacent frames to form an inter-frame difference volume for representing the time sequence change characteristics of the video further comprises: S21: pairing the frames in the standardized frame sequence two by two in the order of video playback to form adjacent frame pairs; S22: performing a pixel-by-pixel difference calculation on the adjacent frame pair to obtain a difference result, the difference result The calculation formula is as follows: ， wherein, is a pixel of the current frame, is a pixel of the previous frame; S23: stacking the difference results along the time dimension to form an inter-frame difference volume.

4. The AI video detection method based on multi-feature branch fusion according to claim 1, characterized in that, In step S3, the fast Fourier transform on each pixel in the time dimension to obtain frequency distribution information and stack to generate a time sequence spectrum volume for identifying frequency domain abnormal features further comprises: S31: extracting the coordinates of each pixel in the standardized frame sequence, collecting the gray scale values of all frames in each pixel coordinate along the time dimension to construct a time sequence for each pixel; S32: performing a fast Fourier transform on each time sequence to obtain a Fourier transform result, and obtaining a frequency amplitude spectrum of the pixel time sequence according to the Fourier transform result; S33: obtaining the time sequence spectrum volume according to the frequency amplitude spectrum of all pixels.

5. The AI video detection method based on multi-feature branch fusion according to claim 1, characterized in that, In step S4, the bilateral filtering decomposition on the standardized frame sequence to obtain an illumination component and a reflection component, and using the illumination component to form an illumination consistency volume for detecting illumination physical consistency defects further comprises: S41: bilateral filtering decomposition is performed on each frame of the standardized frame sequence to obtain an illumination component estimation map, and a bilateral filtering calculation formula is as follows: ， wherein, is a spatial domain Gaussian kernel function, is a value domain Gaussian kernel function, is a normalization coefficient, Ω is a neighborhood window of a pixel , and is a frame of the th normalized frame sequence. S42: a reflection component estimation map is obtained according to the standardized frame sequence and the illumination component estimation map, as follows: ， wherein is a constant; S43: the illumination component map is stacked in the time dimension according to the original time sequence to obtain the illumination consistency volume.

6. The AI video detection method based on multi-feature branch fusion according to claim 1, characterized in that, In step S5, the inter-frame difference volume, the time sequence spectrum volume and the illumination consistency volume are respectively input into a deep convolutional neural network, and further comprising: S51: a convolutional block processes the inter-frame difference volume, the time sequence spectrum volume and the illumination consistency volume to obtain an inter-frame difference advanced feature map, a time sequence spectrum advanced feature map and an illumination consistency advanced feature map; S52: feature compression and global representation are performed on the inter-frame difference advanced feature map, the time sequence spectrum advanced feature map and the illumination consistency advanced feature map to obtain an inter-frame difference feature vector, a time sequence spectrum feature vector and an illumination consistency feature vector; S53: the inter-frame difference feature vector, the time sequence spectrum feature vector and the illumination consistency feature vector are fused to obtain a fusion feature vector, and a fusion feature vector formula is as follows: ， wherein, is the inter-frame difference feature vector, is the temporal spectral feature vector, is the illumination consistency feature vector; S54: the AI forgery probability is obtained according to the fusion feature vector.

7. The AI video detection method based on multi-feature branch fusion according to claim 6, characterized in that, In step S54, the AI forgery probability is obtained according to the fusion feature vector, and further comprising: S541: the fusion feature vector is mapped to a fusion feature representation through a fully connected layer; S542: after Dropout regularization, a regularization feature is obtained, and a forgery probability is calculated based on the regularization feature using a Sigmoid classifier, and the formula is as follows: ， wherein is a Sigmoid function, and is a classifier weight parameter.

8. The AI video detection method based on multi-feature branch fusion according to claim 1, characterized in that, In step S6, the detection accuracy is calculated, and further comprising: The detection accuracy ACC is calculated as follows: ， wherein, represents the number of true positive samples, represents the number of true negative samples, represents the number of false positive samples, represents the number of false negative samples.

9. A computer device, comprising: The computer readable instructions are executed by one or more processors to make the one or more processors execute the steps of the AI video detection method based on multi-feature branch fusion according to any one of claims 1 to 8.

10. A storage medium storing computer readable instructions, wherein, The computer readable instructions are executed by one or more processors to make the one or more processors execute the steps of the AI video detection method based on multi-feature branch fusion according to any one of claims 1 to 8.

Images