Video forgery detection method and device based on progressive frequency domain gating

Abstract

The embodiment of the application provides a video forgery detection method and device based on progressive frequency domain gating, high-frequency residual maps are extracted by element-wise multiplication of two-dimensional discrete Fourier transform and high-pass filter masks, and high-frequency gating maps are obtained through time difference and multi-stage convolution projection, spatial feature maps are extracted in combination with a pre-trained visual base model, fusion feature maps are obtained through depth separable large kernel convolution and high-frequency gating normalization activation modulation superposition, time sequence extreme value features are extracted through spatial global pooling and one-dimensional convolution time sequence correlation modeling, and video authenticity probability distribution is output through a full connection layer, effectively solving the deficiencies of traditional technologies in aspects of fine extraction and gating representation of high-frequency features in frequency domain and time sequence, collaborative modulation fusion of large receptive field spatial features and frequency domain gating, and end-to-end authenticity determination driven by time sequence extreme value features, and providing technical support for accurate and robust detection of video forgery detection in complex and high-quality deep forgery scenes.

Description

Technical Field

[0001] This application relates to the field of data processing, specifically to a video forgery detection method and apparatus based on progressive frequency domain gating. Background Technology

[0002] Existing video forgery detection methods have significant shortcomings. Traditional systems perform poorly in extracting high-frequency features in the video frequency domain and capturing temporal high-frequency changes. They typically rely on spatial texture features or simple inter-frame differences to detect forgery traces. They lack the ability to perform a two-dimensional discrete Fourier transform on each frame to obtain a frequency domain representation, multiply it element-wise with a high-pass filter mask, and then perform an inverse Fourier transform to obtain a high-frequency residual map. Furthermore, they lack the ability to perform absolute difference calculations on the high-frequency residual maps between adjacent frames to obtain a temporal high-frequency difference map, and then input it into a multi-level convolutional projection module for two-dimensional adaptive max pooling to obtain a high-frequency gating map. As a result, high-frequency tampering traces and inter-frame temporal inconsistencies left over from the video forgery process cannot be effectively quantified and perceived, making it difficult to provide a high-quality high-frequency gating foundation for subsequent accurate frequency domain modulation of visual features.

[0003] Furthermore, existing technologies face bottlenecks in spatial feature extraction from visual foundation models and the construction of fusion features using high-frequency gated modulation. Most systems lack the ability to collaboratively fuse large receptive field spatial features with frequency-domain gated adaptive modulation. This involves inputting the video sequence to be detected into a pre-trained visual foundation model to extract features from each frame block, performing dimensionality reshaping to obtain a spatial feature map, inputting a depthwise separable large-kernel convolution to obtain intermediate spatial features, mapping the high-frequency gated map using a normalized activation function, performing element-wise multiplication with the intermediate spatial features, and then superimposing the spatial feature map to obtain a fused feature map. Consequently, spatial domain visual features cannot accurately perceive the guidance of high-frequency forgery signals in the frequency domain, resulting in insufficient feature discrimination capability in local fine-grained tampering areas, thus affecting the detection accuracy of high-quality deepfake videos.

[0004] Existing systems have technical shortcomings in frame-level feature temporal correlation modeling and video authenticity probability output. They lack a mechanism that uses temporal extreme features—obtained by performing spatial global pooling on the fused feature map to obtain frame-level feature vectors, inputting them into a one-dimensional convolutional network to establish temporal correlation, performing one-dimensional adaptive max pooling to obtain temporal extreme features, and then inputting them into a fully connected layer to output the video authenticity probability distribution—to drive end-to-end authenticity determination. This affects the video forgery detection system's ability to globally perceive cross-frame temporal tampering patterns and its robustness in final authenticity determination. Solving these problems is crucial for improving the comprehensive detection capabilities of video forgery detection in frequency domain temporal high-frequency perception, spatial-frequency joint feature fusion, and temporal extreme value determination. Summary of the Invention

[0005] To address the problems in existing technologies, this application provides a video forgery detection method and apparatus based on progressive frequency domain gating. This method effectively solves the shortcomings of traditional technologies in areas such as refined extraction and gating representation of high-frequency temporal features in the frequency domain, fusion of large receptive field spatial features and frequency domain gating collaborative modulation, and end-to-end authenticity determination driven by temporal extreme value features. This provides technical support for accurate and robust detection of video forgery in complex, high-quality deepfake scenarios.

[0006] To solve at least one of the above problems, this application provides the following technical solution: In a first aspect, this application provides a video forgery detection method based on progressive frequency domain gating, including: Two-dimensional discrete Fourier transform is performed on each frame of the video sequence to be detected to obtain a frequency domain representation. The frequency domain representation is multiplied element-wise with a high-pass filter mask and inverse Fourier transform is performed to obtain a high-frequency residual map. The absolute difference between the high-frequency residual maps of adjacent frames is calculated to obtain a temporal high-frequency difference map. The temporal high-frequency difference map is input into a projection module composed of multi-level convolutions and two-dimensional adaptive max pooling is performed at the end to obtain a high-frequency gated map. The video sequence to be detected is input into a pre-trained visual base model to extract block features of each frame and perform dimensionality reshaping to obtain a spatial feature map. The spatial feature map is input into a depthwise separable large kernel convolution to obtain intermediate spatial features. The high-frequency gated map is mapped by a normalized activation function and then multiplied element-wise with the intermediate spatial features and superimposed on the spatial feature map to obtain a fused feature map. The fused feature map is subjected to spatial global pooling to obtain frame-level feature vectors. The frame-level feature vectors are input into a one-dimensional convolutional network to establish temporal correlation and perform one-dimensional adaptive max pooling to obtain temporal extreme value features. The temporal extreme value features are input into a fully connected layer and output as a video authenticity probability distribution after passing through an activation function.

[0007] Furthermore, it also includes: performing a two-dimensional discrete Fourier transform on the pixel matrix of each frame in the video sequence to be detected to obtain a complex spectrum matrix; performing a spectrum center shift on the complex spectrum matrix to concentrate low-frequency components in the central region of the matrix to obtain a centered frequency domain representation; constructing a high-pass filter mask based on a preset cutoff frequency; and resetting the low-frequency weights in the central region of the high-pass filter mask to zero and the weights in the peripheral high-frequency regions to reserved values. The centered frequency domain representation is multiplied element-wise with the high-pass filter mask to obtain the filtered frequency domain representation. The filtered frequency domain representation is then shifted inversely to the center of the spectrum to obtain the restored frequency domain representation. Finally, the restored frequency domain representation is subjected to a two-dimensional inverse Fourier transform, and the real components are extracted to obtain the high-frequency residual map.

[0008] Furthermore, it also includes: reading the high-frequency residual maps of two adjacent frames in chronological order and performing pixel-by-pixel position difference calculation to obtain an inter-frame difference matrix; taking the absolute value of each element in the inter-frame difference matrix to obtain an absolute difference matrix; and stacking the absolute difference matrices corresponding to each adjacent frame in chronological order to obtain a temporal high-frequency difference map. The temporal high-frequency difference map is input into a projection module composed of multiple convolutional layers. In each convolutional layer, the spatial dimension is reduced stepwise and the channel dimension is multiplied stepwise to obtain multi-scale frequency domain features. The multi-scale frequency domain features are then subjected to two-dimensional adaptive max pooling at the end of the projection module, and the response with the largest activation value is extracted within a preset receptive field to obtain a high-frequency gating map.

[0009] Furthermore, it also includes: dividing each frame in the video sequence to be detected into image blocks of fixed size and inputting them into a pre-trained visual base model to obtain block features in the form of a one-dimensional sequence; and performing dimensional reshaping of the block features according to the row and column arrangement of the original image blocks to obtain a spatial feature map with a two-dimensional spatial topology. The spatial feature map is input into a depth-separable convolutional layer with a one-dimensional horizontal kernel and a one-dimensional vertical kernel, respectively, to obtain the horizontal convolutional response and the vertical convolutional response. The horizontal convolutional response and the vertical convolutional response are then added element-wise to obtain the intermediate spatial feature.

[0010] Furthermore, it also includes: inputting the high-frequency gating map into a normalized activation function and mapping each element value to the zero-to-one interval to obtain a gating weight map, and performing a resolution alignment check on the gating weight map and the spatial feature map to ensure that the two have consistent dimensions in the height and width dimensions; The gated weight map is multiplied element-wise with the intermediate spatial features to obtain the gated modulation features, and the gated modulation features are added element-wise with the spatial feature map to obtain the fused feature map.

[0011] Furthermore, it also includes: performing spatial global average pooling along the height and width dimensions of the fused feature map to obtain the frame-level feature vectors corresponding to each frame, and stacking the frame-level feature vectors of each frame in chronological order to obtain a temporal feature sequence; The temporal feature sequence is input into a one-dimensional convolutional network and a sliding convolution is performed along the time dimension to obtain temporal correlation features. The temporal correlation features are then subjected to one-dimensional adaptive max pooling along the time dimension, and the response with the largest activation value on the time axis is extracted to obtain temporal extreme value features.

[0012] Furthermore, it also includes: flattening the temporal extreme value features into a one-dimensional feature vector and inputting it into a fully connected layer to perform a linear transformation to obtain a classification score vector, and establishing a correspondence between each element in the classification score vector and the real category label and the fake category label; The classification score vector is input into the normalized exponential activation function, and the scores of each category are converted into probability values ​​to obtain the authenticity probability distribution of the video. The category with the highest probability value in the authenticity probability distribution of the video is taken as the authenticity determination result of the video sequence to be detected.

[0013] Secondly, this application provides a video forgery detection device based on progressive frequency domain gating, comprising: The high-frequency detection module is used to perform a two-dimensional discrete Fourier transform on each frame of the video sequence to be detected to obtain a frequency domain representation, multiply the frequency domain representation with a high-pass filter mask element by element and perform an inverse Fourier transform to obtain a high-frequency residual map, calculate the absolute difference between the high-frequency residual maps of adjacent frames to obtain a temporal high-frequency difference map, input the temporal high-frequency difference map into a projection module composed of multi-level convolutions and perform two-dimensional adaptive max pooling at the end to obtain a high-frequency gated map; The feature fusion module is used to input the video sequence to be detected into a pre-trained visual basic model to extract block features of each frame and perform dimensionality reshaping to obtain a spatial feature map. The spatial feature map is then input into a depthwise separable large kernel convolution to obtain intermediate spatial features. After the high-frequency gated map is mapped by a normalized activation function, it is multiplied element-wise with the intermediate spatial features and superimposed on the spatial feature map to obtain a fused feature map. The forgery detection module is used to perform spatial global pooling on the fused feature map to obtain frame-level feature vectors, input the frame-level feature vectors into a one-dimensional convolutional network to establish temporal correlations and perform one-dimensional adaptive max pooling to obtain temporal extreme features, and input the temporal extreme features into a fully connected layer and output the video authenticity probability distribution through an activation function.

[0014] Thirdly, this application provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of the video forgery detection method based on progressive frequency domain gating.

[0015] Fourthly, this application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the video forgery detection method based on progressive frequency domain gating.

[0016] Fifthly, this application provides a computer program product, including a computer program / instructions that, when executed by a processor, implement the steps of the video forgery detection method based on progressive frequency domain gating.

[0017] As can be seen from the above technical solution, this application provides a video forgery detection method and device based on progressive frequency domain gating. It extracts a high-frequency residual map by element-wise multiplication of a two-dimensional discrete Fourier transform and a high-pass filter mask, and obtains a high-frequency gating map through temporal difference and multi-level convolution projection. A spatial feature map is extracted by combining a pre-trained visual basic model, and then superimposed with depthwise separable large kernel convolution and high-frequency gating normalized activation modulation to obtain a fused feature map. Temporal extreme features are extracted through spatial global pooling and one-dimensional convolution temporal correlation modeling, and then output as a video authenticity probability distribution through a fully connected layer. This effectively solves the shortcomings of traditional techniques in refined extraction and gating representation of frequency domain temporal high-frequency features, fusion of large receptive field spatial features and frequency domain gating collaborative modulation, and end-to-end authenticity determination driven by temporal extreme features. It provides technical support for accurate and robust detection of video forgery in complex, high-quality deepfake scenarios. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 This is a flowchart illustrating the video forgery detection method based on progressive frequency domain gating in the embodiments of this application; Figure 2 This is a structural diagram of the video forgery detection device based on progressive frequency domain gating in the embodiments of this application. Detailed Implementation

[0020] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0021] The acquisition, storage, use, and processing of data in this application comply with relevant laws and regulations.

[0022] In view of the problems existing in the prior art, this application provides a video forgery detection method and device based on progressive frequency domain gating. The method extracts a high-frequency residual map by multiplying the two-dimensional discrete Fourier transform with a high-pass filter mask element by element, and obtains a high-frequency gating map by temporal difference and multi-level convolution projection. The spatial feature map is extracted by combining a pre-trained visual basic model, and then superimposed with depthwise separable large kernel convolution and high-frequency gating normalized activation modulation to obtain a fused feature map. The temporal extreme value features are extracted by spatial global pooling and one-dimensional convolution temporal correlation modeling, and then output as a video authenticity probability distribution by a fully connected layer. This method effectively solves the shortcomings of traditional techniques in terms of fine extraction and gating representation of frequency domain temporal high-frequency features, fusion of large receptive field spatial features and frequency domain gating collaborative modulation, and end-to-end authenticity determination driven by temporal extreme value features. It provides technical support for accurate and robust detection of video forgery in complex, high-quality deepfake scenarios.

[0023] To effectively address the shortcomings of traditional techniques in areas such as refined extraction and gating of high-frequency temporal features in the frequency domain, fusion of large receptive field spatial features and frequency domain gating collaborative modulation, and end-to-end authenticity determination driven by temporal extremum features, and to provide technical support for accurate and robust video forgery detection in complex, high-quality deepfake scenarios, this application provides an embodiment of a video forgery detection method based on progressive frequency domain gating. See [link to implementation details]. Figure 1 The video forgery detection method based on progressive frequency domain gating specifically includes the following: Step S101: Perform a two-dimensional discrete Fourier transform on each frame of the video sequence to be detected to obtain a frequency domain representation. Multiply the frequency domain representation with a high-pass filter mask element by element and perform an inverse Fourier transform to obtain a high-frequency residual map. Calculate the absolute difference between the high-frequency residual maps of adjacent frames to obtain a temporal high-frequency difference map. Input the temporal high-frequency difference map into a projection module composed of multi-level convolutions and perform two-dimensional adaptive max pooling at the end to obtain a high-frequency gated map. This embodiment sequentially reads image data from each frame of the video sequence to be detected and converts it into a pixel matrix. The video sequence is organized in chronological order of consecutive frames, with each frame having a frame index identifier attached according to its acquisition time. A two-dimensional discrete Fourier transform is performed on the pixel matrix of each frame to transform the spatial domain pixel values ​​to the frequency domain, resulting in a complex spectrum matrix. The complex spectrum matrix contains real and imaginary components, carrying the amplitude and phase information of the image in different frequency directions. To ensure the symmetry of subsequent filtering operations, the complex spectrum matrix is ​​shifted by its spectral center, causing low-frequency components to cluster in the central region of the matrix, resulting in a centered frequency domain representation. This centered frequency domain representation is written into a frequency domain transform buffer, awaiting reading during high-pass filtering operations.

[0024] After the centralized frequency domain representation is generated, a high-pass filter mask is constructed based on a preset cutoff frequency. The high-pass filter mask is organized as a two-dimensional matrix of the same size as the centralized frequency domain representation. The weights of low-frequency components in the central region are set to zero, while the weights of high-frequency components in the outer regions are retained. The cutoff frequency defines the boundary radius between the central and outer regions; frequency components within the radius are suppressed, while those outside the radius are retained. The centralized frequency domain representation is multiplied element-wise with the high-pass filter mask to obtain the filtered frequency domain representation, during which low-frequency semantic information is filtered. The filtered frequency domain representation is then subjected to an inverse spectral center shift to obtain the restored frequency domain representation. A two-dimensional inverse Fourier transform is then performed, and the real components are extracted to obtain a high-frequency residual map. The high-frequency residual map retains only high-frequency details such as edge textures and noise, and is written to the residual map cache for differential calculation.

[0025] Accordingly, high-frequency residual maps of two adjacent frames are read sequentially to perform inter-frame difference calculation. The difference operation is performed on the temporally consecutive frame t and frame t-1, subtracting the values ​​at the same pixel positions element-wise to obtain an inter-frame difference matrix. The absolute value of each element in the inter-frame difference matrix is ​​taken to obtain an absolute difference matrix, eliminating the influence of the positive or negative sign of the difference. This absolute difference matrix represents the amplitude of high-frequency component changes between adjacent frames; abrupt changes at altered edges and flickering artifacts are presented as local high-response regions in this matrix. The absolute difference matrices corresponding to each adjacent frame are stacked along the time dimension to obtain a temporal high-frequency difference map. This temporal high-frequency difference map carries all inter-frame high-frequency fluctuation information in the form of a three-dimensional tensor and is written into the difference map cache for the projection module to read.

[0026] After the temporal high-frequency difference map is generated, it is input into a projection module composed of multiple convolutional layers for feature mapping. The projection module contains multiple convolutional layers connected in series. Each convolutional layer reduces the spatial dimension progressively by setting the stride, and increases the channel dimension progressively by setting the number of convolutional kernels. Frequency domain texture features of different scales are extracted from the temporal high-frequency difference map in each convolutional layer, forming multi-scale frequency domain features. At the end of the projection module, a two-dimensional adaptive max pooling operation is performed on the multi-scale frequency domain features. This operation forcibly extracts the response position with the largest activation value within a preset receptive field, ensuring that weak, faked high-frequency spikes are not smoothly erased during dimensionality reduction. After the two-dimensional adaptive max pooling process, a high-frequency gated map is obtained, whose spatial resolution is strictly aligned with the subsequent spatial feature map.

[0027] Based on the aforementioned projection results, the high-frequency gating map carries a complete structure of time, channel, height, and width dimensions in the form of a four-dimensional tensor. Each element value in the high-frequency gating map represents the intensity of the high-frequency anomalous response at the corresponding spatial location; a higher value indicates a greater likelihood of tampering at that location. The high-frequency gating map, as the output of step S101, is read at the gating fusion entry point in step S102 for element-wise multiplication modulation with spatial features.

[0028] Step S102: Input the video sequence to be detected into the pre-trained visual basic model to extract the block features of each frame and perform dimensionality reshaping to obtain a spatial feature map. Input the spatial feature map into a depthwise separable large kernel convolution to obtain intermediate spatial features. After mapping the high-frequency gated map with a normalized activation function, perform element-wise multiplication with the intermediate spatial features and superimpose the spatial feature map to obtain a fused feature map. This embodiment reads image data from each frame of the video sequence to be detected and performs preprocessing operations. The preprocessing operation scales each frame to the input size required by the pre-trained visual base model and normalizes pixel values ​​according to preset mean and standard deviation. The normalized frames are then divided into fixed-size image blocks, arranged in a raster order from left to right and top to bottom to form a block sequence. This block sequence is input into the pre-trained visual base model to perform forward inference, extracting the block features corresponding to each frame. These block features are organized in a one-dimensional sequence, where each element corresponds to a deep semantic representation of a block in the original image. The block features are written to a feature extraction cache, awaiting retrieval during dimensionality reshaping.

[0029] After the block features are extracted, a dimensionality reshaping operation is performed based on the row and column arrangement of the original image blocks. This reshaping operation restores the one-dimensional sequence of block features to a feature map with a two-dimensional spatial topological structure, where the number of rows and columns corresponds to the number of blocks in the original image in the vertical and horizontal directions, respectively. The resulting spatial feature map is presented as a four-dimensional tensor carrying the time, channel, height, and width dimensions. The feature vectors at adjacent positions in the spatial feature map retain the topological adjacency relationships of adjacent regions in the original image, ensuring the continuity of local fine-grained information such as facial contours and background boundaries in the spatial dimension. The spatial feature map is then written to a spatial feature cache for retrieval by large-kernel convolution processing.

[0030] Accordingly, the spatial feature map is input into a depthwise separable large-kernel convolutional layer to extract long-range spatial responses. The depthwise separable large-kernel convolution employs a decomposition of a horizontal one-dimensional large kernel and a vertical one-dimensional large kernel. The horizontal convolutional kernel size is set to 1 x K, and the vertical convolutional kernel size is set to K x 1, where K is an odd number greater than or equal to seven. The spatial feature map is input into the horizontal and vertical one-dimensional large-kernel convolutional layers respectively to obtain horizontal and vertical convolutional responses. The horizontal convolutional response captures long-range dependencies in the horizontal direction, and the vertical convolutional response captures long-range dependencies in the vertical direction. The horizontal and vertical convolutional responses are element-wise added to obtain intermediate spatial features. These intermediate spatial features have a large receptive field covering facial contours and stitching edges, and are written into the convolutional response buffer for gating modulation operations to read.

[0031] After the intermediate spatial features are generated, the high-frequency gating map output from step S101 is read from the gating fusion input and normalized mapping is performed. The high-frequency gating map is input into the normalized activation function, and each element value is constrained to a closed interval between zero and one after mapping to obtain the gating weight map. Positions with values ​​close to one in the gating weight map indicate significant high-frequency anomalous responses, while positions with values ​​close to zero indicate stable high-frequency responses. Before performing the dot product operation, the gating weight map and the spatial feature map are checked for resolution alignment to ensure that they have consistent dimensions in the height and width dimensions. If the dimensions are inconsistent, bilinear interpolation is performed on the gating weight map to adjust it to the target resolution.

[0032] Based on the aforementioned normalization results, the gated weight map and the intermediate spatial features are multiplied element-wise to obtain gated modulation features. The multiplication operation amplifies the responses at high-frequency anomalous locations within the intermediate spatial features, while suppressing responses in normal regions. The gated modulation features are then added element-wise to the spatial feature map, and the original spatial semantic information is preserved through residual connections to obtain a fused feature map. The fused feature map carries the spatiotemporal joint representation after gated modulation in the form of a four-dimensional tensor, with the spatial location of the tampering trace exhibiting an enhanced response in the channel dimension. The fused feature map, as the output of step S102, is read at the temporal aggregation entry point in step S103 for spatial global pooling and temporal correlation modeling.

[0033] Step S103: Perform spatial global pooling on the fused feature map to obtain frame-level feature vectors, input the frame-level feature vectors into a one-dimensional convolutional network to establish temporal correlations and perform one-dimensional adaptive max pooling to obtain temporal extreme value features, input the temporal extreme value features into a fully connected layer and output the video authenticity probability distribution through an activation function.

[0034] This embodiment reads the fused feature map output from step S102 from the temporal aggregation entry and performs spatial dimension compression. The fused feature map carries a complete structure of time dimension, channel dimension, height dimension, and width dimension in the form of a four-dimensional tensor. For the feature slices corresponding to each frame in the fused feature map, a spatial global average pooling operation is performed along the height dimension and width dimension. The pooling operation takes the arithmetic mean of all activation values ​​of each channel in the spatial plane to obtain a single scalar response for that channel. After spatial global average pooling, the two-dimensional spatial features of each frame are compressed into a one-dimensional vector form to obtain a frame-level feature vector. The dimension of the frame-level feature vector is equal to the number of channels, carrying a global semantic summary of the image of that frame. The frame-level feature vector is written to the frame-level feature cache, waiting to be read by the temporal stacking operation.

[0035] After the frame-level feature vectors are generated, the feature vectors corresponding to each frame are stacked in chronological order. The stacking operation arranges the frame-level feature vectors from the first to the last frame along the time dimension to form a temporal feature sequence, based on the order of the frame index identifiers. This temporal feature sequence is organized in the form of a two-dimensional tensor, with the first dimension corresponding to the number of time frames and the second dimension corresponding to the number of feature channels. The feature vectors at adjacent positions in the temporal feature sequence reflect the semantic state changes of the video at consecutive moments; both tampering mutations and normal transitions between frames are encoded in this sequence structure. The temporal feature sequence is written into a temporal sequence buffer for retrieval by a one-dimensional convolutional network.

[0036] Accordingly, the temporal feature sequence is input into a one-dimensional convolutional network to perform temporal correlation modeling. The one-dimensional convolutional network contains multiple one-dimensional convolutional layers connected in series, each performing a sliding convolution operation along the time dimension. As the convolutional kernel moves along the time axis, it performs weighted aggregation on the feature vectors of several adjacent frames, capturing dynamic change patterns within a local temporal window. After processing by each convolutional layer, temporal correlation features are obtained. These features retain the same or similar length as the input sequence in the time dimension and carry the high-level representation after temporal modeling in the channel dimension. The temporal correlation features contain a mixed response of inter-frame continuity features and abrupt change anomaly features, which are written into a temporal correlation cache for reading by extreme pooling operations.

[0037] After the temporal correlation features are generated, a one-dimensional adaptive max pooling operation is performed along the time dimension to extract extreme responses. One-dimensional adaptive max pooling scans the activation value sequence of each channel across the entire time axis, forcibly extracting the response corresponding to the moment position with the highest activation value. This operation has the ability to lock in sudden forgery defects that only appear in a very small number of frames; the peak responses of short-term anomalies such as micro-expression distortion or momentary flickering on the time axis are preserved rather than diluted by averaging. The temporal extreme features are obtained after the one-dimensional adaptive max pooling process. These temporal extreme features carry a summary of the most significant anomalous responses in the video sequence in the form of a one-dimensional vector. The temporal extreme features are written into an extreme feature cache for the classification layer to read.

[0038] Based on the aforementioned extreme value extraction results, the temporal extreme value features are flattened into a one-dimensional feature vector and input into a fully connected layer for linear transformation. The fully connected layer maps the input feature vector to the classification score space to obtain a classification score vector, which contains two elements, corresponding to the original scores of the real and fake categories, respectively. The classification score vector is input into a normalized exponential activation function, and each element is converted into a probability value after exponential operation and normalization to obtain the video authenticity probability distribution. In the video authenticity probability distribution, the sum of the two probability values ​​is one, and the category with the larger probability value is determined as the authenticity judgment result of the video sequence to be detected. The video authenticity probability distribution is the output product of step S103, which is read by the detection result output module at the judgment result input to generate the final detection report.

[0039] As described above, the video forgery detection method based on progressive frequency domain gating provided in this application can extract a high-frequency residual map by multiplying the two-dimensional discrete Fourier transform with a high-pass filter mask element-wise, and obtain a high-frequency gating map through temporal difference and multi-level convolution projection. Combined with a pre-trained visual basic model to extract spatial feature maps, a fused feature map is obtained by superimposing depthwise separable large kernel convolution with high-frequency gating normalized activation modulation. After extracting temporal extreme value features through spatial global pooling and one-dimensional convolution temporal correlation modeling, the video authenticity probability distribution is output through a fully connected layer. This effectively solves the shortcomings of traditional technologies in terms of refined extraction and gating representation of frequency domain temporal high-frequency features, fusion of large receptive field spatial features and frequency domain gating collaborative modulation, and end-to-end authenticity determination driven by temporal extreme value features. It provides technical support for accurate and robust detection of video forgery in complex, high-quality deep forgery scenarios.

[0040] In one embodiment of the video forgery detection method based on progressive frequency domain gating in this application, the method may further include the following: Step S201: Perform a two-dimensional discrete Fourier transform on the pixel matrix of each frame in the video sequence to be detected to obtain a complex spectrum matrix. Perform a spectrum center shift on the complex spectrum matrix to concentrate the low-frequency components in the central region of the matrix to obtain a centered frequency domain representation. Construct a high-pass filter mask based on a preset cutoff frequency and reset the low-frequency weights in the central region of the high-pass filter mask to zero and the weights in the peripheral high-frequency regions to the reserved values. Step S202: Multiply the centered frequency domain representation with the high-pass filter mask element by element to obtain the filtered frequency domain representation; perform inverse spectral center shift on the filtered frequency domain representation to obtain the restored frequency domain representation; perform two-dimensional inverse Fourier transform on the restored frequency domain representation and extract the real components to obtain the high-frequency residual map.

[0041] This embodiment sequentially reads image data from each frame of the video sequence to be detected and converts it into a grayscale pixel matrix. The pixel matrix is ​​organized in a two-dimensional array structure, with the number of rows and columns corresponding to the vertical and horizontal resolutions of the image, respectively. Each element carries the brightness value of the corresponding pixel position. A two-dimensional discrete Fourier transform is performed on each frame of the pixel matrix to convert the spatial pixel distribution to a frequency domain representation. The two-dimensional discrete Fourier transform performs a one-dimensional Fourier transform on each row of the pixel matrix sequentially, and then performs a one-dimensional Fourier transform on each column of the transformation result, obtaining a complex spectrum matrix with the same size as the original matrix. Each element in the complex spectrum matrix carries the amplitude and phase information of the corresponding frequency component in complex form. Low-frequency components are distributed in the four corner regions of the matrix, and high-frequency components are distributed in the central region of the matrix. The complex spectrum matrix is ​​written into a spectrum transform buffer, awaiting reading during a center shift operation.

[0042] After the complex spectrum matrix is ​​generated, a spectrum center shift operation is performed to adjust the spatial distribution of frequency components. The center shift operation shifts the complex spectrum matrix by half the matrix size both horizontally and vertically, causing low-frequency components originally located at the corners to converge in the central region of the matrix, and high-frequency components originally located in the center to move to the outer region. After the spectrum center shift, a centralized frequency domain representation is obtained, exhibiting a concentric structure of low frequencies in the center and high frequencies on the periphery. This structure facilitates the subsequent construction of filter masks using circular or rectangular region divisions. The centralized frequency domain representation is written to a centralized spectrum buffer for use in the filter mask construction operation.

[0043] Accordingly, a high-pass filter mask is constructed based on a preset cutoff frequency for frequency domain filtering. The high-pass filter mask is organized as a two-dimensional matrix of the same size as the centered frequency domain representation, with each element taking values ​​between zero and one. The cutoff frequency defines the boundary parameter between the low-frequency suppression region and the high-frequency retention region, and a circular boundary is drawn with the center point of the matrix as the center and the pixel distance corresponding to the cutoff frequency as the radius. The weights of elements within the circular boundary in the central region of the high-pass filter mask are reset to zero, and the weights of elements outside the circular boundary in the outer region are reset to the retained values. Setting the retained value to one indicates that the corresponding frequency component is completely retained, and setting it to a gradual value between zero and one can achieve a smooth transition. The high-pass filter mask is written to a mask buffer for element-wise multiplication operations.

[0044] After the high-pass filter mask is constructed, the centralized frequency domain representation is multiplied element-wise with the high-pass filter mask to obtain the filtered frequency domain representation. The element-wise multiplication operation performs complex-to-real multiplication on elements at the same positions in both matrices. Complex components in the centralized frequency domain representation corresponding to low-frequency regions are suppressed with zero weights, while complex components in the corresponding high-frequency regions are preserved with retained weights. After element-wise multiplication, low-frequency semantic information such as background texture and lighting gradients are filtered out, while high-frequency detail information such as tampered edges and generated noise is preserved. The filtered frequency domain representation, in the form of a complex matrix, carries spectral data containing only high-frequency components and is written to the filtered spectrum buffer for inverse transform operations.

[0045] Based on the aforementioned filtering results, the filtered frequency domain representation is subjected to an inverse spectral center shift operation to restore the original frequency distribution position. The inverse spectral center shift uses the same displacement amount as the forward shift but in the opposite direction, resetting the low-frequency regions concentrated in the center to the four corners and the high-frequency regions distributed on the periphery to the center, thus obtaining the restored frequency domain representation. The restored frequency domain representation is then subjected to a two-dimensional inverse Fourier transform to convert the frequency domain data back to the spatial domain representation. The inverse transform result is presented in the form of a complex matrix, from which the real components are extracted to obtain the high-frequency residual map. The high-frequency residual map carries the high-frequency spatial information of the frame image in the form of a real matrix, with edge contours and noise textures appearing as non-zero response regions in the matrix. The high-frequency residual map serves as the output product of step S202, and is read at the residual map entry point for subsequent steps for inter-frame difference calculation and temporal high-frequency difference map construction.

[0046] In one embodiment of the video forgery detection method based on progressive frequency domain gating in this application, the method may further include the following: Step S301: Read the high-frequency residual maps of two adjacent frames in chronological order and perform pixel-by-pixel position difference calculation to obtain the inter-frame difference matrix. Take the absolute value of each element in the inter-frame difference matrix to obtain the absolute difference matrix. Stack the absolute difference matrices corresponding to each adjacent frame in the time dimension to obtain the temporal high-frequency difference map. Step S302: Input the time-series high-frequency difference map into a projection module composed of multiple convolutional layers. Perform stepwise reduction of spatial dimension and stepwise multiplication of channel dimension in each convolutional layer to obtain multi-scale frequency domain features. Perform two-dimensional adaptive max pooling on the multi-scale frequency domain features at the end of the projection module and extract the response with the largest activation value in the preset receptive field to obtain a high-frequency gating map.

[0047] In this embodiment, the high-frequency residual maps of each frame output in step S202 are read from the residual map entry point and a time sequence is established according to the frame index. Each frame's high-frequency residual map is stored in the residual map buffer as a two-dimensional real matrix, with the matrix size consistent with the spatial resolution of the original video frames. The high-frequency residual maps of adjacent frames are read sequentially in time sequence, and the temporally consecutive high-frequency residual maps of the t-th frame and the sub-t-1-th frame are treated as a set of difference calculation objects. For each pair of adjacent frames, element-wise difference calculation is performed at the same pixel position, subtracting the pixel value of the sub-t-1-th frame from the pixel value at the corresponding position in the t-th frame to obtain the inter-frame difference matrix. The positive or negative sign of each element in the inter-frame difference matrix reflects the direction of increase or decrease in the high-frequency response between adjacent frames, and the numerical value reflects the magnitude of the change. The inter-frame difference matrix is ​​written to the difference calculation buffer, awaiting reading during absolute value processing operations.

[0048] After the inter-frame difference matrix is ​​generated, an absolute value operation is performed on each element to eliminate the influence of positive and negative signs. The absolute value operation converts negative elements in the inter-frame difference matrix to their corresponding positive numbers, leaves positive elements unchanged, and maintains the original value of zero elements. The resulting absolute difference matrix consists of non-negative real numbers, representing only the amplitude of high-frequency component changes between adjacent frames without distinguishing the direction of change. The instantaneous jumps at tampered edges and the flickering of inter-frame artifacts appear as local high-response regions in the absolute difference matrix, while smooth transitions of normal video content appear as low-response or zero-response regions. The absolute difference matrix is ​​written to an absolute difference buffer for reading during time-dimension stacking operations.

[0049] Accordingly, the absolute difference matrices corresponding to adjacent frames are stacked along the time dimension to form a three-dimensional tensor structure. The stacking operation follows the temporal order of the frame pairs, placing the absolute difference matrices of the first and second frames in the first layer of the tensor, the absolute difference matrices of the second and third frames in the second layer, and so on until the absolute difference matrix of the last frame is placed in the last layer. After stacking, a temporal high-frequency difference map is obtained. This map is organized in the form of a three-dimensional tensor, with the first dimension corresponding to the number of time frame pairs, and the second and third dimensions corresponding to the spatial height and width. The temporal high-frequency difference map fully carries the high-frequency fluctuation information between all frames in the video sequence, and the continuous slices along the temporal dimension reflect the high-frequency changes of the video at different times. The temporal high-frequency difference map is written into a difference map cache for the projection module to read.

[0050] After the temporal high-frequency difference map is generated, it is input into a projection module composed of multiple convolutional layers to perform progressive feature mapping. The projection module contains multiple convolutional layers arranged in series. The first convolutional layer receives the temporal high-frequency difference map as input, and subsequent convolutional layers sequentially receive the output of the previous layer as input. Each convolutional layer achieves a progressive reduction in spatial dimension by setting a stride greater than one; after each convolutional layer, the height and width of the feature map are halved according to the stride ratio. Each convolutional layer achieves a progressive expansion in channel dimension by setting a doubling number of convolutional kernels; after each convolutional layer, the number of channels in the feature map increases by a preset factor. After processing by each convolutional layer, multi-scale frequency domain features are obtained. These multi-scale frequency domain features have a compressed and compact representation in the spatial dimension and rich frequency domain texture encoding in the channel dimension. The multi-scale frequency domain features are written into the projection feature cache for retrieval by extremum pooling operations.

[0051] Based on the aforementioned projection results, a two-dimensional adaptive max pooling operation is performed on the multi-scale frequency domain features at the end of the projection module. The two-dimensional adaptive max pooling determines the pooling receptive field range according to preset output size parameters, scans all activation values ​​within each receptive field, and extracts the response with the largest value as the output of that receptive field. This operation forcibly retains the most significant anomalous responses within local regions, preventing weak, fabricated high-frequency spikes from being smoothed out by mean during dimensionality reduction. The output size parameters are set to the same height and width as the subsequent spatial feature map, ensuring strict alignment between the two in spatial dimensions. After the two-dimensional adaptive max pooling process, a high-frequency gated map is obtained. This high-frequency gated map carries a complete structure of time, channel, height, and width dimensions in the form of a four-dimensional tensor. This high-frequency gated map, as the output product of step S302, is read at the gate fusion entry point in step S401 for element-wise multiplication modulation with intermediate spatial features.

[0052] In one embodiment of the video forgery detection method based on progressive frequency domain gating in this application, the method may further include the following: Step S401: Divide each frame in the video sequence to be detected into image blocks of fixed size and input them into the pre-trained visual base model to obtain block features in the form of a one-dimensional sequence. Perform dimensional reshaping on the block features according to the row and column arrangement of the original image blocks to obtain a spatial feature map with a two-dimensional spatial topology. Step S402: Input the spatial feature map into a depth-separable convolutional layer with a one-dimensional horizontal kernel and a one-dimensional vertical kernel respectively to obtain the horizontal convolutional response and the vertical convolutional response. Perform element-wise addition of the horizontal convolutional response and the vertical convolutional response to obtain the intermediate spatial feature.

[0053] This embodiment reads image data from each frame of the video sequence to be detected sequentially and performs block partitioning. Block partitioning divides each frame into several fixed-size image blocks based on preset block size parameters. The block size is typically set to a square pixel region matching the input configuration of the pre-trained visual base model. The partitioning process starts from the top left corner of the image and sequentially extracts each image block in raster order from left to right and top to bottom until the entire image area is covered. If there are remaining pixels at the image edges that are less than the size of a complete block, they are padded with filler pixels to reach the complete block size. After block partitioning, each frame is organized into a sequence of image blocks arranged in raster order, with the sequence length equal to the product of the number of blocks in the row direction and the number of blocks in the column direction. The image block sequence is written to the block partitioning buffer, awaiting reading by the visual base model.

[0054] After the image block sequence is generated, it is input into a pre-trained visual base model to perform deep feature extraction. The pre-trained visual base model adopts a network architecture based on a self-attention mechanism. The input layer receives the image block sequence and adds positional encoding information to each block. The encoding layer establishes global dependencies between blocks through multi-layer self-attention operations. The output layer generates deep semantic representations corresponding to each block. The visual base model is pre-trained on a large-scale image dataset and has the ability to extract general visual features. For video forgery detection tasks, a low-rank adaptation fine-tuning method can be used to adapt to the task without destroying the pre-training parameters. After forward inference, one-dimensional sequence-form block features are obtained. Each element in the block features is a fixed-dimensional feature vector, and the vector dimension is equal to the number of channels in the model's hidden layers. The block features are written to the feature extraction cache for dimensionality reshaping operations.

[0055] Accordingly, the block features are reshaped according to the row and column arrangement of the original image blocks. Dimensional reshaping, based on the number of row-direction blocks recorded during the block partitioning stage, restores the one-dimensional sequence of block features to a two-dimensional grid spatial arrangement. The reshaping operation arranges the first to the number of feature vectors in the first row direction of the sequence into the first row of the feature map, arranges the subsequent equal number of feature vectors into the second row of the feature map, and so on, until all feature vectors have been row and column positioned. After dimensional reshaping, a spatial feature map is obtained, which is organized in the form of a four-dimensional tensor. The first dimension corresponds to the number of time frames, the second dimension corresponds to the number of feature channels, the third dimension corresponds to the spatial height, and the fourth dimension corresponds to the spatial width. The feature vectors at adjacent positions in the spatial feature map retain the topological adjacency relationship of adjacent regions in the original image. The spatial feature map is written to a spatial feature cache for reading by depthwise separable large kernel convolution.

[0056] After the spatial feature map is generated, it is input into a horizontal one-dimensional large-kernel convolutional layer and a vertical one-dimensional large-kernel convolutional layer to extract directional long-range responses. The convolution kernel size of the horizontal one-dimensional large-kernel convolutional layer is set to a rectangular structure of size K, where K is an odd number greater than or equal to seven. The convolution kernel has K weight parameters along the horizontal direction and only a single weight position along the vertical direction. The spatial feature map is input into the horizontal one-dimensional large-kernel convolutional layer to perform a sliding convolution operation to obtain a horizontal convolutional response. This horizontal convolutional response captures long-range dependency features spanning K blocks in the horizontal direction. The convolution kernel size of the vertical one-dimensional large-kernel convolutional layer is set to a rectangular structure of size K multiplied by one. The convolution kernel has K weight parameters along the vertical direction and only a single weight position along the horizontal direction. The spatial feature map is input into the vertical one-dimensional large-kernel convolutional layer to perform a sliding convolution operation to obtain a vertical convolutional response. This vertical convolutional response captures long-range dependency features spanning K blocks in the vertical direction. Both the horizontal convolution response and the vertical convolution response are written to the directional convolution cache for fusion operations to read.

[0057] Based on the aforementioned directional convolution results, the horizontal and vertical convolution responses are element-wise added to achieve bidirectional feature fusion. Element-wise addition performs numerical addition on elements at the same position in the two tensors, superimposing and integrating the horizontal long-range features at each position in the horizontal convolution response with the corresponding vertical long-range features in the vertical convolution response. After element-wise addition, intermediate spatial features are obtained. These intermediate spatial features possess large receptive field coverage in both the horizontal and vertical directions, effectively capturing long-range structural features extending in any direction, such as facial contours and stitching edges. The intermediate spatial features carry the same dimensional structure as the spatial feature map in the form of a four-dimensional tensor. These intermediate spatial features, as the output of step S402, are read at the convolution response entry point in step S501 for element-wise multiplication modulation with the high-frequency gating map.

[0058] In one embodiment of the video forgery detection method based on progressive frequency domain gating in this application, the method may further include the following: Step S501: Input the high-frequency gating map into the normalized activation function and map each element value to the interval between zero and one to obtain the gating weight map. Perform resolution alignment verification on the gating weight map and the spatial feature map to ensure that the two have consistent dimensions in the height and width dimensions. Step S502: Perform element-wise multiplication of the gated weight map and the intermediate spatial features to obtain the gated modulation features, and perform element-wise addition of the gated modulation features and the spatial feature map to obtain the fused feature map.

[0059] This embodiment reads the high-frequency gating graph output from step S302 from the gating fusion entry point and performs a normalization mapping operation. The high-frequency gating graph carries a complete structure of time, channel, height, and width dimensions in the form of a four-dimensional tensor, with each element value representing the intensity of the high-frequency anomalous response at the corresponding spatiotemporal location. The high-frequency gating graph is input into a normalization activation function to perform an element-wise nonlinear transformation. The normalization activation function performs exponential operations and normalization on the input values, compressing and mapping input values ​​of any real number range to a closed interval between zero and one. After processing by the normalization activation function, a gating weight graph is obtained. The values ​​of each element in the gating weight graph are between zero and one; values ​​close to one indicate a significant high-frequency anomalous response, while values ​​close to zero indicate a stable high-frequency response. The gating weight graph is written to a weight mapping cache, awaiting reading during the resolution alignment verification operation.

[0060] After the gated weight map is generated, the spatial feature map output in step S401 is read from the spatial feature cache and a resolution alignment check is performed. The resolution alignment check extracts the height and width dimensions of the gated weight map and the spatial feature map, and compares their values ​​in both dimensions. If the gated weight map and the spatial feature map have the same dimensions in both dimensions, the check passes and the alignment is marked as valid. If there is a size difference in either dimension, bilinear interpolation is performed on the gated weight map to adjust it to the same target resolution as the spatial feature map. Bilinear interpolation performs a weighted average of adjacent pixel values ​​based on the distance between the target and source positions, achieving size scaling while maintaining numerical smoothness. After the resolution alignment check, the gated weight map and the spatial feature map have strictly consistent size configurations in the spatial dimensions.

[0061] Accordingly, the intermediate spatial features output in step S402 are read from the convolution response entry point, and element-wise multiplication is performed between the gated weight map and the intermediate spatial features. Element-wise multiplication performs numerical multiplication on elements at the same position in both tensors, multiplying the weight value at each position in the gated weight map with the corresponding feature value in the intermediate spatial features. Positions in the gated weight map with values ​​close to one maintain an approximately unchanged response to the intermediate spatial features, while positions with values ​​close to zero suppress the response to the intermediate spatial features. After element-wise multiplication, a gated modulation feature is obtained. The spatial feature responses corresponding to high-frequency anomalous regions in the gated modulation feature are selectively amplified, while the spatial feature responses corresponding to normal regions are attenuated accordingly. The gated modulation feature carries the spatial semantic representation after gated weight modulation in the form of a four-dimensional tensor and is written into the modulation feature cache for reading during residual fusion operations.

[0062] After the gated modulation features are generated, they are added element-wise to the spatial feature map to achieve residual fusion. Element-wise addition performs numerical addition on elements at the same position in the two tensors. The gated amplified abnormal response in the gated modulation features is superimposed and integrated with the original semantic response in the spatial feature map. The residual connection structure ensures that the original spatial semantic information is completely preserved during the fusion process, and the gated modulation features are superimposed on the original features as incremental information. After element-wise addition, a fused feature map is obtained. The fused feature map carries the spatiotemporal joint representation after cross-modal gated fusion in the form of a four-dimensional tensor. The spatial location of the tampering trace in the fused feature map shows an enhanced response in the channel dimension, while the spatial location of the normal region maintains the original response level. The fused feature map is the output product of step S502, which is read at the temporal aggregation entry point in step S601 for spatial global pooling and frame-level feature vector extraction.

[0063] In one embodiment of the video forgery detection method based on progressive frequency domain gating in this application, the method may further include the following: Step S601: Perform spatial global average pooling along the height and width dimensions of the fused feature map to obtain the frame-level feature vectors corresponding to each frame, and stack the frame-level feature vectors of each frame in chronological order to obtain a temporal feature sequence; Step S602: Input the temporal feature sequence into a one-dimensional convolutional network and perform sliding convolution along the time dimension to obtain temporal correlation features. Perform one-dimensional adaptive max pooling along the time dimension on the temporal correlation features and extract the response with the largest activation value on the time axis to obtain temporal extreme value features.

[0064] This embodiment reads the fused feature map output from step S502 from the temporal aggregation entry and performs spatial dimension compression. The fused feature map is organized in the form of a four-dimensional tensor, with the first dimension corresponding to the number of time frames, the second dimension corresponding to the number of feature channels, the third dimension corresponding to the spatial height, and the fourth dimension corresponding to the spatial width. For the three-dimensional feature slices corresponding to each frame in the fused feature map, spatial global average pooling is performed along the height and width dimensions. Spatial global average pooling performs an arithmetic mean operation on all activation values ​​of each channel across the entire spatial plane, compressing the height multiplied by the width into a single scalar response. After spatial global average pooling, the two-dimensional spatial features of each frame are compressed into a one-dimensional vector form to obtain a frame-level feature vector. The dimension of the frame-level feature vector is equal to the number of channels, and each element carries a global semantic summary of the image in the corresponding channel. The frame-level feature vector is written to the frame-level feature cache, awaiting reading by the temporal dimension stacking operation.

[0065] After the frame-level feature vectors are generated, a stacking operation is performed on the feature vectors corresponding to each frame in chronological order to construct a sequence structure. The stacking operation arranges the frame-level feature vectors according to the order of their frame index identifiers, placing the frame-level feature vector of the first frame at the beginning of the sequence, the frame-level feature vector of the second frame at the next position, and so on until the frame-level feature vector of the last frame is placed at the end of the sequence. After stacking along the temporal dimension, a temporal feature sequence is obtained. This temporal feature sequence is organized in the form of a two-dimensional tensor, with the first dimension corresponding to the number of time frames and the second dimension corresponding to the number of feature channels. The feature vectors at adjacent positions in the temporal feature sequence reflect the global semantic state of the video at continuous moments. Alterations and normal transitions between frames are encoded in the temporal dimension changes of this sequence. The temporal feature sequence is written into a temporal sequence buffer for reading by a one-dimensional convolutional network.

[0066] Accordingly, the temporal feature sequence is input into a one-dimensional convolutional network to perform temporal correlation modeling. The one-dimensional convolutional network comprises multiple one-dimensional convolutional layers arranged in series, each configured with specific kernel length and channel count parameters. The one-dimensional convolutional layers perform sliding convolution operations along the time dimension, with the convolutional kernel moving along the time axis at a preset stride, performing weighted aggregation on the feature vectors of several consecutive frames within the kernel's coverage area. The kernel length defines the temporal receptive field; a larger kernel length can capture dynamic change patterns over a longer time span. After processing by each layer of one-dimensional convolution, temporal correlation features are obtained. These features retain the sequence structure in the time dimension and carry a high-level semantic representation after temporal modeling in the channel dimension. The temporal correlation features contain a hybrid encoding of inter-frame continuity features and abrupt change anomaly features, which are written into a temporal correlation cache for retrieval by extreme pooling operations.

[0067] After the temporal correlation features are generated, a one-dimensional adaptive max pooling operation is performed along the time dimension to extract extreme responses. The one-dimensional adaptive max pooling determines the pooling receptive field range based on a preset output length parameter; when the output length is set to one, the receptive field covers the entire time axis. The pooling operation scans the activation value sequence of each channel within the receptive field, extracting the response corresponding to the moment position with the largest activation value for each channel as the output of that channel. This operation has the ability to lock and capture sudden forgery defects that only appear in a very small number of frames; the peak responses of short-term anomalies such as micro-expression distortion or instantaneous flicker on the time axis are forcibly preserved rather than diluted by averaging. The temporal extreme features are obtained after the one-dimensional adaptive max pooling process. These temporal extreme features carry a summary of the most significant abnormal responses of each channel in the video sequence in the form of a one-dimensional vector. These temporal extreme features are the output of step S602, which is read at the classification input in step S701 for linear transformation of the fully connected layer and calculation of the true / false probability distribution.

[0068] In one embodiment of the video forgery detection method based on progressive frequency domain gating in this application, the method may further include the following: Step S701: Flatten the temporal extreme value features into a one-dimensional feature vector and input it into a fully connected layer to perform a linear transformation to obtain a classification score vector. Establish a correspondence between each element in the classification score vector and the real category label and the fake category label. Step S702: Input the classification score vector into the normalized exponential activation function and convert the scores of each category into probability values ​​to obtain the authenticity probability distribution of the video. Take the category with the highest probability value in the authenticity probability distribution of the video as the authenticity determination result of the video sequence to be detected.

[0069] This embodiment reads the temporal extreme value features output from step S602 from the classification input entry and performs a dimension flattening operation. The temporal extreme value features are organized as a one-dimensional vector, with each element carrying a summary of the extreme response of the video sequence on the corresponding channel. The flattening operation performs continuous memory arrangement on the temporal extreme value features, expanding all elements in the vector into a single-dimensional one-dimensional feature vector in index order. If the temporal extreme value features have batch dimensions or residual time dimensions, the flattening operation merges and compresses these dimensions with the channel dimensions to a uniform feature length. After flattening, a one-dimensional feature vector is obtained, the length of which is equal to the total number of elements in the temporal extreme value features. The one-dimensional feature vector is written to the flattened feature cache, awaiting reading by the fully connected layer.

[0070] After the one-dimensional feature vector is generated, it is input into a fully connected layer to perform a linear transformation operation. The fully connected layer uses a weight matrix and a bias vector as learnable parameters. The number of rows in the weight matrix equals the number of output categories, and the number of columns equals the length of the input feature vector. The linear transformation performs matrix multiplication on the one-dimensional feature vector and the weight matrix, and then adds the multiplication result element-wise to the bias vector to obtain a classification score vector. The dimension of the classification score vector is equal to the preset number of categories, which is set to two in the video forgery detection task, corresponding to the real category and the forged category, respectively. Each element in the classification score vector is an original score value in any real number range, and the value reflects the confidence level that the input video belongs to the corresponding category. The classification score vector is written to a score vector cache for retrieval in the category correspondence establishment operation.

[0071] Accordingly, a correspondence is established between each element of the classification score vector and a preset category label. The first element of the classification score vector is mapped to the real category label, representing the original score if the input video is a real video. The second element of the classification score vector is mapped to the fake category label, representing the original score if the input video is a fake video. The category labels are organized in integer encoding form, with the real category label encoded as zero and the fake category label encoded as one. After the correspondence is established, each element of the classification score vector has a clear semantic meaning and can be used for subsequent probability transformation and category determination. The correspondence between the classification score vector and the category labels is written into a label mapping cache for the normalized exponential activation function to read.

[0072] After establishing the correspondence between the classification score vector and the category label, the classification score vector is input into a normalized exponential activation function to perform probability transformation. The normalized exponential activation function performs exponential operations on each element of the classification score vector to obtain a sequence of exponential values. Then, each exponential value is divided by the sum of all exponential values ​​to obtain a normalized probability value. After processing by the normalized exponential activation function, a video authenticity probability distribution is obtained. This distribution contains two probability value elements, corresponding to the probability of the real category and the probability of the fake category, respectively. Both probability values ​​in the video authenticity probability distribution are between zero and one, and their sum equals one, satisfying the normalization constraint of the probability distribution. This video authenticity probability distribution is written to a probability distribution cache for retrieval during the determination of the judgment result.

[0073] Based on the aforementioned probability transformation results, the category with the highest probability value in the video authenticity probability distribution is determined as the authenticity determination result of the video sequence to be detected. The determination operation compares the probability of the true category and the probability of the fake category in the video authenticity probability distribution, extracting the category label corresponding to the larger probability value as the final determination category. If the probability of the true category is greater than the probability of the fake category, the video sequence to be detected is determined to be a true video. If the probability of the fake category is greater than or equal to the probability of the true category, the video sequence to be detected is determined to be a fake video. The authenticity determination result is output in the form of a combination of category label and corresponding probability value. The video authenticity probability distribution and the authenticity determination result are the output products of step S702, which are read by the detection result output module at the determination result input to generate the final detection report and confidence level label.

[0074] To effectively address the shortcomings of traditional technologies in areas such as refined extraction and gating of high-frequency temporal features in the frequency domain, fusion of large receptive field spatial features and frequency domain gating-based collaborative modulation, and end-to-end authenticity determination driven by temporal extremum features, and to provide technical support for accurate and robust video forgery detection in complex, high-quality deepfake scenarios, this application provides an embodiment of a video forgery detection device based on progressive frequency domain gating for implementing all or part of the aforementioned video forgery detection method. See [link to embodiment]. Figure 2 The video forgery detection device based on progressive frequency domain gating specifically includes the following components: The high-frequency detection module 10 is used to perform a two-dimensional discrete Fourier transform on each frame of the video sequence to be detected to obtain a frequency domain representation, perform element-wise multiplication of the frequency domain representation with a high-pass filter mask and perform an inverse Fourier transform to obtain a high-frequency residual map, perform absolute difference calculation on the high-frequency residual maps between adjacent frames to obtain a temporal high-frequency difference map, input the temporal high-frequency difference map into a projection module composed of multi-level convolutions and perform two-dimensional adaptive max pooling at the end to obtain a high-frequency gated map; The feature fusion module 20 is used to input the video sequence to be detected into a pre-trained visual basic model to extract block features of each frame and perform dimensionality reshaping to obtain a spatial feature map. The spatial feature map is then input into a depth-separable large kernel convolution to obtain intermediate spatial features. After the high-frequency gated map is mapped by a normalized activation function, it is multiplied element-wise with the intermediate spatial features and superimposed on the spatial feature map to obtain a fused feature map. The forgery detection module 30 is used to perform spatial global pooling on the fused feature map to obtain a frame-level feature vector, input the frame-level feature vector into a one-dimensional convolutional network to establish temporal correlation and perform one-dimensional adaptive max pooling to obtain temporal extreme value features, input the temporal extreme value features into a fully connected layer and output the video authenticity probability distribution through an activation function.

[0075] As described above, the video forgery detection device based on progressive frequency domain gating provided in this application can extract a high-frequency residual map by multiplying the two-dimensional discrete Fourier transform with a high-pass filter mask element-wise, and obtain a high-frequency gating map by temporal difference and multi-level convolution projection. Combined with a pre-trained visual basic model to extract spatial feature maps, a fused feature map is obtained by superimposing depthwise separable large kernel convolution with high-frequency gating normalized activation modulation. After extracting temporal extreme value features by spatial global pooling and one-dimensional convolution temporal correlation modeling, the video authenticity probability distribution is output through a fully connected layer. This effectively solves the shortcomings of traditional technologies in terms of refined extraction and gating representation of frequency domain temporal high-frequency features, fusion of large receptive field spatial features and frequency domain gating collaborative modulation, and end-to-end authenticity determination driven by temporal extreme value features. It provides technical support for accurate and robust detection of video forgery in complex, high-quality deep forgery scenarios.

[0076] This invention also provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the video forgery detection method based on progressive frequency domain gating.

[0077] This invention also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the aforementioned video forgery detection method based on progressive frequency domain gating.

[0078] This invention also provides a computer program product, which includes a computer program that, when executed by a processor, implements the aforementioned video forgery detection method based on progressive frequency domain gating.

[0079] In this embodiment of the invention, a high-frequency residual map is extracted by element-wise multiplication of a two-dimensional discrete Fourier transform and a high-pass filter mask, and then a high-frequency gated map is obtained by temporal difference and multi-level convolution projection. The spatial feature map is extracted by combining a pre-trained visual basic model, and then a fused feature map is obtained by superimposing a depthwise separable large kernel convolution with high-frequency gated normalized activation modulation. Temporal extreme features are extracted by spatial global pooling and one-dimensional convolution temporal correlation modeling, and then the video authenticity probability distribution is output through a fully connected layer. This effectively solves the shortcomings of traditional techniques in terms of fine extraction and gating representation of frequency domain temporal high-frequency features, fusion of large receptive field spatial features and frequency domain gating collaborative modulation, and end-to-end authenticity determination driven by temporal extreme features. This provides technical support for accurate and robust detection of video forgery in complex, high-quality deepfake scenarios.

[0080] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0081] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0082] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0083] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0084] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above descriptions are merely specific embodiments of the present invention and are not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A video forgery detection method based on progressive frequency domain gating, characterized in that, The method includes: Two-dimensional discrete Fourier transform is performed on each frame of the video sequence to be detected to obtain a frequency domain representation. The frequency domain representation is multiplied element-wise with a high-pass filter mask and inverse Fourier transform is performed to obtain a high-frequency residual map. The absolute difference between the high-frequency residual maps of adjacent frames is calculated to obtain a temporal high-frequency difference map. The temporal high-frequency difference map is input into a projection module composed of multi-level convolutions and two-dimensional adaptive max pooling is performed at the end to obtain a high-frequency gated map. The video sequence to be detected is input into a pre-trained visual base model to extract block features of each frame and perform dimensionality reshaping to obtain a spatial feature map. The spatial feature map is input into a depthwise separable large kernel convolution to obtain intermediate spatial features. The high-frequency gated map is mapped by a normalized activation function and then multiplied element-wise with the intermediate spatial features and superimposed on the spatial feature map to obtain a fused feature map. The fused feature map is subjected to spatial global pooling to obtain frame-level feature vectors. The frame-level feature vectors are input into a one-dimensional convolutional network to establish temporal correlation and perform one-dimensional adaptive max pooling to obtain temporal extreme value features. The temporal extreme value features are input into a fully connected layer and output as a video authenticity probability distribution after passing through an activation function.

2. The video forgery detection method based on progressive frequency domain gating according to claim 1, characterized in that, The process involves performing a two-dimensional discrete Fourier transform on each frame of the video sequence to be detected to obtain a frequency domain representation. Then, the frequency domain representation is multiplied element-wise by a high-pass filter mask, followed by an inverse Fourier transform to obtain a high-frequency residual map. This includes: A two-dimensional discrete Fourier transform is performed on the pixel matrix of each frame in the video sequence to be detected to obtain a complex spectrum matrix. The complex spectrum matrix is ​​then shifted by the center of the spectrum to concentrate the low-frequency components in the central region of the matrix to obtain a centered frequency domain representation. A high-pass filter mask is constructed based on a preset cutoff frequency, and the low-frequency weights in the central region of the high-pass filter mask are reset to zero, while the weights in the peripheral high-frequency regions are reset to the reserved values. The centered frequency domain representation is multiplied element-wise with the high-pass filter mask to obtain the filtered frequency domain representation. The filtered frequency domain representation is then shifted inversely to the center of the spectrum to obtain the restored frequency domain representation. Finally, the restored frequency domain representation is subjected to a two-dimensional inverse Fourier transform, and the real components are extracted to obtain the high-frequency residual map.

3. The video forgery detection method based on progressive frequency domain gating according to claim 1, characterized in that, The step of performing absolute difference calculation on the high-frequency residual maps between adjacent frames to obtain a temporal high-frequency difference map, inputting the temporal high-frequency difference map into a projection module composed of multi-level convolutions, and performing two-dimensional adaptive max pooling at the end to obtain a high-frequency gated map includes: The high-frequency residual maps of two adjacent frames are read in chronological order and the difference is calculated pixel by pixel to obtain the inter-frame difference matrix. The absolute difference matrix is ​​obtained by taking the absolute value of each element in the inter-frame difference matrix. The absolute difference matrices corresponding to each adjacent frame are stacked in chronological order to obtain the temporal high-frequency difference map. The temporal high-frequency difference map is input into a projection module composed of multiple convolutional layers. In each convolutional layer, the spatial dimension is reduced stepwise and the channel dimension is multiplied stepwise to obtain multi-scale frequency domain features. The multi-scale frequency domain features are then subjected to two-dimensional adaptive max pooling at the end of the projection module, and the response with the largest activation value is extracted within a preset receptive field to obtain a high-frequency gating map.

4. The video forgery detection method based on progressive frequency domain gating according to claim 1, characterized in that, The process of inputting the video sequence to be detected into a pre-trained visual base model to extract block features from each frame and performing dimensionality reshaping to obtain a spatial feature map, and then inputting the spatial feature map into a depthwise separable large kernel convolution to obtain intermediate spatial features, includes: Each frame in the video sequence to be detected is divided into image blocks of fixed size and input into a pre-trained visual base model to obtain block features in the form of a one-dimensional sequence. The block features are then reshaped according to the row and column arrangement of the original image blocks to obtain a spatial feature map with a two-dimensional spatial topology. The spatial feature map is input into a depth-separable convolutional layer with a one-dimensional horizontal kernel and a one-dimensional vertical kernel, respectively, to obtain the horizontal convolutional response and the vertical convolutional response. The horizontal convolutional response and the vertical convolutional response are then added element-wise to obtain the intermediate spatial feature.

5. The video forgery detection method based on progressive frequency domain gating according to claim 1, characterized in that, The step of mapping the high-frequency gated map using a normalized activation function, performing element-wise multiplication with the intermediate spatial features, and then superimposing the spatial feature map to obtain a fused feature map includes: The high-frequency gated graph is input into a normalized activation function and each element value is mapped to the interval between zero and one to obtain a gated weight graph. The gated weight graph and the spatial feature map are subjected to resolution alignment verification to ensure that they have consistent dimensions in the height and width dimensions. The gated weight map is multiplied element-wise with the intermediate spatial features to obtain the gated modulation features, and the gated modulation features are added element-wise with the spatial feature map to obtain the fused feature map.

6. The video forgery detection method based on progressive frequency domain gating according to claim 1, characterized in that, The step of performing spatial global pooling on the fused feature map to obtain a frame-level feature vector, inputting the frame-level feature vector into a one-dimensional convolutional network to establish temporal correlation, and performing one-dimensional adaptive max pooling to obtain temporal extreme value features includes: Spatial global average pooling is performed along the height and width dimensions of the fused feature map to obtain the frame-level feature vectors corresponding to each frame. The frame-level feature vectors of each frame are stacked in chronological order to obtain a temporal feature sequence. The temporal feature sequence is input into a one-dimensional convolutional network and sliding convolution is performed along the time dimension to obtain temporal correlation features. The temporal correlation features are then subjected to one-dimensional adaptive max pooling along the time dimension, and the response with the largest activation value on the time axis is extracted to obtain temporal extreme value features.

7. The video forgery detection method based on progressive frequency domain gating according to claim 1, characterized in that, The step of inputting the temporal extreme value features into a fully connected layer and outputting the video authenticity probability distribution after passing through an activation function includes: The time series extreme value features are flattened into a one-dimensional feature vector and input into a fully connected layer to perform a linear transformation to obtain a classification score vector. The elements of the classification score vector are then associated with the real category label and the fake category label. The classification score vector is input into the normalized exponential activation function, and the scores of each category are converted into probability values ​​to obtain the authenticity probability distribution of the video. The category with the highest probability value in the authenticity probability distribution of the video is taken as the authenticity determination result of the video sequence to be detected.

8. A video forgery detection device based on progressive frequency domain gating, characterized in that, The device includes: The high-frequency detection module is used to perform a two-dimensional discrete Fourier transform on each frame of the video sequence to be detected to obtain a frequency domain representation, multiply the frequency domain representation with a high-pass filter mask element by element and perform an inverse Fourier transform to obtain a high-frequency residual map, calculate the absolute difference between the high-frequency residual maps of adjacent frames to obtain a temporal high-frequency difference map, input the temporal high-frequency difference map into a projection module composed of multi-level convolutions and perform two-dimensional adaptive max pooling at the end to obtain a high-frequency gated map; The feature fusion module is used to input the video sequence to be detected into a pre-trained visual basic model to extract block features of each frame and perform dimensionality reshaping to obtain a spatial feature map. The spatial feature map is then input into a depthwise separable large kernel convolution to obtain intermediate spatial features. After the high-frequency gated map is mapped by a normalized activation function, it is multiplied element-wise with the intermediate spatial features and superimposed on the spatial feature map to obtain a fused feature map. The forgery detection module is used to perform spatial global pooling on the fused feature map to obtain frame-level feature vectors, input the frame-level feature vectors into a one-dimensional convolutional network to establish temporal correlations and perform one-dimensional adaptive max pooling to obtain temporal extreme features, and input the temporal extreme features into a fully connected layer and output the video authenticity probability distribution through an activation function.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps of the video forgery detection method based on progressive frequency domain gating as described in any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the computer program implements the steps of the video forgery detection method based on progressive frequency domain gating as described in any one of claims 1 to 7.

Images