A deepfake video transcoding detection method based on coding domain feature analysis

Abstract

The application relates to the technical field of video forensics, in particular to a deepfake video transcoding detection method based on coding domain feature analysis, which comprises the following steps: acquiring a video to be detected and decoding the video into a video frame sequence; constructing a luminance intra-prediction feature map and a chroma intra-prediction feature map according to HEVC intra-prediction modes respectively, and constructing a quantized residual feature map based on quantized residual coefficients; inputting the luminance and chroma feature maps into a double-flow network, and performing residual quantization attention weighting by using the quantized residual feature map; linearly fusing high-dimensional features output by the double-flow network to obtain frame-level transcoding probability; averaging the probabilities of multiple frames of the same video to output a transcoding detection result according to the voting result. The application jointly analyzes intra-prediction features in the coding domain and quantized residual features, combines a double-flow network and an attention mechanism, and effectively improves the detection accuracy and robustness of deepfake videos in transcoding scenes across different coding standards.

Description

Technical Field

[0001] This invention relates to the field of video forensics, and in particular to a method for detecting deepfake video transcoding based on coding domain feature analysis. Background Technology

[0002] With the rapid development of generative media technology, deepfake videos are becoming increasingly visually difficult to distinguish from real videos. Against this backdrop, to improve the accuracy of video forgery detection, traditional techniques typically employ passive forensics methods, mainly encompassing detection based on frequency domain filtering residuals, noise perception-based detection, and semantic feature analysis-based detection.

[0003] Detection techniques based on frequency domain filtering residuals perform frequency domain transformation on video frames and extract filter residual features, using anomalies in high-frequency components of deepfake content to distinguish between genuine and fake content. However, since this method only focuses on single frequency domain features, it is difficult to effectively capture multi-layered structural information in video frames, and its detection performance drops significantly when faced with transcoded forged videos. Noise-aware detection starts from the noise distribution characteristics of the video, detecting fake regions by analyzing the differences in noise patterns between fake and real regions. However, transcoding introduces additional quantization noise and encoding distortion, causing forgery traces to intertwine with transcoding traces, significantly affecting the generalization ability of this type of method. While semantic feature analysis-based schemes can identify some forged content using deep semantic information, this method mainly relies on pixel-domain or semantic-domain features, making it difficult to stably track the encoding history and editing traces of the video. Furthermore, the aforementioned detection methods all rely on a single feature domain for analysis. When a forged video undergoes transcoding across coding standards such as AVC to HEVC, the spatial and frequency domain traces of the video will change significantly. Even if the forged area still appears visually abnormal, the detection model will find it difficult to effectively extract forgery clues from the features destroyed by transcoding, making it difficult to guarantee continuous and stable detection of deepfake videos.

[0004] In practical applications, the propagation of deepfake videos often involves multiple transcoding operations. During transcoding, the intra-frame prediction modes and quantization residuals of HEVC encoding contain traceable coding domain information, and the intra-frame prediction modes of both the luminance and chrominance components can reflect transcoding traces. However, traditional deepfake detection schemes, once the feature extraction method is determined, are difficult to change, and all detection tasks use the same features. This prevents the detection method from fully utilizing the stable traces left by transcoding operations in the coding domain, and also makes it impossible to effectively distinguish between high-frequency residual anomalies in deepfake regions and coding distortions introduced by transcoding. Consequently, the applicability and robustness of the detection method in transcoding scenarios are significantly insufficient. Summary of the Invention

[0005] The purpose of this invention is to overcome the shortcomings of existing technologies and provide a deepfake video transcoding detection method based on coding domain feature analysis, comprising the following steps: S1: Obtain the video to be detected, decode and parse the encoding domain of the video to be detected to obtain the video frame sequence, the luminance component and chrominance component of each video frame, HEVC intra-frame prediction mode information, transform unit partitioning information and quantization residual coefficients. S2: Based on the HEVC intra-prediction mode information and transform unit partitioning information, construct a luma intra-prediction feature map and a chroma intra-prediction feature map respectively. The luma intra-prediction feature map and the chroma intra-prediction feature map retain the intra-prediction mode values ​​and their spatial distribution information of the corresponding components. S3: Construct a quantization residual feature map based on the quantization residual coefficients to characterize the spatial distribution and local energy changes of high-frequency quantization residuals in the video frame; S4: Input the luminance intra-prediction feature map and the chrominance intra-prediction feature map into the luminance path and chrominance path of the two-stream network respectively, and extract the luminance coding domain features and chrominance coding domain features; S5: Input the quantized residual feature map into the residual coding and attention processing module to generate a residual quantized attention map, and fuse the residual quantized attention map into the spatial attention features of the luminance and chrominance paths to weight the coding domain features related to the deep forgery region; S6: Classify the weighted luminance coding domain features and chrominance coding domain features, and perform linear fusion through learnable weights to obtain the frame-level transcoding probability of the current video frame belonging to the transcoding category; S7: Perform video-level integration on the frame-level transcoding probabilities of several video frames of the same video to be detected to obtain the video-level transcoding probability. S8: Based on the comparison result between the video-level transcoding probability and the preset detection threshold, output the detection result of whether the video to be detected is a deepfake video that has been transcoded.

[0006] Preferably, in step S1, acquiring the video to be detected and parsing the encoded domain information includes: S11: Read the HEVC bitstream of the video to be detected, and parse the sequence parameter set, image parameter set, frame type information, transform unit division information, luminance intra-frame prediction mode and chrominance intra-frame prediction mode. S12: Determine the spatial region corresponding to each intra-frame prediction mode based on the transformation unit partitioning information, and map the intra-frame prediction mode, transformation unit position and video frame coordinates to a unified feature map coordinate system. S13: Extract the quantization residual coefficients and quantization parameters corresponding to each transform unit, and arrange the intra-frame prediction mode information and quantization residual coefficients in the video frame time order; S14: Perform frame-level alignment between the encoded domain information and the decoded video frames, so that the luminance intraprediction feature map, chrominance intraprediction feature map, and quantization residual feature map of each video frame correspond to each other.

[0007] Preferably, in step S2, constructing the luminance intra-prediction feature map and the chrominance intra-prediction feature map includes: Obtain the intra-frame prediction mode index PM of each transform unit for the luminance and chrominance components; PM is mapped to feature values ​​through a preset mapping function. Different feature values ​​correspond to different intra-frame prediction directions, and the feature value differences of the prediction mode indexes of adjacent intra-frames are normalized according to preset rules. The feature values ​​are filled into the corresponding regions according to the spatial boundaries of the transformation unit to form a luminance intra-prediction feature map or a chrominance intra-prediction feature map corresponding to the spatial position of the video frame. When the prediction mode of the chromaticity component is missing or is a shared mode, the corresponding area is filled with a preset fill value; The size of the luminance intraprediction feature map and the chrominance intraprediction feature map is normalized to meet the size requirements of the dual-stream network input channel.

[0008] Preferably, in step S3, constructing the quantized residual feature map includes: The quantization residual coefficients of each transform unit are calculated based on the transform domain residual coefficients and the quantization matrix. The quantization residual coefficients are divided into blocks according to the preset block size, and the high-frequency residual coefficients are determined according to the frequency position of the quantization residual coefficients in the transform domain in each block. The high-frequency residual coefficients whose absolute values ​​are greater than a preset high-frequency threshold are determined as target residual coefficients, and the sum of squares of the target residual coefficients in each block is calculated to obtain the high-frequency residual energy; the local mean and local variance of the high-frequency residual energy are calculated using a sliding window to obtain residual statistics that characterize the non-uniformity of the residuals. The residual statistics are normalized and spatially resampled to generate a quantized residual feature map that is spatially aligned with the brightness intra-prediction feature map.

[0009] Preferably, constructing the quantized residual feature map further includes: The local gradient of high-frequency residual energy is calculated using a sliding window, and the spatial abrupt change characteristics of residual energy are extracted. Based on the spatial abrupt change characteristics, transcoding sensitivity weights are generated, and the residual statistics are weighted and corrected to enhance the representation of residual non-uniformity introduced by recoding, thereby characterizing the high-frequency residual characteristics in the video transcoding process.

[0010] Preferably, in step S4, both the luminance and chrominance paths of the dual-stream network include an activation convolution module, a characterization module, and a classification module, comprising: The luminance prediction feature map is input into the luminance path activation convolution module, and the chrominance prediction feature map is input into the chrominance path activation convolution module. The luminance path activation convolution module uses a first convolution kernel, and the chrominance path activation convolution module uses a second convolution kernel, and the two convolution kernels have different sizes. Batch normalization and SiLU activation are performed sequentially on the convolution output to enhance the nonlinear representation of the spatial distribution of the intra-frame prediction mode. The number of output channels is adjusted by pointwise convolution, and the adjusted features are then input into the corresponding representation modules.

[0011] Preferably, in step S5, generating and fusing residual quantization attention maps includes: The quantized residual feature map is input into the residual encoder, and multi-scale encoding is performed through convolution, normalization and nonlinear activation to obtain the residual encoded features; The residual coding features are input into the residual attention unit, and the responses of the residual coding features in the spatial location dimension and the channel dimension are calculated respectively. A residual spatial attention map is generated based on the spatial location response, a residual channel attention map is generated based on the channel response, and the two are fused to obtain a residual quantized attention map. Adjust the residual quantization attention map to match the size and number of channels of the feature to be weighted, and then weight and fuse it with the spatial attention branch in the spatial-channel hybrid dual attention; The fused spatial attention is input into the representation module to enhance the response of the relevant coding domain features of the deep forgery region.

[0012] Preferably, the characterization module includes a Fused AtMBConv module, comprising: Perform pointwise convolution on the input features to adjust the channel dimensions; Perform depthwise convolution on the pointwise convolution output to extract local spatial features; The depthwise convolution output is input into the channel SE branch and the spatial attention branch respectively to obtain the channel calibration weights and the spatial calibration weights. The residual quantization attention map is fused with the spatial calibration weights to obtain the fused spatial weights; The depthwise convolution output is jointly weighted using channel calibration weights and fusion spatial weights. Perform pointwise convolution on the weighted features and regularize using Dropout; By concatenating the regularized features with the input feature residuals, high-dimensional features calibrated by residual quantization attention are obtained.

[0013] Preferably, in step S6, the luminance coding domain features and chrominance coding domain features are classified and linearly fused, including: Pointwise convolution, global average pooling, and fully connected layers are performed on the high-dimensional features of the luminance and chrominance paths, respectively, to obtain their respective frame-level classification probabilities. Configure pre-trained fusion weights for the two pathways and normalize them; The frame-level classification probabilities of the two paths are weighted and summed according to normalized weights to obtain the frame-level transcoding probability.

[0014] Preferably, video-level integration is performed and the detection results are output, including: N frames are selected from the same video at a preset sampling interval to obtain N frame-level transcoding probabilities; the N frame-level transcoding probabilities are averaged or weighted averaged, and the weighting is based on the frame's time position, frame type, and confidence level. Remove abnormal frames with a frame-level probability lower than the preset confidence level, or resample and detect abnormal frames; The processed frame-level probabilities are integrated at the video level to obtain the video-level transcoding probability, which is then compared with a preset video-level detection threshold. If the video-level transcoding probability is greater than or equal to the threshold, the video to be detected is output as a transcoded deepfake video; otherwise, the video to be detected is output as an untranscoded video.

[0015] Compared with the prior art, the beneficial effects of the present invention are: (1) This invention acquires the video to be detected and parses its coding domain information. Based on the HEVC intra-frame prediction mode information and the transform unit partitioning information, it constructs the luminance intra-prediction feature map and the chrominance intra-prediction feature map respectively, thereby extracting stable traces related to transcoding operations from the coding domain. By utilizing the complementary expressive ability of the luminance component and the chrominance component for transcoding traces, the detection stability can be maintained under different transcoding parameters and encoders.

[0016] (2) The present invention constructs a quantized residual feature map based on the quantized residual coefficient, inputs the quantized residual feature map into the residual coding and attention processing module, generates a residual quantized attention map and integrates it into the spatial attention feature of the two-stream network, thereby achieving weighting of the relevant coding domain features of the deep forgery region, enabling the network to focus on the high-frequency residual anomalies of the deep forgery region during the representation stage, thereby enhancing the ability to express transcoding traces of the forgery region and effectively distinguishing the coding distortion introduced by transcoding from the inherent traces of the forgery region.

[0017] (3) This invention classifies the weighted luminance coding domain features and chrominance coding domain features and performs linear fusion through learnable weights to obtain frame-level transcoding probabilities. Then, it integrates multiple frame-level transcoding probabilities of the same video at the video level and outputs the detection results based on the comparison between the integration results and the preset threshold. This realizes the aggregation of single-frame detection probabilities into video-level detection conclusions, making full use of the multi-frame evidence of the whole video, reducing the impact of single-frame uncertainty on the final judgment, thereby improving the accuracy and robustness of detection. Attached Figure Description

[0018] Various other advantages and benefits will become apparent to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit the invention.

[0019] Figure 1 This is a flowchart of a deepfake video transcoding detection method based on coding domain feature analysis according to the present invention; Figure 2 This is another flowchart of a deepfake video transcoding detection method based on coding domain feature analysis according to the present invention. Detailed Implementation

[0020] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention. Obviously, the described embodiments are only some, not all, of the embodiments described in this application. All other embodiments obtained by those skilled in the art based on the embodiments in this application without creative effort are within the scope of protection of this application.

[0021] Those skilled in the art will understand that, unless otherwise stated, the singular forms “a” and “an” used herein, and “the”, may also include the plural forms. It should be further understood that the term “comprising” as used in this specification means the presence of the stated features, integers, steps, operations, elements, and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0022] First Embodiment Please see Figure 1 and Figure 2As shown, this embodiment provides a deepfake video transcoding detection method based on coding domain feature analysis. By extracting coding domain information such as intra-frame prediction mode and quantization residual from the HEVC bitstream, it constructs intra-luminance prediction feature maps, intra-chrominance prediction feature maps, and quantization residual feature maps, respectively. Utilizing a dual-stream network to fuse residual quantization attention mechanisms, it achieves stable detection of deepfake video transcoding traces, thereby improving the accuracy and generalization ability of forged video recognition in cross-coding standard transcoding scenarios. The method includes the following steps: S1: Obtain the video to be detected, decode and parse the encoding domain of the video to be detected to obtain the video frame sequence, the luminance and chrominance components of each video frame, HEVC intra-frame prediction mode information, transform unit partitioning information and quantization residual coefficients. Specifically, in this embodiment, the video to be detected may come from network platforms, judicial evidence materials, news broadcast videos or other video data that requires authenticity verification.

[0023] Preferably, in step S1, acquiring the video to be detected and parsing the encoded domain information includes: S11: Read the HEVC bitstream of the video to be detected, and parse the sequence parameter set SPS, image parameter set PPS, frame type information, transform unit division information, luminance intra-frame prediction mode and chrominance intra-frame prediction mode. S12: Determine the spatial region corresponding to each intra-frame prediction mode based on the transform unit partitioning information, and map the intra-frame prediction mode, transform unit position and video frame coordinates to a unified feature map coordinate system. S13: Extract the quantization residual coefficients and quantization parameters corresponding to each transform unit, and arrange the intra-frame prediction mode information and quantization residual coefficients in the time order of the video frames; S14: Perform frame-level alignment between the encoded domain information and the decoded video frames, so that the luminance intraprediction feature map, chrominance intraprediction feature map, and quantization residual feature map of each video frame correspond to each other.

[0024] In one alternative implementation, the coding domain resolution process can be performed collaboratively by the decoding module and the coding information extraction module. The decoding module decodes the HEVC bitstream into a sequence of YUV video frames, while the coding information extraction module synchronously extracts the intra-frame prediction mode index, transform unit partitioning information, and quantization residual coefficients of each coding unit during the decoding process. Since the smallest transform unit size in HEVC coding can be 4×4, a 4×4 transform unit can be represented as a pixel unit in the feature map for subsequent feature map construction.

[0025] In one specific implementation, for scenarios requiring the construction of training samples, the original video can first be decoded into a YUV sequence, and then AVC encoded and HEVC transcoded according to a preset bitrate or quantization parameters to form transcoded and untranscoded samples for model training. During the detection phase, the HEVC bitstream of the video to be detected is directly parsed without performing additional encoding operations.

[0026] S2: Based on the HEVC intra-prediction mode information and transform unit partitioning information, construct the luma intra-prediction feature map L-IPFM and the chroma intra-prediction feature map C-IPFM respectively. The luma intra-prediction feature map L-IPFM and the chroma intra-prediction feature map C-IPFM retain the intra-prediction mode values ​​and their spatial distribution information of the corresponding components.

[0027] Preferably, in step S2, constructing the luminance intra-prediction feature map and the chrominance intra-prediction feature map includes: Obtain the intra-prediction mode index of each transform unit for the luminance and chrominance components. ; PM is mapped to feature values ​​through a preset mapping function. Different feature values ​​correspond to different intra-frame prediction directions, and the feature value differences of the prediction mode indexes of adjacent intra-frames are normalized according to preset rules. The feature values ​​are filled into the corresponding regions according to the spatial boundary of the transform unit to form a luminance intra-prediction feature map or chrominance intra-prediction feature map corresponding to the spatial position of the video frame. When the prediction mode of the chromaticity component is missing or is in a shared mode, the corresponding area is filled with a preset fill value; The size of the luminance intra-IPFM and chrominance intra-IPFM is normalized to meet the size requirements of the input channel of the two-stream network.

[0028] The preset mapping function M(·) is defined by the following formula: , in Representing coordinates The internal predicted feature value after mapping; , in For the prediction pattern index (values ​​0-34), when When the value belongs to [0,34], the gray value is assigned according to a linear relationship; when the color prediction mode is missing or is a preset shared mode, a uniform fill value of 255 can be used for filling.

[0029] Furthermore, the grayscale values ​​of the predicted mode are divided and filled into corresponding regions according to the transformation units to obtain the luminance-internal prediction feature map or the chrominance-internal prediction feature map, as shown below: , in This indicates the operation of filling the corresponding area with grayscale values ​​according to the spatial division of N×N transformation units (N can be 4, 8, 16, 32, etc.). The above process is performed on the luminance component and the chrominance component respectively to obtain LIPFM and CIPFM.

[0030] The intra-prediction feature map constructed in the above manner can simultaneously retain the numerical information and spatial distribution information of the intra-prediction mode, so that the spatial pattern of different prediction modes can be completely preserved, providing a structured coding domain feature representation for the subsequent learning of transcoding traces by the dual-stream network.

[0031] S3: Construct a quantization residual feature map (RQM) based on the quantization residual coefficients to characterize the spatial distribution and local energy changes of high-frequency quantization residuals in video frames.

[0032] Preferably, in step S3, constructing the quantized residual feature map (RQM) includes: The quantization residual coefficients of each transform unit are calculated based on the transform domain residual coefficients and the quantization matrix. The quantization residual coefficients are divided into blocks according to the preset block size, and the high-frequency residual coefficients are determined according to the frequency position of the quantization residual coefficients in the transform domain in each block. The high-frequency residual coefficients whose absolute values ​​are greater than the preset high-frequency threshold are determined as the target residual coefficients, and the sum of squares of the target residual coefficients in each block is calculated to obtain the high-frequency residual energy; the local mean and local variance of the high-frequency residual energy are calculated using a sliding window to obtain the residual statistics characterizing the non-uniformity of the residuals. The residual statistics are normalized and spatially resampled to generate a quantized residual feature map (RQM) spatially aligned with the luminance intra-prediction feature map (L-IPFM). Specifically, the quantized residual coefficients are as follows: , in For the first Transform domain residuals (DCT coefficients) of each transform unit. This is the corresponding quantization matrix. Indicates the DCT transform operation. This is for rounding down. For video frames that have undergone AVC transcoding and then HEVC encoding, the transcoding error will alter the residual expression through the spatial domain distortion term. The quantization residual formula is as follows: , in The quantization rounding error introduced by transcoding This is due to loop filter distortion; , in This represents the reduced spatial distortion term. These are pixel coordinates.

[0033] The high-frequency residual energy is shown below: , in To preset a high-frequency threshold, the summation only applies to frequencies exceeding the threshold. The quantization residual coefficients are used to highlight the abnormal high-frequency components in the frequency domain of the deepfake region.

[0034] The local energy variance is as follows: , in It is a sliding window. The number of elements in the window. This represents the mean energy of the high-frequency residuals within the window. The local energy variance further highlights the residual non-uniformity in the deepfake region.

[0035] Preferably, constructing the quantized residual feature map (RQM) further includes: The local gradient of high-frequency residual energy is calculated using a sliding window, and the spatial abrupt change characteristics of residual energy are extracted. Transcoding sensitivity weights are generated based on spatial abrupt change characteristics, and residual statistics are weighted and corrected to enhance the representation of residual non-uniformity introduced by recoding, thereby characterizing the high-frequency residual characteristics in the video transcoding process.

[0036] Since the quantization, rounding, truncation, and loop filtering distortions caused by transcoding are usually relatively uniformly distributed and do not change the significance of the high-frequency quantization residuals in the deepfake region, RQM can be used to locate deepfake traces and help the network distinguish between deepfake anomalies and transcoding anomalies.

[0037] S4: Input the luminance intra-IPFM and chrominance intra-IPFM into the luminance and chrominance paths of the two-stream network, respectively, to extract luminance coding domain features and chrominance coding domain features.

[0038] Preferably, in step S4, both the luminance and chrominance paths of the dual-stream network include an activation convolution module, a characterization module, and a classification module, comprising: The luminance intra-prediction feature map L-IPFM is input into the luminance path activation convolution module, and the chrominance intra-prediction feature map is input into the chrominance path activation convolution module. The luminance path activation convolutional module uses the first convolutional kernel, and the chrominance path activation convolutional module uses the second convolutional kernel. The two convolutional kernels have different sizes: the first convolutional kernel has a size of 2×2, and the second convolutional kernel of the chrominance path has a size of 3×3, in order to expand the perception area of ​​the predicted feature map in different components. Batch normalization and SiLU activation are performed sequentially on the convolution output to enhance the nonlinear expressive power of the spatial distribution of the intra-frame prediction mode. Batch normalization can reduce the risk of overfitting and gradient vanishing, while the SiLU activation function can enhance the nonlinear expressive power. The number of output channels is adjusted by pointwise convolution, and the adjusted features are input into the corresponding representation modules. The pointwise convolution uses a 1×1 convolution kernel to enhance the cross-channel interaction capability of features.

[0039] In this way, the luminance and chrominance pathways can learn the transcoding traces in the luminance and chrominance components respectively. By utilizing the complementary representation ability of the two components on the transcoding traces, the detection stability under different transcoding parameters can be improved.

[0040] S5: Input the quantized residual feature map RQM into the residual coding and attention processing module to generate a residual quantized attention map, and then fuse the residual quantized attention map into the spatial attention features of the luminance and chrominance paths to weight the coding domain features related to the deep forgery region.

[0041] Preferably, in step S5, generating and fusing residual quantization attention maps includes: The quantized residual feature map (RQM) is input into the residual encoder, and multi-scale encoding is performed through convolution, normalization, and nonlinear activation to obtain the residual encoded features. The residual encoded features are input into the residual attention unit, and the responses of the residual encoded features in the spatial location dimension and the channel dimension are calculated respectively. A residual spatial attention map is generated based on the spatial location response, a residual channel attention map is generated based on the channel response, and the two are fused to obtain a residual quantized attention map. Adjust the residual quantization attention map to match the size and number of channels of the feature to be weighted, and then weight and fuse it with the spatial attention branch in the spatial-channel hybrid dual attention; The fused spatial attention is input into the representation module to enhance the response of the relevant coding domain features of the deep forgery region.

[0042] Preferably, the characterization module includes a Fused AtMBConv module, comprising: Perform pointwise convolution on the input features to adjust the channel dimensions; Perform depthwise convolution on the pointwise convolution output to extract local spatial features; The depthwise convolution output is input into the channel SE branch and the spatial attention branch respectively to obtain the channel calibration weights and the spatial calibration weights. The residual quantization attention map is fused with the spatial calibration weights to obtain the fused spatial weights; The depthwise convolution output is jointly weighted using channel calibration weights and fusion spatial weights. Perform pointwise convolution on the weighted features and regularize using Dropout; By concatenating the regularized features with the input feature residuals, high-dimensional features calibrated by residual quantization attention are obtained.

[0043] Through the aforementioned residual quantization attention mechanism, the deep forgery region is enhanced in the spatial attention map, and the network can focus on representing the anomalies in the internal prediction patterns generated in this region, thereby enhancing the ability to express the transcoding traces related to the forgery region.

[0044] S6: Classify the weighted luminance coding domain features and chrominance coding domain features, and perform linear fusion through learnable weights to obtain the frame-level transcoding probability of the current video frame belonging to the transcoding category.

[0045] Preferably, in step S6, the luminance coding domain features and chrominance coding domain features are classified and linearly fused, including: Pointwise convolution, global average pooling, and fully connected layers are performed on the high-dimensional features of the luminance and chrominance paths, respectively, to obtain their respective frame-level classification probabilities. (Brightness Path) and (Chroma path), where t represents the frame index; Configure pre-trained fusion weights for each of the two pathways. , And normalize it; The frame-level classification probabilities of the two paths are weighted and summed according to normalized weights to obtain the frame-level transcoding probability. The formula is shown below: , in For the first The learnable fusion weights corresponding to each pathway This represents the classification probability output by this pathway. Learnable weights are determined during model training and directly loaded for use during the detection phase, requiring no online updates.

[0046] S7: Perform video-level integration on the frame-level transcoding probabilities of several video frames of the same video to be detected to obtain the video-level transcoding probability.

[0047] Preferably, video-level integration includes: N frames are selected from the same video at a preset sampling interval to obtain N frame-level transcoding probabilities. ; The average or weighted average of the N frame-level transcoding probabilities is calculated, with the weighting based on the frame's temporal position, frame type, and confidence level. Remove abnormal frames with a frame-level probability lower than the preset confidence level, or resample and detect abnormal frames; The processed frame-level probabilities are then integrated at the video level to obtain the video-level transcoding probability P, as shown in the following formula: , Where N is the number of selected video frames. For the first Frame-level transcoding probability. In one alternative implementation, different weights can be assigned to the transcoding probabilities of different frames based on their temporal position and frame type (I-frame, P-frame, B-frame). For example, I-frames can be given a higher weight because they contain more complete intra-frame prediction mode information.

[0048] S8: Based on the comparison between the video-level transcoding probability and the preset detection threshold, output the detection result of whether the video to be detected is a deepfake video that has been transcoded.

[0049] Preferably, video-level integration is performed and the detection results are output, including: Compare the video-level transcoding probability with the preset video-level detection threshold; If the video-level transcoding probability is greater than or equal to the threshold, the output video to be detected is a transcoded deepfake video; otherwise, the output video to be detected is an untranscoded video. The preset detection threshold can be set according to the requirements of the false detection rate and false negative rate in the actual application scenario, for example, it can be set to 0.5.

[0050] In a specific application scenario, the video to be detected is a short video circulating online. After the above steps, the video-level transcoding probability is 0.78, which is greater than the preset threshold of 0.5. Based on this, the system determines that the video is a deepfake video that has been transcoded from AVC to HEVC, thereby achieving effective detection and tracing of the fake video.

[0051] Through the above embodiments, the present invention can make full use of the stable traces related to transcoding operations in the HEVC coding domain, and achieve robust detection of deepfake videos in transcoding scenarios across coding standards by using the complementary expression of transcoding traces by luminance and chrominance components, combining the quantization residual feature map to characterize the high-frequency residual anomalies in the deepfake region, and the video-level multi-frame integrated decision mechanism. It has high detection accuracy and generalization ability.

[0052] Second Embodiment In this embodiment, a computer device is provided, including a memory and one or more processors. The memory stores computer code, and when the computer code is executed by one or more processors, the one or more processors perform the steps of a deepfake video transcoding detection method based on coding domain feature analysis in the first embodiment.

[0053] In some embodiments of this application, a computer-readable storage medium is also provided, wherein when the computer-readable instructions are executed by one or more processors, the one or more processors perform the steps of a deepfake video transcoding detection method based on coding domain feature analysis as described in any one of the first embodiments.

[0054] It is understood that, for the aforementioned deepfake video transcoding detection method based on coding domain feature analysis, if all of these methods are implemented as software functional modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer server or a network device, etc.) to execute all or part of the steps of the methods in the various embodiments of this invention. The aforementioned storage medium includes: USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, optical disks, and other media capable of storing program code.

[0055] Computer-readable storage media may include data signals propagated in baseband or as part of a carrier wave, carrying readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A readable storage medium may also be any readable medium other than a readable storage medium that can transmit, propagate, or transfer a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the readable storage medium may be transmitted using any suitable medium, including but not limited to wireless, wired, optical fiber, RF, etc., or any suitable combination thereof.

[0056] The above description is merely a preferred embodiment of the present invention. The scope of protection of the present invention is not limited to the above embodiments. All technical solutions falling within the scope of the present invention's concept are within the scope of protection of the present invention. It should be noted that for those skilled in the art, any improvements and modifications made without departing from the principles of the present invention should also be considered within the scope of protection of the present invention.

Claims

1. A method for detecting deepfake video transcoding based on coding domain feature analysis, characterized in that, Includes the following steps: S1: Obtain the video to be detected, decode and parse the encoding domain of the video to be detected to obtain the video frame sequence, the luminance component and chrominance component of each video frame, HEVC intra-frame prediction mode information, transform unit partitioning information and quantization residual coefficients. S2: Based on the HEVC intra-prediction mode information and transform unit partitioning information, construct a luma intra-prediction feature map and a chroma intra-prediction feature map respectively. The luma intra-prediction feature map and the chroma intra-prediction feature map retain the intra-prediction mode values ​​and their spatial distribution information of the corresponding components. S3: Construct a quantization residual feature map based on the quantization residual coefficients to characterize the spatial distribution and local energy changes of high-frequency quantization residuals in the video frame; S4: Input the luminance intra-prediction feature map and the chrominance intra-prediction feature map into the luminance path and chrominance path of the two-stream network respectively, and extract the luminance coding domain features and chrominance coding domain features; S5: Input the quantized residual feature map into the residual coding and attention processing module to generate a residual quantized attention map, and fuse the residual quantized attention map into the spatial attention features of the luminance and chrominance paths to weight the coding domain features related to the deep forgery region; S6: Classify the weighted luminance coding domain features and chrominance coding domain features, and perform linear fusion through learnable weights to obtain the frame-level transcoding probability of the current video frame belonging to the transcoding category; S7: Perform video-level integration on the frame-level transcoding probabilities of several video frames of the same video to be detected to obtain the video-level transcoding probability. S8: Based on the comparison result between the video-level transcoding probability and the preset detection threshold, output the detection result of whether the video to be detected is a deepfake video that has been transcoded.

2. The deepfake video transcoding detection method based on coding domain feature analysis according to claim 1, characterized in that, In step S1, the video to be detected is acquired and its coding domain information is parsed, including: S11: Read the HEVC bitstream of the video to be detected, and parse the sequence parameter set, image parameter set, frame type information, transform unit division information, luminance intra-frame prediction mode and chrominance intra-frame prediction mode. S12: Determine the spatial region corresponding to each intra-frame prediction mode based on the transformation unit partitioning information, and map the intra-frame prediction mode, transformation unit position and video frame coordinates to a unified feature map coordinate system. S13: Extract the quantization residual coefficients and quantization parameters corresponding to each transform unit, and arrange the intra-frame prediction mode information and quantization residual coefficients in the video frame time order; S14: Perform frame-level alignment between the encoded domain information and the decoded video frames, so that the luminance intraprediction feature map, chrominance intraprediction feature map, and quantization residual feature map of each video frame correspond to each other.

3. The deepfake video transcoding detection method based on coding domain feature analysis according to claim 1, characterized in that, In step S2, the luminance intra-prediction feature map and the chrominance intra-prediction feature map are constructed, including: Obtain the intra-frame prediction mode index PM of each transform unit for the luminance and chrominance components; PM is mapped to feature values ​​through a preset mapping function. Different feature values ​​correspond to different intra-frame prediction directions, and the feature value differences of the prediction mode indexes of adjacent intra-frames are normalized according to preset rules. The feature values ​​are filled into the corresponding regions according to the spatial boundaries of the transformation unit to form a luminance intra-prediction feature map or a chrominance intra-prediction feature map corresponding to the spatial position of the video frame. When the prediction mode of the chromaticity component is missing or is a shared mode, the corresponding area is filled with a preset fill value; The size of the luminance intraprediction feature map and the chrominance intraprediction feature map is normalized to meet the size requirements of the dual-stream network input channel.

4. The deepfake video transcoding detection method based on coding domain feature analysis according to claim 1, characterized in that, In step S3, the quantization residual feature map is constructed, including: The quantization residual coefficients of each transform unit are calculated based on the transform domain residual coefficients and the quantization matrix. The quantization residual coefficients are divided into blocks according to the preset block size, and the high-frequency residual coefficients are determined according to the frequency position of the quantization residual coefficients in the transform domain in each block. The high-frequency residual coefficients whose absolute values ​​are greater than a preset high-frequency threshold are determined as target residual coefficients, and the sum of squares of the target residual coefficients in each block is calculated to obtain the high-frequency residual energy; the local mean and local variance of the high-frequency residual energy are calculated using a sliding window to obtain residual statistics that characterize the non-uniformity of the residuals. The residual statistics are normalized and spatially resampled to generate a quantized residual feature map that is spatially aligned with the brightness intra-prediction feature map.

5. The deepfake video transcoding detection method based on coding domain feature analysis according to claim 4, characterized in that, Constructing quantized residual feature maps also includes: The local gradient of high-frequency residual energy is calculated using a sliding window, and the spatial abrupt change characteristics of residual energy are extracted. Based on the spatial abrupt change characteristics, transcoding sensitivity weights are generated, and the residual statistics are weighted and corrected to enhance the representation of residual non-uniformity introduced by recoding, thereby characterizing the high-frequency residual characteristics in the video transcoding process.

6. The deepfake video transcoding detection method based on coding domain feature analysis according to claim 1, characterized in that, In step S4, both the luminance and chrominance paths of the two-stream network include an activation convolution module, a representation module, and a classification module, including: The luminance prediction feature map is input into the luminance path activation convolution module, and the chrominance prediction feature map is input into the chrominance path activation convolution module. The luminance path activation convolution module uses a first convolution kernel, and the chrominance path activation convolution module uses a second convolution kernel, and the two convolution kernels have different sizes. Batch normalization and SiLU activation are performed sequentially on the convolution output to enhance the nonlinear representation of the spatial distribution of the intra-frame prediction mode. The number of output channels is adjusted by pointwise convolution, and the adjusted features are then input into the corresponding representation modules.

7. The deepfake video transcoding detection method based on coding domain feature analysis according to claim 1, characterized in that, In step S5, residual quantization attention maps are generated and fused, including: The quantized residual feature map is input into the residual encoder, and multi-scale encoding is performed through convolution, normalization and nonlinear activation to obtain the residual encoded features; The residual coding features are input into the residual attention unit, and the responses of the residual coding features in the spatial location dimension and the channel dimension are calculated respectively. A residual spatial attention map is generated based on the spatial location response, a residual channel attention map is generated based on the channel response, and the two are fused to obtain a residual quantized attention map. Adjust the residual quantization attention map to match the size and number of channels of the feature to be weighted, and then weight and fuse it with the spatial attention branch in the spatial-channel hybrid dual attention; The fused spatial attention is input into the representation module to enhance the response of the relevant coding domain features of the deep forgery region.

8. The deepfake video transcoding detection method based on coding domain feature analysis according to claim 7, characterized in that, The characterization module includes the Fused AtMBConv module, which includes: Perform pointwise convolution on the input features to adjust the channel dimensions; Perform depthwise convolution on the pointwise convolution output to extract local spatial features; The depthwise convolution output is input into the channel SE branch and the spatial attention branch respectively to obtain the channel calibration weights and the spatial calibration weights. The residual quantization attention map is fused with the spatial calibration weights to obtain the fused spatial weights; The depthwise convolution output is jointly weighted using channel calibration weights and fusion spatial weights. Perform pointwise convolution on the weighted features and regularize using Dropout; By concatenating the regularized features with the input feature residuals, high-dimensional features calibrated by residual quantization attention are obtained.

9. The deepfake video transcoding detection method based on coding domain feature analysis according to claim 1, characterized in that, In step S6, the luminance coding domain features and chrominance coding domain features are classified and linearly fused, including: Pointwise convolution, global average pooling, and fully connected layers are performed on the high-dimensional features of the luminance and chrominance paths, respectively, to obtain their respective frame-level classification probabilities. Configure pre-trained fusion weights for the two pathways and normalize them; The frame-level classification probabilities of the two paths are weighted and summed according to normalized weights to obtain the frame-level transcoding probability.

10. The deepfake video transcoding detection method based on coding domain feature analysis according to claim 1, characterized in that, Perform video-level integration and output detection results, including: N frames are selected from the same video at a preset sampling interval to obtain N frame-level transcoding probabilities; the N frame-level transcoding probabilities are averaged or weighted averaged, and the weighting is based on the frame's time position, frame type, and confidence level. Remove abnormal frames with a frame-level probability lower than the preset confidence level, or resample and detect abnormal frames; The processed frame-level probabilities are integrated at the video level to obtain the video-level transcoding probability, which is then compared with a preset video-level detection threshold. If the video-level transcoding probability is greater than or equal to the threshold, the video to be detected is output as a transcoded deepfake video; otherwise, the video to be detected is output as an untranscoded video.

Images