Single classification detection of HEVC video recompression based on hybrid heterogeneous network

Abstract

The invention relates to a single-classification detection method for HEVC video recompression based on a hybrid heterogeneous network. The method aims at solving the challenges faced when HEVC double-compressed videos with the same coding parameters are detected in the prior art, especially in low-motion-intensity and static video scenes. According to the method, a heterogeneous hybrid network is constructed, and a shallow convolutional neural network (CNN), a graph neural network (GNN) and a motion sensing layer are combined, so that the dual-voltage video is effectively detected. The network input is fusion features of motion alignment residuals and high-frequency components extracted from a video sequence, training is performed through a single classifier, only single-voltage videos are used, and videos do not need to be forged. According to the technical scheme, the method comprises four parts of a modulation layer, a motion sensing layer, a shallow CNN and a GNN, can effectively overcome the dependence on a large amount of marked video data, and improves the detection precision. Experimental results show that the method is superior to an existing fully supervised learning method and other single classification methods under different video contents, resolution ratios and coding parameters. The method has the advantages that the dependence on the target forged video and the label thereof is reduced, the detection efficiency and accuracy are improved, and the method has important practical application value in the field of judicial evidence collection of videos.

Description

Technical Field

[0001] The present invention relates to the field of video forensics technology, and in particular to a high-efficiency video coding (HEVC) dual compression detection method based on a hybrid heterogeneous network. Specifically, the present invention combines a shallow convolutional neural network (CNN) with a graph neural network (GNN) to propose a single-classification detection method that only requires the original video data for dual compressed videos with the same coding parameters, thereby having important application value in scenarios such as video forensics and video tampering identification. Background Art

[0002] At present, with the continuous development of the electronics industry and multimedia technology, digital video acquisition equipment is becoming more and more common. At the same time, there are many video editing software on the market, such as Premiere, Edius and Final Cut Pro, etc., which make it easier to change the semantic content of the original video, which poses a challenge to the field of video forensics and obviously undermines the credibility and authenticity of digital videos. Therefore, passive forensics technology against malicious video manipulation has become increasingly important. Video tampering inevitably leads to secondary compression of the video, a process that includes decoding the original video (usually compressed before release), performing operational activities on the decoded frame sequence, and recompressing the video using the codec integrated into the editing software. Therefore, it can be considered that video double compression detection is the initial stage of determining whether a suspicious video has been tampered with.

[0003] In order to identify double compression in videos, many algorithms for double compression detection have emerged in the past decade and have flourished in recent years. However, most of them focus on extracting features in the coding domain to distinguish between single compression and double compression in videos, and apply fully supervised methods to obtain good detection performance. In 2021, an OCC-based detection method SPAM-GDOCC[1] was proposed. This method uses the double compression detection task as an outlier detection paradigm, only uses features extracted from untouched videos, and focuses on the detection of MPEG-4 double compression. This method is based on a single Gaussian distribution model, uses a single classifier and only uses target class videos for training to achieve double compression detection. Specifically, the decompressed video frames are first processed to extract the SPAM (Subtractive Pixel Adjacency Matrix) feature detection traces left in the double compression process; since the SPAM features extracted from the original video frames are approximately Gaussian distributed, a single classifier based on Gaussian density is used; due to the diversity of video content, an integrated strategy and voting mechanism are used to improve the robustness of detection. Experimental results show that this method can effectively detect double compression videos with only the original video sequence, and is also better than the fully supervised learning method. Summary of the invention

[0004] The present invention aims to solve the problems existing in the prior art and proposes a hybrid heterogeneous network method based on single classification detection for detecting HEVC double compressed videos with the same coding parameters. The method does not rely on forged video samples and only uses original videos for training, while effectively improving the detection accuracy of low motion intensity videos. To achieve the above purpose, the present invention provides the following technical solutions:

[0005] 1. Modulation layer: Extract the motion alignment residual and high-frequency components of the video, and fuse these two features by pixel-by-pixel multiplication to highlight the coding error and generate fused features for subsequent detection.

[0006] 2. Motion perception layer: Using the motion alignment residual and the amplitude strength of the motion vector of the video sequence, the video is divided into medium-high speed video (MHSV) and low-speed or static video (LSSV) based on the motion changes of the video to meet the detection needs of different motion intensities.

[0007] 3. Shallow CNN: The local characteristics of the fused features are learned through five layers of convolutional blocks, capturing the differences between single-compressed and double-compressed videos in terms of zero-value pixels and other pixel fluctuations.

[0008] 4. Graph Neural Network (GNN): The fused features are divided into block areas of different sizes, and an undirected fully connected graph is constructed with the number of zero-valued pixels in the block as the node feature. The local and global relationships in the fused features are captured through two layers of graph convolution.

[0009] 5. Single classifier: A single classification detection model is constructed based on the Mahalanobis distance, and semi-supervised learning is performed on the fusion features to achieve effective detection of double compressed videos.

[0010] In summary, the present invention combines shallow CNN and GNN to form a heterogeneous hybrid network for HEVC dual-pressure detection with the same coding parameters, effectively overcoming the dependence on a large amount of labeled video data and improving the detection accuracy. By analyzing the trajectories and features of single-pressure and dual-pressure videos, a modulation layer that extracts motion alignment residuals and high-frequency components is designed to highlight coding errors, and a motion perception layer is proposed to select appropriate video categories. At the same time, a hybrid network with two branches is established to respectively characterize the zero-value pixels and other pixels of the fused features, and a fusion strategy is introduced to combine the learned representations, and the Mahalanobis distance is used as a semi-supervised single classifier to complete the detection task. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] Figure 1 is an example of the previous frame, current frame and motion alignment residual, high-frequency components and their fused features;

[0012] Figure 2 is the relationship between the motion direction residual and the motion vector amplitude strength;

[0013] Figure 3 This is an overview diagram of the network architecture. DETAILED DESCRIPTION

[0014] In order to make the purpose and advantages of the present invention more clearly understood, the present invention is specifically described below in conjunction with embodiments. It should be understood that the following text is only used to describe one or several specific embodiments of the present invention, and does not strictly limit the scope of protection of the specific claims of the present invention.

[0015] Example:

[0016] Based on the public video dataset widely used for dual-compression detection, the present invention uses the ffmpeg tool for compression operation, combines shallow CNN and GNN to form a heterogeneous hybrid network, and proposes a HEVC dual-compression detection method with the same coding parameters. Only single-compression video is used in the training process, and the Mahalanobis distance is introduced to realize a single classifier. The input of the network is the fusion feature of the motion alignment residual and high-frequency component extracted from the video sequence. The method only learns features from a single compressed video. Experimental results show that this method is superior to the existing most advanced fully supervised learning method and other more complex single classification methods. The specific implementation method is as follows:

[0017] Modulation layer: This layer mainly includes two parts: extracting motion alignment residuals and high-frequency components. Motion estimation techniques such as Lucas-Kanade are used to calculate the motion vectors of the previous frame and the current frame, and then the previous frame is warped to align with the motion of the current frame. The current frame is then subtracted from the aligned frame to obtain the motion alignment residual. At the same time, starting from the second frame of the video sequence, the high-frequency component is extracted using Fourier transform. Finally, the motion alignment residual is multiplied pixel by pixel with the high-frequency component to obtain the fusion feature. Figure 1 As shown, double compression inevitably leads to more zeros in the high-frequency components of the motion alignment residual.

[0018] Motion perception layer: Since the single classifier is sensitive to the sampling type, when the probability density of the training sample is inconsistent with the probability density of the test sample, the decision result may be incorrect. Therefore, the motion perception layer is designed. By using the motion alignment residual of the video sequence, the motion change of the video is obtained, and the amplitude strength of the motion vector of each video is calculated. It is found that the two are proportional, so the motion change of the root video divides the dataset into two categories: medium-speed video MHSV and low-speed or static video LSSV. Figure 2 shown.

[0019] Shallow CNN: This branch (such as Figure 3 The network (shown at the top) consists of five convolutional blocks. Each convolutional block has a 2D convolutional layer, BatchNorm, ReLU, and Pooling layers. The kernel size of the convolutional layers is 5×5, except for the third one which is 1×1. The pooling layers of the first four blocks use average pooling, and the last block uses global average pooling. Average pooling and global average pooling help alleviate the maximum variance in local areas of high-frequency components, which otherwise may be destroyed by maximum pooling. The last convolutional block outputs a 128-dimensional vector.

[0020] GNN: According to the block segmentation strategy of the HEVC codec, the fused features are divided into different scales (here 256×256) from the smallest 8×8 to the largest power of 2 in integer multiples of 2. The block starts from the upper left corner of the input and is divided into 6 blocks of sizes 8×8, 16×16, 32×32, 64×64, 128×128 and 256×256 respectively. An undirected fully connected graph is constructed by dividing the video frame into regions of different scales as graph nodes and the number of zero-valued pixels as the features of the graph nodes. This strategy helps to present the relationship between the local and the global, reflecting the differences in the positions of zero-valued pixels, and these relationships can be well modeled by GNN. This prompted us to adopt a GNN consisting of 2 graph convolutional layers (such as Figure 3 As shown at the bottom), we get a 6-dimensional vector.

[0021] Dataset construction: 70 original YUV sequences were downloaded from Xiph.org, including 36 CIF format (352×288) videos, 10 720P format (1280×720) videos, and 24 1080P format (1920×1080) videos. This dataset is widely used for double compression detection. According to the motion complexity of the video scene, these videos are divided into two subsets using a motion-aware layer, labeled MHSV (medium-speed and high-speed video) and LSSV (slow-speed or static video). Subsequently, each sequence is divided into non-overlapping YUV segments, each consisting of 100 frames and no more than 600 frames in length. 292 YUV segments are obtained, including 120 CIF format, 57 720P format, and 115 1080P format.

[0022] Single-compression and dual-compression videos are generated by the H.265 / HEVC codec libx265 in ffmpeg based on the same encoding parameters, with -bf set to 0 to ignore b-frames and -qp defined as 22 to control the compression quality in CQP mode. Therefore, for the MHSV subset, single-compression videos in 70 training clips are used as target classes, and 142 test clips, including 71 single-compression videos and corresponding dual-compression videos; for the LSSV subset, 76 single-compression videos are used as training clips, and finally 150 test clips are constructed, including 75 single-compression videos and their corresponding dual-compression videos. The video content between the training set and the test set does not overlap. The experiment uses AUC as the evaluation criterion.

[0023] The above is only a preferred embodiment of the present invention. It should be noted that, for those skilled in the art, several improvements and modifications can be made without departing from the principles of the present invention, and these improvements and modifications should also be considered as the protection scope of the present invention. The structures, devices and operating methods not specifically described and explained in the present invention are implemented according to conventional means in the art unless otherwise specified and limited.

Claims

1. A HEVC dual compression detection method based on a hybrid heterogeneous network, characterized in that: The method comprises the following steps: (1) Extract the motion alignment residual and high-frequency components of the video to generate fusion features; (2) Using the motion perception layer to divide the video into a medium-speed video subset and a low-speed or static video subset according to the video motion intensity; (3) Use a shallow convolutional neural network (CNN) to learn the differences between zero-value pixels and other pixels in the fusion feature; (4) Learning the global and local relationships of zero-value pixel locations in fusion features based on graph neural networks (GNNs); (5) Use a single classifier based on Mahalanobis distance to classify the extracted features and finally determine whether the video is double compressed.

2. The method according to claim 1, characterized in that The method for extracting the motion alignment residual in step (1) includes: The Lucas-Kanade algorithm is used to calculate the motion vectors of the two frames in the video sequence. Warping the previous frame according to the calculated motion vector to generate a motion aligned frame; Calculate the pixel-by-pixel difference between the current frame and the motion-aligned frame to obtain the motion-aligned residual.

3. The method according to claim 1, characterized in that The method for extracting high-frequency components in step (1) includes: Perform Fourier transform on the second frame and subsequent frames in the video sequence to extract high-frequency components; The motion alignment residual is multiplied pixel by pixel with the high-frequency component to generate the fused features.

4. The method according to claim 1, characterized in that: The method of dividing the motion perception layer into subsets according to the video motion intensity in step (2) includes: Calculate the motion change of the video and the amplitude strength of the motion vector; The videos are divided into a medium-speed video subset and a low-speed or static video subset.

5. The method according to claim 1, characterized in that The shallow convolutional neural network in step (3) includes five convolutional blocks, and the settings of the convolutional blocks are: Each convolution block contains a convolution layer, a BatchNorm layer, a ReLU activation function, and a Pooling layer; The Pooling layers of the first to fourth convolutional blocks use average pooling, and the fifth convolutional block uses global average pooling.

6. The method according to claim 1, characterized in that The graph neural network in step (4) is constructed in the following way: Divide the fused features into blocks of different scales (8×8 to 256×256); The number of zero-valued pixels in a block is used as the node feature to construct an undirected fully connected graph; A two-layer graph convolutional network is used to extract features and output a 6-dimensional vector representing the distribution relationship of zero-value pixels.

7. The method according to claim 1, characterized in that In step (5), the single classifier based on Mahalanobis distance uses only single compressed video as training data, and classifies by Mahalanobis distance to determine whether the input video is a double compressed video.

8. A detection system for implementing the method according to any one of claims 1 to 7, characterized in that: The system includes: a modulation module for extracting motion alignment residuals and high-frequency components and generating fusion features; A motion-aware module that partitions videos into subsets based on motion intensity; A shallow convolutional neural network module to learn the fluctuation difference between zero-value pixels and other pixels; A graph neural network module to learn the global and local relationships of fused features; A single classifier module is used to determine whether a video is double compressed based on the Mahalanobis distance.

9. The detection system according to claim 8, characterized in that: The data input of the system comes from public data sets, including video sequences in CIF format, 720P format and 1080P format. The data preprocessing uses the ffmpeg tool for encoding and compression.

Images