Video tampering detection method and system based on multi-scale features and hybrid 3D network

Abstract

The invention provides a video tampering detection method and system based on multi-scale features and a hybrid 3D network, and relates to the technical field of computer vision, and the method comprises the steps: obtaining a to-be-detected video frame, and carrying out the preprocessing of the to-be-detected video frame; feature maps of different scales are obtained through sampling processing; inputting the feature maps of different scales into a multi-scale internal cascade network, and extracting shallow features, time features and spatial features by using a shallow feature extraction module, a time feature extraction module and a spatial feature extraction module respectively; carrying out feature fusion on the shallow features, the time features and the spatial features by adopting an EMA fusion module with multi-scale features to obtain fused features; performing video frame detection by using the fused features to obtain a video tampering detection result; wherein a hybrid three-dimensional convolution structure is introduced into the time feature extraction module. According to the method, whether the video frame is subjected to illegally inserted tampering operation or not can be effectively identified.

Description

Technical Field

[0001] The present invention belongs to the field of computer vision technology, and in particular relates to a video tampering detection method and system based on multi-scale features and hybrid 3D networks. Background Art

[0002] The statements in this section merely provide background information related to the present invention and do not necessarily constitute prior art.

[0003] Video frame interpolation is a video manipulation method used to generate intermediate frames in a video sequence in order to increase the frame rate, smooth motion, or enhance visual effects. Currently, deep video frame interpolation technology is used in many fields, including film post-production, video games, virtual reality, and augmented reality, making it more efficient in generating smooth and realistic intermediate frames.

[0004] However, while it has significant advantages in improving video quality and enhancing visual experience, it may also be abused, bringing some potential risks and hazards. Therefore, it is particularly important to develop frame insertion detection technology, which can not only expose the illegal tampering of video content, but also provide key technical support for legal proceedings and content verification.

[0005] At the same time, due to the increasing demand for efficient frame insertion detection technology in society, the frames generated by modern video frame interpolation technology are becoming increasingly difficult to be detected by traditional detection methods. Currently, the methods for detecting frames generated by video frame interpolation technology mainly use technologies such as multi-scale feature extraction, spatiotemporal analysis, and hybrid three-dimensional networks.

[0006] Multi-scale feature extraction requires determining appropriate scale parameters, which usually involves the selection and adjustment of hyperparameters. When processing high-resolution images or videos, efficient algorithms and powerful hardware support are required, which may increase the complexity of the model and increase the risk of overfitting.

[0007] Spatiotemporal data are often high-dimensional, involving time series and multiple spatial dimensions. Processing and analyzing such data requires efficient algorithms to manage the complexity of the data. Maintaining temporal consistency is a challenge in videos or other dynamic scenes, especially when objects or scenes change rapidly.

[0008] The significant volume problem of 3D convolutional neural networks (3D CNNs) in practical applications has led to the invention of a hybrid convolutional network method called MC3, which combines 3D convolution and 2D convolution for spatiotemporal feature learning. Although the hybrid 3D network combines the advantages of 2D convolution and 3D convolution, its training may be more unstable, especially when using different types of convolution layers, which may cause gradient vanishing or exploding problems. Motion modeling is required in the early layers of the network structure. In contrast, motion modeling may be less critical or even ignored at higher levels of semantic abstraction.

[0009] Video frame insertion technology uses complex techniques such as generative adversarial networks to generate inserted frames that are very similar to the original video frames, thereby evading detection. This adversarial attack makes it difficult for existing detection methods to effectively identify it. Summary of the invention

[0010] In order to overcome the shortcomings of the above-mentioned prior art, the present invention provides a video tampering detection method and system based on multi-scale features and hybrid 3D networks. It aims to effectively identify whether a video frame has been illegally inserted into the tampering operation through the improvement of the hybrid 3D network architecture based on multi-scale feature extraction and the EMA feature fusion module.

[0011] To achieve the above objectives, one or more embodiments of the present invention provide the following technical solutions: A first aspect of the present invention provides a video tampering detection method based on multi-scale features and a hybrid 3D network.

[0012] A video tampering detection method based on multi-scale features and hybrid 3D network, comprising: Obtain the video frame to be detected and perform preprocessing; The preprocessed video frames are sampled to obtain feature maps of different scales; Input feature maps of different scales into the multi-scale internal cascade network, and respectively use the shallow feature extraction module, the temporal feature extraction module and the spatial feature extraction module in the multi-scale internal cascade network to extract shallow features, temporal features and spatial features; The shallow features, temporal features and spatial features are fused using an EMA fusion module of multi-scale features to obtain fused features; Use the fused features to perform video frame detection and obtain video tampering detection results; Among them, a hybrid three-dimensional convolution structure is introduced into the temporal feature extraction module, and the temporal features between multiple adjacent video frames are obtained by parallel two-dimensional convolution and three-dimensional convolution.

[0013] A second aspect of the present invention provides a video tampering detection system based on multi-scale features and a hybrid 3D network.

[0014] A video tampering detection system based on multi-scale features and hybrid 3D networks, comprising: The data acquisition module is configured to acquire the video frame to be detected and perform preprocessing; The feature map acquisition module is configured to sample the preprocessed video frames to obtain feature maps of different scales; The feature extraction module is configured to input feature maps of different scales into the multi-scale internal cascade network, and respectively extract shallow features, temporal features, and spatial features using the shallow feature extraction module, the temporal feature extraction module, and the spatial feature extraction module in the multi-scale internal cascade network; The feature fusion module is configured to fuse the shallow features, temporal features and spatial features using an EMA fusion module of multi-scale features to obtain fused features; The result detection module is configured to use the fused features to perform video frame detection and obtain video tampering detection results; Among them, a hybrid three-dimensional convolution structure is introduced into the temporal feature extraction module, and the temporal features between multiple adjacent video frames are obtained by parallel two-dimensional convolution and three-dimensional convolution.

[0015] The third aspect of the present invention provides a computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the steps in a method as described in the second aspect of the present invention are implemented.

[0016] A fourth aspect of the present invention provides a computer-readable storage medium having a program stored thereon, which, when executed by a processor, implements the steps in a method as described in the second aspect of the present invention.

[0017] A fifth aspect of the present invention provides a computer program product comprising instructions, which, when running on a computer, enables the computer program to implement the steps in a method as described in the second aspect of the present invention when executed by a processor.

[0018] One or more of the above technical solutions have the following beneficial effects: The present invention uses a multi-scale feature extraction framework to extract features of different scales from the video to capture the spatiotemporal features in the video. By extracting features at different scales, the model can capture different levels of information in the image, thereby improving the model's ability to recognize objects of different sizes.

[0019] Through the shallow feature extraction module, not only can the most relevant features in multiple scales be effectively extracted from the original dataset for training, but it can also help reduce network parameters to build a lightweight network.

[0020] Through the temporal feature extraction module, 2D convolution is used in some parts of the network to reduce the computational burden of processing the additional time dimension while still retaining the ability to capture spatial features. This not only maintains computational efficiency, but also improves the performance and generalization ability of the model.

[0021] By building a spatial feature extraction module, the network architecture can be optimized and the model efficiency can be improved through two depth-separable convolutions at three scales. It can reduce the computational load and number of parameters of the model and learn the temporal features between adjacent frames. It can also capture multi-scale and multi-directional features while maintaining fewer parameters.

[0022] The present invention introduces a hybrid network architecture in the temporal feature extraction module, combining 3D convolution and parallel grouped convolution to more effectively extract temporal features from video frames. It helps to obtain temporal features between multiple adjacent frames and reduces the computational burden of processing additional time dimensions. 3D convolution can capture spatiotemporal features at a lower level, while 2D convolution can integrate these features at a higher level. Grouped convolution allows the model to learn different features within different groups, enhancing the model's ability to represent complex spatiotemporal data. By combining with 3D convolution, the network can better capture dynamic and static features in video sequences. Due to the addition of the temporal dimension, the module can more effectively extract temporal features from video frame sequences.

[0023] The present invention realizes the effective fusion of features of different scales and performs final classification to identify forged frames through an improved efficient multi-scale attention module (EMA) technology. It enhances the model's ability to understand and express complex spatiotemporal features, which is crucial for collecting and integrating cross-spatial information from different spatial dimensions.

[0024] Advantages of additional aspects of the present invention will be given in part in the following description, and in part will become obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] The accompanying drawings in the specification, which constitute a part of the present invention, are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute improper limitations on the present invention.

[0026] Figure 1 It is the overall framework of the video tampering detection method based on multi-scale features and hybrid 3D network in the first embodiment of the present invention; Figure 2 This is a schematic diagram of an original data set in Embodiment 1 of the present invention; Figure 3 This is a schematic diagram of the EMA fusion module structure in Example 1 of the present invention. DETAILED DESCRIPTION

[0027] It should be noted that the following detailed descriptions are exemplary and are intended to provide further explanation of the present invention. Unless otherwise specified, all technical and scientific terms used herein have the same meanings as those commonly understood by those skilled in the art to which the present invention belongs.

[0028] It should be noted that the terms used herein are for describing specific embodiments only and are not intended to be limiting of exemplary embodiments according to the present invention.

[0029] In the absence of conflict, the embodiments of the present invention and the features of the embodiments may be combined with each other.

[0030] Embodiment 1 This embodiment discloses a video tampering detection method based on multi-scale features and hybrid 3D networks. In order to improve the recognition ability of illegally inserted video frames, a more robust and generalized feature extraction method is developed, and a model that can resist adversarial attacks is designed, which can more effectively identify, for example, Figure 1 As shown, the specific steps include: Step 1: Get the video frame to be detected and perform preprocessing; Step 2: Sampling the preprocessed video frames to obtain feature maps of different scales; Step 3: Input feature maps of different scales into the multi-scale internal cascade network, and use the shallow feature extraction module, temporal feature extraction module and spatial feature extraction module in the multi-scale internal cascade network to extract shallow features, temporal features and spatial features respectively; Step 4: The shallow features, temporal features and spatial features are fused using an EMA fusion module of multi-scale features to obtain fused features; Step 5: Input the fused features into the classification network to determine whether the image is tampered with by frame insertion.

[0031] In order to more clearly illustrate the present embodiment, a video tampering detection method based on multi-scale features and hybrid 3D network can be specifically described as follows: Step 1: Get the video frame to be detected and perform preprocessing.

[0032] In this embodiment, 26633 videos with an original frame rate of 25 frames per second (fps) from UCF101 and Vimeo90K are selected to construct training and test data in a ratio of 7: 1. A total of 186431 frames (26633 videos × 7 frames / video = 186431 frames) constitute the original data set.

[0033] Using two representative video frame insertion (DVFI) methods, AdaCoF and IFRNet, 7 consecutive frames are taken as a group, such as Figure 2 Therefore, a total of 35,660 frames are used in the training set and 3,760 frames are tested in the test set.

[0034] This rich and diverse dataset provides strong support for our research and enables comprehensive performance evaluation on the DVFI forensics task.

[0035] In this embodiment, the preprocessing is to randomly crop seven consecutive video frames to obtain 256×256 blocks, that is, to obtain a cropped image.

[0036] Step 2: Sampling the preprocessed video frames to obtain feature maps of different scales.

[0037] In this embodiment, specifically: the cropped image in step 1 is subjected to double upsampling and half downsampling processing to obtain a double-scale image and a half-scale image, respectively.

[0038] It can be expressed in mathematical form as follows: ; ; In the formula, F up represents the feature after double upsampling, F down represents the feature after half downsampling, F si Represents the feature map extracted at a specific scale si.

[0039] Step 3: Input feature maps of different scales into the multi-scale internal cascade network, and use the shallow feature extraction module, temporal feature extraction module and spatial feature extraction module in the multi-scale internal cascade network to extract shallow features, temporal features and spatial features respectively.

[0040] In this embodiment, the specific steps are as follows: Step 3-1: Extract shallow features using the shallow feature extraction module in the multi-scale internal cascade network.

[0041] In this embodiment, the shallow feature extraction module includes three groups of convolutions. These features are characterized through training, which can help reduce network parameters and build a lightweight network.

[0042] Among them, the convolution kernel, stride and padding of each group of convolution blocks are set to 3 × 3, 1 and 1 respectively. Each group of convolution blocks includes three operations: Batch Normalization, Relu, and group conv, i.e. batch normalization, activation function and group convolution; BN layer is used to reduce overfitting, and ReLU is used as the activation function, and no pooling operation is used.

[0043] The cropped image, the twice-scale image, and the half-scale image are respectively input into three groups of convolutions for convolution, and the feature map after the first convolution, the feature map after the second convolution, and the feature map after the third convolution are obtained.

[0044] The feature map after the first convolution is then jump-connected with the feature map after the third convolution to obtain more stable shallow features.

[0045] The specific calculation process of this module can be expressed in mathematical form as follows: ; Where W0 represents the weight of the 3 × 3 convolutional layer, represents element-wise addition, Upsample(·) and Downsample(·) represent upsampling and downsampling, respectively.

[0046] By constructing a shallow feature extraction module, it helps to reduce network parameters to build a lightweight network, and can further suppress the content and abnormal noise of the video frame; in addition, this module does not use pooling operations, which effectively avoids the weakening of the interpolation trajectory; and by extracting features at different scales, the model can capture different levels of information in the image, thereby improving the model's ability to recognize objects of different sizes.

[0047] Step 3-2: Extract temporal features using the temporal feature extraction module in the multi-scale internal cascade network.

[0048] In this embodiment, the temporal feature extraction module introduces a hybrid three-dimensional convolutional network structure, which integrates two group convolutions (2D) with 3D convolutions in parallel, and obtains the temporal features between multiple adjacent video frames by parallelizing two-dimensional convolutions and three-dimensional convolutions.

[0049] The specific extraction of time features is as follows: 1. Upsample the half-scale features obtained in step 2 to make them the same size as the original-scale feature map and then splice them; Upsample the original-scale features obtained in step 2 to make them the same size as the double-scale feature map and then splice them, and pass the two spliced ​​feature maps through a point convolution block and a 2D, 3D parallel convolution block respectively; The half-scale feature map directly passes through this convolution block consisting of a point convolution and a 2D, 3D parallel convolution.

[0050] By using a 1x1 convolution kernel to convolve each position of the feature map, the number of channels of the feature map can be effectively changed.

[0051] 2. Input the features extracted by point convolution into two group convolution modules to obtain the feature tensor of the corresponding scale.

[0052] Grouped convolution allows the model to learn different features within different groups, enhancing the model's ability to represent complex spatiotemporal data. Compared with using only 3D convolutions, this approach reduces the computational burden of processing additional time dimensions by using 2D convolutions in some parts of the network while still retaining the ability to capture spatial features.

[0053] 3. The features extracted by point convolution are then fed into the 3D convolution block. 3D convolution can capture spatiotemporal features at a lower level, concatenate the captured feature maps along the time and space dimensions, and integrate these features at a higher level.

[0054] The convolution kernel size of this 3D convolution block is 3×3×3, which has dimensions in both space and time. An additional depth dimension is added to represent the depth in the time dimension, which can capture the relationship between adjacent video frames and obtain a feature tensor containing time information. After the 2D convolution operation, a feature tensor containing spatial information is obtained. The BN layer is used to reduce overfitting, and ReLU is used as the activation function without pooling. The two feature tensors are concatenated along the channel dimension to obtain a feature map with time and space information.

[0055] 3D convolution can capture spatiotemporal features at a lower level, while 2D convolution can integrate these features at a higher level. Grouped convolution allows the model to learn different features within different groups, enhancing the model's ability to represent complex spatiotemporal data.

[0056] The time feature extraction module can be expressed in mathematical form as: ; Where W0 and W1 represent the weights of the 3 × 3 convolutional layer, and W2 represents the weight of the 3 × 3 × 3 convolutional layer.

[0057] By combining 2D with 3D convolutions, building a hybrid 3D convolutional network can better capture dynamic and static features in video sequences. Due to the addition of the time dimension, this module can more effectively extract temporal features from video frame sequences. This approach not only maintains computational efficiency, but also improves the performance and generalization ability of the model.

[0058] Step 3-2: Extract spatial features using the spatial feature extraction module in the multi-scale internal cascade network.

[0059] In this embodiment, the enhanced features obtained after the previous two modules are passed through three groups of sub-networks to estimate the pixel-level parameters of each target pixel on input frames of different scales.

[0060] The spatial feature extraction module contains three convolution operations to extract spatial features between adjacent frames at three scales. That is, for each scale, each convolution operation includes a point convolution and two depth-wise separable convolutions. Point convolution combines feature maps along the channel direction and mixes the features at each position to aggregate temporal features. Depth-wise separable convolution is a combination of depth-wise convolution and point-wise convolution, which is used to reduce the computational load and number of parameters of the model and learn temporal features between adjacent frames. This enables our model to capture multi-scale and multi-directional features while maintaining fewer parameters.

[0061] The specific steps are as follows: 1. Upsample the half-scale features obtained in step 2 to make them the same size as the original-scale feature map and then splice them; Upsample the original-scale features obtained in step 2 to make them the same size as the double-scale feature map and then splice them. The two sets of spliced ​​feature maps are respectively passed through a point convolution block and two depth-wise separable convolution blocks; The half-scale feature map is directly passed through this convolution block consisting of a point convolution and two depth-wise separable convolutions.

[0062] 2. The double-scale features, half-scale features and original-scale features obtained by the previous module are first passed through a point-by-point convolution with a stride of 2, a padding of 1, and a kernel size of 3 × 3 to obtain the multi-scale features after convolution. We use it to combine the feature maps in the channel direction, mix the features at each position point by point, and aggregate the temporal features.

[0063] It is able to mix features in the depth dimension while maintaining the resolution in the spatial dimension.

[0064] The channel direction refers to the depth of the feature map, that is, the number of channels of the feature map. It contains the feature representation of the input data at different levels. Each channel can be regarded as a filter that captures different features in the input data.

[0065] 3. The convolved multi-scale features are passed through two depth-wise separable convolution blocks. Depth-wise separable convolution is a combination of depth-wise convolution and point-wise convolution. The point convolution, stride, and padding are 3×3, 1, and 0, respectively.

[0066] A convolution kernel is applied independently to each input channel of the depth convolution, which keeps the number of channels of the input feature map unchanged, and the spatial features of each channel are extracted and encoded independently. Point-by-point convolution can be regarded as a linear transformation in the channel dimension, which creates a new feature channel by weighting and combining the features of each channel from the depth convolution. A BN layer is used between the two convolutions to reduce overfitting, and ReLU is used as the activation function. After two depth-separable convolution modules, a feature map with spatial information is obtained.

[0067] This module can be expressed mathematically as: ; Where W0 is the weight of the 3 × 3 depthwise separable convolutional layer, and W1 is the weight of the 3 × 3 convolutional layer.

[0068] Step 4: The shallow features, temporal features and spatial features are fused using the multi-scale feature EMA fusion module to obtain fused features.

[0069] In this embodiment, the fusion module adopts an improved efficient multi-scale attention module (EMA), and the EMA module is added to the end of the overall framework without changing the size of the feature vector.

[0070] 1. For any given input feature map X∈R C×H×W , EMA divides X into G sub-features in the channel dimension to learn different semantics; the grouping can be expressed as X = [X0, X1, …X G−1 ], X i ∈R C / / G×H×W .

[0071] Two tensors are introduced, one of which is the output of the 1 × 1 branch and the other is the output of the 5 × 5 branch. Three parallel paths are used to extract the attention weight descriptor of the grouped feature map. In order to better collect multi-scale spatial information, we change the original 3 × 3 branch to a 5 × 5 branch, increasing the capacity of the model.

[0072] Among them, C×H×W is the size of the feature map tensor input to EMA after the previous series of convolutions, which represents a feature map with C channels, H heights, and W widths. X represents the feature map itself, and R represents a set of real numbers, meaning that the data is real-valued.

[0073] EMA groups the number of channels of the input feature map to aggregate information in different spatial dimensions, thereby extracting richer feature representations. The channel dimension C is divided into G groups, each containing C / G channels. Here G is a hyperparameter, which means that the number of channels is evenly divided into G groups. Therefore, the shape of the grouped feature map becomes (C / G)×H×W.

[0074] The above C / G channels are divided into two parts, R1 and R3, where R1 and R3 are two factors of C / G. We need to reorganize the channel dimension C / G into two new dimensions R1 and R3, so that this reorganized feature map can be used in subsequent operations to capture spatial features at different scales and enhance the model's perception of key information through the attention mechanism.

[0075] 2. Use 2D global average pooling to encode the global spatial information in the 1 × 1 branch output, and directly convert the output of the smallest branch into the shape of the corresponding dimension before the channel feature joint activation mechanism; that is, R1 1×C / / G × R3 C / / G×HW .

[0076] The formula for the 2D global pooling operation is: ; In the formula, Z c represents the output associated with the Cth channel. Its purpose is to encode global information and model long-term dependencies.

[0077] Step 5: Input the fused features into the classification network to determine whether the image is tampered with by frame insertion.

[0078] In this embodiment, the details are as follows: the extracted multi-scale features are input into the classification network, and a multi-layer perceptron composed of multiple fully connected layers is used to perform binary classification. Finally, an output unit is included, and a sigmoid activation function is used to output the probability of belonging to two categories.

[0079] Embodiment 2 The purpose of this embodiment is to provide a video tampering detection system based on multi-scale features and hybrid 3D networks, including: The data acquisition module is configured to acquire the video frame to be detected and perform preprocessing; The feature map acquisition module is configured to sample the preprocessed video frames to obtain feature maps of different scales; The feature extraction module is configured to input feature maps of different scales into the multi-scale internal cascade network, and respectively extract shallow features, temporal features, and spatial features using the shallow feature extraction module, the temporal feature extraction module, and the spatial feature extraction module in the multi-scale internal cascade network; The feature fusion module is configured to fuse the shallow features, temporal features and spatial features using an EMA fusion module of multi-scale features to obtain fused features; The result detection module is configured to use the fused features to perform video frame detection and obtain video tampering detection results; Among them, a hybrid three-dimensional convolution structure is introduced into the temporal feature extraction module, and the temporal features between multiple adjacent video frames are obtained by parallel two-dimensional convolution and three-dimensional convolution.

[0080] Embodiment 3 The purpose of this embodiment is to provide a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above method when executing the program.

[0081] Embodiment 4 The purpose of this embodiment is to provide a computer-readable storage medium.

[0082] A computer-readable storage medium stores a computer program, which executes the steps of the above method when executed by a processor.

[0083] Embodiment 5 The purpose of this embodiment is to provide a computer program product containing instructions, which, when running on a computer, enables the computer to execute the methods and functions involved in any of the above embodiments. The steps involved in the apparatus of the above embodiment correspond to the method embodiment 1, and the specific implementation method can refer to the relevant description part of embodiment 1. The term "computer-readable storage medium" should be understood as a single medium or multiple media including one or more instruction sets; it should also be understood to include any medium that can store, encode or carry an instruction set for execution by a processor and enable the processor to execute any method in the present invention.

[0084] Those skilled in the art should understand that the modules or steps of the present invention described above can be implemented by a general-purpose computer device, or alternatively, they can be implemented by a program code executable by a computing device, so that they can be stored in a storage device and executed by the computing device, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. The present invention is not limited to any specific combination of hardware and software.

[0085] Although the above describes the specific implementation mode of the present invention in conjunction with the accompanying drawings, it is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art on the basis of the technical solution of the present invention without creative work are still within the scope of protection of the present invention.

Claims

1. A video tampering detection method based on multi-scale features and hybrid 3D networks, characterized in that: include: Obtain the video frame to be detected and perform preprocessing; The preprocessed video frames are sampled to obtain feature maps of different scales; Input feature maps of different scales into the multi-scale internal cascade network, and respectively use the shallow feature extraction module, the temporal feature extraction module and the spatial feature extraction module in the multi-scale internal cascade network to extract shallow features, temporal features and spatial features; The shallow features, temporal features and spatial features are fused using an EMA fusion module of multi-scale features to obtain fused features; Use the fused features to perform video frame detection and obtain video tampering detection results; Among them, a hybrid three-dimensional convolution structure is introduced into the temporal feature extraction module, and the temporal features between multiple adjacent video frames are obtained by parallel two-dimensional convolution and three-dimensional convolution.

2. The video tampering detection method based on multi-scale features and hybrid 3D network as claimed in claim 1, characterized in that: The preprocessing is to randomly crop the video frame to obtain a cropped image; the preprocessed video frame is sampled to obtain feature maps of different scales, specifically, the cropped image is upsampled by two times and downsampled by one half to obtain a twice-scale image and a half-scale image, respectively.

3. The video tampering detection method based on multi-scale features and hybrid 3D network as claimed in claim 1, characterized in that: The shallow feature extraction module includes three groups of convolutions. The shallow feature extraction module in the multi-scale internal cascade network is used to input feature maps of different scales into the three groups of convolutions for convolution, and obtain the feature map after the first convolution, the feature map after the second convolution, and the feature map after the third convolution. Then, the feature map after the first convolution is jump-connected with the feature map after the third convolution to obtain shallow features; Among them, the convolution kernel, stride and padding of each group of convolution blocks are set to 3 × 3, 1 and 1 respectively. Each group of convolution blocks includes three operations: batch normalization, activation function and group convolution; BN layer is used to reduce overfitting, and ReLU is used as the activation function without pooling operation.

4. The video tampering detection method based on multi-scale features and hybrid 3D network as claimed in claim 1, characterized in that: The temporal feature extraction module is a hybrid three-dimensional convolutional network structure; the network integrates two group convolutions and 3D convolutions in parallel, and obtains the temporal features between multiple adjacent video frames by parallelizing two-dimensional convolutions and three-dimensional convolutions; The temporal feature extraction module is used to extract the temporal feature, specifically: the spliced ​​feature map is passed through a point convolution block and a 2D and 3D parallel convolution block respectively; the features extracted by the point convolution are obtained; the features extracted by the point convolution are input into two group convolution modules and a 3D convolution module respectively, and two feature tensors of corresponding scales are obtained; the two feature tensors of corresponding scales are spliced ​​along the channel dimension to obtain a feature map with temporal and spatial information.

5. The video tampering detection method based on multi-scale features and hybrid 3D network as claimed in claim 1, characterized in that: The spatial feature extraction module includes three convolution operations for extracting spatial features between adjacent frames at three scales; the three convolution operations include a point convolution block and two depth-separable convolution blocks.

6. The video tampering detection method based on multi-scale features and hybrid 3D network as claimed in claim 1, characterized in that: The EMA fusion module using multi-scale features performs feature fusion, and the specific steps are as follows: For any given input feature map, the EMA fusion module divides X into G sub-features in the channel dimension direction; Two tensors are introduced, one of which is the output of the 1 × 1 branch and the other is the output of the 5 × 5 branch, and three parallel paths are used to extract the attention weight descriptor of the grouped feature map; Two-dimensional global average pooling is used to encode the global spatial information in the 1 × 1 branch output, and the output of the smallest branch is directly converted to the shape of the corresponding dimension before the channel feature joint activation mechanism.

7. A video tampering detection system based on multi-scale features and hybrid 3D networks, characterized in that: include: The data acquisition module is configured to acquire the video frame to be detected and perform preprocessing; A feature map acquisition module is configured to sample the preprocessed video frames to obtain feature maps of different scales; The feature extraction module is configured to input feature maps of different scales into the multi-scale internal cascade network, and respectively extract shallow features, temporal features, and spatial features using the shallow feature extraction module, the temporal feature extraction module, and the spatial feature extraction module in the multi-scale internal cascade network; The feature fusion module is configured to fuse the shallow features, temporal features and spatial features using an EMA fusion module of multi-scale features to obtain fused features; The result detection module is configured to use the fused features to perform video frame detection and obtain video tampering detection results; Among them, a hybrid three-dimensional convolution structure is introduced in the time feature extraction module, and the time features between multiple adjacent video frames are obtained by parallel two-dimensional convolution and three-dimensional convolution.

8. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the steps of the method described in any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the method described in any one of claims 1 to 6 are performed.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

Images