Video frame deletion forensics method based on image copy movement features

Abstract

The present application proposes a video frame deletion forensics method based on image copy-move feature, which uses the similarity between ghost features in video frame deletion tampering residual error map and image copy-move tampering features to conduct video frame deletion forensics. First, a Multi-PyRes model is designed, and the model is pre-trained on an image copy-move tampering dataset to learn the features of copy-move tampering. Then, the pre-trained model parameters are migrated to the frame deletion detection task. The method extracts copy-move features from the saliency residual feature sequence map of the frame deletion video by introducing a constraint convolution layer that can learn copy-move features, and uses a residual pyramid convolution layer structure combined with a CBAM attention mechanism to extract copy-move tampering features at multiple scales, thereby realizing video frame deletion tampering detection. The present application extracts copy-move tampering features similar to ghost features of frame deletion tampering at multiple scales for image classification, which can realize video frame deletion tampering detection.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of visual media information security, and particularly to a video frame deletion forensics method based on image copy-move features. BACKGROUND

[0002] Frame deletion tampering of digital video refers to the deletion of certain frames in a video, which can be continuous or discontinuous, to conceal specific content in the video. Typically, these deleted frames may contain important information or segments with special meaning in the video. The emergence of video frame deletion tampering technology poses a challenge to the authenticity and integrity of the video. Therefore, the development of effective forensics methods is crucial for ensuring the credibility of video content and protecting personal privacy.

[0003] Currently, the detection method for video tampering is mainly passive forensics technology. Passive forensics technology relies on the inherent features of video data to restore the editing and tampering process, and does not depend on actively embedded information. Because the traces of video tampering may still remain in the video frames, they provide clues for video forensics.

[0004] Existing frame deletion detection models based on deep neural networks are usually obtained by transferring image classification task models. However, this method has a defect, i.e., it lacks targeted extraction of frame deletion tampering features. How to extract unique features formed by frame deletion tampering is still an important research direction in this field. SUMMARY

[0005] The present application proposes a video frame deletion forensics method based on image copy-move features, which extracts the ghost features similar to image copy-move tampering features existing in the video frame deletion point residual graph in multiple scales, and can realize video frame deletion tampering detection.

[0006] In the technical solution of the present application, the applicant has found that the features in the residual graph of video frame deletion tampering are similar to the image copy-move tampering features. Specifically:

[0007] The "ghost" feature appears on the residual graph of video frame deletion tampering. The reason is that the temporal continuity in the video frame sequence is destroyed. In a normal video, the time interval between adjacent frames is very small, so the inter-frame change is not obvious, and the content difference displayed by the residual graph (the residual feature graph used in the present application is obtained by calculating the difference between the pixels at corresponding positions in adjacent frames) is also not obvious. When there is frame deletion operation, the content change between adjacent frames suddenly increases. These mutations will appear as obvious double contours on the residual graph, i.e., a "ghost" effect is formed.

[0008] The copy-move tampered image also has a ghosting feature. In the process of copy-move tampering, a certain area in the image is copied and pasted to another position of the same image. In this process, the original area remains unchanged, and the copied area appears in the new position, forming two identical or similar image contents, also forming a ghosting effect.

[0009] The technical scheme is specifically as follows.

[0010] The video frame deletion forensics method based on the image copy-move feature uses the similarity between the ghosting feature in the video frame deletion tampering residual error image and the copy-move tampering feature to perform video frame deletion forensics, specifically as follows: first, a Multi-PyRes model is designed, and the model is pre-trained on an image copy-move tampering dataset to learn the copy-move tampering feature; then, the pre-trained model parameters are migrated to the frame deletion detection task; the method extracts the copy-move feature from the saliency residual feature sequence image of the frame deletion video by introducing a constraint convolution layer capable of learning the copy-move feature, and uses a residual pyramid convolution layer structure combined with a CBAM attention mechanism to extract the copy-move tampering feature in multiple scales, thereby realizing video frame deletion tampering detection.

[0011] The method comprises the following steps:

[0012] Step S1, pre-training the Multi-PyRes model combined with the constraint convolution layer and the PyRes_CBAM on the image copy-move tampering dataset;

[0013] Step S2, decoding the video to be detected into an RGB image and converting it into a residual feature map sequence;

[0014] Step S3, saliency of the residual feature map sequence using a feature enhancement algorithm;

[0015] Step S4, inputting the saliency residual feature map into the constraint convolution layer to obtain the copy-move feature, and then superimposing the feature and the original saliency residual feature using an early fusion method;

[0016] Step S5, processing the superimposed feature obtained in step S4 through the PyRes_CBAM network to gradually extract the abstract representation of the copy-move feature;

[0017] Step S6, generating the final prediction result by a neural network containing two fully connected layers using a Softmax function to determine whether the frame is a frame deletion point, thereby realizing video frame deletion tampering detection.

[0018] The ghost feature in the video frame deletion tampering residual error map has similarity with the image copy movement tampering feature, and the step S1 is specifically implemented as follows: the Multi-PyRes model is pre-trained on an image copy movement tampering dataset for not less than 10 rounds, and the model parameters are iterated in the direction of extracting copy movement features through pre-training; and the present application is pre-trained for 10 rounds.

[0019] The step S2 is specifically implemented as follows:

[0020] Step S21, decoding the video sample to be detected into n video frame RGB images;

[0021] Step S22, converting the RGB images into a corresponding pixel residual error sequence of adjacent frames, and the calculation of the pixel residual error feature at the corresponding position between video frames is shown in formula (1).

[0022] R k (i,j)=F k (i,j)-F k-1 (i,j); (1)

[0023] Wherein, F k represents the gray pixel matrix of the Kth frame in the video sequence, i and j are the sequence numbers of the pixel row and column, and R represents the inter-frame pixel residual error matrix;

[0024] Step S23, resizing the residual error map R to 256*256 pixels.

[0025] The step S3 is specifically implemented as follows:

[0026] Step S31, first smoothing and expanding the original residual error map according to formula (2);

[0027]

[0028] Wherein, R is the original inter-frame pixel residual error matrix calculated by formula (1), b is a parameter representing the threshold value of the smoothing processing, and S represents the smoothed and expanded residual error image; formula (2) smoothes the original residual error map, suppresses the noise caused by irregular light changes and object shaking, and retains the main frame deletion tampering traces;

[0029] Then, an adaptive algorithm is used to expand the residual error, further expands the value of the large deformation area caused by tampering, and sets the value still lower than the threshold value after expansion to 0, so as to significantly distinguish the tampering area from the noise area through binarization processing; the parameter b is an empirical threshold value, which needs to avoid being set too high to cause the tampering traces to disappear, and also needs to avoid being set too low to cause the noise influence to be unable to be eliminated, and the parameter b in the present application is set to 100;

[0030] Step S32, the residual image after smoothing expansion is locally binarized by formula (3).

[0031]

[0032] Wherein, T represents the residual feature map after local binarization, S(i, j) less than the empirical threshold Bound is set to 0, that is, the area with small inter-frame change is set to 0; S(i, j) greater than or equal to 255 is set to a smaller number 1, which is used to make the significant inter-frame difference area caused by frame deletion form greater contrast with adjacent areas; the main purpose of this processing is to highlight the main inter-frame change caused by frame deletion.

[0033] The binarization threshold Bound is selected in a small range to avoid too high threshold causing too much information loss, or too low threshold failing to remove noise; the parameter Bound of the embodiment of the present application is 60;

[0034] Step S33, the residual image is further smoothed by median filtering, and the calculation method is shown in formula (4);

[0035]

[0036] Wherein, A represents the filter window centered on (i, j), the median filter window A of the embodiment of the present application is set to 3x3, and Med represents the median of the elements in the window area; the median filter is used to nonlinearly smooth the significant area of tampering traces and noise and other areas, so as to further eliminate residual noise and highlight strong frame deletion tampering traces.

[0037] The step S4 includes the following steps:

[0038] Step S41, the significant residual feature map sequence is convolved with three different 5x5 constraint convolution filters with a step of 1 to generate a predicted residual feature map with a size of 252x252x3, which is used to suppress the content of the image and provide low-level copy-move forensic features for the subsequent steps.

[0039] The filter of the constraint convolution layer estimates the prediction error feature between the center pixel and its adjacent local pixels to generate a feature map by performing the constraint condition of formula (5) below, which corresponds to the prediction pixel error and is used as a low-level forensic trace to adaptively learn the copy-move tampering traces in the image.

[0040]

[0041] Wherein, W k represents the kth filter in the first convolution layer, W k(l, m) represents the filter weight at the (l, m) position, i.e., the filter weight at the center position; in the constraint convolution layer, the initial convolution kernel weight is randomly selected for initialization; this constraint ensures that the center value of each convolution kernel is normalized to -1, and the sum of the values of all other learnable parameters is limited to 1, so that the output of the network layer is a constant value, and the constant value is calculated by the following formula (6), and the constant is the weighted sum of the pixel value and the residual value of the field;

[0042]

[0043] Step S42, superimpose the copied moving feature obtained in step S41 and the original saliency residual feature using an early fusion method.

[0044] The step S5 specifically includes the following steps:

[0045] Step S51, input the superimposed feature map obtained in step S4 into PyRes_CBAM, first pass through a 7x7 convolution with a step size of 2, and then pass through batch normalization and ReLU activation function;

[0046] Step S52, along the network, enter the first stage of PyRes_CBAM, which is composed of three Bottlenecks with the same structure, wherein the Bottleneck first adopts a 1x1 convolution, and batch normalization and ReLU activation function are applied; next, PyConv with kernel sizes of 9x9, 7x7, 5x5 and 3x3 is used, and then batch normalization and ReLU activation function are applied, followed by 1x1 convolution and batch normalization, and the feature information at different scales is integrated to obtain multi-scale feature representation. The multi-scale features are sent to the CBAM attention mechanism, i.e., the multi-scale features are sequentially weighted by the CAM module and the SAM module to obtain the output features. The overall calculation process is shown in the following formulas (7) and (8).

[0047]

[0048]

[0049] wherein F represents the input image feature, represents element-wise multiplication; the CAM module is composed of a global average pooling layer, a maximum pooling layer and a multilayer perceptron (MLP) fully connected layer, and the channel attention calculation is performed by the following formula (9); the SAM module is composed of a group of convolution layers and a Sigmoid function, and the spatial attention calculation is performed by formula (10);

[0050] MCAM (F) = σ(MLP(AvgPool(F)) + MLP(MaxPool(F))) (9);

[0051] M SAM (F) = σ(f 7×7 (AvgPool(F); MaxPool(F)])) (10);

[0052] σ in formula (9), formula (10) represents a Sigmoid function, f 7×7 represents a convolution operation with a filter size of 7x7;

[0053] Finally, the output of the CBAM adjusted feature map is connected in residual with the input feature, fused through an addition operation, and then a ReLU nonlinear activation function is applied; the remaining Bottleneck uses the same structure as above;

[0054] Step S53, enter the second stage of PyRes_CBAM, which is composed of four Bottlenecks with the same structure, wherein the Bottleneck first adopts 1x1 convolution, and batch normalization and ReLU activation function are applied; next, PyConv with kernel sizes of 7x7, 5x5 and 3x3 is used, followed by batch normalization and ReLU activation function, and then 1x1 convolution and batch normalization are applied, and the feature information under different scales is integrated to obtain a multi-scale feature representation; the multi-scale feature is sent to the CBAM attention mechanism, and the multi-scale feature is sequentially weighted through the CAM module and the SAM module to obtain an output feature; the output feature is connected in residual with the input feature, fused through an addition operation, and then a ReLU nonlinear activation function is applied; the remaining Bottleneck uses the same structure as above;

[0055] Step S54, enter the third stage of PyRes_CBAM, which is composed of six Bottlenecks with the same structure, wherein the Bottleneck first adopts 1x1 convolution, and batch normalization and ReLU activation function are applied; next, PyConv with kernel sizes of 5x5 and 3x3 is used, followed by batch normalization and ReLU activation function, and then 1x1 convolution and batch normalization are applied, and the feature information under different scales is integrated to obtain a multi-scale feature representation; the multi-scale feature is sent to the CBAM attention mechanism, and the multi-scale feature is sequentially weighted through the CAM module and the SAM module to obtain an output feature; the output feature is connected in residual with the input feature, fused through an addition operation, and then a ReLU nonlinear activation function is applied. The remaining Bottleneck uses the same structure as above;

[0056] Step S55, enter the fourth stage of PyRes_CBAM, the fourth stage is composed of three Bottleneck structures, wherein the Bottleneck is first applied with 1x1 convolution, and batch normalization and ReLU activation function are applied; next is to use PyConv with a kernel size of 3x3, followed by batch normalization and ReLU activation function, and then apply 1x1 convolution and batch normalization, and the feature information at different scales is integrated to obtain a multi-scale feature representation; the multi-scale feature is sent to the CBAM attention mechanism, that is, the multi-scale feature is sequentially weighted through the CAM module and the SAM module to obtain an output feature; the output feature is then connected with the input feature through residual connection, and the two are fused through an addition operation, and then a ReLU nonlinear activation function is applied; the remaining Bottleneck uses the same structure as above;

[0057] Step S56, in order to reduce the size and the number of parameters of the feature map, an 8x8 average pooling layer with a step size of 1 is used to downsample the feature map, and an abstract representation of the copy-move feature is gradually extracted.

[0058] The step S6 is specifically implemented by the following steps:

[0059] Step S61, the abstract representation of the copy-move feature obtained in step S5 is passed through a fully connected layer containing two neurons, and according to the similarity between the copy-move tampered feature and the frame deletion tampered feature, a Softmax function is used to output the probability that the frame is a frame deletion point, and if the output classification probability score is greater than 0.5, the picture is considered to be a frame deletion point, and the number of frame deletion points is increased by 1.

[0060] Step S62, the number of frame deletion points is detected, and if the frame deletion point is equal to 0, it means that there is no frame deletion point, and the video is an original video; if the frame deletion point is greater than 0, it means that the video is tampered with frame deletion at some place, thereby realizing frame deletion tampering detection of the video.

[0061] Compared with the prior art, the present application has the following beneficial effects:

[0062] (1) The present application finds that the ghost feature in the video frame deletion tampered residual image has a certain similarity with the image copy-move tampered feature, and the parameters trained on the copy-move tampered dataset are migrated to the frame deletion detection task, which improves the extraction ability of the frame deletion tampered feature, thereby better adapting to video detection in various complex scenes and improving the accuracy and robustness of video frame deletion detection.

[0063] (2) The application uses the residual feature highlighted by the processing methods such as targeted smooth expansion, local binaryzation and median filtering, which can highlight the "ghost" area caused by frame deletion tampering, so that the neural network can focus on learning the tampered area, improve the network classification confidence and detection accuracy, and the input of the residual image can accelerate the network convergence speed and reasoning speed, which can effectively compress the data amount of the forensic data set.

[0064] (3) The application designs a network model for extracting copy-move features. The model extracts the tampered features of copy-move by pre-learning the constraint convolution layer of copy-move features, and reduces the interference of non-tampered traces on information. The model combines residual multi-scale structure and attention mechanism to extract more robust copy-move features, effectively improves the sensitivity to frame deletion tampering features, and improves the detection performance. BRIEF DESCRIPTION OF DRAWINGS

[0065] The application will be further described in detail below in combination with the drawings and specific embodiments:

[0066] ATTACHMENT Figure 1 is a comparison diagram of the "ghost feature" similarity in the frame deletion tampered video highlighted residual image and the copy-move tampered image;

[0067] ATTACHMENT Figure 2 is a general flowchart of the application;

[0068] ATTACHMENT Figure 3 is a network architecture diagram of the application;

[0069] ATTACHMENT Figure 4 is a Bottleneck structure diagram of the first stage in the PyRes_CBAM network of the application. DETAILED DESCRIPTION

[0070] As shown in the figure, the video frame deletion forensic method based on image copy-move features, the method uses the similarity of the ghost feature in the video frame deletion tampered residual image and the image copy-move tampered feature to perform video frame deletion forensic, specifically: first, design a Multi-PyRes model, pre-train the model on the image copy-move tampered data set to learn the copy-move tampered features; then, migrate the pre-trained model parameters to the frame deletion detection task; the method extracts the copy-move features from the highlighted residual feature sequence image of the frame deletion video by introducing the constraint convolution layer that can learn the copy-move features, and uses the residual pyramid convolution layer structure combined with the CBAM attention mechanism to extract the copy-move tampered features in multiple scales, so as to realize video frame deletion tampering detection.

[0071] The method comprises the following steps:

[0072] Step S1, pre-training the Multi-PyRes model combined with the constraint convolution layer and the PyRes_CBAM on the image copy-move tampering dataset;

[0073] Step S2, decoding the video to be detected into an RGB image and converting it into a residual feature map sequence;

[0074] Step S3, using a feature enhancement algorithm to highlight the residual feature map sequence;

[0075] Step S4, inputting the highlighted residual feature map into the constraint convolution layer to obtain a copy-move feature, and then superimposing the feature and the original highlighted residual feature using an early fusion method;

[0076] Step S5, processing the superimposed feature obtained in step S4 through the PyRes_CBAM network to gradually extract an abstract representation of the copy-move feature;

[0077] Step S6, generating a final prediction result by using a Softmax function through a neural network containing two fully connected layers, judging whether the frame is a deletion point, and thus realizing video deletion tampering detection.

[0078] As shown in Figure 1 , the ghost feature in the video deletion tampering residual map has certain similarity with the image copy-move tampering feature, so the present application provides a video deletion forensic method based on the image copy-move feature, as shown in Figure 2 , the detection method specifically includes the following steps:

[0079] The ghost feature in the video deletion tampering residual map has similarity with the image copy-move tampering feature, and the specific implementation of step S1 is: pre-training the Multi-PyRes model on the image copy-move tampering dataset for no less than 10 rounds, and through the pre-training, the model parameters are iterated in the direction of extracting the copy-move feature; the present application pre-trains for 10 rounds.

[0080] The specific implementation of step S2 includes the following steps:

[0081] Step S21, decoding the video sample to be detected into n video frame RGB images;

[0082] Step S22, converting the RGB image into a corresponding pixel residual sequence of adjacent frames, and the calculation of the corresponding position pixel residual feature between video frames is shown in formula (1).

[0083] R k (i, j) = F k (i, j) - F k-1 (i, j); (1)

[0084] wherein F k represents the gray pixel matrix of the Kth frame in the video sequence, i and j are the serial numbers of the pixel row and column, and R represents the inter-frame pixel residual matrix;

[0085] Step S23, resize the residual map R to 256x256 pixels.

[0086] The step S3 specifically includes the following steps:

[0087] Step S31, first smooth and expand the original residual map using formula (2);

[0088]

[0089] wherein R is the original inter-frame pixel residual matrix calculated by formula (1), parameter b represents the threshold value of the smoothing processing, and S represents the smoothed and expanded residual image; formula (2) can suppress the noise caused by irregular light changes and object shaking by smoothing the original residual map, and retain the main frame deletion tampering traces;

[0090] Then, an adaptive algorithm is used to expand the residual, further expand the values of the large deformation area caused by tampering, and set the values still lower than the threshold value after expansion to 0, so as to significantly distinguish the tampering area from the noise area through the binarization processing; parameter b is an empirical threshold value, which needs to avoid being set too high to cause the tampering traces to disappear, and also needs to avoid being set too low to cause the noise influence to be unable to be eliminated, and the parameter b of the present application is set to 100;

[0091] Step S32, locally binarize the smoothed and expanded residual image using formula (3).

[0092]

[0093] wherein T represents the locally binarized residual feature map, S(i, j) less than the empirical threshold value Bound is set to 0, that is, the area with small inter-frame changes is set to 0; and S(i, j) greater than or equal to 255 is set to a smaller number 1, which is used to make the significant inter-frame difference area caused by frame deletion form a greater contrast with the adjacent area; the main purpose of this processing is to highlight the main inter-frame changes caused by frame deletion.

[0094] The binarization threshold value Bound is selected in a small range, so as to avoid that a too high threshold value will cause too much information to be lost, or a too low threshold value will be unable to remove the noise; the parameter Bound of the example of the present application is valued at 60;

[0095] Step S33, further smooth the residual image using the median filter, and the calculation mode is shown in formula (4).

[0096]

[0097] Wherein, A represents the filter window centered at the (i, j) point, the median filter window A of the embodiment of the application is set to 3x3, and Med represents the median value of the elements in the window area; the median filter is used to perform nonlinear smoothing on the significant area of the tampering trace and noise and other areas, so as to further eliminate residual noise and highlight the stronger frame deletion tampering trace.

[0098] The specific implementation of the step S4 includes the following steps:

[0099] Step S41, convolve the significant residual feature map sequence with three different 5x5 constraint convolution filters with a step size of 1 to generate a predicted residual feature map with a size of 252x252x3, which is used to suppress the content of the image and provide low-level copy-move forensic features for subsequent steps.

[0100] The filter of the constraint convolution layer estimates the prediction error feature between the center pixel and its adjacent local pixels to generate a feature map by performing the constraint condition of the following formula (5), which corresponds to the prediction pixel error, as a low-level forensic trace, from which the copy-move tampering trace in the image is adaptively learned.

[0101]

[0102] Wherein, W k represents the kth filter in the first convolution layer, W k (l, m) represents the filter weight at the (l, m) position, i.e., the filter weight at the center position; in the constraint convolution layer, the initial convolution kernel weight is randomly selected for initialization; this constraint condition ensures that the center value of each convolution kernel is normalized to -1, and the sum of the values of all other learnable parameters is limited to 1, so that the output of the network layer is a constant value, and the constant value is calculated by the following formula (6), and the constant is the weighted sum of the pixel value and the residual value of the field;

[0103]

[0104] As Figure 3 shown, the copy-move features and the original significant residual features are superimposed using an early fusion method.

[0105] Step S5, the abstract representation of the copy-move features is gradually extracted by the PyRes_CBAM network:

[0106] Step S42, the copy-move features obtained in step S41 are superimposed with the original significant residual features using an early fusion method.

[0107] The step S5 has the following implementation modes:

[0108] As shown in Figure 3 Step S51, the superimposed feature map obtained in step S4 is input into PyRes_CBAM, which first passes through a 7x7 convolution with a step size of 2, and then passes through batch normalization and a ReLU activation function.

[0109] Step S52, along the network, enter the first stage of PyRes_CBAM, which is composed of three Bottlenecks with the same structure. The Bottleneck first adopts a 1x1 convolution, and applies batch normalization and a ReLU activation function. Next, a PyConv with kernel sizes of 9x9, 7x7, 5x5, and 3x3 is used, followed by batch normalization and a ReLU activation function, and then a 1x1 convolution and batch normalization are applied to integrate feature information at different scales to obtain multi-scale feature representation. The multi-scale features are input into the CBAM attention mechanism, i.e., the multi-scale features are sequentially weighted by the CAM module and the SAM module to obtain the output features. The overall calculation process is shown in the following formula (7) and formula (8).

[0110]

[0111]

[0112] wherein F represents the input image feature, denotes element-wise multiplication; the CAM module is composed of a global average pooling layer, a maximum pooling layer, and a multilayer perceptron (MLP) fully connected layer, and the channel attention calculation is performed by the following formula (9); the SAM module is composed of a group of convolution layers and a Sigmoid function, and the spatial attention calculation is performed by formula (10).

[0113] M CAM (F)=σ(MLP(AvgPool(F))+MLP(MaxPool(F))) (9);

[0114] M SAM (F)=σ(f 7×7 (AvgPool(F);MaxPool(F)])) (10);

[0115] In formula (9) and formula (10), σ represents the Sigmoid function, f 7×7 denotes a convolution operation with a filter size of 7x7;

[0116] Finally, the output of the CBAM adjusted feature map is connected with the input feature in residual connection, fused through addition operation, and then a ReLU nonlinear activation function is applied; the rest of the Bottleneck uses the same structure as above;

[0117] Step S53, enter the second stage of PyRes_CBAM, which is composed of four Bottlenecks with the same structure. The Bottleneck first adopts 1x1 convolution, and batch normalization and ReLU activation function are applied; next, PyConv with kernel size of 7x7, 5x5 and 3x3 is used, followed by batch normalization and ReLU activation function, and then 1x1 convolution and batch normalization are applied to obtain multi-scale feature representation by integrating feature information at different scales; the multi-scale features are sent to the CBAM attention mechanism, and the output features are obtained after the multi-scale features pass through the CAM module and the SAM module in turn; the output features are connected with the input features in residual connection, fused through addition operation, and then a ReLU nonlinear activation function is applied. The rest of the Bottleneck uses the same structure as above;

[0118] Step S54, enter the third stage of PyRes_CBAM, which is composed of six Bottlenecks with the same structure. The Bottleneck first adopts 1x1 convolution, and batch normalization and ReLU activation function are applied; next, PyConv with kernel size of 5x5 and 3x3 is used, followed by batch normalization and ReLU activation function, and then 1x1 convolution and batch normalization are applied to obtain multi-scale feature representation by integrating feature information at different scales; the multi-scale features are sent to the CBAM attention mechanism, and the output features are obtained after the multi-scale features pass through the CAM module and the SAM module in turn; the output features are connected with the input features in residual connection, fused through addition operation, and then a ReLU nonlinear activation function is applied. The rest of the Bottleneck uses the same structure as above;

[0119] Step S55, enter the fourth stage of PyRes_CBAM, the fourth stage is composed of three Bottleneck with the same structure, wherein the Bottleneck is firstly applied with 1x1 convolution, and batch normalization and ReLU activation function are applied; next is to use PyConv with kernel size of 3x3, followed by batch normalization and ReLU activation function, and then apply 1x1 convolution and batch normalization, and the feature information at different scales is integrated to obtain multi-scale feature representation; the multi-scale feature is sent into the CBAM attention mechanism, that is, the multi-scale feature is weighted in turn through the CAM module and the SAM module to obtain the output feature; the output feature is connected with the input feature in residual connection, and the two are fused through addition operation, and then ReLU nonlinear activation function is applied; the remaining Bottleneck uses the same structure as above;

[0120] Step S56, in order to reduce the size and the number of parameters of the feature map, an 8x8 average pooling layer with a step of 1 is used to down-sample the feature map, and an abstract representation of the copy-move feature is gradually extracted.

[0121] The step S6 includes the following steps:

[0122] Step S61, as shown in Figure 3 the abstract representation of the copy-move feature obtained in step S5 is passed through a fully connected layer containing two neurons, and according to the similarity between the copy-move tampered feature and the frame deletion tampered feature, the Softmax function is used to output the probability that the frame is a frame deletion point, if the classification probability score is greater than 0.5, it is considered that the picture is a frame deletion point, and the number of frame deletion points is increased by 1.

[0123] Step S62, detect the number of frame deletion points, if the frame deletion point is equal to 0, it means that there is no frame deletion point, and the video is the original video; if the frame deletion point is greater than 0, it means that the video is tampered by frame deletion at some place, thereby realizing the frame deletion tampering detection of the video.

[0124] In this example, the image copy-move tampered dataset is composed of copy-move tampered images from v1 and v2 datasets in CASIA dataset.

Claims

1. A video frame deletion forensics method based on image copy-move feature, characterized in that: The method utilizes the similarity between ghost features in video frame deletion tampered residual images and image copy-move tampered features to perform video frame deletion forensics, specifically: first, a Multi-PyRes model is designed, and the model is pre-trained on a copy-move tampered dataset to learn copy-move tampered features; then, the pre-trained model parameters are migrated to the frame deletion detection task; the method extracts copy-move features from the saliency residual feature sequence map of the frame deletion video by introducing a constraint convolution layer capable of learning copy-move features, and uses a residual pyramid convolution layer structure combined with a CBAM attention mechanism to extract copy-move tampered features at multiple scales, thereby realizing video frame deletion tampered detection; The method comprises the following steps: Step S1, pre-training a Multi-PyRes model combining a constraint convolution layer and a PyRes_CBAM on an image copy-move tampered dataset; Step S2, decoding the video to be detected into RGB images and converting them into residual feature map sequences; Step S3, saliency of the residual feature map sequence using a feature enhancement algorithm; Step S4, inputting the saliency residual feature map into the constraint convolution layer to obtain copy-move features, and then superimposing the features with the original saliency residual features using an early fusion method; Step S5, processing the superimposed features obtained in step S4 through a PyRes_CBAM network to gradually extract abstract representations of copy-move features; Step S6, generating the final prediction result by a neural network containing two fully connected layers using a Softmax function to determine whether the frame is a frame deletion point, thereby realizing video frame deletion tampered detection; The specific implementation of step S4 comprises the following steps: Step S41, convolving the saliency residual feature map sequence with three different 5x5 constraint convolution filters with a step size of 1 to produce a predicted residual feature map with a size of 252x252x3, which is used to suppress the content of the image and provide low-level copy-move forensic features for subsequent steps; The filter of the constraint convolution layer estimates the prediction error feature between the center pixel and its adjacent local pixels to produce a feature map corresponding to the predicted pixel error as a low-level forensic trace, from which the copy-move tampered trace in the image is adaptively learned by executing the constraint condition of the following formula (5); wherein W k represents the kth filter in the first convolutional layer, W k (l, m) represents the filter weight at the (l, m) position, i.e. the filter weight at the center position; in the constrained convolutional layer, the initial convolution kernel weight is randomly selected for initialization; this constraint ensures that the center value of each convolution kernel is normalized to -1, and the sum of the values of all other learnable parameters is limited to 1, so that the output of the network layer is a constant value, and the constant value is calculated by the following formula (6), and the constant is the weighted sum of the pixel value and the residual value of the field; Step S42, superimposing the copy-move features obtained in step S41 with the original saliency residual features using an early fusion method.

2. The video frame deletion forensics method based on image copying movement feature according to claim 1, characterized in that: The ghost features in the video frame deletion tampered residual image and the image copy-move tampered features have similarity, and the specific implementation of step S1 is to pre-train the Multi-PyRes model on the image copy-move tampered dataset for no less than 10 rounds, and through pre-training, the model parameters are iteratively oriented towards extracting copy-move features.

3. The video frame deletion forensics method based on image copying movement feature according to claim 1, characterized in that: The specific implementation of step S2 comprises the following steps: Step S21, decoding the video sample to be detected into n video frame RGB images; Step S22, convert the RGB image into a sequence of pixel residual corresponding to adjacent frames, and the pixel residual feature corresponding to the position between video frames is calculated according to the following formula (1): R k (i,j) = F k (i,j) - F k-1 (i,j); (1) where F k represents the gray pixel matrix of the Kth frame in the video sequence, i and j are the sequence numbers of the pixel row and column, and R represents the inter-pixel residual matrix; Step S23, resize the residual image R to 256x256 pixels.

4. The video frame deletion forensics method based on image copying movement feature according to claim 1, characterized in that: The specific implementation of the step S3 includes the following steps: Step S31, first smooth and expand the original residual image according to formula (2); Wherein, R is the original inter-frame pixel residual matrix calculated by formula (1), parameter b represents the threshold value of smoothing processing, and S represents the residual image after smoothing and expansion; formula (2) can suppress noise caused by irregular light changes and object shaking by smoothing the original residual image, and retain the main frame deletion tampering traces; Then use an adaptive algorithm to expand the residual, further expand the value of the large deformation area caused by tampering, and set the value after expansion to 0 if it is still lower than the threshold value, and cooperate with the binary processing to significantly distinguish the tampering area from the noise area; Parameter b is an empirical threshold value, which needs to avoid setting too high to cause the tampering traces to disappear, and also needs to avoid setting too low to cause the noise influence to be unable to be eliminated; Step S32, perform local binary processing on the smoothed and expanded residual image according to formula (3): Wherein, T represents the residual feature map after local binary processing, and S(i,j) less than the empirical threshold value Bound is set to 0, that is, the area with small inter-frame changes is set to 0; Then set S(i,j) greater than or equal to 255 to a smaller number 1, which is used to make the significant inter-frame difference area caused by frame deletion form a greater contrast with the adjacent area; The main purpose of this processing is to highlight the main inter-frame changes caused by frame deletion; The binary threshold value Bound is selected in a small range to avoid losing too much information due to too high threshold value, or avoiding being unable to remove noise due to too low threshold value; Step S33, further smooth the residual image by median filtering, and the calculation method is shown in the following formula 4: Wherein, A represents the filtering window with (i,j) point as the center, and Med represents the median value of the elements in the window area; The median filtering is adopted to nonlinearly smooth the significant area of tampering traces and noise and other areas, so as to further eliminate residual noise and highlight the strong frame deletion tampering traces.

5. The video frame deletion forensics method based on image copying movement feature according to claim 1, characterized in that: The specific implementation of the step S5 includes the following steps: Step S51, input the superimposed feature map obtained in step S4 into PyRes_CBAM, first pass through 7x7 convolution with a step of 2, and then pass through batch normalization and ReLU activation function; Step S52, along the network, enter the first stage of PyRes_CBAM, the first stage consists of three Bottleneck structures, wherein the Bottleneck adopts 1x1 convolution first, and batch normalization and ReLU activation function are applied; next is to use PyConv with kernel size of 9x9, 7x7, 5x5 and 3x3 respectively, and then batch normalization and ReLU activation function are applied, and then 1x1 convolution and batch normalization are applied, and the feature information at different scales is integrated to obtain multi-scale feature representation; the multi-scale features are sent to the CBAM attention mechanism, and the output features are obtained after the multi-scale features pass through the CAM module and the SAM module in turn; the overall calculation process is shown in the following formula (7) and formula (8): wherein F represents an input image feature, denotes element-wise multiplication; the CAM module is composed of a global average pooling layer, a max pooling layer and a multilayer perceptron fully connected layer MLP, and the channel attention calculation is performed by formula (9) as follows; the SAM module is composed of a group of convolution layers and a Sigmoid function, and the spatial attention calculation is performed by formula (10) as follows; M CAM (F) = σ(MLP(AvgPool(F)) + MLP(MaxPool(F))) (9); M SAM (F) = σ(f 7×7 ([AvgPool(F); MaxPool(F)]) (10); σ in Equation (9), Equation (10) represents a Sigmoid function, f 7×7 denotes a convolution operation with filter size 7x7; Finally, the output of the CBAM adjusted feature map is connected with the input feature in residual connection, and the two are fused through addition operation, and then ReLU nonlinear activation function is applied; the remaining Bottleneck uses the same structure as above; Step S53, enter the second stage of PyRes_CBAM, the second stage consists of four Bottleneck structures, wherein the Bottleneck adopts 1x1 convolution first, and batch normalization and ReLU activation function are applied; next is to use PyConv with kernel size of 7x7, 5x5 and 3x3 respectively, and then batch normalization and ReLU activation function are applied, and then 1x1 convolution and batch normalization are applied, and the feature information at different scales is integrated to obtain multi-scale feature representation; the multi-scale features are sent to the CBAM attention mechanism, and the output features are obtained after the multi-scale features pass through the CAM module and the SAM module in turn; the output features are connected with the input features in residual connection, and the two are fused through addition operation, and then ReLU nonlinear activation function is applied; the remaining Bottleneck uses the same structure as above; Step S54, enter the third stage of PyRes_CBAM, the third stage consists of six Bottleneck structures, wherein the Bottleneck adopts 1x1 convolution first, and batch normalization and ReLU activation function are applied; next is to use PyConv with kernel size of 5x5 and 3x3 respectively, and then batch normalization and ReLU activation function are applied, and then 1x1 convolution and batch normalization are applied, and the feature information at different scales is integrated to obtain multi-scale feature representation; the multi-scale features are sent to the CBAM attention mechanism, and the output features are obtained after the multi-scale features pass through the CAM module and the SAM module in turn; the output features are connected with the input features in residual connection, and the two are fused through addition operation, and then ReLU nonlinear activation function is applied; the remaining Bottleneck uses the same structure as above; Step S54, enter the third stage of PyRes_CBAM, the third stage consists of six Bottleneck structures, wherein the Bottleneck adopts 1x1 convolution first, and batch normalization and ReLU activation function are applied; next is to use PyConv with kernel size of 5x5 and 3x3 respectively, and then batch normalization and ReLU activation function are applied, and then 1x1 convolution and batch normalization are applied, and the feature information at different scales is integrated to obtain multi-scale feature representation; the multi-scale features are sent to the CBAM attention mechanism, and the output features are obtained after the multi-scale features pass through the CAM module and the SAM module in turn; the output features are connected with the input features in residual connection, and the two are fused through addition operation, and then ReLU nonlinear activation function is applied; the remaining Bottleneck uses the same structure as above; Step S55, enter the fourth stage of PyRes_CBAM, the fourth stage is composed of three Bottleneck structures, wherein the Bottleneck is first applied with 1x1 convolution, and batch normalization and ReLU activation function are applied; next is to use PyConv with a kernel size of 3x3, followed by batch normalization and ReLU activation function, and then apply 1x1 convolution and batch normalization, and integrate feature information at different scales to obtain multi-scale feature representation; the multi-scale features are sent to the CBAM attention mechanism, that is, the multi-scale features are sequentially weighted through the CAM module and the SAM module to obtain output features; the output features are then connected with the input features through residual connection, and are fused through an addition operation, and then a ReLU nonlinear activation function is applied; the remaining Bottleneck uses the same structure as above; Step S56, in order to reduce the size and number of parameters of the feature map, an 8x8 average pooling layer with a step of 1 is used to downsample the feature map, and an abstract representation of the copy-move feature is gradually extracted.

6. The video frame deletion forensics method based on image copying movement feature according to claim 1, characterized in that: The step S6 includes the following steps: Step S61, the abstract representation of the copy-move feature obtained in step S5 is input into a fully connected layer containing two neurons, and according to the similarity between the copy-move tampered feature and the frame deletion tampered feature, a Softmax function is used to output the probability that the frame is a frame deletion point, and if the output classification probability score is greater than 0.5, the picture is considered to be a frame deletion point, and the number of frame deletion points is increased by 1; Step S62, detect the number of frame deletion points, if the frame deletion points are equal to 0, it means that there is no frame deletion point, and the video is an original video; if the frame deletion points are greater than 0, it means that the video is tampered by frame deletion at some place, thereby realizing frame deletion tampering detection of the video.

Images