Deep video frame interpolation detection method and device and computer readable storage medium

Abstract

The application discloses a deep video frame interpolation detection method, comprising the following steps: extracting bidirectional optical flow and inter-frame motion abnormal area in a video frame sequence, inputting the bidirectional optical flow into a ParNet network to extract local features, using a motion attention mechanism to focus on the inter-frame motion abnormal area and enhance tampering features; extracting single-frame motion abnormal areas in the video frame sequence, inputting the single-frame motion abnormal areas into a Transformer network for information reorganization, aggregating information with strong correlation to highlight tampering traces, and extracting global features; and inputting the local features and the global features after aggregation into a full connection layer for classification, and outputting a classification result of true and false frames. The application can effectively detect deep frame rate up-conversion tampering, can meet the verification requirements of video authenticity and integrity, and has high discrimination precision. The application also provides a deep video frame interpolation detection device and a computer readable storage medium.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of video information security, and particularly relates to a deep video frame insertion detection method and device and a computer readable storage medium. BACKGROUND

[0002] Video frame rate up-conversion, also known as video frame insertion, improves the frame rate of the original video by generating an interpolated frame between adjacent video frames, and increases the motion continuity of the original video. With the popularity of deep learning, deep frame insertion algorithms have gradually become mainstream. Deep frame insertion algorithms can ensure that the interpolated frame and the adjacent frame have high consistency in the time domain, so that the video after frame rate up-conversion is still natural and smooth, and the observer cannot directly perceive the traces of interpolation by visual observation.

[0003] In related technologies, a bidirectional optical flow is input into a convolutional network to extract discriminative features on enhanced motion vectors, highlight residuals, and then fuse the learned features to obtain a final detection result. However, the compressed workpieces after the interference elimination module are mixed with traces left by frame insertion tampering due to the re-encoding operation, and are also affected by double compression or multiple compression effects, so that the discrimination accuracy is low. Therefore, it is necessary to provide a deep video frame insertion detection method, device and computer readable storage medium to solve the above problems. SUMMARY

[0004] The purpose of the embodiments of the application is to provide a deep video frame insertion detection method, device and computer readable storage medium, which captures weak tampering traces left in a tampered video from a motion area, extracts local features by using a ParNet network, extracts global features by using a Transformer, and finally realizes the classification and detection of samples by using a fully connected layer, so as to realize the discrimination of true and false frames and have high discrimination accuracy.

[0005] To solve the above technical problems, the application is implemented as follows:

[0006] A deep video frame insertion detection method comprises the following steps:

[0007] S1: Extracting bidirectional optical flow and inter-frame motion abnormal area in a video frame sequence, inputting the bidirectional optical flow into a ParNet network to extract local features, and using a motion attention mechanism to focus on the inter-frame motion abnormal area to enhance tampering features;

[0008] S2: Extracting single-frame motion abnormal area in the video frame sequence and inputting the single-frame motion abnormal area into a Transformer network for information reorganization, aggregating information with strong correlation to highlight tampering traces, and extracting global features;

[0009] S3: Inputting the local features and the global features after aggregation into a fully connected layer for classification, and outputting the classification results of true and false frames.

[0010] Preferably, in the step S1, the video frame sequence is represented as: {F1, F2…F i …F N}, wherein Fi represents the i-th video frame, and N represents the total number of video frames. i

[0011] Preferably, the extraction process of the bidirectional optical flow in the video frame sequence is as follows:

[0012] The PWC-Net network is used to obtain the optical flow of the two consecutive frames in time sequence, and the operation process is as follows:

[0013]

[0014]

[0015]

[0016] In the formula, j∈{1, 2…N-1}, represents the forward optical flow of the j-th video frame, represents the backward optical flow of the j-th video frame, represents the bidirectional optical flow of the j-th video frame, and Net(·) represents the optical flow extraction operation using the PWC-Net network.

[0017] Preferably, the extraction process of the inter-frame motion abnormal area is as follows:

[0018] The ConvGRU_Z network is constructed to predict the next frame of each frame in the video frame sequence {F1, F2…F i …F N}, and obtain the predicted frame sequence {F′1, F′2…F′ i …F′ N};

[0019] The last frame F′ N in the predicted frame sequence {F′1, F′2…F′ i …F′ N} is subtracted from the first frame F1 in the video frame sequence {F1, F2…F N …F i}, and the inter-frame abnormal area is obtained, and the operation process is represented as:

[0020] EME=(F′ N -F1)

[0021] In the formula, EME represents the inter-frame motion abnormal area.

[0022] ​Preferably, the ConvGRU_Z network comprises a plurality of prediction units connected in sequence, the prediction unit comprising an encoder and a decoder, the encoder comprising four encoding units arranged in sequence, each of the encoding units comprising a convolution module and a first GRU module, the kernel size of the convolution module and the GRU module is 3*3, the step is 2, and the padding is 1; the decoder comprises four decoding units arranged in sequence, each of the decoding units comprising a second GRU module and a deconvolution module, the kernel size of the second GRU module and the deconvolution module is 4*4, the step is 2, and the padding is 1.

[0023] Preferably, the prediction process of the ConvGRU_Z network is as follows:

[0024] A plurality of continuous frames {F i , F N …F n} in the video frame sequence {F n+1 , F n+k …F n+m} are selected as input frames and input into the ConvGRU_Z network.

[0025] The frame F n in the input frame is taken as the input of the first prediction unit, and a predicted frame F′ n+1 of the next frame F n is output.

[0026] The predicted frame F′ n is symmetrical warped with respect to the frame F n+1 to obtain a symmetrical warped frame The symmetrical warped frame is taken as the input of the second prediction unit, and a predicted frame F′ n+2 of the next frame F n+1 is output, wherein,

[0027] In this way, the predicted frame F′ n+k-1 output by the previous prediction unit is symmetrical warped with respect to the frame F n+k to obtain a symmetrical warped frame The symmetrical warped frame is taken as the input of the current prediction unit, and a predicted frame F′ n+k of the next frame is output, wherein,

[0028] The above process is iterated multiple times until a predicted frame of the next frame F n+m is output, and the prediction of the plurality of continuous frames {F n , F n+1 …F n+k …F n+m} is completed.

[0029] adjusting the value range of consecutive multiple frames {F n ,F n+1 …F n+k …F n+m} to complete the prediction of the entire video frame sequence {F1, F2…F i …F N}.

[0030] Preferably, the ParNet network comprises three parallel sub-networks, each of which is used to process features of different resolutions, and each of the sub-networks is connected with a motion attention mechanism at the end, the motion attention mechanism being formed by two convolution layers in series, the first convolution layer having a kernel size of 3x3, a step of 1, and a padding of 1, and being activated by a Relu function; the second convolution layer having a kernel size of 1x1, a step of 1, and a padding of 0, and being activated by a Sigmoid function; the three sub-networks being fused at the later stage of the ParNet network.

[0031] The channel numbers of the feature maps of the motion attention mechanisms at the ends of the three sub-networks are 90, 180, and 180 respectively.

[0032] Preferably, the extraction process of the single-frame motion abnormal area is as follows:

[0033] The ConvGRU_Z network is used for prediction, and any video frame F i is subtracted from the predicted frame F′ i of the next frame to obtain a single-frame motion abnormal area S i , and the operation process is represented as:

[0034] S i =F i -F i ′.

[0035] The application also provides a deep video frame interpolation detection device, characterized by comprising:

[0036] a local feature extraction module: extracting bidirectional optical flow and inter-frame motion abnormal areas in a video frame sequence, inputting the bidirectional optical flow into a ParNet network to extract local features, and using a motion attention mechanism to focus on the inter-frame motion abnormal areas and enhance tampering features;

[0037] a global feature extraction module: extracting single-frame motion abnormal areas in a video frame sequence and inputting the single-frame motion abnormal areas into a Transformer network for information reorganization, aggregating information with strong correlations to highlight tampering traces, and extracting global features;

[0038] a discrimination module: inputting the aggregated local features and global features into a fully connected layer for classification, and outputting a classification result of true and false frames.

[0039] The application also provides a computer readable storage medium, which stores a computer program, the computer program comprising program instructions, which, when executed by a processor, cause the processor to perform the method described above.

[0040] In the embodiment of the application, local features are extracted by using the ParNet network, ConvGRU_Z is used to strengthen the attention to the motion region, and the tampering features can be enhanced; then the image is divided into blocks by using the Transformer network, the self-attention mechanism is used to reorganize the image, the global information of the input frame is extracted, the tampering traces are highlighted, then the local features and the global features are aggregated, and the classification results of the true and false frames are output by using the full connection layer, the video frame insertion tampering can be effectively detected, the verification requirements of the video authenticity and integrity are met, and high discrimination precision is achieved. BRIEF DESCRIPTION OF DRAWINGS

[0041] Figure 1 is a flow block diagram of a deep video frame insertion detection method provided by the application;

[0042] Figure 2 is an architecture diagram of the ConvGRU_Z network;

[0043] Figure 3 is an architecture diagram of the Transformer network. DETAILED DESCRIPTION

[0044] The technical solutions in the embodiments of the application will be clearly and completely described below with reference to the drawings in the embodiments of the application. Obviously, the described embodiments are some of the embodiments of the application, but not all the embodiments of the application. Based on the embodiments in the application, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the protection scope of the application.

[0045] Please refer to Figure 1 , Figure 1 is a flow block diagram of a deep video frame insertion detection method provided by the application; the deep video frame insertion detection method provided by the application comprises the following steps:

[0046] S1: extracting bidirectional optical flow and inter-frame motion abnormal regions in a video frame sequence, inputting the bidirectional optical flow into a ParNet network to extract local features, and using a motion attention mechanism to focus on the inter-frame motion abnormal regions and enhance tampering features.

[0047] In the embodiment, a video frame sequence {F1, F2…F i …F N} is defined, wherein F irepresents the i-th video frame, and N represents the total number of video frames.

[0048] The extraction process of bidirectional optical flow in video frame sequence is:

[0049] The PWC-Net network is used to obtain the optical flow of two consecutive frames in time sequence. The calculation process is as follows:

[0050]

[0051]

[0052]

[0053] Where, j∈{1,2…N-1}, represents the forward optical flow of the j-th video frame, represents the backward optical flow of the jth video frame, represents the bidirectional optical flow of the jth video frame, and Net(·) represents the use of PWC-N et The network performs optical flow extraction operations.

[0054] It should be noted that the optical flow extraction by the PWC-Net network is a conventional technology in this field. The extraction principle is: first, multi-layer features are obtained through CNN convolution, then the optical flow is estimated from low resolution, and the low-resolution optical flow is upsampled to high resolution. At the same time, the cost volume is constructed and the optical flow of the current resolution is predicted. Finally, the optical flow results of the final resolution are gradually obtained.

[0055] The extraction process of abnormal motion regions between frames is as follows:

[0056] Construct a ConvGRU_Z network to process the video frame sequence {F1, F2…F i …F N} to predict the next frame of each frame, and obtain the predicted frame sequence {F′1, F′2…F′ i …F′ N};

[0057] The predicted frame sequence {F′1, F′2…F′ i …F′ N The last frame F′ in N With the video frame sequence {F1, F2...F i …F N}, and obtain the abnormal region between frames. The calculation process is expressed as: EME = (F′ N -F1)

[0058] Where EME represents the abnormal motion area between frames.

[0059] like Figure 2 As shown, Figure 2 It represents the architecture diagram of the ConvGRU_Z network, such as Figure 2 As shown, the ConvGRU_Z network 100 includes a plurality of prediction units 10 connected in sequence, the prediction unit 10 includes an encoder 11 and a decoder 12, the encoder 11 includes four encoding units (Ecell) 111 arranged in sequence, each of the encoding units 111 includes a convolution module and a first GRU module, the kernel size of the convolution module and the GRU module are both 3×3, the step size is 2, and the padding is 1; the decoder 12 includes four decoding units (Dcell) 121 arranged in sequence, each of the decoding units 121 includes a second GRU module and a deconvolution module, the kernel size of the second GRU module and the deconvolution module is 4×4, the step size is 2, and the padding is 1.

[0060] The prediction process of the ConvGRU_Z network is:

[0061] Select the video frame sequence {F1, F2...F i …F N} in the continuous multiple frames {F n ,F n+1 …F n+k …F n+m} as the input frame and fed into the ConvGRU_Z network;

[0062] The frame F in the input frame n As the input of the first prediction unit, the output is the next frame F n+1 The predicted frame F′ n ;

[0063] Calculate the predicted frame F′ n About frame F n+1 Symmetrical warp frame Distort frames symmetrically As the input of the second prediction unit, the output is the next frame F n+2 The predicted frame F′ n+1 ,in,

[0064] Similarly, calculate the predicted frame F′ output by the previous prediction unit n+k-1 About frame F n+k Symmetrical warp frame Distort frames symmetrically As the input of the current prediction unit, output the predicted frame F' for the next frame n+k ,in,

[0065] Iterate multiple times until the output is F n+m The prediction of the next frame is completed for multiple consecutive frames {F n ,F n+1 …F n+k …F n+m} predictions;

[0066] Adjust multiple consecutive frames {F n ,F n+1 …F n+k …F n+m}, complete the entire video frame sequence {F1, F2…F i …F N} predictions.

[0067] Traditional ConvGRU networks can only be trained by updating network parameters, ignoring the inherent connections between videos in the temporal domain. Within a very short period of time, the same video frame sequence predictions tend to have the same motion trends. The ConvGRU_Z network proposed in this application exploits this trend by calculating symmetrically warped frames of the original frames as network inputs. This approach, considering the temporal domain, makes the predicted frames closer to the real frames.

[0068] Specifically, in this embodiment, the ConvGRU_Z network 100 includes four prediction units 10, which can output predictions for four consecutive frames. The network parameters of the ConvGRU_Z network 100 are shown in Table 1:

[0069] Table 1 Network parameters of ConvGRU_Z network

[0070]

[0071] like Figure 2 As shown, the four consecutive frames {F n ,F n+1 ,F n+2 ,F n+3}’s prediction results.

[0072] The ParNet network 100 is a conventional technology in this field, such as Figure 1 As shown, Figure 1The architecture diagram of the ParNet network is shown. The ParNet network includes three parallel sub-networks, each of which is used to process features of different resolutions. The three sub-networks are fused at the end of the ParNet network 100. A motion attention mechanism is connected to the end of each sub-network. The motion attention mechanism is composed of two convolutional layers in series. The convolution kernel size of the first convolutional layer is 3×3, the stride is 1, the padding is 1, and it is activated by the Relu function; the convolution kernel size of the second convolutional layer is 1×1, the stride is 1, the padding is 0, and it is activated by the Sigmoid function.

[0073] The number of channels of the feature maps of the motion attention mechanisms at the ends of the three sub-networks are 90, 180, and 180, respectively.

[0074] like Figure 1 As shown, the three sub-networks of the ParNet network 200 are the first sub-network 210, the second sub-network 220 and the third sub-network 230. The end of the first sub-network 210 is connected to the first motion attention mechanism 211, the end of the second sub-network 220 is connected to the second motion attention mechanism 221, and the end of the third sub-network 230 is connected to the third motion attention mechanism 231.

[0075] For local feature extraction, the ParNet network serves as the backbone. ParNet is a convolution-based feature extraction network that performs convolutions from left to right and top to bottom with a designed stride length. Traditional neural networks typically achieve better performance by increasing network depth, but this results in increased computational effort and latency. ParNet proposes a lower-depth parallel subnetwork architecture that achieves high performance by expanding the network's "width," effectively reducing computational effort and ensuring faster response.

[0076] The main forensic clues for video frame insertion tampering detection are concentrated in the motion area. The main module for extracting features in the ParNet network is convolution. For video frames, convolution is not targeted, that is, it lacks attention to the motion area. In the technical solution of this application, the motion attention mechanism is integrated into the ParNet network, allowing the network to pay more attention to the motion area, thereby improving network performance. At the same time, the addition of the motion attention mechanism introduces the ReLU function to the ParNet network, which prevents overfitting while also avoiding gradient disappearance.

[0077] S2: Extract the single-frame motion abnormality region in the video frame sequence and input it into the Transformer network for information reorganization, aggregate the strongly correlated information to highlight the tampering traces, and extract the global features;

[0078] The single-frame motion abnormal area is also extracted by the ConvGRU_Z network, and any video frame Fi and the predicted frame F' of the next frame i obtained by difference i The operation process is represented as: S i = F i -F' i .

[0079] The Transformer network adopts conventional techniques in the art, such as Figure 3 as shown in the formula: Figure 3 which represents the architecture diagram of the Transformer network 300, Figure 3 wherein Conv represents a common 2D convolution, S is the step length, GELU represents the GELU activation function, BN represents batch normalization, and DW Conv represents a depth separable convolution.

[0080] As shown in the formula: Figure 3 the Transformer network includes a data enhancement module, a Transformer module, and a feature enhancement module, wherein the operation process in the Transformer module is: I+P=Patch Embed (image)+Position Embed (image)

[0081] wherein I represents picture block information, P represents picture block position encoding, Patch Embed (image) represents a picture block encoding operation, and Position Embed (image) represents a picture position encoding operation. wherein I' represents the reorganized block information after the self-attention mechanism;

[0082] I2=Norm(I+P+I')

[0083] wherein Norm(·) represents normalization processing, and I2 represents the normalized feature.

[0084] I3=Feed forward (I2)

[0085] wherein I3 represents the feature of the reorganized block information after information enhancement; and Feed forward (·) represents forward propagation.

[0086] f globalfeature =Norm(I2+I3)

[0087] wherein f globalfeature represents the feature after information reorganization.

[0088] The block size of the image in the Transformer network is 2x2, and the encoding dimension of each block is 45; the Transformer network adopts a multi-head attention mechanism, the number of heads is set to 2, and the number of blocks is set to 2.

[0089] For global feature extraction, a Transformer is used as the backbone network. First, the video frames are blocked, and the correlation between the blocks is calculated through a self-attention mechanism to aggregate information with strong correlation. For the inserted frame, it has high consistency in the spatial and temporal domains with the adjacent frames, and the main difference from the real frame is concentrated in the motion area. The mechanism of the Transformer network is to calculate the correlation between the blocks, so the synthesized motion area must be significantly different from the remaining area, that is, the correlation is low, so after information reorganization, the extracted features will also be significantly different, which can be used as a kind of tampering trace for detection. Compared with convolution, the Transformer network operates on the entire feature map, rather than focusing on a certain area in the feature map.

[0090] S3: inputting the local feature and the global feature after aggregation into a fully connected layer for classification to output a classification result of the true and false frames.

[0091] In the training process of the network, a CrossEntropyLoss loss function is set to ensure that the network converges stably at a faster speed in the training.

[0092] The application also provides a deep video frame insertion detection device, comprising:

[0093] The local feature extraction module extracts bidirectional optical flow and inter-frame motion abnormal area in the video frame sequence, inputs the bidirectional optical flow into a ParNet network to extract local features, and uses a motion attention mechanism to focus on the inter-frame motion abnormal area to enhance the tampering features.

[0094] The global feature extraction module extracts a single-frame motion abnormal area in the video frame sequence and inputs it into a Transformer network to complete information reorganization, aggregates information with strong correlation to highlight tampering traces, and extracts global features.

[0095] The discrimination module inputs the local feature and the global feature after aggregation into a fully connected layer for classification to output a classification result of the true and false frames.

[0096] The application also provides a computer readable storage medium, which stores a computer program, the computer program comprising program instructions, the program instructions causing the processor to execute the method as described above when executed by the processor.

[0097] Embodiment 1

[0098] A video set DANVIS is selected, which has a total of 210 videos. The 210 original videos are input into a deep video frame rate up-conversion method, such as the AdaCoF algorithm, for video frame rate up-conversion tampering. The training set and the test set are divided according to a ratio of 9:1, that is, 190 videos are used as the training set, and 20 videos are used as the test set. In order to improve the training time of the model, the resolution of the frames is converted to 224x224. In order to obtain the temporal information between videos, the network input uses 4 groups of bidirectional optical flow obtained by completely continuous 4 frames, and the bidirectional optical flow is spliced and used as the input of the local feature extraction network. The continuous 4 frames predicted by the ConvGRU-Z network are used as the input of the global feature extraction network.

[0099] Experimental process:

[0100] The 4 groups of bidirectional optical flow of 4 continuous frames and the predicted continuous 4 frames are used as a group of training data. There are a total of 25660 groups of data in the training set for training, and a total of 2760 groups of data in the test set for testing. CrossEntropyLoss is used as the loss function during training.

[0101] In order to verify the effectiveness of the network, the following comparative experiment schemes are set:

[0102] Scheme one: using bidirectional optical flow as the input of the ParNet network, extracting features and inputting a full connection layer to distinguish between true and false frames;

[0103] Scheme two: using bidirectional optical flow as the input of the ResNet18 network, extracting features and inputting a full connection layer to distinguish between true and false frames;

[0104] Scheme three: using the technical scheme of the present application;

[0105] The detection accuracies of different schemes are shown in Table 1:

[0106] Table 1 Detection accuracy (%) of different schemes

[0107] Scheme 1 Scheme 2 Scheme 3 Detection accuracy 80.69 89.22 99.26

[0108] As can be seen from Table 1, the technical scheme of the present application can obtain better detection accuracy.

[0109] The embodiments of the present application are described above in combination with the drawings, but the present application is not limited to the above specific embodiments, and the above specific embodiments are only illustrative, but not limiting. Those skilled in the art can make many forms under the inspiration of the present application without departing from the scope of the present application and the scope protected by the claims.

Claims

1. A method of depth video frame interpolation detection, the method comprising: Comprising the following steps: S1: extract the bidirectional optical flow and the inter-frame motion abnormal area in the video frame sequence, input the bidirectional optical flow into ParNet The network extracts local features, focuses on the inter-frame motion abnormal area by using a motion attention mechanism, and enhances the tampering features. S2: extract single-frame motion anomaly region input in video frame sequence Transformer The network recombines information, aggregates information with strong correlation to highlight tampering traces, and extracts global features; S3: inputting the aggregated local features and global features into a fully connected layer for classification to output a classification result of the true and false frames; The video frame sequence is represented as: wherein represents the th video frame, represents the total number of video frames; The extraction process of the bidirectional optical flow in the video frame sequence is: Adopting PWC-Net The network obtains the optical flow of two frames in succession in timing, and the operation process is as follows: wherein, , denotes a forward optical flow of the video frame, denotes a backward optical flow of the video frame, denotes a bidirectional optical flow of the video frame, denotes performing an optical flow extraction operation using a PWC-Net network; The extraction process of the inter-frame motion abnormal area is: Build ConvGRU_Z network, for the video frame sequence The next frame of each frame is predicted to obtain the predicted frame sequence ; The predicted frame sequence The last frame in With the video frame sequence The first frame in Subtract and get the abnormal area between frames. The operation process is expressed as: In the formula, represents an inter-frame motion abnormal region; The ConvGRU_Z The network comprises a plurality of prediction units connected in sequence, the prediction units comprising an encoder and a decoder, the encoder comprising four encoding units arranged in sequence, each of the encoding units comprising a convolution module and a first GRU module, the convolution module and the GRU module having a kernel size of 3*3, a step length of 2 and a padding of 1; the decoder comprising four decoding units arranged in sequence, each of the decoding units comprising a second GRU module and a deconvolution module, the second GRU module and the deconvolution module having a kernel size of 4*4, a step length of 2 and a padding of 1.

2. The depth video frame interpolation detection method of claim 1, wherein, The ConvGRU_Z The prediction process of the network is: selecting a plurality of consecutive frames from the sequence of video frames as input frames to the network as input frames to the network ConvGRU_Z ​ frames in the input frames as input to a first said prediction unit, output a predicted frame for a next frame ; computing a predicted frame about a frame symmetrically warped frame symmetrically warped frame as input to a second said prediction unit, outputting a predicted frame for a next frame wherein and so on, the predicted frame output by the previous prediction unit is calculated with respect to the frame of the symmetrically warped frame , the symmetrically warped frame is output as input to the current prediction unit, outputting a predicted frame for the next frame wherein ; Multiple iterations until output pair Prediction of next frame from frame, completing prediction of multiple consecutive frames ​ Adjusting the value range of consecutive multiple frames to complete the prediction of the entire video frame sequence .

3. The depth video frame interpolation detection method of claim 1, wherein, described ParNet The network consists of three parallel sub-networks, each of which is used to process features of different resolutions. The end of each sub-network is connected to a motion attention mechanism, which is composed of two convolutional layers in series. The convolution kernel size of the first convolutional layer is 3×3, the step size is 1, the padding is 1, and it is activated by the Relu function; the convolution kernel size of the second convolutional layer is 1×1, the step size is 1, the padding is 0, and it is activated by the Sigmoid function. The three sub-networks are connected in the ParNet Late integration of the network; The channel numbers of the feature maps of the motion attention mechanisms at the ends of the three sub-networks are 90, 180 and 180 respectively.

4. The depth video frame interpolation detection method of claim 2, wherein, The extraction process of the single-frame motion abnormal area is: Using the ConvGRU_Z network to make predictions, any video frame and the predicted frame for the next frame obtained by differencing single-frame motion anomaly regions The operation process is represented as: 。 5. A depth video frame interpolation detection apparatus, characterized by, Comprising: The local feature extraction module extracts bidirectional optical flow and interframe motion abnormal area in the video frame sequence, and inputs the bidirectional optical flow into ParNet The network extracts local features, focuses on the interframe motion abnormal area by using a motion attention mechanism, and enhances the tampering features. The global feature extraction module extracts a single-frame motion abnormal area in the video frame sequence Transformer The network recombines information, aggregates information with strong correlation to highlight tampering traces, and extracts global features; The discrimination module: inputting the aggregated local features and global features into a fully connected layer for classification to output a classification result of the true and false frames; The video frame sequence is represented as: wherein represents the video frame, represents the total number of video frames; The extraction process of the bidirectional optical flow in the video frame sequence is: Adopting PWC-Net The network obtains the optical flow of two frames in succession in timing, and the operation process is as follows: wherein, , denotes a forward optical flow of the video frame, denotes a backward optical flow of the video frame, denotes a bidirectional optical flow of the video frame, denotes performing an optical flow extraction operation using a PWC-Net network; The extraction process of the inter-frame motion abnormal area is: Build ConvGRU_Z network, for the video frame sequence The next frame of each frame is predicted to obtain the predicted frame sequence ; The predicted frame sequence The last frame in With the video frame sequence The first frame in Subtract and get the abnormal area between frames. The operation process is expressed as: In the formula, represents an inter-frame motion abnormal region; The ConvGRU_Z The network comprises a plurality of prediction units connected in sequence, the prediction units comprising an encoder and a decoder, the encoder comprising four encoding units arranged in sequence, each of the encoding units comprising a convolution module and a first GRU module, the convolution module and the GRU module having a kernel size of 3*3, a step length of 2 and a padding of 1; the decoder comprising four decoding units arranged in sequence, each of the decoding units comprising a second GRU module and a deconvolution module, the second GRU module and the deconvolution module having a kernel size of 4*4, a step length of 2 and a padding of 1.

6. A computer-readable storage medium, characterized in that, The computer readable storage medium stores a computer program, the computer program includes program instructions, the program instructions when being executed by the processor make the processor execute the method as claimed in any one of claims 1-4.

Images