False face video tampering detection method and system based on time domain self-attention mechanism

Abstract

The invention discloses a false face video tampering detection method and system based on a time domain self-attention mechanism, and the method comprises a network training step and a sample testing step, and the network training step mainly comprises the steps of video preprocessing, segmentation network building and training. The sample test mainly comprises the steps of preprocessing a video, building a segmentation network, predicting a tampered area through three modules in sequence, and calculating a facial intersection ratio of a prediction mask to obtain a detection result. According to the invention, color features, noise features and time domain features of a spatial domain are input into a neural network comprising a double-flow feature extraction module, a time domain self-attention module and an up-sampling module to predict a tampered area, a detection object comprises a single picture and a plurality of video frame pictures, and relatively ideal accuracy is obtained in different databases. Compared with other existing algorithms, the cross-database test performance is obviously improved, and the invention has great potential application value.

Description

technical field [0001] The invention relates to the technical field of digital video tampering detection, in particular to a fake face video tampering detection method and system based on a time-domain self-attention mechanism. Background technique [0002] With the rapid development of face recognition technology, the security threat posed by face tampering is getting bigger and bigger, and related fake face videos are also emerging in an endless stream on the Internet. Deepfake face-changing tools mainly use deep neural networks such as autoencoders or confrontation generation networks to generate false faces and then replace the faces of the original video. The present invention mainly detects face-changing videos generated by this type of face tampering technology . [0003] The current mainstream fake face video detection technology is mainly based on the neural network to extract features. Although fake face videos can be identified to a certain extent, especially in ...

AI Technical Summary

Problems solved by technology

[0003] The current mainstream fake face video detection technology is mainly based on the neural network to extract features. Although fake face videos can be identified to a certain extent, especially in the database test, it can achieve a high accuracy rate. However, the cross-database test is accurate. sharp drop in rates

In fact, most of the published face-changing and tampering detection algorithms based on deep networks have such problems, that is, the problem of insufficient generalization ability

In addition, most of the current related algorithms are based on the detection of a single image, and do not use the timing information of the front and back frames in the fake face video

So far, the method of fake face video detection using the color features, noise features and temporal features of the picture space domain, and the neural network constructed by the self-attention mechanism in the temporal domain has not been reported.

Images