DeepFake defense method and system based on visual adversarial reconstruction

Abstract

The invention provides a DeepFake defense method and system based on visual adversarial reconstruction. The method comprises a two-stage method. In a first stage, real face data are converted into potential codes, and real images can be realistically reconstructed through a generator. The problem is regarded as a reverse problem of the GAN, an encoder is trained to generate potential embedding, the potential embedding is used as initialization, and then fine adjustment is carried out on the potential embedding. In a second stage, a search is performed in the neighborhood of the potential embedding obtained in the first stage to obtain an optimal embedding, which can result in a perfect reconstruction and invalidate the DeepFake. In the process, the potential embedding is optimized by using gradient information from the target DeepFake model and is limited in a small modification range so as to meet the visual similarity requirement.

Description

technical field [0001] The invention belongs to the field of DeepFake defense, in particular to a DeepFake defense method and system based on visual confrontation reconstruction. Background technique [0002] With the advent of sophisticated image and video synthesis techniques, especially Generative Adversarial Networks (GAN)~\cite{goodfellow2014generative}, it has become easier and easier to generate high-quality, convincing fake videos. DeepFake~\cite{choi2018stargan,karras2019style,karras2020analyzing} is a new genre of synthetic video in which the face of a subject is modified to a target face to simulate the target subject in a specific environment and create convincing videos of real events. To this end, effective measures should be formulated to combat such DeepFakes to protect personal security and privacy. [0003] Existing deepfake defense techniques mainly focus on passive detection, i.e. exploiting artifacts in generated fake faces to detect them. Specifically...

AI Technical Summary

Problems solved by technology

However, while breaking DeepFake, the added adversarial perturbation is easily detected by the detector and easily broken

Images