A High-Fidelity Real Face Reproduction Method with Hybrid Motion Representation

Abstract

The invention discloses a high-fidelity real face reproduction method with mixed action representation, which belongs to the field of deep face forgery. Including extracting the action unit and posture information of the driving face and the key point information of the source face; using the key point conversion module to convert the key point information of the source face according to the action unit and posture information of the driving face; using a pre-trained segmentation network Separate the source face image into face area and background area; input the action unit, converted key point information, and face area into the reproduction network to generate the target face; input the target face and background area into the background fusion module , which produces the final result. The present invention mixes multiple action representations as a guide signal for face reappearance, and uses spatial adaptive regularization to insert action features, so that the reappearance process can better maintain semantic features; at the same time, combined with the background separation technology, the generated face is further improved. Realism and frame-to-frame continuity for high-fidelity face reproduction.

Description

technical field [0001] The present invention relates to deep face forgery, in particular to a high-fidelity face reproduction method of mixed action representation. Background technique [0002] Face reproduction is the process of generating animation for the source face according to the actions (pose and expression) that drive the face, and has wide application prospects in the fields of film production and augmented reality. Generally speaking, the process consists of three main steps: [0003] 1) Create a representation of the source face identity, [0004] 2) Extract and encode the actions that drive the face, [0005] 3) Combine identity and action representations to generate fake source faces. Each step has a significant impact on build quality. [0006] At present, face reproduction technologies can be mainly divided into synthetic methods based on traditional 3D models and generation methods based on generative adversarial networks (GANs). In 3D face model-based...

AI Technical Summary

Problems solved by technology

Self-supervision-based methods also struggle to distinguish identity from action

AUs-based methods have weak constraints on face shape and are difficult to generate high-quality reproduction results

Images