High-fidelity face reproduction method based on mixed action representation
A technology that drives faces and faces, applied in the field of deep face forgery, can solve problems such as difficulty in generating high-quality reproduction results, difficulty in distinguishing identities and actions, weak face shape constraints, etc., to achieve good retention of background information, The effect of reducing semantic information loss and improving authenticity
Active Publication Date: 2022-06-03
NANJING UNIV OF SCI & TECH
View PDF5 Cites 0 Cited by
- Summary
- Abstract
- Description
- Claims
- Application Information
AI Technical Summary
Problems solved by technology
Self-supervision-based methods also struggle to distinguish identity from action
AUs-based methods have weak constraints on face shape and are difficult to generate high-quality reproduction results
Method used
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View moreImage
Smart Image Click on the blue labels to locate them in the text.
Smart ImageViewing Examples
Examples
Experimental program
Comparison scheme
Effect test
Embodiment Construction
[0020] The present invention will be further illustrated below in conjunction with the accompanying drawings and specific embodiments. The embodiments are implemented on the premise of the technical solutions of the present invention. It should be understood that these embodiments are only used to illustrate the present invention and not to limit the scope of the present invention.
[0021] The high-fidelity real face reproduction method of the mixed action representation of the present invention firstly extracts the action units (Action Units, AUs) and attitude information of the driving face and the key point (landmarks) information of the source face; The action units and pose information of the face convert the keypoint information of the source face; then, use a pretrained segmentation network to separate the source face image into face regions and background regions; convert the action units, transformed keypoint information, and The face area is input to the reproduction...
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More PUM
Login to View More Abstract
The invention discloses a high-fidelity face reproduction method with hybrid action representation, and belongs to the field of deep face counterfeiting. Comprising the following steps: extracting an action unit for driving a human face, posture information and key point information of a source human face; a key point conversion module is used for converting key point information of the source face according to the action unit for driving the face and the posture information; separating the source face picture into a face region and a background region by using a pre-trained segmentation network; inputting the action unit, the converted key point information and the face area into a reproduction network to generate a target face; and inputting the target face and the background region into a background fusion module to generate a final result. According to the method, multiple action representations are mixed to serve as guide signals of face reproduction, and action features are inserted through spatial adaptive regularization, so that semantic features can be better kept in the reproduction process; and meanwhile, in combination with a background separation technology, the authenticity and inter-frame continuity of the generated face are further improved, and high-fidelity face reproduction is realized.
Description
technical field [0001] The present invention relates to deep face forgery, in particular to a high-fidelity face reproduction method of mixed action representation. Background technique [0002] Face reproduction is the process of generating animation for the source face according to the actions (pose and expression) that drive the face, and has wide application prospects in the fields of film production and augmented reality. Generally speaking, the process consists of three main steps: 1) Create a representation of the source face identity, 2) Extract and encode the actions that drive the face, 3) Combine identity and action representations to generate fake source faces. Each step has a significant impact on build quality. [0003] At present, face reproduction technologies can be mainly divided into synthetic methods based on traditional 3D models and generation methods based on generative adversarial networks (GANs). In 3D face model-based methods, the 3D model p...
Claims
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More Application Information
Patent Timeline
Login to View More Patent Type & Authority Applications(China)
IPC IPC(8): G06T17/00G06V40/16
CPCG06T17/00
Inventor 邵长乐耿嘉仪练智超韦志辉
Owner NANJING UNIV OF SCI & TECH