Facial expression migration method based on self-supervised learning and generative adversarial mechanism

Abstract

The invention provides a face expression migration method based on a self-supervised learning and generative adversarial mechanism. The method comprises a face identity and attitude self-supervised decoupling network design step, a generative adversarial network condition generation network design step, a model training step and an overall framework prediction step. In the face identity and posture self-supervision decoupling training step, a multi-frame deformation self-encoder is used for fusing face information of different frames from the same video sequence, a face image capable of representing the identity is estimated, and the image generally tends to the front face, is irrelevant to the posture and keeps the identity information. A self-supervised learning and generative adversarial mechanism is applied to a face expression and posture migration task, face identity and posture information in a video frame sequence is decoupled through a self-supervised method, and a labeled data set does not need to be used; and meanwhile, identity and posture information from different individuals is fused by using a generative adversarial network and a high-quality face is synthesized, sothat migration of facial expressions and postures among the individuals is realized.

Description

technical field [0001] The invention relates to the field of deep learning application technology, in particular to a method for transferring human facial expressions based on self-supervised learning and generation confrontation mechanism. Background technique [0002] With the rapid development of deep learning and image processing technology, facial expression synthesis and transfer are applied in many fields, such as film production, game production, virtual reality, face recognition, etc. At present, the facial expression transfer method mainly adopts the classic model-based parametric modeling method, or the end-to-end data-driven generation method. [0003] In the existing technology, the former is limited to the pre-defined model and its parameters, and it is difficult to fully represent the head pose and facial expression; the latter generally requires a large number of fine facial key point annotations, which is expensive in time and labor. For the field of facial...

AI Technical Summary

Problems solved by technology

[0004] Aiming at the technical problems in the prior art that require a large number of annotations and the quality of the generated images is poor, the present invention provides a face expression transfer method based on self-supervised learning and generation confrontation mechanism, aiming at decoupling the video frame sequence through the self-supervised method. face identity and posture without using labeled data sets, and at the same time use the generative confrontation mechanism to achieve high-quality face synthesis, and realize the transfer of facial expressions and postures from one individual to another

Images