Method for reconstructing a three-dimensional facial expression model based on a monocular video

Abstract

The invention discloses a method for reconstructing a three-dimensional facial expression model based on a monocular video. Extra multi-angle shooting is not needed, A universal 3D face model is directly driven from a neutral expression image frame in an input monocular video to deform so as to generate a personalized three-dimensional face template; and expressing the deformation of the three-dimensional face expressions corresponding to different frames as the change of the personalized three-dimensional face template in the 3D vertex flow of the three-dimensional space, and solving the coarse-scale geometric model of the face expressions through the consistency with the change of the 2D optical flow. While the shape accuracy of a coarse-scale reconstruction model is improved by utilizing dense optical flow, the shooting requirement of an input video is broadened, details are added on a recovered coarse-scale face model through a light and shade recovery shape technology so as to recover fine-scale face geometric data, and a three-dimensional face geometric model with high fidelity is reconstructed.

Description

technical field [0001] The invention relates to a method for reconstructing a three-dimensional facial expression model based on monocular video, and belongs to the technical field of computer virtual reality. Background technique [0002] Realistically reconstructed 3D facial expression models are widely used in computer games, film and television production, social, medical and other fields. Traditional 3D facial model acquisition and reconstruction mostly rely on heavy and expensive hardware equipment and controllable lighting in the laboratory environment. With the rapid entry of virtual reality technology and mobile smart terminals into public life, more and more people hope to obtain high-quality 3D facial expression models in daily life through low-cost equipment and apply them in virtual environments. Using mobile phones and cameras to shoot videos, or directly using Internet videos to reconstruct 3D facial expression models, minimizes the complexity of obtaining eq...

AI Technical Summary

Problems solved by technology

[0007] At present, the 3D facial expression model reconstruction technology based on monocular video relies on the input of 2D and 3D feature point position information of monocular video and the restored camera matrix. The most common method is to project the restored 3D model onto a 2D plane. The optimization of the similarity between the brightness value of the input image and the brightness value of the input image, this type of method cannot get good recovery results for large facial expression changes, illumination changes and slight occlusions

Moreover, the feature points are too sparse to ensure the accuracy of reconstructing the coarse-scale face shape, especially the facial area far away from the feature points.

Images