Human face three-dimensional reconstruction and human face replacing video editing system and method based on deep learning

Abstract

The invention discloses a human face three-dimensional reconstruction and human face replacing video editing system and method based on deep learning, and the system consists of a deep learning module, a three-dimensional reconstruction module, a video preprocessing module, a two-dimensional image generation module, and a video editing module. The deep learning module obtains necessary parameters for the generation of a three-dimensional model through a training convolution neural network. The three-dimensional reconstruction module obtains a corresponding three-dimensional model according to an inputted human face image / video. The video preprocessing module carries out the processing of a target video, and obtains the human face feature points and expression parameters corresponding to each frame. The two-dimensional image generation module enables a three-dimensional model of a target human face to generate a two-dimensional human face image corresponding to each frame. The video editing module enables a new human face image to replace the target image, and carries out the smoothing and adjustment of the illumination conditions.

Description

[0001] Technical field: [0002] The invention belongs to the field of computers, and relates to a three-dimensional reconstruction and video editing system, in particular to a deep learning-based video editing system and method for three-dimensional reconstruction and replacement of human faces. [0003] Background technique: [0004] In recent years, "Avatar"-style virtual character generation technology has been more and more applied in film and television production and electronic games. Among them, the accuracy of facial reconstruction is particularly important for virtual characters. Real-time display of facial expressions can make virtual characters more realistic. At the same time, the rise of deep learning has enabled new breakthroughs in computer vision problems that were difficult to solve in the past. A trained convolutional neural network can be effectively applied to 3D face reconstruction, thereby greatly reducing the requirements for shooting tools and input c...

AI Technical Summary

Problems solved by technology

This patent is only a simple two-dimensional face detection, without generating a three-dimensional model, and there is no different expression of the same face, so expression grafting cannot be achieved, nor can it be used for video editing

[0007] As mentioned above, existing technologies require the use of additional hardware (such as depth-of-field cameras), or face replacement is too simple, without taking into account the 3D model of the face and expression factors

And the technology that can achieve the real-time standard through the above steps is even more scarce

Images