End-to-end three-dimensional face reconstruction method based on neural network

Abstract

The invention discloses an end-to-end three-dimensional face reconstruction method based on a neural network, and belongs to the technical field of three-dimensional face reconstruction. According tothe method, a multi-task loss function and a fusion neural network are applied to a convolutional neural network, so the reconstruction effect of the facial expression is improved; the semantic information of the whole reconstruction process is considered, not only regression face parameters, but also the influences of the camera attitude and the reconstruction model on the whole reconstruction error are considered, so that the accuracy of the whole neural network is improved. According to the three-dimensional face reconstruction method disclosed by the invention, the three-dimensional face shape can be reconstructed from the picture, and three-dimensional recovery can be carried out under the condition of changing illumination or in a face picture with an extreme expression.

Description

technical field [0001] The invention relates to the technical field of three-dimensional face reconstruction, in particular to an end-to-end three-dimensional face reconstruction method based on a neural network. Background technique [0002] Since 3D information is a strong invariant of viewing angle, it is very beneficial to apply it to computer vision. It can solve the problems of posture, expression and illumination changes of face images. However, in these methods, 3D camera systems are ideally used to capture 3D information, however, the high cost and limited effective sensing range of 3D cameras limit their applicability in practice. Therefore, in the industry, it is of universal significance to perform face reconstruction from photos taken under arbitrary lighting and arbitrary camera parameters. Moreover, almost 60% of pictures on the Internet now have face images, so the research on reconstructing 3D face shapes from 2D face images is very extensive. [0003] Wi...

AI Technical Summary

Problems solved by technology

[0004] However, the current technology of using CNNs to reconstruct 3D faces either only focuses on the regression of the parameters themselves, without taking into account the semantic information of the entire parameters, or does not take into account the impact of the camera pose and the reconstruction model on the reconstruction accuracy, and other methods need A large amount of auxiliary work increases the complexity of the method

Images