A method for 3D reconstruction and texture generation from single-view face based on multi-task learning

Abstract

The invention discloses a method for 3D reconstruction and texture generation from single-view face based on multi-task learning, belonging to the computer vision field. The method comprises the following steps of: selecting a special viewpoint for rendering a three-dimensional human face model; generating depth map and texture map as truth value data from special viewpoint; designing an integrated learning and coding network based on feature sharing of depth information and texture information; designing the branch decoding network to recover the depth map from the shared features, and recovering the depth map, designing the mutual information with shared features as latent variables to maximize the generation of antagonistic network, and restoring the texture unwrapping map, adjusting the proportion of each task loss function and training the model; interpolating the depth map and using texture map to reconstruct the 3D face mesh model with texture details. The invention utilizes multi-task learning to carry out single-view face three-dimensional reconstruction, texture generation and style migration, and has the advantages of high speed, low cost and the like.

Description

technical field [0001] The invention relates to the field of computer vision, in particular to a single-view human face three-dimensional reconstruction and texture generation method based on multi-task learning. Background technique [0002] Three-dimensional face models are widely used in security authentication, film and television animation, medical science and other fields. However, it is very expensive to obtain the accurate 3D structure of the face and the complete high-resolution texture map at the same time, and the obtained texture map is not convenient for post-processing, or there are various problems such as that the 3D structure and the high-resolution texture map cannot be obtained at the same time. problem. [0003] For single-view face 3D reconstruction and texture generation using traditional methods, there are usually two techniques: (1) Shape-from-Shading (SFS) method or photometric stereo (Photometric stereo) method is used according to the single-view ...

AI Technical Summary

Problems solved by technology

The texture image restored by this method is better, but the reconstruction effect of the 3D model of the face is not good enough.

[0005] The above-mentioned existing technologies all have the following disadvantages: based on a single face image, the accurate three-dimensional geometric structure of the face and the complete texture cannot be restored at the same time, and for the method of deep learning, the collection cost of the complete texture expansion map data set is very high and inconvenient

Some methods can obtain a relatively complete three-dimensional geometric structure of the face, but the texture part is a texture image obtained directly from the local affine transformation of the input original image, and the texture in the occluded area is obviously wrong

Some methods can restore high-resolution and complete face texture maps, but they are limited to frontal face images or small-angle side face images, and the restoration of the three-dimensional structure of the face is not ideal.

Images