Static portrait three-dimensional reconstruction and dynamic face fusion method based on depth camera

Abstract

The invention discloses a static portrait three-dimensional reconstruction and dynamic face fusion method based on a depth camera, and relates to the technical field of image reconstruction, and the method comprises the steps: obtaining a color sequence frame and a depth sequence frame of a static portrait in a set region range of the depth camera, and employing an iterative nearest point algorithm to calculate the minimum value of the distance difference between two adjacent groups of point clouds in the point cloud sequence, respectively carrying out the segmentation and matting of each color frame in the color sequence frames, obtaining a plurality of human image masks, and projecting the color information of each color frame to the corresponding space voxel according to the relative posture, calculating the average value of the color information and assigning the average value to each triangular patch of the static portrait three-dimensional grid to obtain the static portrait three-dimensional grid with the color information, and constructing the dynamic portrait three-dimensional grid in real time according to the depth information of the depth camera in the dynamic portrait mask, thereby realizing the integrity of the portrait three-dimensional model, real-time driving of the complete portrait three-dimensional model and dynamic real-time three-dimensional reconstructionof the portrait.

Description

technical field [0001] The invention relates to the technical field of image reconstruction, in particular to a depth camera-based three-dimensional reconstruction of a static portrait and a fusion method of a dynamic human face. Background technique [0002] At present, desktop-level real-time 3D portrait reconstruction schemes use body template algorithms to fit 3D portraits through parametric models such as 3DMM and SMPL. Although these methods can reconstruct realistic human bodies, they cannot reconstruct hair, clothing, Accessories and other parts outside the human body, the overall effect is not fine enough. The other type directly meshes and textures the depth information, but lacks the information on the back of the portrait, resulting in an incomplete 3D model. Contents of the invention [0003] In order to solve the deficiencies of the prior art, an embodiment of the present invention provides a method for 3D reconstruction of a static portrait and fusion of a ...

AI Technical Summary

Problems solved by technology

[0002] At present, desktop-level real-time 3D portrait reconstruction schemes use body template algorithms to fit 3D portraits through parametric models such as 3DMM and SMPL. Although these methods can reconstruct realistic human bodies, they cannot reconstruct hair, clothing, Accessories and other parts other than the human body, the overall effect is not fine enough

The other type directly grids and textures the depth information, but lacks the information on the back of the portrait, resulting in an incomplete 3D model