Personalized 3D Face Mesh Finetuning Using Texture Displacement

The patent combines texture generation and mesh fitting into a single integrated process where a neural network simultaneously processes input images to generate both the mesh structure and texture information. This unified approach allows the two previously separate processes to work together synergistically, improving the overall quality and consistency of the 3D face model while maintaining manageable system complexity.

The patent performs preliminary texture generation using a neural network to create an initial texture map from input images. This preliminary texture information is then used to guide and constrain the mesh fitting process, ensuring that the mesh accurately captures facial details while the texture provides a reference for verification. This preliminary action enables faster processing compared to traditional iterative methods while maintaining high precision.

The patent replaces complex mechanical and optical systems (expensive cameras, controlled lighting equipment) with a computational approach using neural networks. The system processes standard images through learned models to extract high-quality texture and geometric information, achieving comparable or superior results to traditional photogrammetry methods while eliminating the need for specialized hardware and controlled environments.