Three-dimensional face and eyeball movement modeling and capturing method and system

Abstract

The invention provides a three-dimensional face and eyeball movement modeling and capturing method and system, and the method comprises the steps: firstly extracting the position information of a feature point of a face in a shot scene image, and obtaining the information through employing a convolutional neural network method; after sparse facial feature points are obtained, a multilinear model of a human face serving as a prior condition of the model, the positions of the feature points detected on the image serving as observation results, and designing a maximum posteriori framework to optimize and solve the geometrical shape and posture of the human face in the image; aiming at the movement of a three-dimensional eyeball, using sparse two-dimensional feature points for obtaining an eyearea image block through cutout, achieving alignment of the image block through position information of the feature points, further marking the areas of an iris and a pupil, and reconstructing the eyeball movement in real time through an analysis method based on synthesis; in combination with the expression parameters obtained in the previous step, combining facial expression motion with eyeballmotion to obtain a complete facial expression animation.

Description

technical field [0001] The invention relates to the field of face recognition, and in particular to a modeling and capturing method and system for three-dimensional human face and eyeball movement. Background technique [0002] In the field of virtual animation, facial animation is an important and hot research problem. Facial animation is not only the movement of facial expressions, but also the movement of eyeballs. Facial animation that covers facial expressions and eye movements will give people a more realistic and realistic animation feeling. Facial animation that ignores eye movement will give people a dull and lifeless feeling, which will cause a strong sense of discomfort. In the current scientific research work, the common method of generating face animation is to capture the face movement from the video of the real person, and add the capture of eye movement at the same time. In terms of hardware cost and resource utilization, using a monocular camera to obtain...

AI Technical Summary

Problems solved by technology

[0004] 1. Using random forest to classify pixels, the classification results obtained are relatively inaccurate, and need to be constrained by the method of central clustering in the later stage;

[0005] 2. The time consumption increases, the memory usage is serious, and the hardware requirements are high, so the system cannot be migrated to the mobile terminal for use

Images