Parametric Eye Model Reconstruction from Sparse Facial Data
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current facial capture technologies focus primarily on facial skin, neglecting important details like the eyes, which are crucial for realistic animations, and existing eye capture methods are often time-consuming and uncomfortable for actors.
Innovation Solution
A parametric eye model is developed, comprising a deformable eyeball, iris, and vein models, allowing for the reconstruction of unique eye shapes and colors from sparse input data, such as single images or multi-view face scanners, using a training database of high-quality eye scans.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If traditional eye capture methods are used, then high-quality eye details can be captured, but the capture process is time-consuming and uncomfortable for actors
Solution Approach 1:
The system performs preliminary actions by capturing facial data during standard face scanning procedures rather than requiring separate eye-specific capture sessions. The eye model is fitted to the captured facial mesh and texture data automatically, eliminating the need for dedicated eye capture time and making the process comfortable for actors during routine filming.
2Manufacturing precision
If detailed eye modeling is implemented, then realistic eye features are achieved, but the model complexity increases
Solution Approach 1:
The eye model is segmented into distinct components including an eyeball model, iris model, and sclera model with vein structures. Each component can be independently parameterized and fitted to the captured data, allowing detailed eye modeling while maintaining manageable complexity through modular architecture. The segmentation enables specific focus on different eye features without requiring the entire model to be overly complex.
Solution Approach 2:
The system uses parameter changes by adjusting model parameters such as pupil dilation, iris color, vein depth, and eyeball shape to achieve realistic eye features. These parameters are derived from the captured facial data and can be modified to create variations in eye appearance without increasing the fundamental model structure complexity, allowing detailed rendering through parameter manipulation rather than model complexity.
3Ease of manufacture
If sparse input data is used, then capture simplicity is maintained, but reconstruction quality may be compromised
Solution Approach 1:
The system introduces an intermediary approach by using a pre-parameterized eye model as a mediator between the sparse captured data and the final detailed eye reconstruction. The captured facial mesh and texture serve as input, the parameterized eye model acts as an intermediary that provides structural constraints and anatomical knowledge, and the output is a high-quality reconstructed eye. This intermediary model guides the reconstruction process to maintain quality even with limited input data.
Data Source
AI summary
Systems and techniques for generating a parametric eye model of one or more eyes are provided. The systems and techniques may include obtaining eye data from an eye model database. The eye data includes eyeball data and iris data corresponding to a plurality of eyes. The systems and techniques may further include generating an eyeball model using the eyeball data. Generating the eyeball model includes establishing correspondences among the plurality of eyes. The systems and techniques may further include generating an iris model using the iris data. Generating the iris model includes sampling one or more patches of one or more of the plurality of eyes using an iris control map and merging the one or more patches into a synthesized texture. The systems and techniques may further include generating the parametric eye model that includes the eyeball model and the iris model.