Gaze Intent Estimation for Real-Time Character Animation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current computer animation techniques struggle to create realistic eye movements for animated figures, leading to unnatural and unconvincing renderings, especially in real-time applications like video games, due to the requirement for extensive processor time and manual corrections.
Innovation Solution
A system using a plurality of cameras with known positions relative to an eye captures images to determine the eye-line and gaze-object, adjusting the eye-line in response to changes, allowing for real-time rendering of realistic eye movements without requiring significant processor or animator time.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If traditional eye animation techniques are used to correct gaze movements, then realistic eye movements can be achieved, but expensive processor time and manual animator time are required
Solution Approach 1:
The patent creates a copy of the actor's eye movements by capturing images with cameras positioned at known locations and generating an eye model that replicates the actor's gaze behavior. This copy can then be applied to animated characters without requiring manual animator intervention for each eye movement correction.
Solution Approach 2:
The patent replaces the mechanical system of manual animator correction with an automated computational system that uses image processing and mathematical modeling to calculate and apply eye movement corrections automatically, eliminating the need for manual frame-by-frame adjustment.
2Reliability
If traditional eye animation techniques are used to correct gaze movements, then realistic eye movements can be achieved, but expensive processor time is required
Solution Approach 1:
The patent performs preliminary actions by capturing eye movement data and creating the eye model in advance during motion capture sessions. This pre-computed eye model can then be efficiently applied to animated characters during rendering without requiring expensive real-time processor calculations for each frame.
Solution Approach 2:
By creating a reusable copy of the eye movement model that can be applied across multiple scenes and characters, the system avoids repeating expensive processor-intensive calculations for each new animation scenario, significantly reducing overall computational requirements.
3Reliability
If manual corrections are used to adjust animated eyes, then realistic eye movements can be achieved, but the process becomes arduous and impractical
Solution Approach 1:
The system performs self-service by automatically capturing eye movement data, processing images, calculating eye lines, and generating corrected eye animations without human intervention. The entire pipeline from data capture to final animation correction is automated, making the process both easy to operate and highly efficient.
Solution Approach 2:
The patent replaces the arduous manual correction process with an automated computational mechanism that handles all eye movement adjustments programmatically, transforming a labor-intensive manual task into an efficient automated process that requires minimal operator input.
4Reliability
If expensive eye specific animation is added to body and face animation, then realistic eye movements can be achieved, but device complexity and cost increase
Solution Approach 1:
The patent merges the eye animation process with the existing motion capture and facial animation pipeline by using the same camera system and data processing infrastructure. The eye model is integrated into the existing animation workflow, eliminating the need for separate expensive eye-specific animation systems and reducing overall device complexity.
Data Source
AI summary
Any image capturing system, and a method to use same, that includes one or more image capturing devices that have known and fixed positions relative to one another and the eye of a subject, wherein the image capturing devices capture a plurality of images of eye of a subject during a performance is disclosed. The images of the eye, along with the known and fixed relative positions of the image capturing devices are used define eye-lines for the eye of the subject. The eye-lines can then be used to manually or automatically retarget the eye of a rendered or animated figure based on the performance of the subject to retarget the gaze intent of the rendered or animated figure within a scene.