Eye State Variable Determination via Cornea-Free 3D Modeling
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing eye tracking technologies face challenges in accurately determining eye state variables such as eyeball position, gaze direction, and pupil size due to issues like corneal refraction, size-distance ambiguity, and high computational requirements, especially in head-mounted devices.
Innovation Solution
A method and system using a 3D eye model that accounts for corneal refraction, employing a non-iterative algorithm to determine eye state variables by generating synthetic images and establishing relationships between pupil image characteristics and model parameters, allowing for real-time, low-complexity calculations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If glint-based methods with complex optical setups (IR LEDs, stereo cameras) are used to determine eye state variables, then measurement precision is improved, but device complexity increases and additional hardware requirements are needed
Solution Approach 1:
The patent extracts and eliminates the cornea from the eye model to create a simplified cornea-free eye model. This allows gaze estimation to be performed without detecting corneal reflections (glints), thereby removing the need for complex IR LED illumination systems and stereo camera setups while maintaining measurement precision through alternative pupil-based methods
Solution Approach 2:
The patent uses synthetic eye images generated from a simplified eye model as a substitute for complex real-world optical setups. These synthetic images are used to train machine learning models that can estimate gaze from simple monocular images without requiring glint detection, effectively copying the essential features needed for accurate gaze estimation
2Measurement precision
If iterative numerical optimization methods are used to fit 3D eye models to time series eye observations, then measurement precision is improved, but computing power requirements increase and real-time performance is limited
Solution Approach 1:
The patent performs preliminary action by pre-training machine learning models offline using synthetic eye images generated from 3D eye models. This pre-training phase captures the complex relationships between pupil appearance and gaze parameters, allowing the trained model to make rapid real-time predictions without requiring computationally intensive iterative optimization during actual gaze estimation
Solution Approach 2:
The patent replaces the mechanical iterative numerical optimization process with a machine learning-based system. The trained neural network model directly maps pupil image features to gaze parameters in a single forward pass, substituting the step-by-step iterative optimization mechanism with a learned direct mapping that requires minimal computational resources
3Device complexity
If cornea-free eye models are used to simplify the estimation process, then device complexity is reduced, but measurement precision deteriorates due to inability to account for corneal refraction effects
Solution Approach 1:
The patent introduces machine learning models as an intermediary between the simplified cornea-free eye model and the complex optical reality. These models learn to compensate for corneal refraction effects by training on synthetic data that incorporates refraction physics, allowing the simple model structure to achieve high measurement precision through the intelligent intermediary layer
4Ease of operation
If traditional geometric eye models with fixed calibration are used, then ease of operation is improved, but reliability deteriorates when the eye tracker slips or moves on the user's head
Solution Approach 1:
The patent transitions from static fixed calibration to dynamic adaptive calibration. The system continuously estimates eyeball center coordinates and updates the mapping between pupil positions and gaze directions in real-time, allowing the eye tracker to adapt to movements and slippage on the user's head while maintaining reliability and accuracy
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
Enables accurate and efficient determination of eye state variables with reduced computational effort, overcoming corneal refraction effects and size-distance ambiguity, suitable for head-wearable devices.
Implementation Method 1
providing a first 3D eye model modeling corneal refraction
Data Source
Figure 1A~1C
Figure 2A~2C
Figure 3
AI summary
Methods, devices and systems for generating data suitable for determining at least one eye state variable of at least one eye of a subject, and methods and systems for determining such eye state variables are provided. The eye comprises an eyeball, an iris defining a pupil, and a cornea, the at least one eye state variable being derivable from at least one image of the eye taken with a camera of known camera intrinsics. In a simulation scenario, synthetic image data of a first 3D model eye which models corneal refraction can be generated for different sets of eye state variables and a given algorithm may be used to determine said eye state variables using a further 3D eye model comprising at least one parameter, in particular this model can be simpler and does not need to explicitly model corneal refraction. A characteristic of the pupil image in the synthetic images can be determined and relationships can be established between hypothetically optimal values of the at least one model parameter and the characteristic, for later use by a given algorithm to determine eye state variables based on image data of real eyes and under use of the further 3D eye model, while taking into account effects of corneal refraction of the real eyes via the characteristic-dependent eye model parameter.