Current Gaze Direction Tracking with 3D Reprojection Compensation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing gaze tracking technologies in 3D scenes face latency issues due to reprojections, leading to inaccurate determination of user gaze direction and limiting interaction granularity, especially in dynamic environments.
Innovation Solution
A method and system that determine a modified gaze direction by considering both the physical gaze direction and reprojection transformation applied to the projection image, using a reprojection means and modified gaze direction detection means to account for transformations downstream of rendering.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If reprojection transformation is applied to the projection image to adapt to dynamic 3D scenes, then the adaptability of the gaze tracking system is improved, but latency increases and measurement precision deteriorates
Solution Approach 1:
The system performs preliminary actions by predicting the reprojection transformation that will be applied to the projection image before the actual reprojection occurs. By anticipating the transformation based on scene dynamics and camera movements, the gaze direction can be pre-adjusted to compensate for the upcoming reprojection, thereby reducing latency and maintaining precision in dynamic 3D environments.
2Adaptability or versatility
If reprojection transformation is applied to account for dynamic scene changes, then adaptability is improved, but measurement precision of gaze direction deteriorates due to latency
Solution Approach 1:
The system implements feedback by continuously monitoring the actual reprojection transformation applied to the projection image and using this information to refine gaze direction predictions. The detected reprojection transformation feeds back into the gaze direction determination process, allowing the system to compensate for transformations and maintain accurate gaze measurements even in dynamically changing 3D scenes.
Solution Approach 2:
By predicting the reprojection transformation in advance based on scene dynamics and camera movements, the system performs preliminary adjustments to the gaze direction determination process. This preliminary action allows the system to pre-compensate for upcoming reprojections, maintaining measurement precision while adapting to dynamic 3D environments.
3Device complexity
If traditional gaze tracking is used without considering reprojection transformation, then device complexity is reduced, but adaptability to dynamic 3D scenes deteriorates
Solution Approach 1:
The system introduces an intermediary component that detects the reprojection transformation applied to the projection image and uses this information to adjust gaze direction determinations. This intermediary layer acts as a bridge between the traditional gaze tracking system and the dynamic 3D scene requirements, adding adaptability without requiring complete system redesign. The intermediary processes transformation information and translates it into compensatory adjustments for gaze tracking.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
This approach provides accurate and timely gaze direction determination, enhancing user interaction precision and reducing latency in dynamic 3D scenes by accounting for reprojections.
Implementation Method 1
This can be done, for example, by illuminating a region in which the eye is sought with infrared radiation; capturing an image of the region; and detecting bright spots in the image that derive from the pupil and cornea of the eye.
Implementation Method 2
A separate, smaller bright spot (also referred to as a glint) is created by the cornea.
Data Source
AI summary
Method for determining a current gaze direction of a user in relation to a three-dimensional (“3D”) scene, the 3D scene being sampled by a rendering function to produce at least one projection image of the 3D scene, wherein the method comprises the steps: determining, by a reprojection means, a reprojection transformation to be applied to the projection image before being displayed so as to be visible to the user at a gaze time point; determining, by a gaze direction detection means, a physical gaze direction of the user at said gaze time point; and determining a modified gaze direction of the user at said gaze time point, the modified gaze direction being determined in relation to the 3D scene based on both the physical gaze direction and the reprojection transformation. The disclosure also relates to a system and to a computer software function.