Gaze Interaction Bitmap for 3D Scene Focus Target Detection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing gaze tracking technologies face inefficiencies in 3D environments, particularly in computer games and virtual reality, due to the complexity of handling numerous occluders and interactable objects, leading to high computational costs and inaccuracies when determining if a user's gaze is directed at specific zones, especially when objects are not visually rendered or move out of the view frustum.
Innovation Solution
The solution involves creating a dynamic 'gaze interaction bitmap' that stores information about gaze-interactable zones in a 2D viewport context, rather than the 3D world, allowing for efficient generation of interaction masks and reducing computational load by considering only relevant zones, and using separate visual and gaze-detection cameras with different viewports to track gaze direction effectively.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If traditional gaze tracking methods are used in 3D environments with multiple occluders and interactable objects, then the system can detect gaze direction, but the computational cost becomes excessively high and accuracy decreases
Solution Approach 1:
The patent segments the 3D scene into a 2D projection map that represents only the interactable zones from the camera's viewpoint. This segmentation reduces the complexity by transforming the three-dimensional problem into a two-dimensional problem, where gaze tracking only needs to consider the projected 2D positions of interactable objects rather than their full 3D spatial relationships, thereby improving computational efficiency while maintaining accuracy.
Solution Approach 2:
The patent employs dimensionality reduction by projecting 3D interactable objects onto a 2D plane that corresponds to the camera's viewport. This dimensional transformation allows the system to track gaze by comparing the 2D gaze direction with the 2D projected positions of interactable zones, significantly reducing computational requirements while preserving the essential spatial information needed for accurate gaze detection.
2Loss of information
If the system tracks all objects in the 3D scene including occluders, then complete scene understanding is achieved, but the processing load increases significantly
Solution Approach 1:
The patent extracts only the essential information needed for gaze interaction by projecting interactable zones onto a 2D map from the camera's viewpoint. Non-interactive occluders and irrelevant scene elements are excluded from the processing, as they do not contribute to gaze interaction decisions. This extraction approach maintains the necessary scene information for interaction while dramatically reducing processing complexity.
Solution Approach 2:
The patent applies local quality by creating a specialized 2D projection map that contains detailed information only about interactable zones in the regions where gaze interaction is possible. Rather than processing the entire 3D scene uniformly, the system focuses computational resources on the specific 2D regions corresponding to interactable objects, optimizing the balance between information retention and processing efficiency.
3Productivity
If gaze tracking is limited to visually rendered objects within the view frustum, then processing is simplified, but gaze tracking of invisible or off-screen objects is lost
Solution Approach 1:
The patent creates a universal 2D projection map that can represent interactable zones regardless of their visibility in the current camera view. The projection system maintains information about all interactable zones in the scene, allowing the gaze tracking system to detect gaze direction toward objects that may be off-screen, occluded, or not currently visible, thereby extending the versatility of gaze interaction beyond what traditional viewport-limited systems can achieve.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
This approach enables accurate and efficient determination of a user's gaze direction in 3D scenes, reducing computational expenses and handling occluders and world geometry complexities, while allowing gaze tracking of objects not visually rendered, thereby improving interaction detection in complex 3D environments.
Implementation Method 1
illuminating a region in which the eye is sought with infrared radiation
Implementation Method 2
light enters the eye and is reflected or absorbed and re-emitted through the pupil
Implementation Method 3
capturing an image of the region; and detecting bright spots in the image
Data Source
Figure 1~2
Figure 3~4
Figure 5
AI summary
A method for determining a focus target of a user's gaze in a three-dimensional ("3D") scene (10) is disclosed. The method may include determining a first gaze direction (805) of a user into a 3D scene (10), where the 3D scene (10) includes a plurality of components. The method may also include executing a first plurality of line traces (810) in the 3D scene (10), where each of the first plurality of line traces (810) is in proximity to the first gaze direction (805). The method may further include determining a confidence value for each component intersected by at least one of the first plurality of line traces (810). The method may additionally include identifying as a focus target of the user the component having the highest confidence value of all components intersected by at least one of the first plurality of line traces (810).