3D Gaze Detection Using Interactor Masks for Occlusion Handling
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing gaze tracking technologies face inefficiencies in 3D scenarios due to complex occluder management and high computational costs, especially when dealing with numerous interactive objects and invisible or occluded gaze-interactable zones, leading to inaccurate and processing-intensive solutions.
Innovation Solution
A method and apparatus for dynamically generating an 'interactor mask' using a bitmap that stores information about gaze-interactable zones in a 3D scene, allowing for efficient gaze tracking by considering only relevant zones and leveraging existing graphics rendering capabilities, reducing the burden on developers and improving interaction in 3D environments.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If traditional gaze tracking methods are used in 3D scenes with multiple occluders, then gaze direction can be detected, but computational cost increases significantly and processing efficiency decreases
Solution Approach 1:
The patent segments the complex 3D scene into simplified 2D screen space representations. By projecting 3D occluders and gaze-interactable zones onto a 2D bitmap, the system divides the complex spatial reasoning problem into manageable 2D geometric operations, significantly reducing computational complexity while maintaining detection accuracy.
Solution Approach 2:
The patent creates a 2D copy (bitmap) of the 3D scene geometry that contains all necessary occluder and zone information. This bitmap serves as a simplified representation that captures the essential spatial relationships without requiring full 3D processing, enabling efficient gaze detection through 2D line-intersection tests.
2Reliability
If all objects in the 3D scene are considered as potential occluders, then gaze tracking accuracy is maintained, but device complexity and processing burden increase
Solution Approach 1:
The patent extracts only the essential geometric information (occluder positions and gaze-interactable zones) from the full 3D scene and represents them in a simplified 2D bitmap. This extraction process filters out unnecessary 3D details while retaining the critical spatial relationships needed for accurate gaze tracking.
Solution Approach 2:
The patent transforms the problem from 3D spatial reasoning to 2D geometric processing by changing the dimensional parameter. The bitmap encoding converts complex 3D occluder management into simple 2D screen space coordinates, making the system more reliable and easier to manage.
3Adaptability or versatility
If gaze-interactable zones are not visible in the visual scene, then invisible interactions can be enabled, but detection accuracy becomes more difficult to maintain
Solution Approach 1:
The patent introduces a separate gaze detection bitmap as an intermediary representation that independently tracks gaze-interactable zones regardless of their visual visibility. This mediator bitmap maintains the spatial information of invisible zones, allowing the system to detect gaze direction toward them accurately without relying on visual scene information.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
This approach enables efficient and accurate determination of a user's gaze direction towards interactable zones in 3D scenes, reducing computational costs and simplifying implementation by converting the complex 3D occluder management into a 2D screen space issue, thus enhancing the incorporation of gaze interaction in 3D applications.
Implementation Method 1
illuminating a region in which the eye is sought with infrared radiation
Implementation Method 2
light enters the eye and is reflected or absorbed and re-emitted through the pupil
Implementation Method 3
capturing an image of the region; and detecting bright spots in the image
Data Source
Figure 1~2
Figure 3~4
Figure 5
AI summary
A method for determining if a user's gaze is directed in the direction of a zone of interest in a 3D scene comprises: providing a 3D scene containing a zone of interest; associating a property with the zone of interest; creating a bitmap representing the location of the zone of interest in a projected view of the 3D scene, each pixel of the bitmap to which the zone of interest is projected storing the property of the zone of interest; detecting the direction of the user's gaze; using the bitmap to determine if the detected user's gaze is directed in the direction of the zone of interest.