Extended Reality Eye-Gaze Interaction for Object Selection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
XR systems face challenges in detecting user's field of view and gaze accurately, particularly in determining which object the user desires to interact with, and current interaction methods such as hand gestures or joysticks are cumbersome.
Innovation Solution
The system identifies objects in the user's field of view and detects eyelid motion to regenerate the object with a modified level of detail, allowing users to interact with objects using eyelid motions and voice commands, and provides opacity-based indicators to guide gaze shifts.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If hand gestures or joystick are used for navigation and interaction in XR environment, then user can control the environment, but the operation becomes cumbersome and inconvenient, taking away from the XR experience
Solution Approach 1:
The patent replaces mechanical interaction methods (hand gestures, joysticks) with physiological-based control through eye tracking technology. The system uses ocular sensors to detect gaze direction, saccades, and fixation patterns, allowing users to navigate and interact with XR environments through natural eye movements rather than mechanical controls, thereby improving ease of operation while reducing interaction complexity
Solution Approach 2:
The system enables self-service interaction by automatically detecting user intent through eye movements and autonomously executing navigation or object selection actions. The eye tracking system continuously monitors gaze patterns and interprets them as interaction commands without requiring explicit user input, allowing the system to serve itself by translating physiological signals directly into environmental control
2Measurement precision
If pupil dilation and constriction are monitored to determine user gaze, then some information can be obtained, but it is not a reliable way to determine gaze or field of view since users cannot control their pupils
Solution Approach 1:
The patent transitions from monitoring involuntary physiological parameters (pupil dilation/constriction) to tracking controllable ocular parameters (eye position, gaze direction, saccades, fixation duration). By changing the measured parameters from those beyond user control to those that reliably indicate intentional gaze, the system achieves both precise and reliable gaze detection for determining user field of view and object interaction intent
Solution Approach 2:
The system introduces an intermediary layer of analysis by using ocular sensors to capture raw eye movement data and then processing this data through algorithms that interpret saccades and fixations as meaningful interaction signals. This intermediary processing transforms involuntary eye movements into reliable gaze indicators, bridging the gap between physiological motion and intentional user focus
3Adaptability or versatility
If multiple objects are present in the user's field of view, then the environment is rich and interactive, but it becomes difficult to determine which object the user desires to interact with
Solution Approach 1:
The system implements feedback by continuously monitoring eye gaze patterns and providing real-time interpretation of user intent. When the user's gaze fixates on a specific object within the field of view, the system detects this through sustained fixation patterns and saccade termination, then uses this feedback to determine which object the user wishes to interact with, effectively resolving the ambiguity of multiple objects in the environment
Solution Approach 2:
The system performs preliminary action by pre-processing and analyzing eye movement data to predict user intent before explicit interaction occurs. By detecting gaze direction and fixation patterns in advance, the system prepares to identify the target object, enabling seamless transition from viewing multiple objects to selecting the intended interaction target without requiring additional user effort
Data Source
AI summary
Systems and methods are described for extended reality environment interaction. An extended reality environment including an object is generated for display, and a first sensor is used to detect that a gaze has shifted from a first portion of the extended reality environment to a second portion of the extended reality environment, where the object is excluded from the first portion of the extended reality environment and included in the second portion of the extended reality environment. An indicator of the shift in the gaze is generated for display within the extended reality environment in response to detecting the gaze shift, and a voice command is detected by a second sensor while the indicator is in a vicinity of the object. In response to detecting the voice command, an action corresponding to the voice command may be executed.