XR Eye-Motion Interaction for Reliable Object Selection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
XR systems face challenges in detecting user gaze and field of view changes, and users often need cumbersome input methods like hand gestures or joysticks to interact, making it difficult to determine which object is of interest and obtain information in the XR environment.
Innovation Solution
The system detects eyelid motion to regenerate objects with modified detail, performs actions based on eyelid motion identifiers, and uses gaze shift indicators and voice commands to enhance interaction in XR environments.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If hand gestures or joystick are used for navigation and interaction in XR environment, then user can control and interact with objects, but operation becomes cumbersome and inconvenient, taking away from the immersive experience
Solution Approach 1:
The system monitors the user's own physiological signals (eye movements, pupil dilation, blink patterns) to automatically determine gaze direction and trigger interactions, eliminating the need for external control devices. The user's natural eye movements become the control mechanism, making the system self-serve the user's intent without requiring manual intervention
Solution Approach 2:
The patent replaces mechanical interaction devices (hand gestures, joysticks) with optical detection methods (eye tracking cameras, pupil monitoring). By substituting mechanical control systems with optical sensing of physiological signals, the system achieves more natural and less cumbersome interaction while maintaining full control capabilities
2Measurement precision
If pupil dilation and constriction are monitored to determine user gaze, then field of view information can be obtained, but reliability is reduced because user cannot control pupil and it varies with lighting conditions
Solution Approach 1:
The system introduces an intermediary reference frame (the iris pattern) that is directly linked to eye orientation. By tracking the position and rotation of the iris relative to the eye socket, the system obtains a more reliable gaze indicator that is not confounded by pupil size changes due to lighting or accommodation, effectively using the iris as a mediator between the eye's mechanical movement and the detection system
Solution Approach 2:
The patent focuses detection on specific local features of the eye (iris pattern, eyelid position, corneal reflection) rather than relying on global changes like pupil size. By concentrating measurement on local structures that maintain consistent geometric relationships with gaze direction, the system achieves more reliable and lighting-independent detection
3Adaptability or versatility
If multiple objects are present in user field of view, then environment complexity increases, but it becomes difficult to determine which object user desires to interact with
Solution Approach 1:
The system uses visual feedback indicators (such as highlighting, bounding boxes, or color changes) to mark the currently selected object based on gaze detection. This visual distinction helps the user clearly identify which object will be activated by the next blink command, eliminating ambiguity when multiple objects are present in the field of view
4Ease of operation
If eyelid motion is detected to select objects and perform actions, then interaction becomes more natural and convenient, but system complexity increases due to sensor integration and motion analysis
Solution Approach 1:
The eye tracking camera and sensor system serve multiple functions: detecting gaze direction, monitoring pupil size, tracking eyelid position, and identifying blink patterns. By designing a single multi-functional detection system rather than separate sensors for each function, the patent reduces overall system complexity while enabling comprehensive eye-based interaction
Data Source
AI summary
Systems and methods are described for extended reality environment interaction. An extended reality environment including an object is generated for display, and a table of eyelid motion identifiers and corresponding actions performable on the object in the extended reality environment is stored in memory. An eyelid motion is detected by using a sensor, and the detected eyelid motion is matched to one of the stored eyelid motion identifiers. An updated version of the extended reality environment, based on the action that corresponds to the matched eyelid motion, is generated for display, in response to matching the detected eyelid motion to one of the stored eyelid motion identifiers.