Virtual Keyboard Selection Using Gaze and Hand Motion Confirmation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional XR input interfaces in artificial reality environments suffer from inaccuracy, user fatigue, and ergonomic challenges due to reliance on single input modalities like hand movement or gaze, leading to frustration and discomfort.
Innovation Solution
Combining user gaze and hand motion inputs to trigger character selections on a virtual keyboard, where hand motion meets a trigger criterion to resolve selections based on gaze input, optimizing keyboard placement for reduced eye and neck strain.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If single input modality (hand movement or gaze) is used for virtual keyboard selection, then device complexity is reduced, but input accuracy and user comfort deteriorate due to fatigue and inaccuracy
Solution Approach 1:
The patent combines two distinct input modalities (gaze tracking and hand motion tracking) into a unified input system. The gaze tracker identifies the target key while the hand motion provides confirmation, creating a hybrid input mechanism that leverages the strengths of both modalities to improve input accuracy while distributing the ergonomic burden.
Solution Approach 2:
The system creates a multi-functional input interface where the same virtual keyboard can be interacted with through multiple modalities (gaze alone, hand motion alone, or combined). This universal design allows the system to adapt to different user needs and situations, improving reliability without being constrained to a single input method.
2Ease of operation
If virtual keyboard is placed within easy reach for hand interaction, then ease of operation improves, but eye and neck strain increase due to suboptimal viewing distance
Solution Approach 1:
By merging gaze-based selection with hand motion confirmation, the system enables the virtual keyboard to be positioned at an optimal viewing distance for the eyes while still allowing reliable input through the combination of modalities. The hand motion component compensates for the increased distance, maintaining ease of operation without sacrificing ergonomic viewing conditions.
3Object-affected harmful factors
If virtual keyboard is placed at optimal viewing distance for reduced eye strain, then harmful factors decrease, but ease of operation worsens due to reduced accessibility
Solution Approach 1:
The hand motion tracking serves as an intermediary mechanism that bridges the gap between optimal viewing distance and accessibility. Even when the keyboard is positioned farther away to reduce eye strain, the hand motion tracker can still detect and interpret hand movements accurately, maintaining ease of operation despite the increased distance.
4Productivity
If conventional single modality input is used, then device complexity is reduced, but productivity decreases due to user frustration and slow input
Solution Approach 1:
The combination of gaze and hand motion inputs creates a more efficient character input system by allowing users to quickly identify keys through gaze while confirming selections through natural hand movements. This reduces the time and cognitive load required for each input action, improving overall productivity despite the increased system complexity.
Data Source
AI summary
Aspects of the present disclosure are directed to triggering virtual keyboard selections using multiple input modalities. An interface manager can display an interface, such as a virtual keyboard, to a user in an artificial reality environment. Implementations of the interface manager can track user eye gaze input and user hand input (e.g., hand or finger motion). The interface manager can resolve a character selection on the virtual keyboard according to the tracked user gaze input based on detection that the user's hand motion meets a trigger criteria. For example, the interface manager can: detect that the tracked user hand motion meets the trigger criteria at a given point in time; and resolve a selection from the virtual keyboard (e.g., selection of a displayed character) according to the tracked user gaze on the virtual keyboard at the given point in time.