Gaze-Assisted Gesture Control for AR and VR Input
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing AR and VR systems face challenges with gesture tracking, including noisy or biased trajectories, inefficiency, user fatigue, and poor visual feedback, which impede accurate and swift command input.
Innovation Solution
The implementation of gaze-assisted gesture control systems that combine gesture tracking with implicit eye gaze tracking, allowing the system to determine when a user fixates on a location and adjust cursor speed accordingly, thereby improving gesture control accuracy and speed.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If gesture tracking is used for control in AR/VR systems, then user input capability is provided, but the trajectories become noisy or biased reducing accuracy
Solution Approach 1:
The patent combines gesture tracking data with eye gaze tracking data to control the cursor. By merging these two input modalities, the system leverages the spatial information from gestures and the intent information from gaze, resulting in more accurate and reliable cursor control than either modality alone could provide.
Solution Approach 2:
The patent introduces an intermediary processing layer that fuses gesture and gaze data. This intermediary system processes both input streams, determines their compatibility, and generates a unified control signal, acting as a mediator between the raw inputs and the final cursor movement.
2Productivity
If traditional gesture tracking is used, then control input is possible, but the system becomes inefficient and slow
Solution Approach 1:
The patent uses eye gaze tracking to predict the user's intended target before the gesture is completed. By determining the fixation position in advance, the system can prepare for the upcoming gesture and reduce the time needed to process and respond to the input, thereby improving efficiency and reducing input time.
Solution Approach 2:
The patent dynamically adjusts cursor speed based on the compatibility between gesture and gaze data. When the gesture direction aligns with the gaze direction, the cursor moves faster; when they conflict, the system slows down or corrects the trajectory. This dynamic speed adjustment optimizes both productivity and input time.
3Measurement precision
If precise gesture movements are required for accuracy, then input precision improves, but user fatigue increases
Solution Approach 1:
The eye gaze tracker serves as an intermediary that reduces the precision burden on the user's hand movements. By using gaze to determine intent and gesture to confirm and execute, the system allows users to make larger, more comfortable gestures while maintaining high input accuracy, thereby reducing fatigue.
Solution Approach 2:
The patent requires only partial precision from the gesture component, as the gaze component provides the primary directional intent. The gesture needs only to confirm the general direction and trigger the action, rather than requiring precise positioning throughout the entire movement trajectory, reducing the physical effort and fatigue for the user.
4Loss of information
If gesture tracking alone is used, then control is possible, but visual feedback is poor
Solution Approach 1:
The patent implements enhanced visual feedback by displaying the predicted fixation position (based on gaze) and showing how the gesture will move the cursor toward that target. This feedback loop helps users understand the system's interpretation of their intent and the expected outcome, improving control clarity and reducing uncertainty.
Data Source
AI summary
The disclosed computer-implemented method may include capturing, by a computing device, a current position of a wearable device and a current position of user eye gaze at a current time. The method may also include determining that a user eye gaze is at a fixation position based on a speed of gaze movement. Additionally, the method may include calculating a direction of cursor movement by comparing a current cursor position with a previous cursor position at a previous time. Furthermore, the method may include calculating a likelihood of the fixation position being a target cursor position based on a difference between the direction of cursor movement and a direction from the current cursor position to the fixation position. Finally, the method may include increasing a speed of cursor movement toward the fixation position based on the likelihood. Various other methods, systems, and computer-readable media are also disclosed.