Gaze-guided UI Element Selection in XR Interfaces
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current methods for interacting with user interface elements in computer-generated environments are not efficient or intuitive, particularly when it comes to one-handed selection and actuation, which can hinder user experience.
Innovation Solution
The implementation of gestures such as pinch and release, pinch and hold, and eye gaze data to select and manipulate user interface elements, allowing for indirect selection and manipulation without the need for direct proximity, enhancing usability and efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If traditional direct manipulation methods are used for user interface elements, then selection precision is improved, but operation complexity and time consumption increase
Solution Approach 1:
The system performs preliminary actions by automatically identifying and highlighting potential user interface elements based on eye gaze data before the user completes the selection gesture. This pre-positioning of the selection target reduces the time required for precise selection while maintaining accuracy through the subsequent pinch and release confirmation gesture.
Solution Approach 2:
Eye gaze data serves as an intermediary that bridges the user's attention and the selection target. The system uses gaze tracking to predict which UI element the user intends to select, allowing the interface to prepare and confirm the selection without requiring the user to manually navigate to the precise location, thus reducing time consumption while maintaining selection precision.
2Measurement precision
If direct proximity interaction is required for user interface elements, then selection accuracy is improved, but ease of operation deteriorates
Solution Approach 1:
Eye gaze tracking acts as an intermediary mechanism that captures the user's attention direction without requiring physical proximity to the UI element. The system translates gaze data into selection predictions, allowing users to indicate their intent from a distance while maintaining high selection accuracy through the combination of gaze prediction and confirmation gesture.
Solution Approach 2:
The patent replaces the mechanical requirement of physical proximity for selection with an optical/biological mechanism (eye gaze tracking). This substitution eliminates the need for users to move their hands close to the display while maintaining selection accuracy through gaze-based prediction and gesture confirmation.
3Reliability
If multiple interaction steps are used for element selection, then reliability of selection is improved, but productivity deteriorates
Solution Approach 1:
The system performs preliminary analysis of eye gaze data to predict the user's intended selection before the confirmation gesture is completed. This pre-processing of selection intent reduces the effective interaction time while maintaining reliability, as the system is already prepared to confirm or correct the selection based on the predicted target.
Solution Approach 2:
The system provides continuous feedback by monitoring eye gaze data and predicting selection targets in real-time. This feedback mechanism allows the system to guide the user through the interaction process, confirming the predicted target before final selection, thereby maintaining high reliability while reducing the perceived number of interaction steps through seamless feedback loops.
Data Source
AI summary
Displaying and manipulating user interface elements in a computer-generated environment is disclosed. In some embodiments, a user is able to use a pinch and hold gesture to seamlessly and efficiently display and isolate a slider and then manipulate that slider without having to modify the pinch and hold gesture. In some embodiments, gaze data can be used to coarsely identify a focus element, and hand movement can then be used for fine identification of the focus element.