Mixed Reality Headset Gaze and Gesture Selection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Developing a simple and intuitive paradigm for user interaction with mixed reality devices in mixed reality environments has proven elusive, as existing technologies struggle to effectively engage users with virtual and physical objects.
Innovation Solution
A method utilizing head pose data to identify gross selectable targets, combined with eye-tracking data to specify fine selectable targets, and gesture data to perform operations on these targets, providing visual feedback through distinct highlighting and intuitive gesture recognition.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If traditional interaction methods are used in mixed reality environments, then device functionality is available, but user interaction complexity and difficulty increase
Solution Approach 1:
The patent replaces traditional mechanical interaction devices (controllers, keyboards, mice) with a gaze-based selection system. Users interact with virtual objects by simply looking at them with the head-mounted display device, eliminating the need for separate control mechanisms and simplifying the interaction paradigm while maintaining full device functionality
Solution Approach 2:
The system allows users to interact with virtual objects using their natural line of sight and head movements. The head-mounted display device automatically tracks gaze direction and interprets it as selection intent, enabling users to interact without requiring learning of complex control schemes or additional input devices
2Adaptability or versatility
If multiple selectable targets are identified in the mixed reality environment, then user engagement increases, but target selection precision becomes difficult
Solution Approach 1:
The patent introduces a hierarchical selection system with two levels: gross selectable targets identified by head pose data and fine selectable targets identified by eye-tracking data. This multi-dimensional approach allows the system to manage multiple targets effectively by first narrowing down to a region of interest using head position, then precisely identifying the specific target using eye gaze direction
Solution Approach 2:
The target selection process is divided into two distinct stages: gross target selection based on head pose and fine target selection based on eye tracking. This segmentation allows each stage to focus on a specific level of precision, with head pose data identifying general areas and eye-tracking data pinpointing exact targets, thereby improving overall selection accuracy
Data Source
AI summary
Embodiments that relate to presenting a mixed reality environment via a mixed reality display device are disclosed. For example, one disclosed embodiment provides a method for presenting a mixed reality environment via a head-mounted display device. The method includes using head pose data to generally identify one or more gross selectable targets within a sub-region of a spatial region occupied by the mixed reality environment. The method further includes specifically identifying a fine selectable target from among the gross selectable targets based on eye-tracking data. Gesture data is then used to identify a gesture, and an operation associated with the identified gesture is performed on the fine selectable target.