3D Virtual Object Selection with Head Pose and Hand Gestures
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing VR, AR, and MR technologies face challenges in providing a comfortable and precise interaction with virtual objects in three-dimensional space, often leading to user fatigue due to the difficulty in controlling head poses for selection and the need for precise movements.
Innovation Solution
A wearable system that switches between user input modes based on user pose and hand gestures, allowing preliminary object identification through head poses and precise selection using hand gestures, and adapts input modes based on contextual information and object characteristics.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If head pose control is used for object selection in VR/AR/MR, then interaction versatility is improved, but measurement precision deteriorates leading to user fatigue
Solution Approach 1:
The patent segments the interaction process into two distinct phases: a first input mode using head pose for preliminary object identification (coarse selection), and a second input mode using hand gestures for precise object selection (fine selection). This segmentation allows each mode to excel at its specific task while avoiding the weaknesses of using a single mode for the entire interaction sequence.
2Measurement precision
If precise hand gestures are required for object selection, then measurement precision is improved, but ease of operation deteriorates due to user fatigue
Solution Approach 1:
The system performs preliminary object identification using head pose before requiring hand gestures for final selection. This preliminary action narrows down the target objects, so users only need to perform precise hand gestures for a limited set of candidates rather than searching through all virtual objects, thereby reducing overall fatigue.
3Device complexity
If single user input mode is used, then device complexity is reduced, but adaptability deteriorates for different interaction scenarios
Solution Approach 1:
The patent implements a dynamic input mode switching mechanism that automatically transitions between the first input mode (head pose-based) and the second input mode (hand gesture-based) based on the interaction context and object characteristics. This dynamic adaptation allows the system to optimize for either versatility or precision depending on the situation without requiring manual configuration.
Data Source
AI summary
Systems and methods for interacting with virtual objects in a three-dimensional space using a wearable system are disclosed. The wearable system can be programmed to permit user interaction with interactable objects in a field of regard (FOR) of a user. The FOR includes a portion of the environment around the user that is capable of being perceived by the user via the AR system. The system can determine a group of interactable objects in the FOR of the user and determine a pose of the user. The system can update, based on a change in the pose or a field of view (FOV) of the user, a subgroup of the interactable objects that are located in the FOV of the user and receive a selection of a target interactable object from the subgroup of interactable objects. The system can initiate a selection event on the target interactable object.