Virtual Controls With Gaze and Hand Tracking for 3D Object Interaction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for interacting with virtual/augmented reality environments are cumbersome, inefficient, and complex, leading to a significant cognitive burden on users and energy wastage, particularly in battery-operated devices.
Innovation Solution
The system enhances interactions by using electronic devices with improved methods and interfaces that reduce the number and nature of user inputs through gaze and hand tracking, enabling efficient manipulation of virtual objects and user interface elements, including selectable and slider elements, and providing enhanced visual feedback.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If conventional input methods are used to interact with virtual/augmented reality environments, then basic functionality is achieved, but user cognitive burden increases and interaction efficiency decreases
Solution Approach 1:
The system provides visual feedback by displaying virtual controls and affordances within the augmented reality environment that correspond to physical controls. This feedback loop helps users understand the connection between their physical inputs and device responses, reducing cognitive burden and improving interaction efficiency.
Solution Approach 2:
The patent introduces virtual controls and affordances as intermediaries between physical controls and virtual/augmented reality elements. These virtual controls serve as a mediating layer that simplifies complex interactions by providing intuitive visual representations and guidance within the AR/VR environment.
2Productivity
If multiple input steps are required to achieve desired outcomes, then precise control is achieved, but interaction time increases and energy consumption rises
Solution Approach 1:
The system performs preliminary actions by automatically detecting user intent through gaze tracking and hand tracking, then pre-positioning virtual controls or preparing expected operations. This reduces the number of explicit input steps required, thereby decreasing interaction time and energy consumption while maintaining productivity.
Solution Approach 2:
The system serves itself by using sensors to automatically detect and interpret user intentions without requiring multiple explicit inputs. The gaze tracking and hand tracking systems enable the device to understand user intent autonomously, reducing interaction steps and improving interaction speed.
3Measurement precision
If detailed control mechanisms are provided, then manipulation precision is improved, but interface complexity increases
Solution Approach 1:
The patent applies local quality by providing detailed control mechanisms only where and when needed within the augmented reality environment. Virtual controls and affordances are contextually displayed based on user gaze and interaction state, offering precision control at specific locations without overwhelming the entire interface with complexity.
Solution Approach 2:
The system adds spatial dimensionality to control precision by using gaze direction and hand position in three-dimensional space to determine control actions. This dimensional approach allows precise control through natural spatial gestures rather than complex two-dimensional interface elements, reducing perceived interface complexity.
Data Source
AI summary
In some embodiments, an electronic device enhances interactions with virtual objects in a three-dimensional environment. In some embodiments, an electronic device enhances interactions with selectable user interface elements. In some embodiments, an electronic device enhances interactions with slider user interface elements. In some embodiments, an electronic device moves virtual objects in a three-dimensional environment and facilitates accessing actions associated with virtual objects.