3D Virtual Object Selection Using Eye and Hand Tracking
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for interacting with augmented and virtual reality environments are cumbersome, inefficient, and create a significant cognitive burden on users, often requiring multiple inputs and providing insufficient feedback, leading to errors and energy wastage, particularly in battery-operated devices.
Innovation Solution
The implementation of computer systems with improved user interfaces that utilize touch-sensitive displays, eye-tracking, hand-tracking, and tactile output generators to reduce the number and complexity of user inputs, providing intuitive interactions by correlating inputs with device responses and enhancing visual feedback.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If conventional input methods are used for interacting with virtual objects, then the system can process user inputs, but the user input complexity and cognitive burden increase significantly
Solution Approach 1:
The system automatically determines user intent by tracking eye gaze and hand position, eliminating the need for explicit selection inputs. The interface serves itself by interpreting implicit user attention and movement patterns, reducing input complexity while maintaining operational efficiency
Solution Approach 2:
The patent replaces traditional mechanical input devices (buttons, keyboards, explicit gestures) with optical tracking systems (eye-tracking cameras, hand-tracking sensors). This substitution captures natural user behavior passively, reducing the cognitive burden of learning and executing complex input sequences
2Productivity
If multiple input steps are required to achieve desired outcomes, then the system can perform precise operations, but the interaction time and energy consumption increase
Solution Approach 1:
The system performs preliminary tracking of user eye gaze and hand position continuously, so when a user looks at or reaches for a virtual object, the system is already prepared to interpret and respond to the intent. This eliminates intermediate selection steps and reduces interaction time
Solution Approach 2:
The patent maintains continuous tracking of user attention and movement throughout the interaction session, allowing seamless transitions between objects and operations without requiring re-selection or re-positioning inputs. This continuous action maintains productivity while minimizing time loss
3Loss of information
If conventional feedback mechanisms are used, then the system can provide operational information, but the feedback sufficiency and user understanding decrease
Solution Approach 1:
The system provides real-time visual feedback by displaying indicators of user gaze position and hand tracking status, allowing users to understand what the system is detecting and how their actions are being interpreted. This transparent feedback loop improves user understanding without requiring complex feedback mechanisms
Solution Approach 2:
The patent uses color-coded visual indicators to represent different interaction states (e.g., target object highlight, selection status, tracking accuracy). These color changes provide intuitive, immediate feedback about system state and user intent recognition, enhancing understanding with simple visual cues
4Measurement precision
If traditional selection methods are used for virtual objects, then the system can maintain selection state, but the selection precision and error rate increase
Solution Approach 1:
The patent introduces an intermediary tracking system that captures intermediate user intent signals (eye gaze direction, hand movement trajectory) before final selection occurs. This intermediary layer filters out accidental inputs and provides a buffer for intent verification, reducing selection errors while maintaining precision
Solution Approach 2:
The system performs preliminary verification of user intent by analyzing the stability and duration of gaze fixation or hand position before confirming selection. This preliminary action filters out accidental or premature selections, improving reliability without compromising selection precision
Data Source
AI summary
In some embodiments, a computer system performs different object selection-related operations. In some embodiments, a computer system places objects at locations in a displayed region based on attention of a user. In some embodiments, a computer system displays a container virtual object with curvature in a three-dimensional environment.