3D Environment Application Display via Wrist Gesture Recognition
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing computer systems for augmented reality environments are cumbersome, inefficient, and limited, with inadequate feedback for interacting with virtual objects, requiring multiple inputs for desired outcomes, and being complex, tedious, and error-prone, leading to a significant cognitive burden on users.
Innovation Solution
The system uses a computer-generated three-dimensional environment where application icons are displayed at positions corresponding to a user's wrist, allowing inputs directed to the wrist to activate applications, and the system automatically adjusts display sizes, orientations, and interaction models based on user hand postures and gestures to enhance interaction efficiency and user experience.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If conventional methods for interacting with virtual objects are used, then the system can provide basic functionality, but the user interface becomes cumbersome and requires multiple inputs achieving desired outcomes
Solution Approach 1:
The system automatically determines user intent and executes actions based on hand gesture recognition without requiring explicit command sequences. The virtual assistant proactively identifies when an object should be opened or activated based on contextual hand movements, eliminating the need for users to manually navigate through multiple input steps.
Solution Approach 2:
The patent replaces complex mechanical input sequences with intuitive hand gesture recognition. Instead of requiring users to press buttons, navigate menus, or combine multiple inputs, the system uses visual recognition of hand movements to directly trigger actions, substituting mechanical input complexity with a more natural interaction paradigm.
2Loss of information
If detailed feedback is provided for each input, then user understanding improves, but the system becomes more complex and time-consuming
Solution Approach 1:
The system provides immediate visual feedback by highlighting the selected virtual object or action in response to hand gestures. This feedback mechanism confirms to users that their gesture was correctly interpreted and executed, eliminating uncertainty without requiring lengthy explanations or multiple confirmation steps.
Solution Approach 2:
The virtual assistant anticipates user needs and pre-performs actions based on contextual hand gestures before the user completes their intended interaction. This preliminary action reduces the time users would otherwise spend waiting for system responses or navigating through multiple input stages.
3Measurement precision
If the system requires multiple inputs to achieve desired outcomes, then input precision can be maintained, but the cognitive burden on users increases significantly
Solution Approach 1:
The virtual assistant acts as an intermediary between the user's hand gesture and the desired system action. It translates the visual hand movement into the appropriate virtual action automatically, mediating the interaction so that users don't need to mentally process complex input sequences or understand the underlying system logic.
Solution Approach 2:
The system changes the interaction parameters from multiple discrete inputs to a single continuous hand gesture. By recognizing gestures in terms of movement trajectory, speed, and position rather than requiring separate button presses or selections, the system maintains input precision while dramatically reducing cognitive effort.
4Productivity
If conventional input methods are used, then the system can process commands, but energy consumption increases due to longer interaction times
Solution Approach 1:
The system continuously monitors hand gestures and environmental context to maintain readiness for interaction without requiring discrete activation. This continuous monitoring enables faster response times by eliminating the need for users to initiate separate command sequences, thereby reducing overall interaction time and associated energy consumption.
Solution Approach 2:
The interaction system dynamically adjusts its responsiveness based on the detected hand gesture and contextual situation. When a gesture indicates intent, the system rapidly transitions to execution mode, providing dynamic response timing that optimizes both speed and energy efficiency by avoiding unnecessary processing during idle states.
Data Source
AI summary
A computer system detects presence of a respective portion of a user. In response to detecting the presence of the respective portion of the user, and in accordance with a determination that first criteria are met by the presence of the respective portion of the user, wherein the first criteria require an inner side of the respective portion of the user is facing toward a viewpoint corresponding to the view of the three-dimensional environment that is provided via the display generation component, the computer system displays a first user interface object. After displaying the first user interface object, the computer system detects that the respective portion of the user has changed from satisfying the first criteria to satisfying second criteria different than the first criteria, and in response, the computer system switches to displaying status information for the computer system.