3D Virtual Object Gesture Reconfiguration for Lower Input Burden
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for interacting with virtual and augmented reality environments are cumbersome, inefficient, and place a significant cognitive burden on users, requiring multiple inputs and lacking sufficient feedback, leading to errors and reduced user experience.
Innovation Solution
The system employs gesture recognition, simulated visual interplay between virtual and physical objects, and voice-based virtual assistants to enhance user interaction, reducing the number and complexity of inputs through intuitive and efficient human-machine interfaces.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If traditional input devices (cameras, controllers, joysticks, touch-sensitive surfaces) are used to interact with virtual reality environments, then the system can process user inputs, but the interaction becomes cumbersome, inefficient, and creates significant cognitive burden on users
Solution Approach 1:
The patent replaces traditional mechanical input devices (controllers, joysticks, touch surfaces) with a voice-based digital assistant system that processes natural language commands. This substitution eliminates the need for physical manipulation of input devices, reducing cognitive burden and simplifying user interaction with virtual reality environments.
Solution Approach 2:
The digital assistant acts as an intermediary between the user and the virtual reality system. Instead of directly manipulating virtual objects through complex input devices, users communicate their intentions through natural language to the digital assistant, which then translates and executes the appropriate actions, simplifying the interaction process.
2Productivity
If multiple inputs are required to achieve desired outcomes in augmented reality environments, then the system can perform complex operations, but the process takes longer than necessary and wastes energy
Solution Approach 1:
The digital assistant continuously listens and anticipates user needs, maintaining a ready state to process commands immediately when spoken. This preliminary preparation eliminates delays associated with activating input devices or navigating through multiple input steps, allowing users to achieve desired outcomes with single voice commands.
Solution Approach 2:
The digital assistant provides a universal interface that can execute multiple types of operations through natural language commands. Instead of requiring different input methods for different tasks, a single voice-based interface handles diverse operations, reducing the number of steps needed to achieve desired outcomes and improving overall interaction efficiency.
3Reliability
If traditional user interfaces are used in virtual reality environments, then the system can provide feedback, but the feedback is insufficient and manipulation of virtual objects becomes complex, tedious and error-prone
Solution Approach 1:
The digital assistant provides immediate verbal and visual feedback to confirm command recognition and execution status. This feedback mechanism reduces errors by clearly communicating whether the system has correctly interpreted user intentions and what actions are being performed, making manipulation more reliable and less error-prone.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
While displaying a virtual object with a first spatial location in a three-dimensional environment, a computer system detects a first hand movement performed by a user. In accordance with a determination that the first hand movement meets first gesture criteria, the computer system performs a first operation in accordance with the first hand movement, without moving the virtual object away from the first spatial location; and in accordance with a determination that the first hand movement meets second gesture criteria, the computer system displays a first visual indication that the virtual object has transitioned into a reconfiguration mode, and further detects a second hand movement performed by the user. In accordance with a determination that the second hand movement meets the first gesture criteria, the computer system moves the virtual object from the first spatial location to a second spatial location in accordance with the second hand movement.