Virtual Joystick Hand Control for Out-of-Reach AR/VR Objects
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing augmented and virtual reality systems lack effective interaction techniques for manipulating virtual objects beyond arm's reach without the need for physical controllers.
Innovation Solution
A virtual joystick system that utilizes hand gestures to create a vector-based control mechanism, allowing users to manipulate virtual objects in AR or VR environments by performing targeting and manipulation gestures to apply velocity to virtual objects based on hand movements.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If direct hand tracking is used to grab and manipulate virtual objects, then the interaction feels natural and intuitive, but the virtual objects are constrained to being within arm's reach
Solution Approach 1:
A virtual joystick is introduced as an intermediary tool between the user's hand and the virtual object. The joystick is positioned at the tip of the user's virtual hand and serves as a mediator that translates hand movements into object manipulation commands, enabling interaction with objects beyond arm's reach while maintaining intuitive control
Solution Approach 2:
The patent replaces the direct mechanical grabbing interaction with a vector-based control system. Instead of physically grasping objects, the system uses detected hand gestures to generate velocity vectors that propel virtual objects, substituting mechanical manipulation with a computational control mechanism
2Length of moving object
If a physical controller is used to manipulate virtual objects beyond arm's reach, then the manipulation distance is unlimited, but the interaction loses naturalness and requires additional hardware
Solution Approach 1:
The system uses the user's own hand as the control interface. By tracking the user's hand gestures and using them to generate control vectors, the system eliminates the need for external physical controllers while maintaining unlimited manipulation distance. The user's hand serves both as the identifier of intent and the source of control input
3Measurement precision
If hand gestures are used to create vector-based control, then unlimited manipulation distance is achieved with precision, but the system complexity increases
Solution Approach 1:
The system transforms hand gesture parameters (position, movement direction, speed) into velocity vector parameters for object manipulation. By changing the parameter representation from direct spatial coordinates to velocity vectors, the system achieves precise control over objects at any distance while managing computational complexity through parameter transformation
Data Source
AI summary
Systems and methods are provided. For example, a method includes determining a position of a user's hand and identifying a manipulation gesture performed by the user targeting a virtual object. The method also includes determining a three-dimensional (3D) origin point based on the position of the user's hand when the manipulation gesture is performed, and determining a 3D end point based on a movement of the user's hand from the origin point. The method additionally includes deriving a 3D vector based on the 3D origin point and the 3D end point, and applying an action to the targeted virtual object based on the 3D vector, wherein the targeted virtual object is at a distance greater than the user's arm reach.