Augmented Reality Direct Interaction via Hand Tracking
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing augmented reality systems face challenges in user interaction, as users feel detached from the environment when using indirect interaction devices, and current display techniques require proper head alignment for accurate spatial registration.
Innovation Solution
An augmented reality system with a user-interaction region, a partially transparent display, and image sensors that allow direct interaction by tracking object movement, enabling users to manipulate virtual objects in a 3D environment as if they were real, using a processor to update the virtual environment accordingly.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If indirect interaction devices (mouse or stylus) are used to control virtual objects, then the user can interact with the augmented reality environment, but the user feels detached from the environment rather than being part of it
Solution Approach 1:
The patent uses the user's actual hand as an intermediary between the user and the virtual environment. Instead of using a mouse or stylus, the hand itself becomes the interaction medium, with virtual representations of the hand appearing in the augmented reality scene to directly manipulate virtual objects, creating a sense of presence while maintaining interaction capability
Solution Approach 2:
The system creates virtual copies of the user's hand that appear within the augmented reality environment. These hand representations are tracked in real-time and used to interact with virtual objects, allowing the user to feel part of the environment while maintaining direct control through natural hand movements
2Manufacturing precision
If large spatially aligned optical elements (transparent screens, holograms, or video-projectors) are used to combine virtual graphics with the real world, then higher resolution and larger display area are achieved, but the user's head must be properly aligned with the display for correct spatial registration
Solution Approach 1:
The system dynamically adjusts the virtual graphics to match the user's head position and orientation in real-time. Rather than requiring the user to align their head with a fixed display, the display content adapts to the user's movements, maintaining accurate spatial registration while allowing freedom of movement
Solution Approach 2:
The system uses cameras to track the user's hand position and the environment, providing feedback that enables accurate spatial registration of virtual objects with the real world. This feedback mechanism allows the system to maintain precision without requiring strict head alignment
Data Source
AI summary
Augmented reality with direct user interaction is described. In one example, an augmented reality system comprises a user-interaction region, a camera that captures images of an object in the user-interaction region, and a partially transparent display device which combines a virtual environment with a view of the user-interaction region, so that both are visible at the same time to a user. A processor receives the images, tracks the object's movement, calculates a corresponding movement within the virtual environment, and updates the virtual environment based on the corresponding movement. In another example, a method of direct interaction in an augmented reality system comprises generating a virtual representation of the object having the corresponding movement, and updating the virtual environment so that the virtual representation interacts with virtual objects in the virtual environment. From the user's perspective, the object directly interacts with the virtual objects.