Hand Gesture Virtual Element Deck for AR Interaction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current virtual and augmented reality interfaces lack intuitive and natural hand interaction methods for navigating and exploring content, particularly in displaying and interacting with virtual elements like a gallery of images or documents, relying on outdated desktop metaphors.
Innovation Solution
A system that tracks hand motion using a tracking device and outputs virtual elements in VR or AR, allowing users to perform gestures such as turning a palm to reveal a card deck, grabbing to extend, and moving hands to arrange elements in three dimensions, incorporating physics simulations for dynamic visualization and gamification.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If desktop metaphors (windows, documents) are used in VR/AR interfaces, then familiar interface patterns are provided, but intuitive and natural hand interaction is lost
Solution Approach 1:
The patent copies the familiar card deck metaphor from the physical world into the virtual environment. Users interact with virtual elements as if they were physical cards - holding them in their hand, fanning them out, selecting them - creating an intuitive interface that feels natural rather than requiring learning of abstract desktop metaphors
Solution Approach 2:
The patent transitions from 2D desktop metaphors to 3D spatial interactions. Virtual elements are arranged in three-dimensional space around the user, allowing natural hand movements and gestures to navigate and manipulate content, eliminating the need for flat screen-based interactions
2Adaptability or versatility
If virtual elements are displayed in 3D space extending between hands, then multi-dimensional data access is enabled, but tracking and interaction precision requirements increase
Solution Approach 1:
The patent introduces visible reference elements (such as card edges, selection indicators, and spatial anchors) that serve as intermediaries between the user's hands and the virtual elements. These references provide visual feedback and alignment cues, making precise tracking and interaction achievable even in 3D space
Solution Approach 2:
The system provides continuous visual feedback about hand position, element selection, and spatial arrangement. Virtual elements respond to hand movements in real-time, showing selection states, navigation progress, and spatial relationships, which helps users maintain precision without requiring perfect tracking
Data Source
Figure 1~4
Figure 5~6
Figure 7~8
AI summary
A tracking device tracks hand motion of a user. An output device, for example an augmented reality headset or a virtual reality headset, is configured for the output of virtual elements in augmented or virtual reality. A processor recognizes a first gesture when the user turns a palm of a first hand (H1) towards his face. The processor then displays a set of virtual elements (EN) above the palm of the first hand, the virtual elements being arranged above each other like a card deck, using the output device. In other words, the virtual elements can be visualized as cards on the palm of the first hand. In most embodiments, the virtual elements are attached to at least one of the user's hands. The method provides a user interface for hand interaction with virtual elements, for example for viewing a content gallery or data exploration. The virtual elements each have a primarily flat appearance and can be displayed as windows, documents, search results, images, etc. The method allows the user to use his hands in an intuitive and natural way, as if he would hold a card deck or a slinky in his hands.