Mapped Hand Regions for Precise Bimanual Virtual Control
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing technologies struggle to effectively utilize bimanual hand gestures for precise control of virtual and graphical elements on displays, particularly in augmented and virtual reality environments, lacking intuitive and efficient interaction methods.
Innovation Solution
A system utilizing an eyewear device with dual cameras and image processing to detect and track bimanual hand gestures, defining mapped regions and executing predefined actions based on recognized gestures, enhancing interaction with virtual and graphical elements.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If bimanual hand gestures are used for controlling virtual elements, then interaction intuitiveness is improved, but control precision deteriorates
Solution Approach 1:
The patent introduces virtual control elements (buttons, sliders, dials) as intermediaries between the user's hand gestures and the system control. These virtual elements are displayed at the user's hand location and provide precise control interfaces, bridging the gap between intuitive gesture input and precise control output.
Solution Approach 2:
The system creates a virtual copy of the user's hand environment by projecting control elements at the hand's location in augmented reality. This allows the user to interact with virtual representations of controls that mirror real-world object positions, maintaining spatial context while enabling precise manipulation.
2Adaptability or versatility
If hand tracking is implemented for gesture recognition, then interaction versatility is improved, but system complexity increases
Solution Approach 1:
The system implements a universal hand tracking framework that can recognize multiple gesture types (pinch, grasp, point, tap) and apply them across different interaction scenarios. The same tracking infrastructure serves multiple functions including gesture recognition, virtual element placement, and control activation, reducing overall system complexity despite enhanced versatility.
Data Source
AI summary
Example systems, devices, media, and methods are described for controlling the presentation of one or more virtual or graphical elements on a display in response to bimanual hand gestures detected by an eyewear device that is capturing frames of video data with its camera system. An image processing system detects a first hand and defines a mapped region relative to a surface of the detected first hand. The system presents a virtual target icon at a position within or near the mapped region. The image processing system detects a series of bimanual hand shapes, including the detected first hand and at least one fingertip of a second hand. In response to determining whether the detected series of bimanual hand shapes matches a predefined hand gesture, the system executes a selecting action that includes presenting one or more graphical elements on the display.