Free-Space Gestural Interaction via Virtual Touch Planes
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current human-computer interaction methods, especially in augmented reality environments, are limited and cumbersome, requiring users to rely on keyboards, mice, or simple hand gestures, which restricts interaction precision and freedom, and are often costly and impractical for general consumer use.
Innovation Solution
The system enables free-space gestural interactions by tracking the motion of control objects, such as fingers or pointing devices, to interact with virtual interfaces without physical contact, using motion-capture technology to define virtual touch planes and surfaces, allowing users to select and manipulate controls with greater precision and freedom.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If traditional input devices (keyboards, mice, remote controllers) are used, then device complexity is reduced and ease of manufacture is improved, but interaction precision and freedom are limited
Solution Approach 1:
The patent replaces physical input devices (mechanical keyboards, mice, remote controllers) with a motion capture system that uses optical sensors and computer vision to detect hand and finger movements. This substitution eliminates the need for mechanical components while achieving higher interaction precision through 3D spatial tracking of gestures in free space.
Solution Approach 2:
The patent introduces virtual control constructs (virtual planes, virtual surfaces, virtual buttons) as intermediaries between the user's physical gestures and the digital interface. These virtual constructs mediate the interaction by translating complex 3D hand movements into simple 2D interface commands, thereby improving interaction freedom while maintaining system manageability.
2Ease of operation
If touch screens are used for direct interaction, then interaction precision is improved, but the requirement for physical contact limits freedom of movement and accessibility
Solution Approach 1:
The patent transitions from 2D touch screen interaction to 3D free-space gesture recognition. By adding the third dimension (depth) to the interaction space, users gain freedom of movement while the system maintains precision through stereo vision and depth sensing technologies that accurately track hand positions in three-dimensional space.
Solution Approach 2:
The patent creates a virtual copy of the touch interface in 3D space. Virtual control constructs replicate the functionality of physical buttons and sliders but exist in free space, allowing users to interact with them through gestures without physical contact. This copying approach preserves the precision of touch interaction while eliminating the need for physical contact.
3Adaptability or versatility
If advanced gesture recognition systems are implemented, then interaction precision and freedom are improved, but cost and practicality for general consumer use worsen
Solution Approach 1:
The patent designs a gesture recognition system that can serve multiple functions using a single setup. The motion capture system not only recognizes hand gestures but also tracks eye movements, head position, and body orientation. The virtual control constructs can represent various interface elements (buttons, sliders, menus) depending on the application context, making the system versatile while cost-effective for general consumer use.
Data Source
AI summary
During control of a user interface via free-space motions of a hand or other suitable control object, switching between control modes can be facilitated by tracking the control object's movements relative to, and its contact with a “virtual touch plane or surface” (i.e., a plane, portion of a plane, and/or surface computationally defined in space, or corresponding to any physical surface).