3D Gesture User Interface for Non-Tactile Object Manipulation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current user interface technologies lack efficient methods for non-tactile interaction with computer systems, particularly in selecting and manipulating interactive objects using three-dimensional gestures, which limits user interaction flexibility and accuracy.
Innovation Solution
A non-tactile zoom-based user interface utilizing a 3D sensing device to capture hand gestures and translate them into interactions with a hierarchical data structure of interactive objects, allowing users to perform transverse, grab, longitudinal, and execute gestures to select and operate on objects, enabling intuitive control of media players and text input.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If traditional tactile interface devices (keyboard, mouse, touch screen) are used, then ease of operation is maintained, but user interaction flexibility and accuracy deteriorate due to lack of intuitive gesture-based control
Solution Approach 1:
The patent replaces traditional mechanical tactile input devices (keyboard, mouse, joystick) with a 3D sensing system that captures hand gestures in three-dimensional space. The sensing device detects hand position, orientation, and movement trajectories to recognize gestures such as transverse gestures for selection, grab gestures for manipulation, longitudinal gestures for navigation, and execute gestures for confirmation, eliminating the need for physical contact with mechanical interfaces.
Solution Approach 2:
The patent transitions from two-dimensional touch screen interaction to three-dimensional gesture-based interaction by utilizing depth-perceptive sensors. The system captures hand gestures in 3D space, allowing users to perform intuitive gestures such as reaching, grabbing, and manipulating virtual objects in three dimensions, thereby enhancing interaction flexibility and accuracy beyond flat touch interfaces.
2Measurement precision
If 3D gesture recognition is implemented, then user interaction flexibility and accuracy improve, but device complexity increases due to additional sensing and processing requirements
Solution Approach 1:
The patent employs a hierarchical data structure that serves multiple functions: it organizes interactive objects in a tree structure with parent-child relationships, manages the state of gesture recognition, and coordinates the sequence of gestures (transverse, grab, longitudinal, execute). This universal data structure reduces the need for separate processing mechanisms for different gesture types and interaction states.
Solution Approach 2:
The patent divides the gesture recognition process into distinct sequential phases: transverse gesture for object selection, grab gesture for object manipulation, longitudinal gesture for navigation or parameter adjustment, and execute gesture for confirmation. Each gesture type is recognized and processed independently through the hierarchical data structure, simplifying the overall complexity by breaking down complex interaction into manageable segments.
3Ease of operation
If hierarchical data structure is used to organize interactive objects, then navigation and control improve, but information processing complexity increases
Solution Approach 1:
The patent implements a hierarchical data structure where interactive objects are organized in a tree structure with parent nodes and child nodes. Each node can contain sub-nodes, creating nested levels of organization. This allows users to navigate through nested menus and object hierarchies using intuitive gestures, such as expanding or collapsing nested structures, while the system efficiently manages the nested data relationships through the hierarchical framework.
Data Source
AI summary
A method includes arranging, by a computer, multiple interactive objects as a hierarchical data structure, each node of the hierarchical data structure associated with a respective one of the multiple interactive objects, and presenting, on a display coupled to the computer, a first subset of the multiple interactive objects that are associated with one or more child nodes of one of the multiple interactive objects. A sequence of three-dimensional (3D) maps including at least part of a hand of a user positioned in proximity to the display is received, and the hand performing a transverse gesture followed by a grab gesture followed by a longitudinal gesture followed by an execute gesture is identified in the sequence of three-dimensional (3D) maps, and an operation associated with the selected object is accordingly performed.