Gesture-Based UI for Large Screens Using 3D Hand Distinction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing gesture-based user interfaces face challenges such as user fatigue due to extensive hand movement, difficulty in distinguishing between right and left hands, especially when they overlap, and restrictive usage constraints, leading to limitations in pointing accuracy on large screens.
Innovation Solution
A computer-implemented method and apparatus that employs a noncontact capture unit to recognize human body poses and gestures, using a human body pose estimation unit and gesture recognition unit to generate pose and gesture commands, allowing for robust information selection without the need for explicit hand position detection, enabling simultaneous use of both hands and reducing movement requirements.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If hand position detection is used for information selection, then pointing operation is intuitive and readily understandable, but difficulty arises in distinguishing between right and left hands when they overlap
Solution Approach 1:
The patent transitions from two-dimensional image processing to three-dimensional spatial reasoning by incorporating depth information. The hand distinction unit determines hand laterality based on the three-dimensional spatial relationship between the hand and the camera, specifically whether the hand is located in the left or right half of the image plane in three-dimensional space. This dimensional transition enables accurate distinction between right and left hands even when they overlap in the two-dimensional image, resolving the measurement precision problem while maintaining intuitive operation.
2Measurement precision
If extensive hand movement is required for information selection on large screens, then pointing accuracy is improved, but user fatigue increases
Solution Approach 1:
The patent replaces the mechanical pointing system with a gesture-based interaction system. Instead of requiring precise hand movements to the screen location, the system captures hand gestures in three-dimensional space and translates them into selection operations. The gesture recognition unit identifies gestures such as index finger pointing, palm facing, or fist formation, and the information selection unit maps these gestures to screen locations or selection actions. This substitution dramatically reduces the physical effort required from the user while maintaining or improving pointing accuracy through spatial gesture recognition.
3Measurement precision
If constraints are imposed on hand usage (disabling simultaneous use of both hands or crossing), then hand distinction accuracy is improved, but adaptability and versatility of the interface is reduced
Solution Approach 1:
The patent changes the parameters for determining hand laterality from binary constraints to continuous spatial analysis. Instead of disabling simultaneous hand use or crossing, the system continuously monitors the three-dimensional positions of both hands and dynamically determines which hand is the primary target based on spatial relationships. The hand distinction unit analyzes the relative positions of hands, the camera, and the target object to intelligently identify the intended hand, allowing simultaneous hand usage and crossing movements without compromising distinction accuracy. This parameter change enables the system to adapt to various usage scenarios while maintaining high measurement precision.
Data Source
Figure 1
Figure 2
Figure 3~4
AI summary
An apparatus and method provide logic for providing gestural control. In one implementation, an apparatus includes a receiving unit configured to receive a first spatial position associated with a first portion of a human body, and a second spatial position associated with a second portion of the human body. An identification unit is configured to identify a group of objects based on at least the first spatial position, and a selection unit is configured to select an object of the identified group based on the second spatial position.