Mobile Terminal Depth-Camera Gesture Control Without Touch
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing mobile terminal interaction methods, such as touch and voice recognition, face limitations in accuracy and usability, particularly when used in stand states, leading to potential device movement or fall risks and suboptimal recognition rates.
Innovation Solution
Implementing a vision-based UI using a depth camera to quickly and accurately extract a user's hand image, measure finger direction, and distance, allowing precise control through finger pointing without physical contact.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If touch interface is used in stand state, then user can interact with device, but device position may change or device may fall
Solution Approach 1:
The patent replaces the mechanical touch interface with a vision-based interaction system using a depth camera. The system captures depth images to detect hand gestures and finger pointing directions, allowing users to control the device without physical contact. This substitution eliminates the risk of device movement or falling while maintaining ease of operation through gesture recognition.
2Ease of operation
If voice recognition is used to solve touch interface problem, then user can interact without contact, but recognition rate performance is insufficient
Solution Approach 1:
The patent replaces voice recognition with a depth camera-based gesture recognition system. The depth camera captures three-dimensional hand positions and finger orientations, providing precise spatial information for intent detection. This approach achieves contactless interaction with superior recognition accuracy compared to voice recognition, as it directly measures physical gesture parameters.
Solution Approach 2:
The patent transitions from two-dimensional touch screen coordinates to three-dimensional spatial gesture detection using depth information. By incorporating the Z-axis distance measurement, the system can accurately determine finger pointing directions and hand positions in 3D space, enabling more precise gesture recognition and higher recognition rates.
3Productivity
If depth image processing is performed to extract hand image quickly, then image processing time is reduced, but processing accuracy must be maintained
Solution Approach 1:
The patent segments the depth image processing into distinct stages: first extracting the hand region using depth thresholding, then isolating the finger portion from the hand, and finally determining the pointing direction. This segmentation allows each processing stage to focus on specific features, improving both processing speed and accuracy by avoiding unnecessary computation on irrelevant image regions.
Solution Approach 2:
The patent applies different processing strategies to different regions of the depth image based on their importance. The hand region is extracted using depth thresholding, while the finger tip region undergoes more detailed analysis to determine pointing direction. This local quality approach optimizes processing resources by applying appropriate complexity only where needed, balancing speed and accuracy.
Data Source
Figure 1A
Figure 1B
Figure 1C
AI summary
The present invention relates to a device and a control method therefor and, more specifically, the device comprises: a memory for storing at least one command; a depth camera for capturing at least one hand of a user; a display module; and a controller for controlling the memory, the depth camera, and the display module. The controller controls the depth camera so as to capture the at least one hand of a user and controls the display module so as to output a visual feedback that changes on the basis of the captured hand of a user.