Optical Gesture Interface for Contactless Screen Control
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Touch screen devices suffer from slow reaction times, poor accuracy, and reliability issues, and frequent use leads to performance deterioration due to residue accumulation, limiting effective gesture-based control in human-machine interfaces, particularly in controlling unmanned air vehicles.
Innovation Solution
A gesture-based human machine interface system using multiple cameras to capture and analyze the scene in front of the screen, allowing a processor to determine the position and distance of an object from the screen, enabling precise control of on-screen interactions without physical contact, such as zooming and object manipulation, by interpreting gestures like finger movements.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If touch screen devices are used for gesture-based control, then gesture input capability is enabled, but reaction time becomes slow and accuracy deteriorates
Solution Approach 1:
The patent replaces the mechanical touch screen input system with an optical gesture recognition system using cameras and image processing. This substitution eliminates physical contact requirements while improving response speed and accuracy by using non-contact optical detection methods.
Solution Approach 2:
The patent introduces an intermediary gesture recognition system that captures hand gestures through cameras and translates them into control commands. This intermediary layer between the user and the controlled system enables more reliable and accurate control compared to direct touch screen interaction.
2Ease of operation
If touch screens are frequently used, then gesture-based control functionality is maintained, but performance deteriorates due to residue accumulation
Solution Approach 1:
The patent extracts the gesture detection function from the touch screen surface and relocates it to external cameras. This separation removes the vulnerability to residue accumulation on the screen surface while preserving gesture-based control functionality, ensuring stable long-term performance.
3Measurement precision
If multiple cameras are used to capture scene and determine object position and distance, then control accuracy is improved, but device complexity increases
Solution Approach 1:
The patent uses multiple cameras to capture gestures from different spatial dimensions, enabling three-dimensional position and distance determination. This dimensional approach improves measurement precision by triangulating object positions while distributing the complexity across multiple sensing perspectives.
Solution Approach 2:
The patent creates multiple optical copies of the gesture scene through multiple cameras, allowing the system to analyze the same gesture from different angles simultaneously. This copying approach enables accurate depth and distance measurement without requiring complex single-sensor systems.
Data Source
Figure 1
Figure 2
Figure 3A~3D
AI summary
The present invention relates to a gesture-based human machine interface, for example a graphical user interface for controlling a program executing on a computer. Gestures of the user are monitored and a response is provided that is based upon the detected gesture. An object is used to point to information displayed on a screen. The information displayed on the screen is modified in response not only to a determination of to where the object is pointing, but also in response to the distance of the object from the screen.