Touchless Input Using Depth Camera Finger Tracking
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Finger-based touch-input technologies face challenges such as visual occlusion and difficulty in resolving the large finger pad surface to a single point, making effective interaction with touch screens cumbersome.
Innovation Solution
A touchless input system using a depth camera to create depth maps of a scene, identifying nonstatic pixels with shallowest depth to map the position of a pointed finger to a cursor on a display, enabling control of cursor operations and multitouch gestures without physical contact.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If finger-based touch-input is used, then direct interaction with user interface is enabled, but visual occlusion occurs and finger pad surface cannot be resolved to a single point
Solution Approach 1:
A camera system captures images of the finger and processes them through image processing algorithms to determine touch location. The camera acts as an intermediary between the finger and the touch detection system, allowing indirect measurement of touch position through image analysis rather than direct electrical contact detection.
Solution Approach 2:
The patent replaces mechanical touch detection systems with an optical system. Instead of using capacitive or resistive touch sensors that directly detect finger contact, the system uses a camera to capture visual information about the finger and computationally determines touch location, substituting mechanical sensing with optical sensing and image processing.
2Ease of operation
If finger-based touch screen is used, then user interface interaction is achieved, but finger visually occludes aspects of the user interface
Solution Approach 1:
The camera system serves as an intermediary that captures finger position information without requiring the finger to physically contact the display surface. This allows the user interface to remain visually accessible while still enabling interaction through computational determination of touch location from captured images.
Solution Approach 2:
Instead of requiring direct physical contact with the display, the system creates a visual copy or representation of the finger's position through camera imaging. The touch location is determined by analyzing the captured image data, allowing interaction without the finger physically blocking the display surface.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
Enables precise and intuitive control of cursor positions and gestures on a display by tracking the finger's position in three-dimensional space, overcoming occlusion and surface size limitations, and allowing for seamless multitouch interactions.
Implementation Method 1
A depth camera is used to capture a series of depth maps imaging a scene
Data Source
AI summary
A series of depth maps image a scene including a human subject. Each depth map includes a plurality of pixels, where each of the plurality of pixels defines a depth of a surface imaged by that pixel. The human subject is modeled with a virtual skeleton including a virtual hand joint. A position of the virtual hand joint is mapped to a corresponding portion of one or more depth maps in the series of depth maps. One or more fingers are modeled from depth map information of the corresponding portion of the one or more depth maps. A finger gesture is interpreted from the modeled fingers.