TOF Gesture Recognition for Passive Surface Displays
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing user interaction technologies with video displays struggle to detect passive gestures and hovering interactions without physical contact, especially on large screens, and are not easily retrofittable to existing displays, leading to inaccuracies and limitations in gesture recognition.
Innovation Solution
The implementation of time-of-flight (TOF) sensing systems, augmented by triangulation or structured light, to detect user interactions on large screens, allowing for passive user gestures and hovering recognition without the need for active contact or embedded sensors, and enabling retrofitting to existing displays.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If capacitive sensing lines are used for gesture recognition, then user interaction detection is enabled, but the display cannot detect hovering gestures without physical contact
Solution Approach 1:
The patent replaces capacitive sensing (which requires physical contact) with time-of-flight optical sensing. The TOF system uses infrared light emission and detection to measure the distance and position of user objects, enabling detection of hovering gestures without mechanical contact. This substitution allows the system to detect gestures in mid-air and on the display surface equally well.
Solution Approach 2:
The patent introduces an intermediary optical field (infrared light) between the user and the display. Instead of direct electrical capacitive interaction, the user's hand reflects infrared light back to sensors, creating an optical intermediary that enables non-contact detection. This intermediary allows gesture detection through the air and on the surface without requiring physical contact with sensing elements.
2Ease of operation
If capacitive sensing grid is implemented, then touch interaction is detected, but the system cannot recognize passive user objects without capacitance
Solution Approach 1:
The patent replaces capacitive sensing with optical time-of-flight sensing. Instead of detecting electrical capacitance from active user objects, the TOF system detects reflected infrared light from passive objects. This allows any object (fingers, styluses, gloves, even bare hands) to be detected as long as it reflects the infrared light, greatly enhancing adaptability to different user scenarios.
Solution Approach 2:
The patent changes the detection parameter from electrical capacitance to optical reflectivity. By measuring the time of flight of infrared light and analyzing reflected light patterns, the system can identify both the position and presence of passive objects. This parameter change enables recognition of objects without requiring them to generate electrical signals, expanding the system's versatility.
3Ease of operation
If dedicated capacitive sensing display is used, then gesture recognition is achieved, but the system is not easily retrofittable to existing displays
Solution Approach 1:
The patent segments the gesture recognition function from the display hardware itself. Instead of requiring capacitive sensing lines to be embedded in the display manufacturing process, the TOF sensing system is implemented as a separate, attachable component. This segmentation allows existing displays to be retrofitted with gesture recognition capability through external optical sensors and processing units, rather than requiring display replacement.
Solution Approach 2:
The patent introduces an intermediary optical sensing layer that operates independently of the display's electrical sensing infrastructure. The TOF system uses infrared light as an intermediary to bridge the gap between existing display hardware and gesture recognition functionality. This intermediary approach enables retrofitting by adding optical sensing components to displays that were not originally designed with capacitive sensing capabilities.
4Measurement precision
If multiple TOF systems are used for accurate gesture detection, then detection accuracy improves, but device complexity increases
Solution Approach 1:
The patent merges multiple TOF sensing functions into a single integrated system. Instead of using separate sensing systems for different detection tasks, the invention combines multiple TOF sensors and processes their data through a unified processing architecture. This merging approach maintains high detection accuracy while reducing overall system complexity through shared optical components and coordinated signal processing.
Solution Approach 2:
The patent implements a universal TOF sensing system that performs multiple functions: detecting single-finger gestures, multi-finger gestures, hovering interactions, and surface contact. By designing the system to handle diverse interaction modes through a common optical sensing framework, the patent reduces the need for separate specialized sensors for each function, thereby managing complexity while maintaining comprehensive detection capability.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
This solution enables accurate detection of user interactions, including gestures and hovering, on both small and large screens, with passive user objects, improving usability and versatility by allowing interaction without physical contact and facilitating retrofitting to existing displays.
Implementation Method 1
Each TOF system includes an emitter of optical energy and a sensor array that detects a fraction of the emitted optical energy that is reflected-back by an object in close proximity to or touching the surface of the display screen
Implementation Method 2
detects a fraction of the emitted optical energy that is reflected-back by an object
Data Source
AI summary
A system recognizes user-object gesture interactions with the surface of a monitor display, with hover space defined spaced-apart from the display surface, or in virtual scroll regions defined on the periphery of the monitor display. The system recognizes user-object interactions, e.g., gestures, and can affect what is displayed commensurately. The system includes at least a first time-of-flight (TOF) system and at least one of a second TOF, a two-dimensional camera, and a mirror, each TOF system processing at least one of z-depth data and A-brightness data. User-object interactions, e.g., touching(s) of the display surface, location(s) in a hover region, or location(s) in a virtual scroll region, are recognized passively in that the user-object need not have capacitance, resistance, exert force, or deform during gesture interaction. The system may be attached retroactively to the monitor, which may be a large (>22 cm) monitor, or a small cell phone sized monitor.