Visible Light Gesture Control via Display Reflection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing in-air gesture recognition methods on mobile devices raise privacy concerns and require specialized hardware, with Wi-Fi-based solutions being sensitive to background noise and camera-based methods relying on extra hardware.
Innovation Solution
A method utilizing visible light signals emitted from the display device, reflected by hand gestures, and analyzed using a system with a frame processing unit, signal processing unit, and gesture recognition unit to achieve gesture recognition without additional hardware, leveraging the display device and ambient light sensor for spatial resolution and classification.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If camera-based or microphone-based methods are used for in-air gesture recognition, then gesture recognition capability is achieved, but privacy concerns arise and specialized hardware is required
Solution Approach 1:
The display device serves multiple functions: it acts as both the user interface for showing content and the light source for gesture recognition. The ambient light sensor similarly serves dual purposes by detecting both display content and reflected light signals from gestures. This eliminates the need for separate specialized hardware components.
Solution Approach 2:
The mobile device uses its own built-in components (display and ambient light sensor) to perform gesture recognition, rather than requiring external specialized hardware. The display emits light that reflects off gestures and is detected by the device's own sensor, creating a self-contained gesture recognition system.
2Adaptability or versatility
If Wi-Fi-based solutions are used for gesture recognition, then gesture control is achieved, but the system becomes sensitive to background noise and requires specialized hardware
Solution Approach 1:
The patent replaces electromagnetic wave-based detection (Wi-Fi) with optical detection using visible light. Light signals from the display reflect off gestures and are detected by the ambient light sensor, providing a more reliable and noise-resistant method compared to Wi-Fi-based approaches.
3Measurement precision
If high-frequency light signals are generated beyond human perception, then spatial resolution information is improved, but the screen refresh rate limitation is encountered
Solution Approach 1:
The display alternates between showing normal content and displaying invisible high-frequency light patterns at rates beyond human perception. This periodic switching enables the generation of high-frequency light signals that provide spatial resolution information while maintaining compatibility with the screen's refresh capabilities.
Solution Approach 2:
The system changes the frequency parameter of light signals emitted by the display to exceed human visual perception thresholds. By operating at frequencies above what humans can detect, the system can encode spatial resolution information without affecting the user's viewing experience or being constrained by standard refresh rates.
4Ease of operation
If embedded light signals are hidden in display content, then user viewing experience is maintained, but signal detection precision must be sufficient
Solution Approach 1:
The gesture itself acts as an intermediary that modulates the reflected light signal. When a gesture enters the detection zone, it reflects the embedded light signals back to the sensor, creating a detectable modulation pattern that preserves both user experience and detection precision.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
Enables gesture recognition on commodity mobile devices with high accuracy and low power consumption, breaking screen refresh rate limitations and maintaining user experience, while avoiding the need for specialized hardware and ensuring privacy.
Implementation Method 1
The light signal is emitted from the display device, then reflected by hand gesture and finally collected and analyzed
Implementation Method 2
The light signal is emitted from the display device, then reflected by hand gesture and finally collected and analyzed
Implementation Method 3
collected by the light sensor... The signal processing unit preprocesses the light intensity signal collected by the light sensor
Data Source
AI summary
An in-air gesture control method based on visible light signals, which transmits light signals through a display device and collects its light signals reflected by hand, and after analysis, realizes gesture recognition; the transmitted light signals are in the visible light band; which can realize the gesture recognition and control on mobile devices without modifying hardware and a visible light-based gesture recognition system on commercial mobile devices; compared with the existing gesture recognition methods on commercial mobile devices, the invention avoids special hardware modification, that is, it does not require any additional components on the mobile device, such as a depth camera, and protects the user's privacy well.