Through-Glass Virtual Mouse Using Structured Light Gesture Sensing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing storefront displays lack interactive capabilities, with existing solutions being vulnerable to vandalism, requiring physical contact, and limited by inferior image quality and integration with interior computing devices.
Innovation Solution
A system using sensors and structured light to detect gestures through glass, allowing users to interact with digital interfaces without touching the glass, and enabling flexible placement of screens and intuitive gesture recognition.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If sensors are placed outside the storefront window to enable interaction, then user interaction capability is improved, but sensors become vulnerable to vandalism and environmental damage
Solution Approach 1:
The patent inverts the conventional sensor placement by moving sensors from the outside of the glass to the inside. The system uses infrared LEDs and sensors that can detect hand gestures through the glass from the interior side, protecting sensors while maintaining interaction capability. This inversion resolves the contradiction by placing sensors in a protected environment while still enabling external user interaction.
Solution Approach 2:
The patent introduces infrared radiation as an intermediary medium that allows communication between users outside and sensors inside. The infrared LEDs emit radiation that passes through the glass, and the system detects reflected or transmitted infrared signals to identify hand gestures. This intermediary enables interaction without direct sensor exposure to external elements.
2Adaptability or versatility
If capacitive touch-sensing systems are used with rear-projected images, then interaction capability is improved, but image quality deteriorates due to sunlight and reflections
Solution Approach 1:
The patent replaces the optical projection system with electronic display technology. Instead of rear-projecting images onto glass, the system uses LED or LCD displays mounted on the interior side of the glass. This substitution eliminates image quality degradation from sunlight and reflections while maintaining interaction capabilities through the same gesture recognition system.
Solution Approach 2:
The patent changes the operating parameters of the display system by using active electronic displays rather than passive projection. The displays can control their own illumination, adjust brightness dynamically, and eliminate reflections by not relying on external light sources. This parameter change resolves the image quality issue while preserving interaction functionality.
3Measurement precision
If users must physically touch the glass to engage capacitive sensing, then sensing capability is improved, but ease of operation deteriorates and hygiene concerns arise
Solution Approach 1:
The patent extracts the sensing function from the glass surface itself and places it on the interior side. By using infrared sensors that detect hand gestures through the glass, the system removes the requirement for physical contact. Users can interact by making hand gestures visible to the sensors, improving ease of operation and hygiene while maintaining sensing precision.
Solution Approach 2:
The patent replaces the capacitive touch mechanism (which requires physical contact) with an optical detection system using infrared sensors. The system detects changes in infrared radiation patterns caused by hand gestures, eliminating the need for mechanical or physical contact with the glass surface. This substitution improves user convenience and hygiene while preserving interaction capability.
4Reliability
If a closed system with interior monitor is used, then sensor protection is improved, but adaptability and integration with other window areas deteriorate
Solution Approach 1:
The patent designs a system that serves multiple functions: it protects sensors while enabling interaction, displays content on the glass surface, and can be integrated with interior computing devices. The LED/LCD display and gesture recognition system can work together with other smart window features, creating a universal platform that enhances rather than limits adaptability.
Solution Approach 2:
The patent implements a feedback loop where the system detects user gestures, processes the information, and provides visual feedback through the display. This feedback mechanism enables seamless integration with interior computing devices and other smart window systems, allowing the protected interior system to communicate effectively with users outside while maintaining system-wide adaptability.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
Enables two-way communication and interactive experiences, enhancing user engagement and flexibility in display design while protecting sensors from environmental factors.
Implementation Method 1
a source of radiating energy directed into a predefined area, at least one directionally sensitive sensor that detects the radiation when it's reflected off of an object within the predefined area
Data Source
AI summary
A gesture-based control system utilizes real-time camera input and artificial intelligence to interpret user hand gestures for controlling a digital interface. The system includes a camera and processor configured to analyze image data using machine learning models trained on a diverse set of stored hand gesture representations. Gestures are recognized based on positional attributes, handedness, and motion vectors, and are mapped to input commands such as swipe left, right, up, or down. The system is designed for simplicity, using natural, intuitive gestures, such as swiping, thumbs up, thumbs down, and the “OK” sign, requiring no memorization or complex training. These familiar motions enable touch-free navigation and interaction with digital content, including support for hierarchical menu structures. The system can operate with or without visual gesture feedback. Methods and non-transitory computer-readable media are also disclosed for performing gesture detection and command execution.