Near-Plane Gesture Recognition Using IR Difference Imaging
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Head-mounted display (HMD) devices lack a low-cost and energy-efficient method for gesture recognition in various lighting environments, as existing gesture recognition techniques rely on depth cameras that are expensive and power-intensive, and voice commands may not be appropriate in all situations.
Innovation Solution
The use of an IR light source to illuminate a specific range within a near-field environment, capturing images with and without illumination, and generating a difference image to eliminate background noise, allowing for hand and finger gesture recognition using integrated color and IR image sensors.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If depth cameras are used for gesture recognition, then gesture recognition capability is improved, but device cost and energy consumption increase
Solution Approach 1:
The patent replaces the mechanical/optical depth camera system with an infrared imaging system that uses active infrared illumination and standard color camera sensors. This substitution eliminates the need for expensive depth camera hardware while achieving gesture recognition through image processing of reflected infrared light patterns.
Solution Approach 2:
The patent creates a virtual depth representation by processing reflected infrared light patterns from objects in the scene. Instead of directly capturing depth information with a depth camera, the system captures infrared reflections and computationally derives depth and gesture information from these reflected patterns, effectively copying depth data from optical reflections.
2Reliability
If depth cameras are used for gesture recognition, then gesture recognition capability is improved, but device cost increases
Solution Approach 1:
The patent replaces the mechanical/optical depth camera system with an infrared imaging system that uses active infrared illumination and standard color camera sensors. This substitution eliminates the need for expensive depth camera hardware while achieving gesture recognition through image processing of reflected infrared light patterns.
Solution Approach 2:
The patent makes the infrared illumination system serve multiple functions: it provides active illumination for the camera, creates depth information through reflected light patterns, and enables gesture recognition. This multi-functionality reduces the need for separate dedicated components, lowering overall system cost.
3Measurement precision
If active infrared illumination is used to illuminate a specific range, then gesture recognition accuracy is improved, but light intensity decay limits the illumination range
Solution Approach 1:
The patent applies local quality by concentrating infrared illumination on the specific near-field region where gestures occur. The active infrared illumination is directed to illuminate only the immediate surroundings of the device, creating a localized illumination zone that provides sufficient light intensity for accurate gesture recognition without requiring wide-area illumination.
Solution Approach 2:
The patent uses partial action by providing just enough infrared illumination intensity for the near-field gesture recognition requirement. Rather than illuminating the entire field of view, the system applies illumination selectively to the necessary near-field region, optimizing the balance between illumination intensity and energy consumption.
4Measurement precision
If difference imaging is used to eliminate background noise, then gesture detection accuracy is improved, but image processing complexity increases
Solution Approach 1:
The patent applies preliminary action by capturing an infrared image before the gesture occurs and using this pre-captured image as a reference for subtraction. This preliminary infrared capture allows the system to establish a baseline of the scene without gestures, enabling subsequent difference imaging to isolate gesture-related changes from background variations.
Solution Approach 2:
The patent extracts gesture information by subtracting the pre-captured infrared image from the current infrared image. This difference imaging process extracts only the changes caused by gestures, separating gesture signals from background noise and static scene elements, thereby isolating the relevant gesture data for recognition.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
Enables effective and energy-efficient hand and finger gesture recognition in HMD devices, functioning across a wide range of lighting conditions without the need for expensive depth cameras, thus providing a robust natural user interface.
Implementation Method 1
emitting IR light from a mobile device into an environment
Implementation Method 2
capturing images with and without illumination... detecting reflections of the projected IR light
Data Source
Figure 1
Figure 2A
Figure 2B
AI summary
Methods for recognizing gestures within a near-field environment are described. In some embodiments, a mobile device, such as a head-mounted display device (HMD), may capture a first image of an environment while illuminating the environment using an IR light source with a first range (e.g., due to the exponential decay of light intensity) and capture a second image of the environment without illumination. The mobile device may generate a difference image based on the first image and the second image in order to eliminate background noise due to other sources of IR light within the environment (e.g., due to sunlight or artificial light sources). In some cases, object and gesture recognition techniques may be applied to the difference image in order to detect the performance of hand and/or finger gestures by an end user of the mobile device within a near-field environment of the mobile device.