Gesture Recognition with Skin Reflection Segmentation and Depth Maps
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing gesture detection systems face challenges in reliability and resource/cost efficiency due to high variation in hand appearance, lighting conditions, and the complexity of combining 2D and 3D sensors, particularly in distinguishing hands from backgrounds and handling variations in hand shape and color.
Innovation Solution
A detector system using an illumination source to project an illumination pattern on an area, an optical sensor to capture reflection features, and an evaluation device to determine depth maps and segment objects based on biological tissue reflection, allowing for reliable gesture detection with reduced technical complexity and cost.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If 2D image processing methods are used for gesture detection, then the system complexity is reduced, but the reliability and accuracy of gesture detection deteriorates due to high variation in hand appearance, shadows, and lighting conditions
Solution Approach 1:
The patent combines 2D image data from a CMOS sensor with depth information from a time-of-flight sensor to create a fused representation of the hand. This merging of data from multiple sensing modalities allows the system to maintain low complexity while improving reliability, as the depth information provides robustness against lighting variations and appearance changes that plague 2D-only methods.
Solution Approach 2:
The patent transitions from 2D image processing to 3D gesture detection by incorporating depth maps from a time-of-flight sensor. This addition of the depth dimension enables the system to distinguish hands from backgrounds more reliably and detect gestures in varying lighting conditions, thereby improving detection reliability without significantly increasing system complexity.
2Reliability
If 3D depth maps are used to improve gesture detection accuracy, then the reliability of gesture detection improves, but the device complexity and cost increases due to comprehensive fusion concepts and careful calibration requirements
Solution Approach 1:
The patent performs preliminary segmentation of the hand from the background using depth information before detailed gesture analysis. This preliminary action simplifies subsequent processing by focusing computational resources only on the relevant hand region, thereby reducing the overall complexity of the fusion system while maintaining high reliability in gesture detection.
Solution Approach 2:
The patent segments the hand region from the background using depth maps and reflection beam profile analysis. This segmentation approach simplifies the complexity of fusing 2D and 3D data by first isolating the hand, then applying gesture recognition algorithms only to the segmented region, thereby reducing computational burden and calibration requirements.
3Ease of manufacture
If traditional hand recognition methods are used, then the system is simpler to implement, but the ability to distinguish hands from backgrounds deteriorates due to low contrast and varying hand shapes
Solution Approach 1:
The patent replaces traditional 2D image-based hand recognition with a method that uses optical reflection characteristics and depth information. By analyzing the reflection beam profile of laser light off the hand and combining it with depth map data, the system achieves high precision in distinguishing hands from backgrounds without requiring complex mechanical or optical setups.
Solution Approach 2:
The patent changes the parameters used for hand detection from 2D color and shape features to 3D depth and optical reflection characteristics. By measuring the reflection beam profile and depth information, the system achieves high detection precision while maintaining implementation ease, as these parameters are directly available from the sensor fusion approach.
4Measurement precision
If multiple sensors are combined for comprehensive gesture detection, then the measurement precision improves, but the loss of time and computational resources increases due to data fusion and calibration requirements
Solution Approach 1:
The patent performs preliminary segmentation of the hand using depth information before detailed gesture analysis. This preliminary action reduces the amount of data that requires intensive processing, thereby reducing computational time and resource loss while maintaining high measurement precision through the use of both 2D and 3D sensor data.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
The system provides robust gesture detection by accurately identifying hand gestures with low resource requirements, overcoming challenges of varying lighting and hand shapes, and simplifying sensor integration.
Implementation Method 1
an optical sensor having at least one light-sensitive area, wherein the optical sensor is configured for determining at least one image of the area, wherein the image comprises a plurality of reflection features generated by the area in response to illumination by the illumination features
Implementation Method 2
the evaluation device is configured for determining at least one depth map of the area by determining at least one depth information for each of the reflection features
Data Source
Figure 1
Figure 2A~2C
AI summary
Detector (110) for gesture detection comprising - an illumination source (112) configured for projecting an illumination pattern comprising a plurality of illumination features on an area (114) comprising an object (116), wherein the object (116) comprises a face; - an optical sensor (118) having a light-sensitive area (126), wherein the optical sensor (118) is configured for determining an image (128) of the area, wherein the image (128) comprises a plurality of reflection features (130) generated by the area (114) in response to illumination by the illumination features; - an evaluation device (136), wherein the evaluation device (136) is configured for determining a depth map of the area by determining depth information for the reflection features (130), wherein the evaluation device (136) is configured for identifying a reflection feature (130) as being generated by illuminating skin by comparing its intensity distribution with a predetermined or predefined intensity distribution, wherein the evaluation device (136) is configured for segmenting the image (128) of the area by using a segmentation algorithm, wherein the evaluation device (136) is configured for determining an orientation of the object (116) in space considering the segmented image and the depth map.