Conveyance Gesture Control via Depth-Color Fusion
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing conveyance devices, such as elevators, face challenges in accurately capturing passenger behaviors due to limitations in current 2D and 3D sensors, which struggle to distinguish between passengers and background objects, especially in shadow regions or when passengers wear similar clothing, and fail to provide sufficient resolution for explicit and implicit gesture recognition.
Innovation Solution
A system that combines structured-light based 3D depth sensing with video streams to recognize gestures, using an array of sensors to cover a wide field of view and provide high spatial resolution, allowing for accurate detection and classification of gestures by fusing depth and color information.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If 2D sensors operating on color or intensity information are used, then the system can detect passenger presence, but the sensors cannot distinguish between passengers wearing similar colored clothing or discriminate between passengers and background objects of similar color
Solution Approach 1:
The patent transitions from 2D color/intensity sensing to 3D depth sensing by introducing a time-of-flight camera that measures the time for light to travel to and from objects. This adds a temporal dimension to the sensing process, enabling depth measurement and creating a three-dimensional representation of the environment that resolves ambiguities in two-dimensional color-based detection
2Loss of information
If 3D sensors providing depth information are used, then the system can capture spatial information, but the sensors cannot generate depth estimates in shadow regions due to distance differences between emitter and receiver
Solution Approach 1:
The patent combines multiple sensing modalities - time-of-flight depth sensing, stereo vision, and traditional 2D imaging - into a unified sensor system. By merging these complementary technologies, the system can cross-validate depth measurements and compensate for the weaknesses of individual sensors in shadow regions, achieving more complete and accurate three-dimensional reconstruction
3Adaptability or versatility
If traditional sensors are used for conveyance control, then the system can detect basic passenger presence, but the sensors are unable to capture many passenger behaviors including explicit and implicit gestures
Solution Approach 1:
The patent implements dynamic gesture recognition by continuously capturing sequences of three-dimensional data over time and analyzing temporal patterns of motion. The system adapts to different gesture types and passenger behaviors by processing dynamic changes in depth information, enabling recognition of both explicit intentional gestures and implicit behavioral patterns
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
Enables precise and reliable gesture-based control of conveyance devices, improving passenger experience by accurately sensing gestures over a wide range and field of view, enhancing elevator operation and security monitoring.
Implementation Method 1
3D sensors that provide depth information may be unable to generate an estimate of depth in a so-called 'shadow region' due to a difference in distance between an emitter/illuminator (e.g., an infrared (IR) laser diode) and a receiver/sensor (e.g., an IR sensitive camera)
Data Source
Figure 1
Figure 2~3
Figure 4
AI summary
A method includes generating a depth stream from a scene associated with a conveyance device; processing, by a computing device, the depth stream to obtain depth information; recognizing a gesture based on the depth information; and controlling the conveyance device based on the gesture.