Gesture Keyboard with Stereo Cameras for 3D Recognition
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current gesture recognition technologies, such as those using Kinect-like devices, face challenges in regions close to the screen or lower in the field-of-view, leading to poor performance and user discomfort, and are less effective for precise motion tracking and diverse gesture lexicons, making them unsuitable for non-game applications.
Innovation Solution
Integrating sensors into keyboards with a wide field-of-view, allowing users to activate gesture recognition by lifting their hands above the keyboard, and partitioning three-dimensional space into interaction zones for both static and dynamic gestures, enabling comfortable and natural interaction.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If cameras are placed in peripheral devices for gesture recognition, then gesture control capability is enabled, but sensors do not work well in regions close to the screen or lower in the field-of-view
Solution Approach 1:
The patent transitions from traditional 2D camera-based gesture recognition to 3D spatial gesture recognition by incorporating depth information through time-of-flight sensors. This dimensional change enables accurate gesture detection in previously problematic regions close to the screen and in the lower field-of-view, resolving the reliability issue while maintaining versatility.
Solution Approach 2:
The patent introduces an intermediary processing layer that combines data from multiple sensors including time-of-flight cameras and traditional cameras. This intermediary processing compensates for individual sensor limitations and improves overall gesture recognition reliability in challenging regions.
2Area of stationary object
If cameras are placed in the bezel of a screen with wide field-of-view, then more regions can be viewed, but some regions of desired interaction still cannot be viewed by the camera
Solution Approach 1:
By adding the depth dimension through time-of-flight sensing, the system extends its effective interaction volume beyond what 2D cameras can capture. This enables gesture recognition in regions that would otherwise be outside the camera's field-of-view, effectively increasing the usable interaction space.
Solution Approach 2:
The patent divides the interaction space into multiple zones with different gesture recognition capabilities. By segmenting the 3D space above the keyboard into distinct interaction zones, the system can optimize gesture detection for each zone, ensuring comprehensive coverage of desired interaction regions.
3Ease of operation
If coarse body gestures are used for game control, then natural interaction is enabled, but precise motion tracking and diverse gesture lexicon are not achieved
Solution Approach 1:
The patent changes the parameters of gesture recognition by utilizing time-of-flight depth information and 3D spatial coordinates instead of 2D image coordinates. This parameter change enables both natural coarse gestures for games and precise motion tracking for professional applications, simultaneously achieving ease of operation and measurement precision.
Solution Approach 2:
The system dynamically adjusts its gesture recognition sensitivity and precision based on the application context. For gaming, it accepts coarse natural gestures, while for precision applications, it utilizes detailed 3D motion data, making the system adaptable to different operational requirements.
4Adaptability or versatility
If users lift hands above the keyboard to activate gesture recognition, then gesture interaction is enabled, but user fatigue may occur with prolonged use
Solution Approach 1:
The patent implements partial action by requiring only minimal hand lifting above the keyboard to activate gesture recognition, rather than requiring full arm movements. This reduces the physical effort and endurance required while still enabling gesture interaction capability.
Solution Approach 2:
The system automatically detects when hands are lifted above the keyboard and activates gesture recognition mode without requiring explicit user commands. This self-service approach reduces cognitive load and makes the transition between typing and gesture modes seamless, reducing overall user fatigue.
Data Source
AI summary
A gesture-enabled keyboard and method are defined. The gesture-enabled keyboard includes a keyboard housing including one or more keyboard keys for typing, a pair of stereo camera sensors mounted within the keyboard housing, a field of view of the pair of stereo camera sensors projecting substantially perpendicularly to the plane of the keyboard housing, and a pair of micro-electronic machines, each of the micro-electronic machines being coupled with, and adapted to adjust a position of a corresponding one of the pair of stereo cameras. A background of the field of view is updated when one or more alternative input devices are in use. A gesture region including a plurality of interaction zones and a virtual membrane defining a region of transition from one of the plurality of interaction zones to another of the plurality of interaction zones is defined within the field of view of the pair of stereo camera sensors. Gesture interaction is enabled when one or more gesture objects are positioned within the gesture region, and when one or more alternative input devices are not in use.