Camera-Based Finger Pose Detection for Virtual Keyboard Input
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing input methods for portable devices, such as on-screen keyboards, physical keyboards, and voice keyboards, face challenges including high error rates, inconvenience, spatial limitations, and the need for separate devices or power sources, necessitating an efficient and accurate interface for character input.
Innovation Solution
An electronic apparatus equipped with a camera, display, and processor that detects fingers and their motions to recognize and input characters through a virtual interface, allowing for accurate and quick input without separate physical devices by identifying key presses based on finger positions and motions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Weight of moving object
If an on-screen keyboard is used, then the device portability is improved, but the input accuracy and error rate deteriorate
Solution Approach 1:
The patent replaces the mechanical on-screen keyboard interface with an optical recognition system using camera-based finger pose detection. The system captures images of the user's hand and fingers, processes the visual data to identify finger positions and movements, and translates these into keyboard inputs, thereby eliminating the need for a traditional on-screen keyboard while maintaining portability.
Solution Approach 2:
The patent creates a virtual representation of a physical keyboard layout through software, mapping detected finger positions to corresponding keyboard keys. This virtual keyboard model allows the system to interpret natural hand gestures as keyboard inputs without requiring actual physical keys or a displayed on-screen keyboard.
2Measurement precision
If a physical keyboard is used, then the input accuracy is improved, but the device complexity and portability deteriorate
Solution Approach 1:
The patent substitutes the mechanical physical keyboard with an optical sensing system. Instead of requiring physical keys, the system uses a camera to detect finger positions and movements in three-dimensional space, processing these visual signals to generate keyboard inputs, thereby eliminating the need for additional physical input devices.
Solution Approach 2:
The patent enables the camera, originally designed for photography or video functions, to serve multiple purposes including finger pose detection, gesture recognition, and keyboard input generation. This multi-functional approach allows the device to maintain physical keyboard input capabilities without adding dedicated keyboard hardware.
3Ease of operation
If voice keyboard is used, then the ease of operation is improved, but the reliability and recognition accuracy deteriorate
Solution Approach 1:
The patent replaces the acoustic voice recognition system with an optical gesture recognition system. Instead of processing spoken words through microphones and speech-to-text algorithms, the system uses camera-based visual sensing to detect finger positions and movements, translating these physical gestures into text inputs with higher reliability.
Data Source
AI summary
Disclosed are an electronic apparatus and a method of controlling the same. The electronic apparatus includes a camera, a display, a memory, and a processor configured to execute at least one instruction to: detect a plurality of fingers in a plurality of first image frames obtained through the camera and, in response to an identification that a pose of the plurality of detected fingers corresponds to a trigger pose, enter a character input mode, detect a first motion of a finger among the plurality of fingers in a plurality of second image frames obtained through in the character input mode, identify a key corresponding to the first motion, from among a plurality of keys mapped to the finger, based on a position of the finger by the first motion and a reference point set to the finger, and control the display to display information corresponding to the identified key.