Virtual Keyboard Finger Tracking via Optical Detection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Users face difficulties in typing on virtual keyboards due to finger coverage of multiple keys and hand blocking the display, leading to increased typing mistakes and difficulty in selecting proper keys on small electronic device screens.
Innovation Solution
A system utilizing a camera to capture sequences of images of a user's fingers and a virtual keyboard displayed on the screen, with a video feature extraction module detecting finger motion relative to the keyboard, generating sensor actuation data from virtual sensors, and a gesture pattern matching module recognizing user gestures to input data into a computing device.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If a virtual keyboard is displayed on a small electronic device screen, then the device can provide keyboard input functionality, but the user's finger covers multiple virtual keys making it difficult to select the proper key
Solution Approach 1:
The patent replaces the mechanical touch interaction system with an optical detection system. Instead of relying on physical contact between the finger and the specific key location, the camera captures images of the finger's position, and software algorithms determine which virtual key is being targeted based on the finger's location in the captured image. This substitution of mechanical touch with optical detection resolves the accuracy problem on small screens.
Solution Approach 2:
The patent introduces an intermediary system consisting of the camera and image processing software between the user's finger and the virtual keyboard selection. The camera acts as a mediator that captures the finger's position, and the software serves as an intermediary that translates the visual information into key selection data. This intermediary system allows for more precise key selection than direct touch on small screens.
2Productivity
If a user types on a virtual keyboard displayed on a small screen, then data input is possible, but the user's hand blocks portions of the display making it difficult to determine whether a proper key has been selected
Solution Approach 1:
The patent creates a visual copy of the finger's position as captured by the camera and displays it overlaid with the virtual keyboard. This copy allows the user to see exactly which key is being targeted without moving their hand away from the typing position. The visual feedback is provided through the camera image itself rather than requiring the user to look at a separate display area.
3Measurement precision
If the camera captures a sequence of images to detect finger motion, then gesture recognition accuracy is improved, but the processing time and computational resources increase
Solution Approach 1:
The patent processes a sequence of images rather than relying on a single frame, which provides more data for accurate gesture recognition. By analyzing multiple frames and detecting motion patterns across the sequence, the system achieves higher accuracy in determining finger position and gesture intent. The excessive action of capturing and processing multiple frames compensates for the increased processing time by providing more reliable recognition data.
Data Source
AI summary
The present disclosure provides systems, methods and apparatus, including computer programs encoded on computer storage media, for providing virtual keyboards. In one aspect, a system includes a camera, a display, a video feature extraction module and a gesture pattern matching module. The camera captures a sequence of images containing a finger of a user, and the display displays each image combined with a virtual keyboard having a plurality of virtual keys. The video feature extraction module detects motion of the finger in the sequence of images relative to virtual sensors of the virtual keys, and determines sensor actuation data based on the detected motion relative to the virtual sensors. The gesture pattern matching module uses the sensor actuation data to recognize a gesture.