On-Screen Keyboard with Optical Finger Position Feedback
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional on-screen keyboards are inefficient for typing large amounts of data due to slow data entry speed, lack of real-time feedback on finger position, and inaccurate finger detection, which hinders touch typing training and usability on head-up, head-mounted, and virtual reality displays.
Innovation Solution
A method for displaying an on-screen keyboard that includes real-time feedback of keystroke information and finger standby positions using a physical keyboard connected to a computing device with sensors and light sources, allowing users to focus on the screen without looking at the physical keyboard, and correcting finger position detection errors.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If a conventional on-screen keyboard is used to type data by clicking keys one by one with a mouse, then the keyboard can replace a physical keyboard, but the data typing speed is extremely slow
Solution Approach 1:
The patent replaces the mechanical mouse-clicking operation with an optical detection system. A camera captures images of the physical keyboard and user fingers, and image processing algorithms automatically determine finger positions and keystrokes, substituting manual mouse operations with automated optical-mechanical recognition.
Solution Approach 2:
The patent creates a visual copy of the physical keyboard layout on the display screen, showing real-time finger positions and keystroke feedback. This visual copy allows users to see their typing actions reflected on screen, enabling touch typing without looking down at the physical keyboard.
2Loss of information
If the on-screen keyboard displays real-time feedback of finger positions, then touch typing training is improved, but the system complexity increases due to image processing requirements
Solution Approach 1:
The patent implements real-time feedback by displaying the user's finger positions and keystroke information on the screen through an on-screen keyboard interface. The system continuously captures images, processes finger positions, and updates the display to show current typing status, providing immediate visual feedback for touch typing training.
Solution Approach 2:
The patent introduces an image processing module as an intermediary between the camera and the display system. This module processes captured images to extract finger position information and generates the visual feedback displayed on screen, mediating between raw image data and the final user interface.
3Measurement precision
If a camera is used to detect finger positions on the physical keyboard, then real-time feedback is achieved, but the camera may erroneously determine finger positions
Solution Approach 1:
The patent processes only the necessary portions of the captured image data - specifically focusing on identifying finger positions relative to keyboard keys - rather than analyzing the entire image. This selective processing reduces computational complexity while maintaining detection accuracy for the relevant information.
Solution Approach 2:
The patent adjusts image processing parameters such as threshold values, recognition sensitivity, and coordinate mapping to optimize finger position detection accuracy. By tuning these parameters, the system adapts to different lighting conditions, keyboard layouts, and user typing styles to improve detection reliability.
Data Source
AI summary
A display method of an on-screen keyboard, applied to a computing device connected to a physical keyboard, a screen and a sensor, the physical keyboard including a plurality of keys, and the sensor being configured to detect and obtain an operating position image, where the display method of the on-screen keyboard includes: displaying an on-screen keyboard at a topmost layer of a displayed image of the screen; reading the operating position image obtained by the sensor; converting the operating position image into an operating position data, where the operating position data includes relative position data of a key which is touched or is approached; and according to the operating position data, displaying a first key mark at a position corresponding to the key which is touched or is approached on the on-screen keyboard.