Touch screen gesture recognition keyboard system

Abstract

The invention discloses a keyboard with touch screen keys, which realizes real-time sensing and display of finger positions of a user through cooperation of a touch screen key module and a position sensing module. When a user wears the head equipment and cannot directly check the keyboard, the user only needs to touch the keys on the touch screen key module, and the positions of fingers can be seen on the output screen connected with the touch screen key module in real time, so that the user can conveniently and accurately carry out keyboard input. The method has the advantages of being high in input efficiency, high in adaptability, convenient to use and the like, and is particularly suitable for application scenes such as virtual reality and augmented reality.

Description

Technical Field

[0001] The present invention relates to the technical field of computer input devices, and in particular to a keyboard with touch screen keys, which can sense finger positions by touch and demonstrate them in real time on an output screen. The keyboard is particularly suitable for users who cannot directly view the finger positions on the keyboard when wearing head devices (such as VR helmets, AR glasses, etc.). Background Art

[0002] With the rapid development of virtual reality (VR) technology, users often need to wear head devices when using these technologies, which makes it impossible to directly see the finger position on the keyboard, thus affecting input efficiency and accuracy. Therefore, the market is in urgent need of a keyboard that can solve this problem. Summary of the invention

[0003] The purpose of the present invention is to provide a keyboard with touch screen keys, which can sense the finger position by touch and display the finger position in real time on an output screen connected thereto, thereby facilitating users to accurately input keyboard input when wearing a head device.

[0004] To achieve the above object, the present invention adopts the following technical solutions: A keyboard with touch screen buttons, including a keyboard body, a touch screen button module, a position sensing module, a communication module and a power module. The keyboard body is a traditional keyboard structure, which is used to provide a key input function; The touch screen key module covers the key area of ​​the keyboard body and can sense the touch position of the user's finger; The position sensing module is connected to the touch screen button module and is used to receive and process the finger position information transmitted by the touch screen button module; The communication module is connected to the position sensing module and is used to send the processed finger position information to the output screen connected thereto; The power module provides power support for the entire keyboard.

[0005] The gesture recognition algorithm module, as the core innovation of the present invention, includes two algorithm principles: Fingerprint recognition algorithm: Through the pre-entered user fingerprint information, the algorithm can quickly and accurately recognize the user's gestures. Efficient gesture recognition is possible even when the keyboard cannot be viewed directly while wearing a head device. Gesture recognition model constructed by feature extraction: This model analyzes the movement trajectory of the user's finger on the touch screen, the touch force, The touch duration and other features are used to build a gesture recognition model. When the user performs gesture operations, the model can extract these features in real time. The system can detect gestures and simulate the user's gestures to achieve accurate gesture recognition.

[0006] When the user is wearing a head device and cannot directly view the keyboard, just touch the key on the touch screen key module. The sensing module can obtain the finger position information in real time and send the information to the output screen through the communication module. Based on the received information, the finger position is displayed on the screen in real time, thus helping users to input the keyboard accurately. Beneficial Effects

[0007] 1. The present invention realizes the real-time perception and display of the user's finger position through the cooperation of the touch screen key module and the position sensing module, which greatly improves the keyboard input efficiency and accuracy of the user when wearing the head device 2. The communication module of the present invention can be flexibly connected to various output screens, has strong adaptability and is easy to use. 3. The power module of the present invention is reasonably designed to ensure the stable operation and long-term use of the keyboard DETAILED DESCRIPTION The following is a detailed description of a specific implementation of the present invention - a touch screen gesture recognition keyboard system: 1. Keyboard body structure: The keyboard body adopts the standard QWERTY layout, but it can also be customized to other layouts such as AZERTY, QWERTZ, etc. The keyboard body has mechanical or membrane 2. Touch screen button module integration: The touch screen key module adopts highly sensitive capacitive touch screen technology and covers the key area of ​​the keyboard body; The touch screen corresponds to the keys on the keyboard body one by one, ensuring that the input when the user touches the screen is consistent with the key input; The touch screen module is connected to the position sensing and processing module through a flexible circuit board to achieve signal transmission. 3. Location perception and processing module design: The position sensing and processing module is built into the keyboard body, using a high-performance microprocessor as the core; The module receives the finger touch position information from the touch screen button module and generates accurate finger position data through algorithm processing; The module also has the ability to recognize additional information such as touch strength and touch duration, providing data support for gesture recognition. 4. Communication module implementation: The communication module supports three or at least one of the following communication protocols: Bluetooth, Wi-Fi and USB, ensuring compatibility with a variety of external output devices; The module is built into the keyboard body and communicates wirelessly with external devices via Bluetooth, or connects to wired devices via a USB interface; The communication module adopts low power consumption design to ensure long-term use of the keyboard system 5. Power module design: The power module adopts a rechargeable lithium battery design, which has enough capacity to support the long-term use of the keyboard system; There is a charging port on the keyboard body to facilitate users to charge; The battery management circuit is responsible for monitoring the battery level and reminding the user to charge when the battery is low 6. Gesture recognition module implementation: The gesture recognition module is built into the position sensing and processing module or the communication module, and uses advanced fingerprint recognition algorithm and feature extraction and gesture simulation algorithm. Fingerprint recognition algorithm is used to quickly identify the user's fingerprint information and improve the accuracy of gesture recognition; The feature extraction and gesture simulation algorithm recognizes gestures by analyzing the movement trajectory, touch strength, touch duration and other features of the user's finger on the touch screen; the gesture recognition module can recognize a variety of gesture operations, such as sliding, clicking, long pressing, etc., and execute corresponding commands based on the recognition results. 7. Finger position simulation image generation and display: The position sensing and processing module generates a finger position simulation image for display on a screen of an external output device according to the processed finger position data; The simulacrum is displayed on the screen in a transparent or semi-transparent manner to ensure that the user can clearly see the correspondence between the finger position and the keyboard keys; The communication module sends the generated simulacrum graphics to the external output device for display, achieving real-time finger position feedback 8. System workflow: When the user touches the touch screen on the keyboard, the touch screen key module senses and transmits the finger touch position information to the position sensing and processing module; The position sensing and processing module processes the received information, generates finger position data, and may perform gesture recognition; If gesture recognition is required, the position sensing and processing module passes the gesture recognition task to the gesture recognition module for processing; The gesture recognition module recognizes the gesture operation according to the algorithm, and sends the recognition result to the external output device through the communication module to execute the corresponding command; At the same time, the position sensing and processing module generates a finger position simulacrum and sends it to an external output device for display through the communication module; Through the above specific implementation methods, the present invention realizes an intelligent keyboard system integrating a touch screen and gesture recognition, which improves the interactivity and user experience of the keyboard and is suitable for efficient input needs in various scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] Figure 1 1 is a schematic diagram of a key structure with a touch screen of the present invention, 1 is a touch screen, 2 is a position sensing module; Figure 2 1 is a schematic diagram of the left side structure of the present invention, 1 is the touch screen, 2 is the position sensing module, and 3 is the keyboard body; Figure 3 1 is a rear cross-sectional view of the present invention, 1 is a communication module, 2 is a transmission circuit board, 3 is a power module, and 4 is a keyboard housing.

Claims

1. A keyboard system with touch screen buttons, characterized in that: include: The keyboard body has the key layout of a traditional keyboard; The touch screen key module covers the key area of ​​the keyboard body and can sense the touch position of the user's finger on the keyboard; The position sensing and processing module is electrically connected to the touch screen button module and is used to receive finger touch signals from the touch screen button module. Position information, and process it to generate finger position data; The communication module is electrically connected to the position sensing and processing module and is used to send the processed finger position data to an external output device. The device is provided so as to display a simulated image of the finger position on a screen of an output device in real time; Power module, providing power support for the entire keyboard system; and a finger position simulacrum generation algorithm, which is built into the position sensing and processing module or the communication module and can A finger position simulacrum image for display on a screen of an output device is generated according to the processed finger position data.

2. The keyboard system with touch screen keys according to claim 1, characterized in that: The touch screen button module adopts Use capacitive touch screen technology to achieve high-precision finger touch position sensing.

3. The keyboard system with touch screen keys according to claim 1, characterized in that: The communication module supports Bluetooth, At least one communication protocol, such as Wi-Fi and USB, can be used to establish communication connections with various types of external output devices.

4. The keyboard system with touch screen keys according to claim 1, characterized in that: The power module is rechargeable The keyboard system also includes a battery management circuit to monitor the battery charge and manage the battery charging and discharging process. Procedure.

5. The keyboard system with touch screen keys according to claim 1, characterized in that: The finger position simulacrum The algorithms include: A data receiving step, for receiving the finger position data processed by the position sensing and processing module; A coordinate conversion step, converting the received finger position data into display coordinates on the screen of the output device; A graphics generation step, generating a simulated image of the finger position according to the converted display coordinates; And a graphics output step, sending the generated simulacrum graphics to an external output device through a communication module for display.

6. The keyboard system with touch screen keys according to claim 5, characterized in that: The finger position simulacrum The algorithm also includes an error correction step for performing error correction on the generated finger position simulation image to improve the display accuracy. Certainty.

7. The keyboard system with touch screen keys according to claim 5, characterized in that: include: a. Fingerprint identification steps: By using the pre-recorded user fingerprint information, the fingerprint recognition algorithm can quickly and accurately identify the user's gestures, so that Efficient gesture recognition can be achieved even when the head-mounted device cannot directly view the keyboard; b. Feature extraction and gesture simulation Steps: Build a gesture recognition model based on feature extraction, which analyzes the movement trajectory of the user's finger on the touch screen. Touch force, touch duration and other features to identify gestures, and extract these features in real time when the user performs gesture operations, simulating The user's gestures can be detected to achieve accurate gesture recognition.

8. The gesture recognition method according to claim 7, characterized in that: The fingerprint recognition step further includes: a. Entry stage: when the user uses it for the first time or at a preset time point, the user's fingerprint information is entered as the recognition basis; b. Matching stage: when the user performs gesture operation, the fingerprint information on the touch screen is collected in real time and matched with the entered fingerprint information to recognize the user's gesture.

9. The gesture recognition method according to claim 7, characterized in that: The feature extraction and gesture simulation steps further include: a. Model training stage: training the gesture recognition model by collecting a large number of users' finger movement trajectories, touch strength, touch duration and other data; b. Real-time recognition stage: when the user performs gesture operations, these features on the touch screen are extracted in real time and input into the gesture recognition model, and the model outputs the recognized gesture.

Images