Keyboard input fingering determination method, system and device and medium

Abstract

The invention discloses a keyboard input fingering determination method, system and device and a storage medium. The method comprises the following steps: when a keyboard is knocked by a target finger, obtaining a target screenshot at the keyboard; detecting position information of a key position adjacent to the target key position in the target screenshot and a color identifier of a target finger; wherein the target key position is the key position knocked by the current target finger; and determining a keyboard input fingering according to the position information and the color identifier. Meanwhile, according to the method, the operation key position is determined through the key position adjacent to the shielded finger, participation of naked eyes can be reduced, and the key position recognition reliability and recognition efficiency are improved. The method can be applied to the technical field of data processing.

Description

Technical Field

[0001] The present invention relates to the technical field of data processing, and in particular to a method, system, device and storage medium for determining keyboard input fingerings. Background Art

[0002] Currently, the practice of keyboard input fingerings is a process in which a user aligns the index finger with a specific key position, and then continuously corrects the user's own input fingerings by observing the type of finger used when the user strikes each key on the keyboard and the difference between the corresponding key struck and a preset input fingering table.

[0003] In related technologies, most of the keyboard input fingering practices are usually implemented using corresponding software. Keyboard input fingering software usually guides through the key position diagram on the screen, and relies on the user's own naked eyes to identify whether the type of finger and the struck key position conform to the specifications. This kind of keyboard input fingering practice has low efficiency and low reliability. There are some methods that use sensors worn on the fingers, and then use the sensors to sense the specific type of finger and the struck key position. However, this method requires wearing on the fingers, resulting in a decrease in finger flexibility, reducing the training effect and having a relatively high cost. Therefore, there are still technical problems to be solved in related technologies. Summary of the Invention

[0004] An object of the present invention is to solve at least to some extent one of the technical problems existing in the prior art.

[0005] To this end, an object of an embodiment of the present invention is to provide a method, system, device and storage medium for determining keyboard input fingerings, and this solution can improve the reliability and recognition efficiency of key position recognition.

[0006] To achieve the above technical object, the technical solution adopted by the embodiment of the present invention includes: A method for determining keyboard input fingerings, including: when the keyboard is struck by a target finger, obtaining a target screenshot at the keyboard; detecting position information of adjacent key positions of a target key position in the target screenshot and a color identifier of the target finger; wherein, the target key position is the key position currently struck by the target finger; determining keyboard input fingerings according to the position information and the color identifier.

[0007] In the process of the keyboard being struck by a target finger in the embodiment of the present invention, the position information of adjacent key positions of the key position currently struck by the target finger and the color identifier of the target finger can be obtained; according to the color identifier and the position information, keyboard input fingerings are determined. The keyboard input fingerings of the present invention can use a color identifier instead of a sensor, which can maintain the flexibility of keyboard input while reducing the training cost. At the same time, the method of the present invention determines the corresponding operation key position through the position information of adjacent key positions, which can reduce the participation of the human naked eye and improve the reliability and recognition efficiency of key position recognition.

[0008] In addition, according to the method for determining keyboard input fingering in the above embodiments of the present invention, the following additional technical features may also be included:

[0009] Further, in the embodiments of the present invention, the detecting the position information of adjacent key positions of the target key position in the target screenshot includes: obtaining the key value corresponding to the target key position; determining the adjacent key positions based on a pre-established mapping table according to the key value; identifying the adjacent key positions in the target screenshot, and extracting the position information of the adjacent key positions.

[0010] Further, in the embodiments of the present invention, the determining the keyboard input fingering according to the color identifier and the position information includes: determining the target position information of the target key position in the target screenshot according to the position information; determining the finger category of the target finger according to the color identifier; and determining the keyboard input fingering according to the finger category and the target position information.

[0011] Further, in the embodiments of the present invention, when it is determined that the adjacent key position is the key position adjacent above the target key position, the determining the target position information of the target key position in the target screenshot according to the position information includes: determining the first X coordinate and the first Y coordinate of the adjacent key position according to the position information; taking the sum value between the first X coordinate and the left-right key position offset as the X coordinate of the target position information; and taking the difference value between the first Y coordinate and the up-down key position spacing as the Y coordinate of the target position information.

[0012] Further, in the embodiments of the present invention, when it is determined that the adjacent key position is the key position adjacent below the target key position, the determining the target position information of the target key position in the target screenshot according to the position information includes:

[0013] determining the first X coordinate and the first Y coordinate of the adjacent key position according to the position information;

[0014] taking the difference value between the first X coordinate and the left-right key position offset as the X coordinate of the target position information;

[0015] taking the sum value between the first Y coordinate and the up-down key position spacing as the Y coordinate of the target position information.

[0016] Further, in the embodiments of the present invention, the left-right key position offset is obtained through the following steps:

[0017] determining the left-right key position spacing between any two adjacent key positions on the left and right;

[0018] Multiply the left - right key position spacing by a preset correction parameter to obtain a first product;

[0019] Use the first product as the left - right offset of the key position.

[0020] Further, in the embodiment of the present invention, when it is determined that the adjacent key is the key adjacent to the left of the target key, the determining the target position information of the target key in the target screenshot according to the position information includes: determining the first X - coordinate and the first Y - coordinate of the adjacent key according to the position information; using the sum value between the first X - coordinate and the left - right key position spacing as the X - coordinate of the target position information; using the first Y - coordinate as the Y - coordinate of the target position information.

[0021] Further, in the embodiment of the present invention, when it is determined that the adjacent key is the key adjacent to the right of the target key, the determining the target position information of the target key in the target screenshot according to the position information includes: determining the first X - coordinate and the first Y - coordinate of the adjacent key according to the position information; using the difference value between the first X - coordinate and the left - right key position spacing as the X - coordinate of the target position information; using the first Y - coordinate as the Y - coordinate of the target position information.

[0022] Further, in the embodiment of the present invention, the determining the finger category of the target finger according to the color identifier includes:

[0023] Extract a regional screenshot of the area where the color identifier is located in the target screenshot;

[0024] Convert all pixels in the regional screenshot into HSV pixel points;

[0025] Determine the target color of the target finger according to the HSV pixel points;

[0026] Determine the finger category according to the target color.

[0027] Further, in the embodiment of the present invention, the determining the target color of the target finger according to the HSV pixel points includes:

[0028] Classify the colors of all the HSV pixel points in the area; where each HSV pixel point corresponds to a color;

[0029] Determine the color corresponding to the most HSV pixel points in the area as the target color.

[0030] Further, in the embodiments of the present invention, the method further includes: comparing the keyboard input fingering with a preset standard fingering, and if the keyboard input fingering is different from the preset standard fingering, generating an alarm message.

[0031] On the other hand, an embodiment of the present invention further provides a keyboard input fingering determination system, including:

[0032] A first processing unit, configured to obtain a target screenshot at the keyboard when the keyboard is tapped by a target finger;

[0033] A second processing unit, configured to detect the position information of adjacent key positions of a target key position in the target screenshot and the color identifier of the target finger; wherein, the target key position is the key currently tapped by the target finger;

[0034] A third processing unit, configured to determine the keyboard input fingering according to the position information and the color identifier.

[0035] On the other hand, the present invention further provides a keyboard input fingering determination device, including:

[0036] At least one processor;

[0037] At least one memory, configured to store at least one program;

[0038] When the at least one program is executed by the at least one processor, the at least one processor implements the keyboard input fingering determination method described in any one of the invention content.

[0039] In addition, the present invention further provides a computer-readable storage medium, in which processor-executable instructions are stored, and the processor-executable instructions are used to execute the keyboard input fingering determination method described in any one of the above when executed by a processor.

[0040] The advantages and beneficial effects of the present invention will be partially given in the following description, partially become obvious from the following description, or be understood through the practice of the present invention:

[0041] During the process of the keyboard being tapped by a target finger, the present invention can obtain the position information of adjacent key positions of the key currently tapped by the target finger and the color identifier of the target finger. According to the color identifier and the position information, the keyboard input fingering is determined. The keyboard input fingering of the present invention can use the color identifier instead of the sensor, which can maintain the flexibility of keyboard input while reducing the training cost. At the same time, the method of the present invention determines the corresponding operation key position through the position information of adjacent key positions, which can reduce the participation of the human eye and improve the reliability and recognition efficiency of key position recognition. Description of the Drawings

[0042] Figure 1 Schematic diagram of the steps of the keyboard input fingering determination method in a specific embodiment of the present invention;

[0043] Figure 2 Schematic diagram of the steps of detecting the position information of adjacent keys of the target key in the target screenshot in a specific embodiment of the present invention;

[0044] Figure 3 Schematic diagram of the steps of determining the keyboard input fingering according to the color identification and position information in a specific embodiment of the present invention;

[0045] Figure 4 Schematic diagram of the steps of determining the target position information of the target key in the target screenshot according to the position information when it is determined that the adjacent key is the key adjacent to the upper part of the target key in a specific embodiment of the present invention;

[0046] Figure 5 Schematic diagram of the steps of determining the target position information of the target key in the target screenshot according to the position information when it is determined that the adjacent key is the key adjacent to the lower part of the target key in a specific embodiment of the present invention;

[0047] Figure 6 Schematic diagram of the steps of determining the left - right offset of the key position in a specific embodiment of the present invention;

[0048] Figure 7 Schematic diagram of the steps of determining the target position information of the target key according to the position information when the key adjacent to the left side of the target key is used as the adjacent key in a specific embodiment of the present invention;

[0049] Figure 8 Schematic diagram of the steps of determining the target position information of the target key according to the position information when the key adjacent to the right side of the target key is used as the adjacent key in another specific embodiment of the present invention;

[0050] Figure 9 Schematic diagram of the steps of determining the finger category of the target finger according to the color identification in another specific embodiment of the present invention;

[0051] Figure 10 Schematic diagram of the steps of determining the target color of the target finger according to the HSV pixel points in a specific embodiment of the present invention;

[0052] Figure 11 Schematic diagram of the steps of the keyboard input fingering determination method in another specific embodiment of the present invention;

[0053] Figure 12 Schematic diagram of the sticker identification colors of the left and right hands during the keyboard input fingering determination process in a specific embodiment of the present invention;

[0054] Figure 13 It is a schematic structural diagram of a keyboard input fingering determination system in a specific embodiment of the present invention;

[0055] Figure 14 It is a schematic structural diagram of a keyboard input fingering determination device in a specific embodiment of the present invention. Specific embodiments

[0056] The following describes in detail the embodiments of the present invention with reference to the accompanying drawings. The principles and processes of the keyboard input fingering determination method, system, device, and storage medium in the embodiments of the present invention are described as follows.

[0057] First, explain the terms that appear in the present invention:

[0058] OpenCV: OpenCV (Open Source Computer Vision Library) is an open-source cross-platform computer vision and machine learning software library. The functional features of OpenCV can include: image processing functions, computer vision functions, machine learning support functions, and deep learning integration functions. Image processing functions: Provide basic operations such as image loading, saving, displaying, cropping, scaling, rotation, etc., as well as advanced functions such as filtering, morphological operations, edge detection, and histograms. Computer vision functions: Cover algorithms such as feature detection, object recognition, object tracking, camera calibration, etc., as well as feature detection and descriptor matching algorithms such as SIFT, SURF, and ORB. Machine learning support functions: Combined with machine learning libraries (such as MLPack, Scikit-Learn), various machine learning tasks such as classification, regression, and clustering can be realized, including algorithms such as support vector machines, K-nearest neighbors, and decision trees. Deep learning integration functions: Support using deep learning frameworks (such as TensorFlow, PyTorch) for tasks such as image recognition and object detection.

[0059] OCR technology: OCR (Optical Character Recognition) technology. OCR technology is a technology that converts the text content in an image into an editable text format. It is widely used in multiple fields such as finance, education, medical care, government, and retail, greatly improving the efficiency and accuracy of information processing. The implementation process of OCR technology mainly can include steps such as image preprocessing, text detection, and text recognition. Among them, image preprocessing can include image binarization, denoising, tilt angle detection and correction, etc., and these steps are crucial for improving the recognition rate of OCR. In the.NET environment, the OCR function can also be realized, usually by using third-party libraries and other methods.

[0060] HSV: HSV is a color space model, standing for Hue, Saturation, Value, also known as the HSB model, where B stands for Brightness. V and B essentially represent the same thing: describing the brightness of a color. The HSV model organizes color information into a cone-shaped representation, where hue (H) is the angle around the cone's central axis, saturation (S) is the radius from the cone's vertex to the cone's edge, and value (V) is the cone's vertical height.

[0061] Secondly, the technical problems existing in the relevant technologies are explained:

[0062] In the related art, most keyboard input fingering exercises are usually implemented using corresponding software. However, keyboard input fingering software relies on the guidance of the key map on the screen, and the user relies on the naked eye to identify whether the fingering conforms to the prescribed fingering. This type of keyboard input fingering training is inefficient and unreliable. Some methods use sensors worn on the fingers to detect the specific fingering. However, this method requires wearing the fingers, resulting in reduced finger flexibility, reduced training effectiveness, and relatively high cost. Therefore, there are still problems in the related art that need to be solved urgently.

[0063] Then, in view of the defects of the above-mentioned prior art, Figure 1 , Figure 1 This is a schematic diagram of the steps of a method for determining a keyboard input fingering provided by an embodiment of the present invention. Figure 1 In the method for determining the keyboard input fingering, the method may include but is not limited to steps S101 to S103.

[0064] S101. When a keyboard is tapped by a target finger, a target screenshot of the keyboard is obtained.

[0065] It is understandable that the target screenshot can be a screenshot of any key on the keyboard being tapped. During the process of the keyboard being tapped by the target finger, since the key positions tapped by different types of fingers constantly change, this embodiment can generate many target screenshots.

[0066] In some feasible embodiments of the present application, when a target finger taps a keyboard, a user can configure a camera to capture a video stream of the keyboard being tapped by the target finger. The camera can establish a wired or wireless connection with the processor, and the keyboard can also establish a wired or wireless connection with the processor. Each time the user taps a key on the keyboard, the tapped key can generate a signal, which enables the processor to obtain a corresponding target screenshot from the video stream.

[0067] It should be noted that the above-mentioned wired connection methods may include the connection between the mobile device and the processing module, and may also include the connection between the processing module and the hardware device, as well as the wired connection between other currently known or future-developed devices and the processing module; and the above-mentioned wireless connection methods may include, but are not limited to, 3G / 4G / 5G connections, WiFi connections, Bluetooth connections, WiMAX connections, Zigbee connections, UWB (Ultra Wide Band) connections, and other currently known or future-developed wireless connection methods.

[0068] S102. Detect the position information of adjacent keys of the target key in the target screenshot and the color identifier of the target finger; where the target key is the key struck by the current target finger.

[0069] It can be understood that each struck target key may include one or more adjacent keys. Among them, these adjacent keys are two keys that are not blocked by the finger during the process of the keyboard being struck by the target finger. For example, for the keys at the four corner positions of the keyboard, there are two adjacent keys. The struck key can be the key at the upper right corner of the keyboard. The adjacent keys of the key at the upper right corner of the keyboard may include the adjacent key below and the adjacent key on the left. The struck key can be the key at the upper left corner of the keyboard. The adjacent keys of the key at the upper left corner of the keyboard can be the adjacent key below and the adjacent key on the right. The struck key can be the key at the lower left corner of the keyboard. The adjacent keys of the key at the lower left corner of the keyboard can be the adjacent key above and the adjacent key on the right. The struck key is the key at the lower right corner of the keyboard. The adjacent keys of the key at the lower right corner of the keyboard can be the adjacent key above and the adjacent key on the left. The struck key can be the row at the top of the keyboard and not the leftmost or rightmost key. Its adjacent keys can be the adjacent key below and the adjacent keys on the left and right. The struck key can be the row at the bottom of the keyboard and not the leftmost or rightmost key. Its adjacent keys can be the adjacent key above and the adjacent keys on the left and right. The struck key can be the row at the leftmost of the keyboard and not the topmost or bottommost key. Its adjacent keys can be the adjacent key on the right and the adjacent keys above and below. The struck key can be the row at the rightmost of the keyboard and not the topmost or bottommost key. Its adjacent keys can be the adjacent key on the left and the adjacent keys above and below.

[0070] Each adjacent key position can correspond to a position information, which can be coordinates in a coordinate system on a screenshot. Among them, the coordinate system can be a coordinate system established with the center of a certain key on the keyboard as the origin, the horizontal direction of the key as the X-axis of the coordinate system, and the vertical direction of the key as the Y-axis of the coordinate system. The coordinate system can also be a coordinate system established with the key corresponding to any side edge of the screenshot. Each key position can be a coordinate corresponding to it in the coordinate system. The coordinate of the key position can also be the coordinate of the center of the key position.

[0071] The target finger can be the finger used in each tap during the user's keyboard tapping process. During a tapping process, the target finger can be continuously switched, and the target finger can be any finger of both hands. Each target finger can be set with a color identifier. The color identifier can be set on the back of the finger, specifically on the fingernail of the finger. The color identifier can be formed by pasting a color sticker on the fingernail of the finger, or can be formed by coating the fingernail of the finger with color paint, or can also be formed by putting a color finger cot on the fingertip of the whole finger. Preferably, in order to reduce costs and facilitate operation at the same time, the color identifier in this embodiment is formed by pasting a color sticker on the fingernail of the finger. The color identifiers corresponding to each finger can be different from each other. And since the thumb can be used for positioning or tapping specific key positions during keyboard input, the color identifiers of the thumbs corresponding to both hands can be the same, while the color identifiers corresponding to other fingers can be different from each other, and the color identifier of any finger can be different from that of the thumb. The color identifier in this embodiment in the form of a sticker can be quickly pasted and removed, improving the efficiency of finger method training. Moreover, the sticker is lightweight and does not affect the smoothness of finger operations during keyboard input, further improving the training efficiency.

[0072] Exemplarily, the two thumbs can be set with red identifiers. The other 8 fingers can be set to magenta, orange, yellow, green, blue, cyan, purple, and brown. As for which color the left little finger, left ring finger, left middle finger, left index finger, right little finger, right ring finger, right middle finger, and right index finger specifically correspond to, the user can choose according to needs. The color differences between the above 9 colors are relatively large, which can effectively improve the accuracy of image recognition. Moreover, these 9 colors have a relatively large difference from the keyboard background color, which can effectively reduce the interference of the keyboard background color and the keyboard base color, further improving the accuracy of image recognition.

[0073] In some feasible embodiments of the present invention, the processor can detect the position information of the adjacent key positions of the target key position in the target screenshot and the color identifier of the target finger. Among them, the target key position can be the key position tapped by the current target finger. Since one or more adjacent key positions will not be completely blocked during the tapping process, the adjacent key positions in this embodiment can be directly detected by the camera.

[0074] Exemplarily, the user can use a mobile phone camera or a USB camera, in cooperation with a fixed bracket, to photograph the keyboard from above the keyboard to obtain a video stream of the keyboard being tapped for input. Then, the processor can be equipped with corresponding software, and this software can monitor keyboard input events in real time through the mobile phone camera or the USB camera. During a period when the user taps the keyboard for input, this software can read the real-time video stream information or real-time video frame information at each tap. By analyzing this real-time video stream information or real-time video frame information, the processor can obtain the position information of the adjacent key positions corresponding to each tap and the color identifier corresponding to the target finger that tapped the keyboard.

[0075] S103. Determine the keyboard input fingering according to the position information and the color identifier.

[0076] It can be understood that the color identifier can correspondingly identify whether the finger that taps the keyboard each time belongs to the thumb, index finger, middle finger, ring finger or little finger. The position information of the adjacent key positions can determine the key position being tapped;

[0077] In some feasible embodiments of the present invention, after obtaining the color identifier and the position information, the processor obtains the finger type corresponding to each tap according to the color identifier. The processor can determine the tapped key position according to the position information. After obtaining the key position and finger for each tap, the processor can obtain the complete fingering corresponding to the process of the keyboard being tapped by the target finger within a period of time.

[0078] In summary, during the process of the keyboard being tapped by the target finger, this embodiment can obtain the position information of the adjacent key positions of the key position currently tapped by the target finger and the color identifier of the target finger. Then, according to the color identifier and the position information, the keyboard input fingering is determined. The keyboard input fingering of this embodiment can use the color identifier to replace the sensor, which can maintain the flexibility of keyboard input while reducing the training cost. At the same time, the method of the present invention determines the corresponding operation key position through the position information of the adjacent key positions, which can reduce the participation of the human eye, improve the reliability and recognition efficiency of key position recognition.

[0079] Further, referring to Figure 2 , Figure 2 is a schematic diagram of the steps for detecting the position information of the adjacent key positions of the target key position in the target screenshot in an embodiment of the present invention. In Figure 2 , this step may include but is not limited to step S201 - step 203.

[0080] S201. Obtain the key value corresponding to the target key position.

[0081] S202. Based on the key value, determine the adjacent key positions based on a pre-established mapping table.

[0082] S203: Identify adjacent key positions in the target screenshot and extract position information of the adjacent key positions.

[0083] It is understandable that the key value corresponding to the target key position cannot be directly obtained by the processor, but the key value is combined with a pre-established mapping table to determine the adjacent key corresponding to the target key position.

[0084] In some feasible embodiments of the present invention, the processor may first obtain a key value corresponding to a target key position. After obtaining the key value, the processor may determine adjacent keys based on the key value and a pre-established mapping table. After determining the adjacent keys, the processor may identify adjacent keys of the target key position in the target screenshot and extract position information of the adjacent keys.

[0085] For example, if the key value of the number 1 is XXX, the mapping relationship can be expressed as: "XX X":{top:null,left:"~",right:'2',bottom:'Q'}. The key value of Q is YY, and the mapping relationship is: "YY":{top:'1',left:"左Tab",right:'W',bottom:'A'}. It should be noted that auxiliary keys such as tab, shift, ctrl, or alt need to distinguish between left and right, and the key values corresponding to the left and right auxiliary keys are different.

[0086] In summary, this embodiment can determine adjacent keys based on keystroke values and identify key position information in a screenshot when a target finger is tapping on the keyboard, thereby reducing detection and recognition errors caused by instability during keyboard tapping in traditional visual detection. This embodiment can also improve the accuracy of adjacent key recognition.

[0087] Further, refer to Figure 3 , Figure 3 This is a schematic diagram of the steps of determining keyboard input fingering according to color identification and position information in an embodiment of the present invention. Figure 3 In the embodiment, the steps may include S301-S303.

[0088] S301: Determine target position information of a target key in a target screenshot according to the position information.

[0089] S302: Determine the finger category of the target finger according to the color identifier.

[0090] S303: Determine keyboard input fingering according to the finger type and target position information.

[0091] It can be understood that the target position information and the position information of adjacent key positions can be in the same coordinate system. The finger categories can be specifically divided into 10 categories: left thumb, left index finger, left middle finger, left ring finger, left little finger, right thumb, right index finger, right middle finger, right ring finger, and right little finger. In some embodiments, the finger categories can also be divided into 9 categories: thumb, left index finger, left middle finger, left ring finger, left little finger, right index finger, right middle finger, right ring finger, and right little finger.

[0092] In some feasible embodiments of the present invention, the processor can determine the target position information of the target key position in the target screenshot according to the position information. At the same time, the processor can also determine the finger category of the target finger according to the color identifier. After obtaining the target position information and the finger category, the processor can determine the keyboard input fingering according to the finger category and the target position information.

[0093] It should be noted that the keyboard input fingering can be presented in the form of a list. Among them, the row category corresponding to the list can be 9 finger types, and the column type corresponding to the list can be the target key positions of the keyboard. Similarly, the column category corresponding to the list can also be 9 finger types, and the row type corresponding to the list can also be the target key positions for each keystroke.

[0094] In summary, in this embodiment, during the process of the keyboard being struck by the target finger, the position information of the struck key position can be determined according to the position information of adjacent key positions. This embodiment can also determine the finger type of the keyboard input according to the color identifier. The keyboard input fingering of this embodiment can use the color identifier to replace the sensor to determine the finger type, which can improve the recognition efficiency while maintaining the finger flexibility. At the same time, the method of the present invention can determine the corresponding operation key position through the position information of adjacent key positions, which can improve the reliability and recognition efficiency of key position recognition.

[0095] Further, referring to Figure 4 , Figure 4 is a schematic diagram of the steps for determining the target position information of the target key position in the target screenshot according to the position information when it is determined that the adjacent key position is the key position adjacent to the upper side of the target key position in an embodiment of the present invention. In Figure 4 , this step can include but is not limited to steps S401 - S403.

[0096] S401. Determine the first X coordinate and the first Y coordinate of the adjacent key position according to the position information.

[0097] S402. Use the sum of the first X coordinate and the left - right offset of the key position as the X coordinate of the target key position.

[0098] S403. Use the difference between the first Y coordinate and the up - down spacing of the key position as the Y coordinate of the target key position.

[0099] It is understandable that the position information can be coordinates in a coordinate system or specific grid lines. When the position coordinates are specific grid lines, they can also be converted into plane coordinate system coordinates through the corresponding mapping table. The left - right offset of the key position can be data determined by the keyboard tilt pair and the distance between the left and right key positions. The up - down distance between key positions can be the distance between two adjacent upper - lower key positions, which can be calculated from the Y coordinates of two adjacent upper - lower key positions. The first X coordinate can be the X coordinate of the key position adjacent to the key being struck in the same coordinate system. The first Y coordinate can be the Y coordinate of the key position adjacent to the key being struck in the same coordinate system. And both the key position adjacent to the key being struck and the key being struck can be in the same coordinate system.

[0100] In some feasible embodiments of the present invention, the processor can first determine the positions of all adjacent key positions. When the target key position is struck and the unobstructed adjacent key position is the key position adjacent above the target key position being struck, the processor can use the adjacent key position above as the key position required for calculating the position of the target key position and extract its corresponding X coordinate and Y coordinate. After obtaining the X coordinate and Y coordinate of the adjacent key position, the processor can calculate or directly extract the left - right offset of the key position and the up - down distance between key positions from the database. After obtaining the left - right offset of the key position and the up - down distance between key positions, the processor can use the sum of the X coordinate and the left - right offset as the corresponding X coordinate of the target key position being struck. At the same time, the processor can use the difference between the Y coordinate and the up - down distance between key positions as the corresponding Y coordinate of the target key position being struck.

[0101] Exemplarily, after extracting the X coordinate and Y coordinate of the adjacent key position above in this embodiment, they can be input into the corresponding formula group to obtain the corresponding X coordinate and Y coordinate of the target key position. The corresponding formula group can include:

[0102] X t = X1 + L1

[0103] Y t = Y1 - S1

[0104] In the above formula group, X t is the X coordinate of the target key position, X1 is the X coordinate of the adjacent key position above, L1 is the left - right offset of the key position, Y t is the Y coordinate of the target key position, and Y1 is the Y coordinate of the adjacent key position above.

[0105] In summary, in this embodiment, the key adjacent to the target key above can be used as the adjacent key. At the same time, in this embodiment, the X and Y coordinates of the corresponding target key in the same coordinate system can be calculated through the key adjacent to the target key above. According to the characteristics of keyboard fingering, this embodiment can reduce the use of the left, right, and lower keys that are more likely to be blocked to calculate the coordinates of the target key, which can improve the accuracy of data processing.

[0106] Further, referring to Figure 5 , Figure 5 is a schematic diagram of the steps for determining the target position information of the target key in the target screenshot according to the position information when the adjacent key is the key adjacent to the target key below in an embodiment of the present invention. In Figure 5 , this step may include but is not limited to steps S501 - S503.

[0107] S501. Determine the first X coordinate and the first Y coordinate of the adjacent key according to the position information.

[0108] S502. Use the difference between the first X coordinate and the left - right offset of the key as the X coordinate of the target key.

[0109] S503. Use the sum of the first Y coordinate and the up - down spacing of the keys as the Y coordinate of the target key.

[0110] It can be understood that the left - right offset of the key can be data determined by the keyboard tilt and the spacing between the left and right keys. The up - down spacing of the keys can be the spacing between two adjacent keys above and below, which can be calculated through the Y coordinates of two adjacent keys above and below. The first X coordinate can be the X coordinate of the key adjacent to the key being struck in the same coordinate system. The first Y coordinate can be the Y coordinate of the key adjacent to the key being struck in the same coordinate system. And the key adjacent to the key being struck and the key being struck are both in the same coordinate system.

[0111] In some feasible embodiments of the present invention, the processor can first determine the positions of all adjacent keys. When the adjacent key that is not blocked when the target key is struck is the key adjacent to the target key above, the processor can use the adjacent key above as the key required for calculating the position of the target key and extract its corresponding X coordinate and Y coordinate. After obtaining the X coordinate and Y coordinate of the adjacent key, the processor can calculate or directly extract the left - right offset of the key and the up - down spacing of the keys from the database. After obtaining the left - right offset of the key and the up - down spacing of the keys, the processor can use the difference between the X coordinate and the left - right offset as the corresponding X coordinate of the target key being struck. At the same time, the processor can use the sum of the Y coordinate and the up - down spacing of the keys as the corresponding Y coordinate of the target key being struck.

[0112] Exemplarily, after extracting the X coordinate and Y coordinate of the adjacent key position below, the corresponding X coordinate and Y coordinate of the target key position can be obtained by inputting them into the corresponding formula group. The corresponding formula group may include:

[0113] X t = X2 - L1

[0114] Y t = Y2 + S1

[0115] In the above formula group, X t is the X coordinate of the target key position, X2 is the X coordinate of the adjacent key position below, L1 is the left - right offset of the key position, and Y t is the Y coordinate of the target key position, Y2 is the Y coordinate of the adjacent key position below.

[0116] It should be noted that the target key position can be the key position in the upper - left corner of the keyboard. Due to the special position of the target key position, when the key position in the upper - left corner of the keyboard is generally struck, the adjacent key position below is not easily blocked. Therefore, the adjacent key position below can be used as the key position for calculating the coordinates of the target key position. Similarly, the key position in the upper - right corner of the keyboard can also use this method to determine the specific position information. Similarly, several target key positions on the left edge of the keyboard can also use this method to determine the specific position information. Similarly, several target key positions on the right edge of the keyboard can also use this method to determine the specific position information.

[0117] In summary, in this embodiment, the adjacent key position below the target key position is used as the adjacent key position. At the same time, in this embodiment, the X and Y coordinates of the corresponding target key position in the same coordinate system can be calculated through the adjacent key position below the target key position. According to the characteristics of keyboard fingering, for key positions at specific locations such as the upper - left and upper - right corners of the keyboard, algorithms different from other key positions are adopted, which can effectively reduce the influence of different key position on the algorithm accuracy.

[0118] Further, referring to Figure 6 , Figure 6 is a schematic diagram of the steps for determining the left - right offset of the key position in an embodiment of the present application. In Figure 6 , the left - right offset of the key position can be obtained through step S601 - step S603.

[0119] S601. Determine the left - right key position distance between any two adjacent key positions on the left and right.

[0120] S602. Multiply the left - right key position distance by the preset correction parameter to obtain the first product.

[0121] S603. Use the first product as the left - right offset of the key position.

[0122] In some feasible embodiments of the present invention, the processor may first determine the left - right key - position spacing between any two adjacent left - right key positions. After obtaining the left - right key - position spacing, the processor may multiply the left - right key - position spacing by a preset correction parameter to obtain a first product, and use the first product as the left - right offset of the key position.

[0123] Exemplarily, the left - right offset of the key position may also satisfy the relational expression:

[0124] L1 = A * L2

[0125] In the above formula, L1 is the left - right offset of the key position, L2 is the left - right key - position spacing between any two adjacent left - right key positions, A is the preset correction parameter, and 0 < A < 1.

[0126] It should be noted that the preset correction parameter may be different values according to different tilts of the keyboard. In this embodiment, A is preferably 0.2.

[0127] The left - right offset of the key position in this embodiment can be adjusted according to different tilts of the keyboard, which can further improve the accuracy of the algorithm for determining the target key - position coordinates.

[0128] Further, referring to Figure 7 , Figure 7 is a schematic diagram of the steps for determining the target position information of the target key position according to the position information when the key position adjacent to the left side of the target key position is used as the adjacent key position in an embodiment of the present invention. In Figure 7 , this step may include but is not limited to steps S701 - S703.

[0129] S701. Determine the first X - coordinate and the first Y - coordinate of the adjacent key position according to the position information.

[0130] S702. Use the sum of the first X - coordinate and the left - right key - position spacing as the X - coordinate of the target key position.

[0131] S703. Use the first Y - coordinate as the Y - coordinate of the target key position.

[0132] It can be understood that the left - right key - position spacing may be the spacing between any two adjacent left - right key positions. It can be obtained by subtracting the X - coordinate of the left - hand key position from the X - coordinate of the right - hand key position among any two adjacent left - right key positions. The left - right key - position spacings between all two adjacent left - right key positions in the keyboard may be equal.

[0133] In some feasible embodiments of the present invention, the processor may first determine the position where the target key is located. When the target key is struck and the adjacent key that is not blocked is the key adjacent to the left of the struck target key, the processor may determine the X coordinate and the Y coordinate in the position information of the adjacent key. After obtaining the X coordinate and the Y coordinate of the adjacent key, the processor may calculate or directly extract the left and right key spacing from the database. After obtaining the left and right key spacing, the processor may use the sum of the X coordinate and the left and right key spacing as the corresponding X coordinate of the struck target key. At the same time, the processor may use the Y coordinate of the left adjacent key as the Y coordinate corresponding to the struck target key.

[0134] Exemplarily, after extracting the X coordinate and the Y coordinate of the left adjacent key in this embodiment, they may be input into the corresponding formula group to obtain the X coordinate and the Y coordinate of the corresponding target key. The corresponding formula group may include:

[0135] X t = X3 + L2

[0136] Y t = Y3

[0137] In the above formula group, X t is the X coordinate of the target key, X3 is the X coordinate of the left adjacent key, L2 is the left and right key spacing, Y t is the Y coordinate of the target key, and Y3 is the Y coordinate of the left adjacent key.

[0138] It should be noted that the target key may be several keys at the right edge of the keyboard. Due to the special position of the target key, when several keys at the right edge of the keyboard are generally struck, the adjacent keys on the left are not easily blocked. Therefore, the adjacent keys on the left can be used as the keys for calculating the coordinates of the target key.

[0139] In summary, this embodiment can, according to the characteristics of keyboard fingering, adopt an algorithm different from that of other keys to determine the coordinates of keys at specific positions, such as the keys at the right edge of the keyboard. This embodiment can improve the accuracy of key position calculation.

[0140] Further, referring to Figure 8 , Figure 8 is a schematic diagram of the steps for determining the target position information of the target key according to the position information when the key adjacent to the right of the target key is used as the adjacent key in an embodiment of the present invention. In Figure 8 , this step may include but is not limited to step S801 - step S803.

[0141] S801. According to the position information, determine the first X coordinate and the first Y coordinate of the adjacent key.

[0142] S802. Use the difference between the first X coordinate and the left - right key - position spacing as the X coordinate of the target key position.

[0143] S803. Use the first Y coordinate as the Y coordinate of the target key position.

[0144] It can be understood that the left - right key - position spacing can be the spacing between any two adjacent left - right key positions. The left - right key - position spacing can be obtained by subtracting the X coordinate of the left - hand key position from the X coordinate of the right - hand key position among any two adjacent left - right key positions. The left - right key - position spacings between all two adjacent left - right key positions within the keyboard can be equal.

[0145] In some feasible embodiments of the present invention, the processor can first determine the position where the target key position is located. When the target key position is tapped and the adjacent key position that is not blocked is the adjacent key position on the right side of the tapped target key position, the processor can determine the X coordinate and the Y coordinate in the position information of the adjacent key position. After obtaining the X coordinate and the Y coordinate of the adjacent key position, the processor can calculate or directly extract the left - right key - position spacing from the database. After obtaining the left - right key - position spacing, the processor can use the difference between the X coordinate and the left - right key - position spacing as the corresponding X coordinate of the tapped target key position. At the same time, the processor can use the Y coordinate of the adjacent key position on the right side as the corresponding Y coordinate of the tapped target key position.

[0146] Exemplarily, after extracting the X coordinate and the Y coordinate of the adjacent key position on the right side in this embodiment, the X coordinate and the Y coordinate of the adjacent key position on the right side can be input into the corresponding formula group to obtain the corresponding X coordinate and Y coordinate of the target key position. The corresponding formula group can include:

[0147] X t = X4 - L2

[0148] Y t = Y4

[0149] In the above formula group, X t is the X coordinate of the target key position, X4 is the X coordinate of the adjacent key position on the right side, L2 is the left - right key - position spacing, Y t is the Y coordinate of the target key position, and Y4 is the Y coordinate of the adjacent key position on the right side.

[0150] It should be noted that the target key positions can be several key positions on the left - hand edge of the keyboard. Due to the special position of the target key positions, when the several key positions on the left - hand edge of the keyboard are generally tapped, the adjacent key positions on the right side are not easily blocked. Therefore, the adjacent key positions on the right side can be used as the key positions for calculating the coordinates of the target key positions.

[0151] In summary, according to the characteristics of keyboard fingering, in this embodiment, for key positions at specific locations, such as key positions on the left edge of the keyboard, algorithms different from those of other key positions are used to determine the coordinates of the key positions. This embodiment can improve the accuracy of key position calculation.

[0152] Further, referring to Figure 9 , Figure 9 is a schematic diagram of the steps for determining the finger category of the target finger according to the color identifier in an embodiment of the present invention. In Figure 9 , this step may include but is not limited to steps S901 - S904.

[0153] S901. Extract the regional screenshot of the area where the color identifier is located in the target screenshot.

[0154] S902. Convert all pixels in the regional screenshot into HSV pixel points.

[0155] S903. Determine the target color of the target finger according to the HSV pixel points.

[0156] S904. Determine the finger category according to the target color.

[0157] It can be understood that the area where the color identifier is located may be an area with a preset length and width including the color identifier, and the area where the color identifier is located may completely cover the entire color identifier. Specifically, the length of the area where the color identifier is located may be greater than or equal to the length of the color identifier, and the width of the area where the color identifier is located may be greater than or equal to the width of the color identifier. The HSV pixel points may be pixel points each having three parameters: hue, saturation, and value.

[0158] In some feasible embodiments of the present invention, the processor may first extract the regional screenshot of the area where the color identifier is located. After obtaining the regional screenshot, the processor may convert the RGB pixel points of the regional screenshot into HSV pixel points through the HSV model. After obtaining the HSV pixel points, the processor may obtain the specific color of the color identifier, that is, the target color corresponding to the target hand value. According to the target color and the preset mapping table, the finger category corresponding to the color can be mapped.

[0159] It can be understood that there may be a mapping relationship between the finger types of the left and right hands and different colors in the preset mapping table. Since there is only one same color for the two thumbs, this mapping table may include 9 mapping relationships.

[0160] Exemplarily, the processor may crop out the area of the key position being tapped from the screenshot. Convert the cropped image area from the RGB color to the HSV color space. Determine the target color corresponding to the target hand value within the color space. Finally, through the target color and the preset mapping table, the finger category corresponding to the color can be mapped.

[0161] In summary, in this embodiment, the finger color is determined through HSV pixels, which can reduce the interference of the keyboard background color on the color judgment result of the color identifier, improve the accuracy of color judgment, and thus can more accurately identify the type of finger pressing the keyboard.

[0162] Further, referring to Figure 10 , Figure 10 is a schematic diagram of the steps for determining the target color of the target finger according to HSV pixels in an embodiment of the present invention. In Figure 10 , this step may include but is not limited to steps S111 - S112.

[0163] S111. Classify the colors of all HSV pixels in the area; wherein, each HSV pixel corresponds to a color.

[0164] S112. Determine the color corresponding to the most HSV pixels in the area as the target color.

[0165] It can be understood that each color corresponds to a hue, a saturation, and a brightness. Different colors can be distinguished by hue, saturation, and brightness. ]>[[ID=

[22] ]

[0166] In some feasible embodiments of the present invention, the processor can classify the colors of all HSV pixels in the area where the color identifier is located to obtain the color of each pixel. Then the processor counts the pixels and determines the color corresponding to the most HSV pixels in the area where the color identifier is located as the target color.

[0167] In summary, this embodiment can determine the color corresponding to the most HSV pixels in the area as the target color, which can reduce the interference of the finger color on the color judgment result of the color identifier, improve the accuracy of the color judgment of the color identifier, and thus can more accurately identify the type of finger pressing the keyboard.

[0168] Further, referring to Figure 11 , Figure 11 is a schematic diagram of the steps of another method for determining keyboard input fingerings in the present invention. In Figure 11 , the method for determining keyboard input fingerings may further include but is not limited to step S104.

[0169] S104. Compare the keyboard input fingering with a preset standard fingering. If the keyboard input fingering is different from the preset standard fingering, generate an alarm message.

[0170] It can be understood that during the process of comparing the keyboard input fingering with the preset standard fingering, if the finger types or key positions corresponding to the keyboard input fingering are different, the processor will determine it as a fingering error and generate an alarm message. The alarm message can be displayed on the screen of the monitor to remind the user to correct the fingering. The alarm message can also be conveyed to the user through sound information such as a buzzer to remind the user to correct the fingering.

[0171] In some feasible embodiments of the present invention, the processor can compare the keyboard input fingering with the preset standard fingering to obtain a comparison result. If the comparison result shows that the keyboard input fingering is different from the preset standard fingering, the processor can generate an alarm message and convey the alarm message to the user in a way that can be heard or seen by the user's naked eyes to remind the user to correct the fingering.

[0172] In some other feasible embodiments, while generating the alarm message, the processor will also count the number of keyboard inputs, the number of errors, the error rate, and the error ranking of each finger on the same day, and generate a corresponding list to remind the user to correct the fingering later.

[0173] In summary, this embodiment can generate an alarm message according to the comparison result between the keyboard input fingering and the preset standard fingering to remind the user to correct the fingering in real time. This embodiment can improve the real-time performance of fingering correction and enhance the efficiency of fingering correction.

[0174] The following describes the specific calculation principle of the present invention with reference to the accompanying drawings:

[0175] In this embodiment, the left-right offset of the key position is 0.2, the target key position is "J", the adjacent key position is the upper adjacent key position "1", and the color identifier is a sticker with red, magenta, orange, yellow, green, blue, cyan, purple, and brown colors. Among them, referring to Figure 12 , the stickers corresponding to the thumbs of both hands are red stickers (red), the sticker corresponding to the left little finger is a green sticker (green), the sticker corresponding to the left ring finger is a blue sticker (blue), the sticker corresponding to the left middle finger is a yellow sticker (yellow), the sticker corresponding to the left index finger is a cyan sticker (cyan), the sticker corresponding to the right little finger is a magenta sticker (magenta), the sticker corresponding to the right ring finger is an orange sticker (orange), the sticker corresponding to the right middle finger is a purple sticker (purple), and the sticker corresponding to the right index finger is a brown sticker (brown).

[0176] In this embodiment, the method for determining and correcting the keyboard input fingering is implemented through a system. The system can include a mobile phone camera or a USB camera and a system processor. The process of determining and correcting the keyboard input fingering can include steps S1 to S8.

[0177] S1. Use a mobile phone camera or a USB camera, together with a fixed bracket, to take a picture of the keyboard from above the keyboard and obtain a video stream.

[0178] S2. Prepare 10 small stickers of different colors. Among them, 2 stickers of the same color are pasted on the thumb nails, and the remaining 8 stickers of different colors are pasted on the nails of the other fingers. Except for the thumb, which does not distinguish between left and right hands, each finger of the left and right hands has a corresponding color.

[0179] S3. The system can start a specific software, which can monitor keyboard input events in real time and can read video stream information in real time.

[0180] S4. When a keyboard input occurs, for example, after the user inputs the key signal of "J", the system reads the video stream screenshot information at this time through the program, and can identify the position of the letter "J" on the keyboard through ORC technology. Since the "J" key is blocked by the finger, the position of the adjacent key is used to obtain the position of the target key "J" through an algorithm.

[0181] S5. After the system calculates the position of the target key on the screenshot through the algorithm, it then determines the target color corresponding to the sticker on the finger when the target key "J" is struck in the screenshot of the video stream through the OpenCV software of the system.

[0182] S6. Through the above step S5, the system can determine which finger the user used to strike the "J" key. By comparing with the preset standard fingering key position correspondence table, it can be judged whether the user's keyboard input fingering is incorrect.

[0183] S7. When the user's keyboard input fingering is incorrect, the system can give an alarm prompt sound to remind the user that the fingering is incorrect.

[0184] S8. The system can count the number of keyboard inputs, the number of errors, the error rate, and the error ranking of each finger on the same day, so as to help the user continuously optimize the fingering.

[0185] Further, in this embodiment, the above step S4 may include steps S41 to S44.

[0186] S41. In the initial stage of the program, first identify the coordinate positions of the number 1, 2 keys and the Q key through ORC technology. And establish a coordinate system with the 1 key as the origin.

[0187] S42. Subtract the x-axis coordinates of the number 1 and 2 keys to obtain the distance between adjacent keys on the left and right. Subtract the Y-axis of the 1 and Q keys to obtain the distance between adjacent keys up and down.

[0188] S43. Set up a mapping table to store what keys are above, below, to the left, and to the right of the keys for numbers, letters, and symbols. For example, the representation of the key for number 1 is: "1 or the key value of 1": {top: null, left: "~", right: '2', bottom: 'Q'}. The mapping relationship for Q is: "Q or the key value of Q": {top: '1', left: "left Tab", right: 'W', bottom: 'A'}. It can be understood that auxiliary keys such as tab, shift, ctrl, and alt need to distinguish between the left and right keys.

[0189] S44. When a keyboard input occurs, the system obtains the key signal of the "J" key when the keyboard input occurs through a program, and this key signal can enable the system to take a screenshot of the video stream. At this time, the target key "J" in the video stream screenshot may be blocked by a finger, but according to ergonomics, the area above the target key "J" will not be blocked. Therefore, it can be obtained from the mapping table that the key above the target key J is the "U" key. The system obtains the coordinates of the "U" key, and adding the vertical adjacent key spacing to the Y coordinate of the "U" key and adding an offset of 0.2 times the horizontal adjacent key spacing to the X axis can obtain the coordinate position of the target key "J". If there is no key above the target key, the coordinates of the target key can be calculated through the key to the left, right, or below the target key. Specifically:

[0190] The X coordinate of the position of the target key "J" = the X coordinate of the upper key "U" + the horizontal adjacent key spacing * 0.2.

[0191] The Y coordinate of the position of the target key "J" = the Y coordinate of the upper key "U" + the vertical adjacent key spacing.

[0192] Furthermore, in this embodiment, the above step S5 may include steps S51 to S56.

[0193] S51. Determine the corresponding HSV range according to the RGB values of the 9 colors designed for the sticker and the preset color_ranges table. color_ranges = {

[0194] 'red': ([0, 100, 100], [10, 255, 255]), # thumb

[0195] 'green': ([50, 100, 100], [70, 255, 255]), # left - little finger

[0196] 'blue': ([110, 100, 100], [130, 255, 255]), # left - ring finger

[0197] 'yellow':([25,100,100],[35,255,255]), # Left-middle finger

[0198] 'cyan':([80,100,100],[100,255,255]), # Left-index finger

[0199] 'magenta':([140,100,100],[160,255,255]), # Right-little finger

[0200] 'orange':([15,100,100],[25,255,255]), # Right-ring finger

[0201] 'purple':([130,100,100],[140,255,255]), # Right-middle finger

[0202] 'brown':([10,50,50],[20,150,150]) # Right-index finger

[0203] }

[0204] S52. The system crops the area of the J key position from the screenshot.

[0205] S53. The system converts the cropped image area from the RGB color space to the HSV color space.

[0206] S54. The system can traverse the above color_ranges table, and within each color range, filter each pixel corresponding to the color in the area by calling the cv2.inRange function to obtain the color corresponding to each pixel. Then, the cv2.countNonZero function is used to count the number of corresponding pixels.

[0207] S55. The system can compare the one with the most pixels among the 9 colors in the area, which is the target color.

[0208] S56. The system can determine the type of the corresponding target finger according to the target color.

[0209] In summary, the present invention has the following beneficial effects.

[0210] 1. The present invention can implement real-time finger gesture monitoring and correction. The present invention can capture keyboard input and finger positions through a camera, and through real-time software analysis, timely detect finger gesture errors and give prompts to help users correct the errors.

[0211] 2. The present invention can accurately identify finger types through color identification. The present invention can accurately determine the finger types for key presses through stickers of different colors and the color recognition algorithm of the system's OpenCV software. Moreover, the present invention can use 9 colors different from the finger colors and the keyboard color as color identifiers to reduce the interference of finger colors and keyboard colors on the sticker color, and further improve the recognition accuracy.

[0212] 3. The present invention can adopt OCR technology to identify key coordinates, and use a preset key mapping table and adjacent position calculation algorithm to improve the finger occlusion problem and achieve accurate positioning of keys.

[0213] 4. The present invention can implement a personalized data feedback function. The present invention can provide daily keyboard input statistics, error times, and finger usage conditions to help users conduct targeted typing training and improve finger method standardization.

[0214] 5. The present invention does not need to modify the keyboard or other devices, and the system uses common hardware and software solutions. The color identifiers use stickers that are convenient to paste and remove, which can reduce the implementation cost and improve the device applicability and convenience.

[0215] In addition, referring to Figure 13 , corresponding to the method of Figure 1 In an embodiment of the present invention, a keyboard input finger method determination system is further provided. The system may include a first processing unit 1001, a second processing unit 1002, and a third processing unit. During the process of the keyboard being struck by a target finger, the first processing unit 1001 can obtain a target screenshot of the keyboard. The second processing unit 1002 can detect the position information of adjacent keys of the target key on the target screenshot and the color identifier of the target finger; wherein, the target key is the key currently struck by the target finger. The third processing unit 1003 can determine the keyboard input finger method according to the position information and the color identifier.

[0216] It should be noted that the first processing unit can be any integrated circuit unit or microprocessor unit obtained by integrating a processing function chip and its peripheral circuits through existing integration technologies. The first processing unit and the second processing unit can also be any integrated circuit module or microprocessor module obtained by integrating a processing function chip and its peripheral circuits through existing integration technologies. The first processing unit and the second processing unit may further include one or more memories. One or more memories can be used to store the specific algorithms for determining the keyboard input finger method in the present invention.

[0217] In some embodiments of the present invention, the first processing unit 1001 and the second processing unit 1002 may be disposed in the same gateway or a device with a processor. The first processing unit 1001 may obtain a target screenshot at the keyboard during the process of the target finger tapping the keyboard. The first processing unit 1001 may be any unit connected to the processor inside the gateway or the device. After obtaining the target screenshot through a wired or wireless connection with the processor of the first processing unit 1001, the second processing unit 1002 may detect the position information of the adjacent key positions of the target key position in the target screenshot and the color identification of the target finger. Wherein, the target key position may be the key currently tapped by the target finger. After obtaining the position information and the color identification, the third processing unit 1003 may determine the keyboard input fingering according to the position information and the color identification. The specific device connection manner between the first processing unit 1001 and the second processing unit 1002 and the setting of the device between the second processing unit 1002 and the third processing unit 1003 are not limited.

[0218] In some feasible embodiments of the present invention, the keyboard input fingering determination system may further include a fourth processing unit 1004. The fourth processing unit 1004 may be connected to the third processing unit 1003 in a wired or wireless manner. The fourth processing unit 1004 may compare the keyboard input fingering with a preset standard fingering. If the keyboard input fingering is different from the preset standard fingering, the fourth processing unit 1004 may generate an alarm message.

[0219] It should be noted that the content in the above embodiments of the keyboard input fingering determination method is applicable to the embodiments of the present keyboard input fingering determination system. The functions specifically implemented by the embodiments of the present keyboard input fingering determination system are the same as those of the above embodiments of the keyboard input fingering determination method, and the beneficial effects achieved are also the same as those of the above embodiments of the keyboard input fingering determination method.

[0220] Corresponding to Figure 1 the method, an embodiment of the present invention further provides a keyboard input fingering determination device. The specific structure of the keyboard input fingering determination device may refer to Figure 14 . The device may include:

[0221] At least one processor 1011;

[0222] At least one memory 1012 for storing at least one program;

[0223] When at least one program is executed by at least one processor, the keyboard input fingering determination method implemented by the at least one processor.

[0224] The content in the above method embodiments is applicable to the device embodiments of the present invention. The functions specifically implemented in the device embodiments of the present invention are the same as those in the above method embodiments, and the beneficial effects achieved are also the same as those in the above method embodiments.

[0225] Corresponding to Figure 1 the method, an embodiment of the present invention further provides a computer-readable storage medium storing instructions executable by a processor, and the instructions executable by the processor are used to execute a method for determining keyboard input fingering.

[0226] The content in the above method embodiments for determining keyboard input fingering is applicable to the storage medium embodiments of the present invention. The functions specifically implemented in the storage medium embodiments of the present invention are the same as those in the above method embodiments for determining keyboard input fingering, and the beneficial effects achieved are also the same as those in the above method embodiments for determining keyboard input fingering.

[0227] In some alternative embodiments, the functions / operations mentioned in the block diagrams may not occur in the order mentioned in the operation diagrams. For example, depending on the functions / operations involved, two consecutive blocks shown may actually be executed substantially simultaneously or the blocks can sometimes be executed in the reverse order. Additionally, the embodiments presented and described in the flowcharts of the present invention are provided by way of example for the purpose of providing a more comprehensive understanding of the technology. The disclosed methods are not limited to the operations and logical flows presented herein. Alternative embodiments are contemplated where the order of various operations is changed and where sub-operations described as part of a larger operation are executed independently.

[0228] Furthermore, although the present invention has been described in the context of functional modules, it should be understood that, unless otherwise stated to the contrary, one or more of the functions and / or features may be integrated in a single physical device and / or software module, or one or more functions and / or features may be implemented in separate physical devices or software modules. It can also be understood that a detailed discussion of the actual implementation of each module is not necessary for understanding the present invention. Rather, considering the attributes, functions, and internal relationships of the various functional modules in the devices disclosed herein, the actual implementation of the modules will be understood within the ordinary skills of an engineer. Therefore, those skilled in the art can implement the present invention as set forth in the claims without undue experimentation. It can also be understood that the specific concepts disclosed are illustrative only and are not intended to limit the scope of the present invention, which is determined by the full scope of the appended claims and their equivalents.

[0229] If a function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or a part of this technical solution can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several programs for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods of various embodiments of the present invention. The aforementioned storage medium includes: USB flash drives, mobile hard disks, read-only memories (ROM, Read-Only Memory), random access memories (RAM, Random Access Memory), magnetic disks, or optical discs, etc., various media that can store program codes.

[0230] The logic and / or steps represented in the flowchart or described in other ways herein, for example, can be considered as a sequenced list of executable programs for implementing logical functions, and can be specifically implemented in any computer-readable medium for use by a program execution system, apparatus, or device (such as a computer-based system, a system including a processor, or other systems that can retrieve and execute programs from a program execution system, apparatus, or device), or in combination with these program execution systems, apparatuses, or devices. For the purposes of this specification, a "computer-readable medium" can be any device that can contain, store, communicate, propagate, or transport a program for use by or in combination with a program execution system, apparatus, or device.

[0231] More specific examples (non-exhaustive list) of computer-readable media include the following: electrical connection parts with one or more wirings (electronic devices), portable computer disk cartridges (magnetic devices), random access memories (RAM), read-only memories (ROM), erasable programmable read-only memories (EPROM or flash memories), optical fiber devices, and portable compact disc read-only memories (CDROM). Additionally, a computer-readable medium can even be paper or other suitable media on which a program can be printed, because the program can be obtained electronically, for example, by optically scanning the paper or other media, followed by editing, interpretation, or other suitable processing as necessary, and then stored in a computer memory.

[0232] It should be understood that various parts of the present invention can be implemented by hardware, software, firmware, or a combination thereof. In the above-described embodiments, multiple steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable program execution system. For example, if implemented by hardware, as in another embodiment, any one or a combination of the following techniques well known in the art can be used: discrete logic circuits having logic gate circuits for implementing logical functions on data signals, application specific integrated circuits having appropriate combinational logic gate circuits, programmable gate arrays (PGAs), field programmable gate arrays (FPGAs), and the like.

[0233] In the above description of this specification, the description with reference to the terms "one embodiment / example", "another embodiment / example", or "certain embodiments / examples", etc. means that the specific features, structures, materials, or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described can be combined in any one or more embodiments or examples in a suitable manner.

[0234] Although the embodiments of the present invention have been shown and described, those of ordinary skill in the art can understand that various changes, modifications, substitutions, and variations can be made to these embodiments without departing from the principles and spirit of the present invention, and the scope of the present invention is defined by the claims and their equivalents.

[0235] The above is a specific description of the preferred embodiments of the present invention, but the present invention is not limited to the embodiments. Those skilled in the art can also make various equivalent deformations or substitutions without departing from the spirit of the present invention, and these equivalent deformations or substitutions are all included within the scope defined by the claims of the present invention.

Claims

1. A method for determining keyboard input fingering, characterized in that Including: When the keyboard is tapped by a target finger, obtaining a target screenshot at the keyboard; Detecting the position information of adjacent key positions of the target key position in the target screenshot and the color identifier of the target finger; wherein, the target key position is the key position currently tapped by the target finger; Determining the keyboard input fingering according to the position information and the color identifier.

2. The method for determining keyboard input fingering according to claim 1, characterized in that The detecting the position information of adjacent key positions of the target key position in the target screenshot includes: Obtaining the key value corresponding to the target key position; Based on the key value, determining the adjacent key positions according to a pre-established mapping table; Identifying the adjacent key positions in the target screenshot and extracting the position information of the adjacent key positions.

3. The method for determining keyboard input fingerings according to claim 2, wherein The determining the keyboard input fingering according to the color identifier and the position information includes: According to the position information, determining the target position information of the target key position in the target screenshot; According to the color identifier, determining the finger category of the target finger; Determining the keyboard input fingering according to the finger category and the target position information.

4. The method for determining keyboard input fingering according to claim 3, characterized in that When it is determined that the adjacent key position is the key position adjacent above the target key position, the determining the target position information of the target key position in the target screenshot according to the position information includes: According to the position information, determining the first X coordinate and the first Y coordinate of the adjacent key position; Taking the sum value between the first X coordinate and the left-right key position offset as the X coordinate of the target position information; Taking the difference value between the first Y coordinate and the up-down key position spacing as the Y coordinate of the target position information.

5. The method for determining keyboard input fingering according to claim 3, wherein, When it is determined that the adjacent key position is the key position adjacent below the target key position, the determining the target position information of the target key position in the target screenshot according to the position information includes: According to the position information, determining the first X coordinate and the first Y coordinate of the adjacent key position; Taking the difference value between the first X coordinate and the left-right key position offset as the X coordinate of the target position information; Taking the sum value between the first Y coordinate and the up-down key position spacing as the Y coordinate of the target position information.

6. The method for determining keyboard input fingering according to claim 4 or 5, characterized in that The left-right key position offset is obtained through the following steps: Determining the left-right key position spacing between any two adjacent left-right key positions; Multiplying the left-right key position spacing by a preset correction parameter to obtain a first product; Taking the first product as the left-right key position offset.

7. The method for determining keyboard input fingering according to claim 3, wherein When it is determined that the adjacent key position is the key position adjacent to the left of the target key position, the determining the target position information of the target key position in the target screenshot according to the position information includes: According to the position information, determining the first X coordinate and the first Y coordinate of the adjacent key position; Taking the sum value between the first X coordinate and the left-right key position spacing as the X coordinate of the target position information; Taking the first Y coordinate as the Y coordinate of the target position information.

8. The method for determining keyboard input fingerings according to claim 3, characterized in that, When it is determined that the adjacent key position is the key position adjacent to the right of the target key position, the determining the target position information of the target key position in the target screenshot according to the position information includes: According to the position information, determining the first X coordinate and the first Y coordinate of the adjacent key position; Use the difference between the first X coordinate and the left - right key spacing as the X coordinate of the target position information; Use the first Y coordinate as the Y coordinate of the target position information.

9. The method for determining keyboard input fingerings according to claim 3, characterized in that The determining the finger category of the target finger according to the color identifier includes: Extract a regional screenshot of the area where the color identifier is located in the target screenshot; Convert all pixels in the regional screenshot into HSV pixel points; Determine the target color of the target finger according to the HSV pixel points; Determine the finger category according to the target color.

10. The method for determining keyboard input fingering according to claim 9, wherein The determining the target color of the target finger according to the HSV pixel points includes: Classify the colors of all the HSV pixel points in the area; where each HSV pixel point corresponds to a color; Determine the color corresponding to the most HSV pixel points in the area as the target color.

11. The method for determining keyboard input fingering according to claim 1, wherein The method further includes: comparing the keyboard input fingering with a preset standard fingering, and if the keyboard input fingering is different from the preset standard fingering, generating an alarm message.

12. A keyboard input fingering determination system, characterized in that, Includes: A first processing unit, configured to obtain a target screenshot at the keyboard when the keyboard is tapped by a target finger; A second processing unit, configured to detect the position information of adjacent keys of a target key in the target screenshot and the color identifier of the target finger; where the target key is the key currently tapped by the target finger; A third processing unit, configured to determine the keyboard input fingering according to the position information and the color identifier.

13. A device for determining keyboard input fingering, characterized in that Includes: At least one processor; At least one memory, configured to store at least one program; When the at least one program is executed by the at least one processor, the at least one processor implements the keyboard input fingering determination method according to any one of claims 1 - 11.

14. A computer-readable storage medium storing instructions executable by a processor, characterized in that, The instructions executable by the processor, when executed by the processor, are used to execute the keyboard input fingering determination method according to any one of claims 1 - 11.

Images