Input error correction method, computing device and medium for analyzing hand tremor mistouch

By analyzing the video stream and touch information of the user and the touch screen of the smart device, and generating and correcting encoded strings, it solves the input error problem caused by hand tremor on the smart device, improves input accuracy and speed, and provides patients with hand tremor symptoms. More accurate input experience.

CN113190160BActive Publication Date: 2024-11-08TSINGHUA UNIVERSITY
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202110220942.0
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2021-02-26
Publication Date
2024-11-08
Estimated Expiration
2041-02-26

AI Technical Summary

Technical Problem

The coding correction methods provided by the prior art for patients with hand tremor symptoms on the touch screen of smart devices are weak, making it difficult to effectively solve the problem of input errors and multiple clicks, affecting the accuracy and speed of input.

Method used

By obtaining the video stream and touch information of the user and the touch screen, analyzing the possibility of false touch caused by hand tremor, combining the video stream and touch coordinate frequency information, a corrected encoded string is generated to provide more accurate candidates to improve input accuracy and Error tolerance.

Benefits of technology

It effectively reduces the impact of false touch caused by tremor, reduces the number of times the input characters needs to be deleted, improves input speed and accuracy, and provides a more accurate input experience for patients with hand tremor symptoms.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN113190160B_ABST
    Figure CN113190160B_ABST
Patent Text Reader

Abstract

A computer-implemented input error correction method includes: using touch information on a touch screen to receive a coded string input by a user and the coordinates and time of each character input; analyzing the coordinates and frequency of adjacent touch interactions based on the received string information, and analyzing the possibility that a certain touch interaction is an accidental touch due to hand tremor; converting the possibility of an accidental touch through the adjacent touch interaction coordinate frequency analysis and the received coded string to obtain a corresponding candidate item. Compared with the common input method string correction method, the present invention can effectively reduce the impact of accidental touches caused by tremor on the input string, including large touch deviations and multi-touch problems caused by tremor; the processed candidate items can effectively filter out invalid inputs from users and correct inaccurate inputs, providing users with more expected strings, so as to help users improve the error tolerance rate during use, reduce the number of times input characters need to be deleted, and improve input speed.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention generally relates to the technical field of computer information input, and in particular to a method and device for coding error correction during information input. Background Art

[0002] For users who input text in languages ​​such as English and Chinese on smart touch screen devices, they generally need to interact with the smart terminal through an input method program: for example, when a user inputs a coded string through a virtual touch keyboard, the applied input method program will map the input string into candidates for the corresponding language based on the language, keyboard, algorithm and other standards set by the user, so that the user can confirm the input information.

[0003] Due to the non-discrete nature of the coordinate points that users can touch on the touch screen, there is often an error between the string input by the user and the target string due to inaccurate clicks. Therefore, unlike mechanical keyboards, touch screen virtual keyboards usually require the fuzzification of key coordinates, that is, to correct the error of user clicks through an automatic correction algorithm. The QWERTY keyboard is used in many language input methods, such as Chinese Pinyin and English. Due to the dense key density of the virtual QWERTY keyboard on the touch screen of smart devices, the string correction algorithm is an indispensable part of the input method.

[0004] More specifically, for people with hand tremors, such as patients with Parkinson's disease, the error in touch interaction and the false touch rate of multiple touches will increase due to hand tremors. Therefore, a more targeted and accurate string correction algorithm is particularly critical.

[0005] When the input algorithm is used in combination with the string error correction algorithm, the input method program will map the input string to the candidate items of the corresponding language according to the set algorithm and correction algorithm while the user is inputting, so that the user can confirm the input information.

[0006] Patent publication CN101276245A discloses a method and system for prompting coding error correction during input. The invention receives the coding string input by the user, converts it into corresponding candidate items, and determines whether there are candidate items obtained by confusable coding equivalent methods in the candidate items. If so, it provides coding error correction information to help users actively improve the input accuracy of the coding string during use, thereby reducing the user's dependence on ambiguous sounds. This method will have a weak correction effect when dealing with repeated points and multiple clicks of hand tremor symptoms.

[0007] Patent publication CN106896931A discloses an input method error correction method and device, which corrects the coded characters input by the terminal by sensing vibration parameters, that is, the coded characters corresponding to the error correction key are combined with the coded characters input before and / or after the vibration occurs to obtain the error correction coded string, and the string is placed into the candidate entry. This method will have a weak correction effect when dealing with repeated clicks and multiple clicks of hand tremor symptoms. Summary of the invention

[0008] In view of the above problems in the prior art, one of the technical problems to be solved by the present invention is to provide a coding correction method for people with hand tremor symptoms during the process of text input using the touch screen of a smart device. The method can obtain a correction code based on the user's input character string that may have errors, so as to help the user improve the fault tolerance rate during use, reduce the number of times the input characters need to be deleted, and increase the input speed.

[0009] According to one embodiment of the present invention, there is provided a computer-implemented input error correction method for touch screen input, comprising: obtaining a video stream of user interaction with the touch screen; using touch information on the touch screen to receive a coded string input by the user and the coordinates and time of each character input; analyzing the possibility that a touch interaction is a false touch due to hand tremor based on multiple video frame images in the received interactive video stream; analyzing the coordinates and frequency of adjacent touch interactions based on the received string information, and analyzing the possibility that a touch interaction is a false touch due to hand tremor; converting to obtain corresponding candidate items based on the false touch possibility analyzed through the video stream, the false touch possibility analyzed through the coordinate frequency of adjacent touch interactions, and the received coded string.

[0010] Optionally, the input error correction method may further include: determining whether there are additional candidate items in the candidate item database that can be obtained by equivalent means of easily confusable encoding; if so, adding the additional candidate items to the candidate item set.

[0011] Optionally, obtaining a video stream of the user's interaction action with the screen includes obtaining a video stream of the user's finger or touch pen interacting with the touch screen using a front camera and a mirror reflection device.

[0012] Optionally, analyzing the possibility that a touch interaction is a false touch due to hand tremor based on multiple video frame images in the received interactive video stream may include: when a user is detected performing a screen touch interaction, obtaining the timestamp of this touch-screen interaction, and in the video stream of the user's finger or touch pen interacting with the touch screen obtained by using the front camera mirror reflection device, intercepting the video stream n seconds before the corresponding timestamp, and analyzing the images frame by frame in the video stream, where n is a positive integer, and if the number of video streams is less than n, taking the largest number of video streams, and obtaining the distance between the fingertip or the top of the touch pen and the screen related to the timestamp after analyzing the video stream frame by frame; determining the possibility that a touch interaction is a false touch due to hand tremor based on the change in the fluctuation amplitude of the distance between the fingertip or the top of the touch pen and the screen within a previously determined time.

[0013] Optionally, obtaining the distance between the fingertip or the top of the touch pen and the screen related to the timestamp includes: determining the plane area of ​​the mobile phone screen according to the installation position of the camera reflection device; performing image area segmentation according to color, pixel difference between image frames or other factors to obtain the outline of the user's finger or touch pen; estimating the distance between the fingertip or the top of the touch pen and the screen according to the width of the outline area and the position in the image, using the following formula:

[0014] δ=l·d

[0015] Where δ is the estimated distance between the screen and the fingertip or the top of the touch pen; l is the distance between the front camera and the finger or the top of the touch pen.

[0016] Optionally, the distance l between the front camera and the top of the finger or stylus is calculated using the following formula:

[0017] l=α·w+β

[0018] Where w is the width of the contour area, α is a constant, and users are required to calibrate before use. The calibration process requires users to touch the four corners of the screen to complete it; d is the pixel value of the interval between the fingertip or the top of the touch pen and the screen in the image. This pixel value reflects the tangent value of the angle between the fingertip or the top of the touch pen and the mirror reflection device and the mobile phone screen.

[0019] Optionally, the method of determining the possibility that a touch interaction is a false touch due to hand tremor based on the change in the fluctuation amplitude of the distance between the fingertip or the top of the touch pen and the screen within a previously determined time includes: for a specific touch interaction, the closer the distance between the fingertip or the top of the touch pen and the screen before the interaction, that is, the smaller the average value of the above δ value, the higher the possibility that the touch interaction is a false touch; the larger the fluctuation amplitude of the falling edge of the distance between the fingertip or the top of the touch pen and the screen before the interaction, the lower the possibility that the touch interaction is a false touch.

[0020] Optionally, analyzing the possibility that a touch interaction is a false touch due to hand tremor based on multiple video frame images in the received interactive video stream may include: combining the fingertip or touch pen top position information obtained by analyzing the video stream with the touch point coordinate information obtained by the software to determine the degree of match between the two positions, and calculating the possibility that the user made a false touch with a non-primary operating finger or touch pen based on the degree of match.

[0021] Optionally, based on the received character string information, analyzing the coordinates and frequency of adjacent touch interactions, analyzing the possibility that a touch interaction is a false touch due to hand tremor may include: obtaining interaction information of n times before the current interaction based on the records, including the touch coordinates and time of each interaction, where n is an integer greater than or equal to 2; if the time difference between the current interaction and the previous interactions meets the tremor frequency condition of 4-6Hz, that is, the time difference between adjacent interactions is approximately 0.15-0.25 seconds, then compared with the case where this tremor frequency is not met, the possibility that the current interaction is a false touch is higher; and under the premise of meeting the tremor frequency condition, the closer the touch coordinates of adjacent interactions, the higher the possibility that the current interaction is a false touch.

[0022] Optionally, analyzing the possibility that a touch interaction is a false touch due to hand tremor may include: using SVM or artificial neural network training to generate a binary classifier, and during testing, obtaining the coordinates and time of the fingertip or touch pen tip in contact with the screen and inputting them into the binary classifier, and taking its probability output value as a numerical criterion, as the probability value of this interaction being a false touch.

[0023] According to another embodiment of the present invention, a computing device is provided, including a memory and a central processing unit, wherein computer executable instructions are stored in the memory, and when the instructions are executed by the central processing unit, the above method is executed.

[0024] According to another embodiment of the present invention, a computer-readable storage medium is provided, on which computer-executable instructions are stored. When the instructions are executed by a computer, the above method is executed.

[0025] According to another embodiment of the present invention, there is provided a computer-implemented input error correction method, comprising: utilizing touch information on a touch screen to receive a coded string input by a user and the coordinates and time of each character input; analyzing the coordinates and frequency of adjacent touch interactions based on the received string information, and analyzing the possibility that a touch interaction is an accidental touch due to hand tremor; converting the possibility of an accidental touch obtained through the adjacent touch interaction coordinate frequency analysis and the received coded string to obtain corresponding candidate items.

[0026] Optionally, analyzing the coordinates and frequency of adjacent touch interactions based on the received character string information, and analyzing the possibility that a touch interaction is a false touch due to hand tremor may include: obtaining interaction information of n times before the current interaction based on the records, including the touch coordinates and time of each interaction, where n is an integer greater than or equal to 2; if the time difference between the current interaction and the previous interactions satisfies the tremor frequency condition of 4-6Hz, that is, the time difference between adjacent interactions is approximately 0.15-0.25 seconds, then the possibility that the current interaction is a false touch is higher than that of the case where this tremor frequency is not satisfied; and under the premise of satisfying the tremor frequency condition, the closer the touch coordinates of adjacent interactions, the higher the possibility that the current interaction is a false touch.

[0027] Optionally, analyzing the possibility that a touch interaction is a false touch due to hand tremor may include: using SVM or artificial neural network training to generate a binary classifier, and during testing, obtaining the coordinates and time of the fingertip or touch pen tip in contact with the screen and inputting them into the binary classifier, and taking its probability output value as a numerical criterion, as the probability value of this interaction being a false touch.

[0028] Compared with the common input method character string correction method, the method for predicting false touches based on touch screen input and correcting input errors accordingly according to the embodiment of the present invention can effectively reduce the impact of false touches caused by tremors on input characters, including large touch deviations and multi-touch problems caused by tremors; the processed candidate items can effectively filter out invalid user inputs and correct inaccurate inputs, providing users with more expected character strings, so as to help users improve the fault tolerance rate during use, reduce the number of times input characters need to be deleted, and improve input speed. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 A flow chart of an input error correction method for touch screen input executed by a computer according to an embodiment of the present application is shown.

[0030] Figure 2 A flow chart of an input error correction method for simultaneously correcting touch screen input using touch information on a touch screen and a video stream according to another embodiment of the present invention is shown.

[0031] Figure 3 A schematic diagram of the arrangement of a technology for obtaining a video stream by using a front camera and a plane mirror according to an embodiment of the present invention is shown.

[0032] Figure 4 A schematic diagram showing parameters used in estimating the distance between a fingertip or a touch pen tip and a screen according to an embodiment of the present invention is shown.

[0033] Figure 5 A block diagram of an electronic device according to an embodiment of the present application is shown. DETAILED DESCRIPTION

[0034] Below, the exemplary embodiments according to the present application will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application, and it should be understood that the present application is not limited to the exemplary embodiments described here.

[0035] Figure 1 A flow chart of an input error correction method for touch screen input executed by a computer according to an embodiment of the present application is shown.

[0036] like Figure 1 As shown, in step S110, the touch information on the touch screen is used to receive the encoded character string input by the user and the coordinates and time of each character input.

[0037] Each touch by the user has the touch coordinates, time, and corresponding characters. By recording these, we can get the encoded string input by the user and the coordinates and time of each character input.

[0038] In step S120, the coordinates and frequencies of adjacent touch interactions are analyzed according to the received character string information, and the possibility of a touch interaction being a false touch due to hand tremor is analyzed.

[0039] Since the frequency of hand tremors caused by Parkinson's disease is mostly 4-6 Hz, the probability that the current interaction is a false touch can be estimated based on the information of several adjacent interactions.

[0040] The n previous interactions are obtained according to the records, including the touch coordinates and time of each interaction, where n is an integer greater than or equal to 2 and includes the touch coordinates and time of several interactions. In one example, n is 3, and if it is less than 3, the maximum value is taken.

[0041] If the time difference between this interaction and the previous interactions meets the tremor frequency condition of 4-6Hz, that is, the time difference between adjacent interactions is about 0.15-0.25 seconds, then compared with the case where this tremor frequency is not met, the possibility that this interaction is a false touch is higher; and under the premise of meeting the said tremor frequency condition, the closer the touch coordinates of adjacent interactions, the higher the possibility that this interaction is a false touch.

[0042] In one example, analyzing the possibility that a touch interaction is a false touch due to hand tremor includes: using SVM or artificial neural network training to generate a binary classifier, and during testing, the coordinates and time of the fingertip or touch pen tip in contact with the screen are input into the binary classifier, and the probability output value is taken as a numerical criterion as the probability value of this interaction being a false touch. Specifically, the data set used in SVM or artificial neural network training requires a one-to-one correspondence between the original input information collected and whether it is a false touch (false touch is marked as 1, non-false touch is marked as 0), which can ensure that the probability output result will fall in the closed interval [0,1] when applied. When applied, a threshold value ∈ needs to be set. If the output probability value is greater than the threshold, the interaction is considered to be a false touch. In one example, ∈ is taken as 0.8.

[0043] In step S130, corresponding candidate items are converted according to the false touch possibility obtained through the frequency analysis of adjacent touch interaction coordinates and the accepted encoding character string.

[0044] In one example, based on the probability of mis-touch obtained through the frequency analysis of adjacent touch interaction coordinates, the probability that a specific character in the received coded string needs to be deleted is obtained, and a set of automatically deleted coded strings is obtained in combination with the received complete coded string. This step can effectively filter out the redundant input characters caused by mis-touch. The set of automatically deleted coded strings is compared with the existing vocabulary, and based on the similarity and the frequency of the words in the vocabulary, the candidate items with high similarity and relatively high frequency are converted, so as to achieve the purpose of correcting incorrect input and providing a more expected string.

[0045] Compared with the common input method character string correction method, the method for predicting false touches based on touch screen input and correcting input errors accordingly according to the embodiment of the present invention can effectively reduce the impact of false touches caused by tremors on input characters, including large touch deviations and multi-touch problems caused by tremors; the processed candidate items can effectively filter out invalid user inputs and correct inaccurate inputs, providing users with more expected character strings, so as to help users improve the fault tolerance rate during use, reduce the number of times input characters need to be deleted, and improve input speed.

[0046] Figure 2 A flow chart of an input error correction method for simultaneously correcting touch screen input using touch information on a touch screen and a video stream according to another embodiment of the present invention is shown.

[0047] In step S210, a video stream of the user's interaction with the touch screen is obtained.

[0048] Figure 3The schematic diagram of the arrangement of the technology of obtaining a video stream by using a front camera and a plane mirror according to an embodiment of the present invention is shown. In this example, a video stream of the interaction between a user's finger or a touch pen and a touch screen is obtained by using a front camera mirror reflection device. The light emitted by the user's finger directly reaches the front camera or enters the front camera after being reflected by the plane mirror, and the relevant video stream is obtained.

[0049] In step S220, the touch information on the touch screen is used to receive the encoded character string input by the user and the coordinates and time of each character input.

[0050] The description of this step can refer to the previous Figure 1 The description of step S110 is performed.

[0051] In step S230, the possibility of a touch interaction being an accidental touch due to hand tremor is analyzed based on a plurality of video frame images in the received interactive video stream.

[0052] Specifically, in one example, when a user performs a touch screen interaction, the timestamp of this touch screen interaction is obtained, and in the video stream of the interaction between the user's finger or touch pen and the touch screen obtained by the front camera mirror reflection device, the video stream n seconds before the corresponding timestamp (in one example, n is 2, and if it is less than 2, the maximum value is taken) is intercepted, and the image frame by frame in the video stream is analyzed.

[0053] The analysis may include:

[0054] (1) Determine the plane area of ​​the mobile phone screen according to the installation position of the camera reflection device;

[0055] (2) Segmenting the image region based on color, pixel difference between image frames, or other factors to obtain the outline of the user's finger or touch pen;

[0056] (3) Estimate the distance between the fingertip or the top of the stylus and the screen based on the width of the contour area and its position in the image. In one example, the following formula is used for estimation

[0057] δ=l·d

[0058] Wherein δ is the estimated distance between the screen and the fingertip or the top of the touch pen; l is the distance between the front camera and the finger or the top of the touch pen, which can be estimated by l=α·w+β, where w is the width of the contour area and α is a constant. Users are required to calibrate before use, and the calibration process can be completed by requiring the user to touch the four corners of the screen; d is the pixel value of the interval between the fingertip or the top of the touch pen and the screen in the image. According to the characteristics of the camera's spherical imaging, this pixel distance will directly reflect the tangent value of the angle between the fingertip or the top of the touch pen and the mirror reflection device and the mobile phone screen. Figure 4A schematic diagram showing parameters used in estimating the distance between a fingertip or a touch pen tip and a screen according to an embodiment of the present invention is shown.

[0059] After analyzing the video stream frame by frame, the distance between the fingertip or the top of the touch pen and the screen related to the timestamp can be obtained. Due to the symptoms of hand tremor patients, this distance value usually shows a regular fluctuation of a certain amplitude, and the fluctuation amplitude is related to the severity of the user's hand tremor. For a specific touch interaction, if the distance between the fingertip or the top of the touch pen and the screen is close before this interaction, that is, the average value of the above δ value is small, then the possibility of this touch interaction being a false touch is high; if the distance between the fingertip or the top of the touch pen and the screen shows a falling edge greater than the regular fluctuation amplitude before this interaction, then the possibility of this touch interaction being a false touch is low. In addition, there is the possibility that the user uses a non-primary operating finger or touch pen to make an accidental touch. In this case, it is necessary to combine the fingertip or touch pen top position information analyzed in the video stream with the touch point coordinate information sensed by the touch screen for judgment. If they do not match, the possibility of a false touch is high.

[0060] As an example of a specific feasible quantification method, a support vector machine (SVM) or a machine learning algorithm can be used to generate a binary classifier, and the δ-t curve of the relationship between the distance between the fingertip or the top of the touch pen and the screen and time obtained by video stream analysis is used as input, and its probability output value is taken as a numerical criterion as the probability value of this interaction being a false touch.

[0061] The data set used for SVM or artificial neural network training requires that the δ-t curve obtained after analyzing the known video stream collected corresponds to the manually calibrated false touch (false touch is marked as 1, and non-false touch is marked as 0), that is, it can be guaranteed that the probability output result will fall in the closed interval [0,1] when applied. When applied, a threshold ∈ needs to be set (for example, ∈ is 0.8 in this solution). If the output probability value is greater than the threshold, the interaction is considered to be a false touch.

[0062] In step S240, the coordinates and frequencies of adjacent touch interactions are analyzed based on the received character string information, and the possibility of a touch interaction being a false touch due to hand tremor is analyzed.

[0063] Regarding the implementation of step S240, please refer to the previous Figure 1 Description of step S120 in .

[0064] In step S250, corresponding candidate items are converted according to the possibility of mis-touch analyzed through the video stream, the possibility of mis-touch analyzed through the frequency of adjacent touch interaction coordinates, and the received encoding string.

[0065] Specifically, the probability that a specific character in the received coded string needs to be deleted can be obtained based on the possibility of mis-touch obtained through video stream analysis and the possibility of mis-touch obtained through adjacent touch interaction coordinate frequency analysis. Combined with the received complete coded string, a set of automatically deleted coded strings can be obtained. This step can effectively filter out the redundant input characters caused by mis-touch. The set of automatically deleted coded strings is compared with the existing vocabulary, and based on the similarity and the frequency of the words in the vocabulary, candidates with high similarity and relatively high frequency are converted to achieve the purpose of correcting incorrect input and providing a more expected string.

[0066] In one example, the input error correction method further includes: determining whether there are additional candidate items in the vocabulary that can be obtained by equivalent means of easily confusing codes; if so, adding the additional candidate items to the candidate item set. If there are multiple satisfying items in the vocabulary, the strings are sorted and displayed from high to low according to the frequency of occurrence of each string in the vocabulary statistics.

[0067] Compared with the common input method string correction method, the input correction method of the embodiment of the present invention can effectively reduce the impact of false touches caused by tremors on the input string, including large touch deviations and multi-touch problems caused by tremors. The processed candidate items can effectively filter out invalid inputs from users and correct inaccurate inputs, providing users with strings that are more in line with expectations, so as to help users improve the error tolerance rate during use, reduce the number of times input characters need to be deleted, and increase input speed.

[0068] Below, reference Figure 5 To describe an electronic device according to an embodiment of the present application.

[0069] Figure 5 A block diagram of an electronic device according to an embodiment of the present application is illustrated.

[0070] like Figure 5 As shown, the electronic device 10 includes one or more processors 11 and a memory 12 .

[0071] The processor 11 may be a central processing unit (CPU) or other forms of processing units having data processing capabilities and / or instruction execution capabilities, and may control other components in the electronic device 10 to perform desired functions.

[0072] The memory 12 may include one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory may include, for example, random access memory (RAM) and / or cache memory (cache), etc. The non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage medium, and the processor 11 may run the program instructions to implement the data storage method and data recovery method of the various embodiments of the present application described above and / or other desired functions. Various contents such as data blocks, data buckets, etc. may also be stored in the computer-readable storage medium.

[0073] In one example, the electronic device 10 may further include: an input device 13 and an output device 14, and these components are interconnected via a bus system and / or other forms of connection mechanisms (not shown).

[0074] For example, the input device 13 may be a keyboard, a mouse, etc.

[0075] The output device 14 can output various information to the outside, including storage results and recovery results, etc. The output device 14 can include, for example, a display, a speaker, a printer, a communication network and a remote output device connected thereto, etc.

[0076] Of course, to simplify, Figure 5 Only some of the components related to the present application in the electronic device 10 are shown, and components such as a bus, an input / output interface, etc. are omitted. In addition, according to specific application situations, the electronic device 10 may also include any other appropriate components.

[0077] In addition, an embodiment of the present application may also be a computer-readable storage medium on which computer program instructions are stored. When the computer program instructions are executed by a processor, the processor executes the steps of the data storage method and data recovery method according to various embodiments of the present application described in the above "Exemplary Method" section of this specification.

[0078] The computer readable storage medium can adopt any combination of one or more readable media. The readable medium can be a readable signal medium or a readable storage medium. The readable storage medium can include, for example, but is not limited to, a system, device or device of electricity, magnetism, light, electromagnetic, infrared, or semiconductor, or any combination of the above. More specific examples (non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0079] The basic principles of the present application are described above in conjunction with specific embodiments. However, it should be noted that the advantages, strengths, effects, etc. mentioned in the present application are only examples and not limitations, and it cannot be considered that these advantages, strengths, effects, etc. are required by each embodiment of the present application. In addition, the specific details disclosed above are only for the purpose of illustration and ease of understanding, not for limitation, and the above details do not limit the present application to being implemented by adopting the above specific details.

[0080] The block diagrams of the devices, apparatuses, equipment, and systems involved in this application are only illustrative examples and are not intended to require or imply that they must be connected, arranged, and configured in the manner shown in the block diagram. As will be appreciated by those skilled in the art, these devices, apparatuses, equipment, and systems can be connected, arranged, and configured in any manner. Words such as "including", "comprising", "having", etc. are open words, referring to "including but not limited to", and can be used interchangeably with them. The words "or" and "and" used here refer to the words "and / or" and can be used interchangeably with them, unless the context clearly indicates otherwise. The words "such as" used here refer to the phrase "such as but not limited to", and can be used interchangeably with them.

[0081] It should also be noted that in the apparatus, device and method of the present application, each component or each step can be decomposed and / or recombined. Such decomposition and / or recombination shall be regarded as an equivalent solution of the present application.

[0082] The embodiments of the present invention have been described above, and the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. Therefore, the scope of protection of the present invention should be subject to the scope of protection of the claims.

Claims

1. A computer-implemented input error correction method for touch screen input, comprising: Get the video stream of the user's interaction with the touch screen; Using the touch information on the touch screen to receive the encoded string input by the user and the coordinates and time of each character input; Analyzing the possibility that a touch interaction is an accidental touch due to hand tremor based on multiple video frame images in the received interactive video stream; According to the received character string information, the coordinates and frequencies of adjacent touch interactions are analyzed, and the possibility of a touch interaction being an accidental touch due to hand tremor is analyzed; According to the possibility of false touch analyzed through the video stream, the possibility of false touch analyzed through the frequency of adjacent touch interaction coordinates, and the accepted encoding string, the corresponding candidate items are converted.

2. The input error correction method according to claim 1, further comprising: Determine whether there are additional candidates in the vocabulary that can be obtained by confusable encoding equivalents; If so, add the additional candidates to the candidate set.

3. The input error correction method according to claim 1, wherein: Acquiring a video stream of the user's interaction with the screen includes acquiring a video stream of the user's finger or touch pen interacting with the touch screen using a front camera and a mirror reflection device.

4. The input error correction method according to claim 1, wherein: The possibility of analyzing a touch interaction as an accidental touch due to hand tremor according to multiple video frame images in the received interactive video stream includes: When a user touches the screen, the timestamp of the touch screen interaction is obtained. In the video stream of the user's finger or touch pen interacting with the touch screen obtained by the front camera mirror reflection device, the video stream n seconds before the corresponding timestamp is intercepted, and the image in the video stream is analyzed frame by frame. n is a positive integer. If the video stream is less than n, the maximum video stream is taken. After analyzing the video stream frame by frame, the distance between the fingertip or the top of the touch pen and the screen related to the timestamp is obtained; The possibility that a touch interaction is a false touch due to hand tremor is determined based on a change in the fluctuation amplitude of the distance between the fingertip or the top of the stylus and the screen within a previously determined time.

5. The input error correction method according to claim 4, wherein obtaining the distance between the fingertip or the top of the touch pen and the screen related to the timestamp comprises: Determine the plane area of ​​the mobile phone screen according to the installation position of the camera reflection device; Segment the image area based on color, pixel difference between image frames or other factors to obtain the outline of the user's finger or touch pen; According to the width of the contour area and its position in the image, the distance between the fingertip or the top of the stylus and the screen is estimated using the following formula: δ=l·d Where δ is the estimated distance between the screen and the fingertip or the top of the touch pen; l is the distance between the front camera and the finger or the top of the touch pen; d is the pixel value of the interval between the fingertip or the top of the touch pen and the screen in the image. This pixel value reflects the tangent value of the angle between the fingertip or the top of the touch pen and the mirror reflection device and the mobile phone screen.

6. The input error correction method according to claim 5, wherein the distance l between the front camera and the tip of the finger or the touch pen is calculated using the following formula: l=α·w+β Where w is the width of the contour area, α is a constant, and users are required to calibrate before use. The calibration process requires users to touch the four corners of the screen to complete.

7. The input error correction method according to claim 4, wherein the determining the possibility that a touch interaction is a false touch due to hand tremor based on the fluctuation amplitude change of the distance between the fingertip or the top of the touch pen and the screen within a previously determined time comprises: For a specific touch interaction, if the distance between the fingertip or the top of the touch pen and the screen before the interaction is closer, and the estimated average distance between the screen and the fingertip or the top of the touch pen is smaller, the possibility that the touch interaction is a false touch is higher; if the fluctuation amplitude of the falling edge of the distance between the fingertip or the top of the touch pen and the screen before the interaction is larger, the possibility that the touch interaction is a false touch is lower.

8. The input error correction method according to claim 1, wherein the analyzing the possibility that a touch interaction is an accidental touch due to hand tremor based on multiple video frame images in the received interactive video stream comprises: The fingertip or touch pen tip position information obtained by analyzing the video stream is combined with the touch point coordinate information obtained by the software to determine the degree of coincidence between the two positions, and the possibility of the user's accidental touch with a non-primary operating finger or touch pen is calculated based on the degree of coincidence.

9. The input error correction method according to claim 1, wherein the step of analyzing the coordinates and frequencies of adjacent touch interactions based on the received character string information and analyzing the possibility that a touch interaction is an accidental touch due to hand tremor comprises: Obtain the interaction information of the n previous interactions according to the records, including the touch coordinates and time of each interaction, where n is an integer greater than or equal to 2; If the time difference between this interaction and the previous interactions meets the tremor frequency condition of 4-6Hz, that is, the time difference between adjacent interactions is 0.15-0.25 seconds, then the possibility that this interaction is a false touch is higher than that of not meeting this tremor frequency; and On the premise that the vibration frequency condition is met, the closer the touch coordinates of adjacent interactions are, the higher the possibility that the interaction is a false touch.

10. The input error correction method according to any one of claims 1 to 9, wherein the possibility of analyzing a touch interaction as an accidental touch due to hand tremor comprises: Use SVM or artificial neural network training to generate binary classifiers. During the test, the coordinates and time of the contact between the fingertip or the top of the touch pen and the screen are input into the binary classifier, and the probability output value is taken as a numerical criterion as the probability value of the current interaction being a false touch.

11. A computing device comprising a memory and a central processing unit, wherein the memory stores computer executable instructions, and when the instructions are executed by the central processing unit, the method according to any one of claims 1 to 10 is executed.

12. A computer-readable storage medium having computer-executable instructions stored thereon, wherein when the instructions are executed by a computer, the method according to any one of claims 1 to 10 is performed.