Handwriting input method, handwriting input device and storage medium
By using gesture correction methods in handwriting input devices, and displaying candidate character sets based on text shape and contextual information, the complexity of correcting character recognition errors in handwriting input is reduced, and the user experience is improved.
Patent Information
- Application Number
- CN202310627600.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-05-30
- Publication Date
- 2024-12-03
AI Technical Summary
In existing technologies, text recognition errors during handwriting input require users to manually delete the errors and re-enter the text, which reduces the user experience.
The gesture correction method in the handwriting input device determines the text to be corrected by using the correction gesture in the handwriting input interface, and displays a candidate character set based on the shape and context information of the text. The user then selects a character to replace the erroneous text.
It simplifies the handwriting input error correction process, avoids manual deletion and re-entry, and improves the user experience.
Smart Images

Figure CN119065577A_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to the field of handwriting input, and in particular to a handwriting input method, a handwriting input device and a storage medium. Background Art
[0002] Stylus pens can greatly enrich the application scenarios of terminals such as tablets. Using stylus pens for handwriting input has become a writing habit for more and more young people, including but not limited to handwriting notes, annotating files and other explicit handwriting scenarios. However, due to different writing habits, different people write the same word in very different shapes, which makes it difficult for text recognition algorithms to recognize, resulting in recognition errors and typos.
[0003] In the related art, when a typo occurs during handwriting input, the user needs to call out the virtual keyboard of the terminal, delete the typo using the function keys on the keyboard, and re-enter the text by handwriting or typing the text through the virtual keyboard, which loses the advantage of handwriting input and greatly reduces the user experience. Summary of the invention
[0004] In order to overcome the problems existing in the related art, the present disclosure provides a handwriting input method, a handwriting input device and a storage medium.
[0005] According to a first aspect of an embodiment of the present disclosure, a handwriting input method is provided, comprising: in response to receiving a correction gesture in a handwriting input interface, determining a text to be corrected based on the correction gesture; based on the glyph and / or context information of the text to be corrected, determining and displaying a set of characters to be selected; in response to a target character in the set of characters to be selected being selected, replacing the text to be corrected with the target character.
[0006] In one embodiment, the error correction gesture includes a graphic that frames the text.
[0007] In one implementation, the error correction gesture includes a triangle, and the base of the triangle is parallel to a text line corresponding to the input text in the handwriting input interface.
[0008] In one embodiment, determining the text to be corrected based on the correction gesture includes: determining that the area selected by the correction gesture includes text information, and determining the text to be corrected based on the text information; the text information includes one or more of the starting point of the text line, the text size, and the text font.
[0009] In one embodiment, the determining and displaying of a set of characters to be selected based on the glyph and / or context information of the characters to be corrected includes: based on the glyph of the characters to be corrected, determining characters whose glyph matching degree and confidence between the glyph of the characters to be corrected are greater than a first confidence threshold as the first set of characters to be selected, determining the first set of characters to be selected as the set of characters to be selected and displaying them; or based on the context of the characters to be corrected, determining characters whose semantics match the context semantics and whose confidence is greater than a second confidence threshold as the second set of characters to be selected, determining the first set of characters to be selected as the set of characters to be selected and displaying them; or based on the glyph of the characters to be corrected, determining characters whose glyph matching degree and confidence between the glyph of the characters to be corrected are greater than the first confidence threshold as the first set of characters to be selected, and based on the context of the characters to be corrected, determining characters whose semantics match the context semantics and whose confidence is greater than the second confidence threshold as the second set of characters to be selected, determining the union of the first set of characters to be selected and the second set of characters to be selected as the set of characters to be selected and displaying them.
[0010] In one implementation, the candidate character set is displayed in the following manner: determining the confidence level corresponding to each candidate character in the candidate character set; and displaying all candidate characters in the candidate character set on the handwriting input interface in descending order of confidence level.
[0011] According to a second aspect of an embodiment of the present disclosure, a handwriting input device is provided, comprising: a determination unit for, in response to receiving a correction gesture in a handwriting input interface, determining a text to be corrected based on the correction gesture; a processing unit for determining and displaying a set of characters to be selected based on the glyph and / or context information of the text to be corrected; and a replacement unit for, in response to a target text in the set of characters to be selected being selected, replacing the text to be corrected with the target text.
[0012] In one embodiment, the error correction gesture includes a graphic that frames the text.
[0013] In one implementation, the error correction gesture includes a triangle, and the base of the triangle is parallel to a text line corresponding to the input text in the handwriting input interface.
[0014] In one embodiment, the determination unit determines the text to be corrected based on the correction gesture in the following manner: determine that the area selected by the correction gesture includes text information, and determine the text to be corrected based on the text information; the text information includes one or more of the starting point of the text line, the text size, and the text font.
[0015] In one embodiment, the processing unit determines and displays a set of characters to be selected based on the glyph and / or context information of the characters to be corrected in the following manner: based on the glyph of the characters to be corrected, characters whose glyph matching degree and whose confidence between the glyph of the characters to be corrected is greater than a first confidence threshold are determined as the first set of characters to be selected, and the first set of characters to be selected is determined as the set of characters to be selected and displayed; or based on the context of the characters to be corrected, characters whose semantics match the context semantics and whose confidence is greater than a second confidence threshold are determined as the first set of characters to be selected. A second set of characters to be selected, determining the first set of characters to be selected as the set of characters to be selected and displaying it; or based on the glyphs of the characters to be corrected, determining characters whose glyph matching degree and whose confidence between the glyphs of the characters to be corrected is greater than a first confidence threshold, as the first set of characters to be selected, and based on the context of the characters to be corrected, determining characters whose semantics match the context semantics and whose confidence is greater than a second confidence threshold, as the second set of characters to be selected, determining the union of the first set of characters to be selected and the second set of characters to be selected as the set of characters to be selected and displaying it.
[0016] In one embodiment, the set of characters to be selected is displayed by the processing unit in the following manner: determining the confidence corresponding to each character to be selected in the set of characters to be selected; and displaying all characters to be selected in the set of characters to be selected on the handwriting input interface in descending order of confidence.
[0017] According to a third aspect of an embodiment of the present disclosure, there is provided a handwriting input device, comprising: a processor; and a memory for storing processor executable instructions; wherein the processor is configured to: execute the handwriting input method described in the first aspect or any one of the implementations of the first aspect.
[0018] According to a fourth aspect of an embodiment of the present disclosure, a storage medium is provided, in which instructions are stored. When the instructions in the storage medium are executed by a processor, the processor is enabled to execute the handwriting input method described in the first aspect or any one of the embodiments of the first aspect.
[0019] The technical solution provided by the embodiments of the present disclosure may include the following beneficial effects: when a typo is input using a handwriting input method, the text to be corrected is determined by the correction gesture. According to the glyph and / or context information of the text to be corrected, a text set containing the text that the user expects to input is determined. In response to the user selecting the expected input text, the text to be corrected is replaced with the expected input text. This avoids the need for the user to delete the typo and repeat the handwriting input or call out the virtual keyboard for input when the user needs to correct an error in the handwritten text. This simplifies the complexity of error correction in the case of handwritten text recognition errors, reduces complexity to a simple task, and saves user energy.
[0020] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS
[0021] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.
[0022] Figure 1 is a schematic diagram showing a handwriting input according to an exemplary embodiment of the present disclosure.
[0023] Figure 2 It is a schematic diagram showing a handwriting font recognition conversion according to an exemplary embodiment of the present disclosure.
[0024] Figure 3 It is a schematic diagram showing a method of deleting text to be corrected according to an exemplary embodiment of the present disclosure.
[0025] Figure 4 It is a schematic diagram showing a text error correction according to an exemplary embodiment of the present disclosure.
[0026] Figure 5 The figure is a flow chart of a handwriting input method according to an exemplary embodiment.
[0027] Figure 6 It is a schematic diagram showing a method of determining text to be corrected according to an exemplary embodiment of the present disclosure.
[0028] Figure 7 The present invention is a flowchart of a method for determining text to be corrected according to an exemplary embodiment.
[0029] Figure 8 The present invention is a flowchart showing a method for determining and displaying a set of characters to be selected according to an exemplary embodiment.
[0030] Fig. 9 is a flowchart showing a method for determining and displaying a set of characters to be selected according to yet another exemplary embodiment.
[0031] Fig.10 It is a schematic diagram showing a set of characters to be selected according to an exemplary embodiment of the present disclosure.
[0032] Fig.11 It is a schematic diagram showing a method of replacing text to be corrected according to an exemplary embodiment of the present disclosure.
[0033] Fig.12 is a block diagram of a handwriting input device according to an exemplary embodiment.
[0034] Fig.13It is a block diagram of a device for handwriting input according to an exemplary embodiment. DETAILED DESCRIPTION
[0035] Here, exemplary embodiments will be described in detail, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present disclosure.
[0036] The handwriting input method provided by the embodiments of the present disclosure is applied to a scenario where a terminal such as a tablet computer corrects a text to be corrected after a typo occurs during handwriting input.
[0037] At present, more and more people are using stylus to write in large-screen mobile terminals such as tablets. This writing method is very similar to the writing method of paper and pen, so it is very convenient to use. The stylus is often combined with the text recognition function to recognize and convert the handwritten text for storage, which can bring great convenience to users. The stylus can greatly enrich the application scenarios of terminals such as tablets, and using a stylus for handwriting input has become a writing habit for more and more young people. With the increasingly rich functions of note applications in tablet terminals and the gradual improvement of handwriting functions by manufacturers, users gradually tend to use the tablet's note function to record key points and their own opinions in meeting scenarios or learning scenarios, including but not limited to explicit handwriting scenarios such as handwritten notes and annotations to files. At the same time, the global handwriting function in terminals such as tablets can directly use a stylus to write and call the handwritten text recognition function for most input boxes in the tablet. The importance and usage rate of this function are self-evident.
[0038] The process from using a stylus to input handwriting to finally getting the desired result can be divided into three steps:
[0039] In the first step, the user writes text with a stylus. Due to the wide variety of personal writing habits and the difference between the handwriting touch on a tablet and that on paper and pen, the written text will inevitably have a different shape from the printed text. This is also the main reason why the text recognition algorithm (Optical Character Recognition, OCR) obtains errors in the target text after recognizing the handwritten font.
[0040] In the second step, the text recognition algorithm recognizes the handwritten text and converts it into printed text. The operation of each manufacturer is similar in this process. The main difference lies in the text recognition algorithm itself.
[0041] The current function of saving after handwritten input and text recognition conversion is applied in terminal products of different manufacturers. In the system software notes of terminals such as tablets, basic handwritten text recognition can be achieved, and certain operations can be performed on the recognized text, such as translation, sharing, etc. However, there is still a major problem with this function, that is, the accuracy of handwritten text recognition. Since handwritten text input is greatly affected by the user's handwriting habits, different people writing the same character can result in very different outcomes, which increases the difficulty of recognition and reduces the recognition accuracy. For example, Figure 1 in which the user handwrites "上善若水" (The highest good is like water), Figure 2 is recognized as the printed text "上喜岩水" by the text recognition algorithm in
[0042] In the third step, the user confirms whether the recognition result of the text recognition algorithm is accurate. If it is accurate, the user can accept it. If it is not accurate, the user needs to first delete the incorrect text and then rewrite it by hand or input it using the keyboard.
[0043] Currently, manufacturers of various terminal products do not have corresponding optimization strategies for correcting spelling mistakes in handwritten input. As shown in the schematic diagram of deleting the text to be corrected in Figure 3 and the schematic diagram of text correction in Figure 4 , the user can only call the virtual keyboard to use the function key to delete the misspelled word "喜岩" (Xi Yan), or use the pen gesture function to delete the misspelled word. Rewrite the text by hand and recognize it again through the text recognition algorithm of the terminal. Due to the functional limitations of the text recognition algorithm itself, for some difficult-to-write characters, it may happen that after rewriting multiple times, the text still cannot be recognized as the word the user wants. At this time, the virtual keyboard can only be called and input using the virtual keyboard, inputting the target word "善若" (Shan Ruo) to replace the misspelled word, but this loses the advantage of using the stylus input and reduces the user experience.
[0044] In view of this, the present disclosure proposes a handwritten input method. By using a correction pen gesture to select the text to be corrected and displaying a set of candidate words corresponding to the text to be corrected, after the user selects the target word in the set, the text to be corrected is replaced to complete text correction.
[0045] Figure 5 is a flowchart of a handwritten input method shown according to an exemplary embodiment. As shown in Figure 5 , this method includes steps S101 to S103.
[0046] In step S101, in response to receiving a correction pen gesture in the handwritten input interface, the text to be corrected is determined based on the correction pen gesture.
[0047] In the embodiments of the present disclosure, the user can select multiple adjacent texts to be corrected through the correction pen gesture. For example, Figure 6As shown in the schematic diagram for determining the text to be corrected, the text to be corrected, "Xi Yan", is selected by a pen gesture.
[0048] In step S102, based on the glyph and / or context information of the text to be corrected, a set of candidate characters is determined and displayed.
[0049] In an embodiment of the present disclosure, the set of candidate characters is a set of characters that are similar in glyph to the text to be corrected and have a certain semantic logical relationship with the text before and after the text to be corrected. The set of candidate characters includes the target character that the user expects to input.
[0050] Among them, the glyph of a character is a set of strokes and radicals. The character recognition method adopted in the present disclosure is based on the composition of characters and can split characters into strokes and radicals (there is adhesion between different strokes of characters, and some radicals cannot be split into strokes). Although different people have different writing habits, resulting in differences in the fonts written, the strokes and radicals of handwritten characters have inherent structural and geometric features. By splitting characters and recognizing the strokes and radicals that make up the characters, the recognition of the glyph of the characters is realized, so as to determine the input character corresponding to the user's handwritten character.
[0051] In step S103, in response to the selection of the target character in the set of candidate characters, the text to be corrected is replaced with the target character.
[0052] In an embodiment of the present disclosure, the user can select the target character in the set of candidate characters by a stylus and replace the text to be corrected with the target character.
[0053] In an embodiment of the present disclosure, the text to be corrected is determined based on a correction pen gesture, and a set of candidate characters is displayed. After the target character in the set is selected, the text to be corrected is replaced. Only by the pen gesture, without having to call the virtual keyboard, the correction of the text to be corrected can be completed, ensuring the superiority of the stylus input and improving the user experience.
[0054] In an embodiment of the present disclosure, the terminal itself supports some pen gesture functions, such as inserting and deleting text, changing lines, selecting text areas, etc. It can be understood that the pen gestures corresponding to different functions should not conflict. In the following embodiments of the present disclosure, the pen gesture for selecting the text to be corrected will be described.
[0055] In one implementation manner of an embodiment of the present disclosure, the correction pen gesture includes a graphic that can enclose the text.
[0056] In an example, the correction pen gesture includes a triangle, and the bottom side of the triangle is parallel to the text line corresponding to the text already input in the handwritten input interface.
[0057] In an embodiment of the present disclosure, the correction pen gesture is a geometric graphic with the ability to enclose. Through the enclosed area of the correction pen gesture, the text to be corrected in the enclosed area can be accurately judged. For example Figure 6As shown in the schematic diagram for determining the text to be corrected, the text to be corrected, "Xiyan", is selected by a pen gesture.
[0058] In the embodiments of the present disclosure, the terminal has multiple pen gesture functions. Each function corresponds to one or more specific pen gestures. The determination of the pen gesture should meet two requirements. One is that it is simple and easy to draw, and the user can call a specific function through the pen gesture without spending too much effort. For example, inserting a space corresponds to a vertical line drawn from top to bottom, and deleting a space corresponds to a vertical line drawn from bottom to top. The other is that there should be no conflict with other pen gestures. The relationship between the pen gesture and the function can be many-to-one, that is, the same function can correspond to multiple pen gestures. For example, the correction pen gestures corresponding to the correction function can be various pen gestures with the ability to select, such as a triangle, a rectangle, and an ellipse. However, it cannot be a one-to-many relationship, that is, the same pen gesture cannot correspond to multiple functions. For example, both the pen gesture for highlighting key text and the correction pen gesture need to select the text area. However, the above-mentioned various pen gestures with the ability to select, such as a triangle, a rectangle, and an ellipse, can only be used to select the correction area, and other pen gestures with the function of selecting text, such as a rhombus and a trapezoid, must be used to highlight key text.
[0059] In the embodiments of the present disclosure, when the user uses the correction pen gesture to determine the text to be corrected, the correction pen gesture may not completely enclose the text to be corrected, or may enclose a part of adjacent text or adjacent lines of text. Due to the user's operation error, it is necessary to ensure that when the user draws the correction pen gesture, only the text to be corrected is determined. The following steps for selecting the text to be corrected after enclosing the correction area are described in the embodiments of the present disclosure.
[0060] Figure 7 is a flowchart of a method for determining the text to be corrected according to an exemplary embodiment. As Figure 7 shown, the method includes steps S201 to S202.
[0061] In step S201, it is determined that the area enclosed by the correction pen gesture includes text information.
[0062] In step S202, the text to be corrected is determined based on the text information.
[0063] Among them, the text information includes one or more of the starting point of the text line, the text size, and the text font.
[0064] In an exemplary embodiment of the present disclosure, in the printed text recognized and displayed by the character recognition algorithm, each character has a corresponding number of pixels. The text to be corrected can be determined based on the ratio of the number of pixels occupied by the selected part of the character in the character. For example, if the preset threshold is 70%, the character whose ratio of the number of pixels occupied by the selected part of the character in the character is greater than 70% is determined as the text to be corrected. As Figure 6As shown in the schematic diagram for determining the text to be corrected, the text to be corrected, "Xiyan", is selected by a pen gesture. The text to be corrected is not fully enclosed, but only most of the pixels of the text to be corrected are included.
[0065] In the embodiments of the present disclosure, the area selected by the correction pen gesture can be used to determine the area of the text to be corrected. At this time, the terminal system can obtain the text within the area to be corrected according to this area and information such as the starting point of the text line and the font size of the text. And obtain information such as the font and font size of the text to be corrected to ensure that the target text used to replace the text to be corrected has the same font and font size.
[0066] In the embodiments of the present disclosure, a set of target texts is determined based on the glyph of the text to be corrected and the context semantics of the text to be corrected. It can be understood that there will be certain differences between the set of texts determined based on the glyph of the text to be corrected and the set of texts determined based on the context semantics of the text to be corrected. It is necessary to integrate the sets of texts obtained by different methods respectively to obtain a set of candidate texts containing the target text.
[0067] In the embodiments of the present disclosure, the method for determining and displaying the set of candidate texts will be described below.
[0068] Figure 8 is a flowchart of a method for determining and displaying a set of candidate texts according to an exemplary embodiment. As Figure 8 shown, the method includes step S301, step S302A, step S302B, and step S302C.
[0069] In step S301, in response to receiving a correction pen gesture in the handwriting input interface, the text to be corrected is determined based on the correction pen gesture.
[0070] In step S302A, based on the glyph of the text to be corrected, texts with a confidence level greater than the first confidence threshold between the glyph matching degree and the glyph of the text to be corrected are determined as the first set of candidate texts, and the first set of candidate texts is determined as the set of candidate texts and displayed.
[0071] In step S302B, based on the context of the text to be corrected, texts that semantically match the context semantics and have a confidence level greater than the second confidence threshold are determined as the second set of candidate texts, and the first set of candidate texts is determined as the set of candidate texts and displayed.
[0072] In step S302C, based on the glyph of the text to be corrected, texts with a confidence level greater than the first confidence threshold between the glyph matching degree and the glyph of the text to be corrected are determined as the first set of candidate texts, and based on the context of the text to be corrected, texts that semantically match the context semantics and have a confidence level greater than the second confidence threshold are determined as the second set of candidate texts. The union of the first set of candidate texts and the second set of candidate texts is determined as the set of candidate texts and displayed.
[0073] Among them, step S302A, step S302B and step S302C are selected to be executed.
[0074] In the disclosed embodiment, based on the glyph of the text to be corrected, similar characters with similar glyphs to the text to be corrected are determined. The glyph matching degree and the confidence between the glyphs of the text to be corrected represent the degree of similarity between the text to be selected and the text to be corrected. The first confidence threshold is the critical value of whether other characters and the text to be corrected are similar characters. The similar characters of the text to be corrected can be used as the text to be selected. Similarly, the second confidence threshold is the critical value of expressing clear semantics by connecting the context of the text to be corrected. The text that can express clear semantics by connecting the context information can be used as the text to be selected.
[0075] In the disclosed embodiment, a character set of similar characters and a character set with suitable semantics that include the character to be selected are obtained based on the shape of the character to be corrected and the contextual semantics of the character to be corrected, and then the character set to be selected is determined and displayed.
[0076] In an exemplary embodiment of the present disclosure, in response to obtaining the text to be corrected using the correction gesture, the intelligent correction function evokes the intelligent correction algorithm, and inputs the text to be corrected and the text before and after into the correction algorithm model. The correction algorithm model is divided into two parts, one part is similar to the cloze test, using a model obtained based on natural language processing (NLP), and according to the semantic information of the text before and after the text to be corrected, the missing text in the middle is inferred, and the natural language processing model will output multiple results, which are arranged from high to low according to the size of the confidence. The other part is a similar character search, that is, according to the glyph of the text to be corrected, the text similar in glyph to the text to be corrected is searched, and the text written by the user can be recognized even if the recognition error is wrong. The result similar to the glyph of the text written by the user can be recognized. Similarly, a series of texts to be selected arranged from high to low according to the confidence is also given, and finally the arrangement result obtained based on the glyph and the arrangement result obtained based on the semantics are combined to obtain the final result, and the confidence of each text to be selected is regenerated.
[0077] In the disclosed embodiment, glyph recognition and semantic recognition are different dimensions. The confidence determined based on the similarity of glyphs and the confidence determined based on semantic logical relationships are not in the same ranking system. Therefore, it is necessary to combine the two confidences to obtain a text set that includes both glyph confidence and semantic confidence. The method for determining and displaying a set of characters to be selected is further described in the disclosed embodiment below.
[0078] Fig. 9 FIG. 1 is a flow chart showing a method for determining and displaying a set of characters to be selected according to another exemplary embodiment. Fig. 9As shown, the method includes step S401 to step S402.
[0079] In step S401, determine the confidence level corresponding to each candidate character in the candidate character set.
[0080] In step S402, display all the candidate characters in the candidate character set on the handwriting input interface in descending order of confidence level.
[0081] In the embodiments of the present disclosure, the candidate character set determined based on the degree of similarity in glyphs and the candidate character set determined based on the semantic logical relationship both have their respective weights. Based on the semantic confidence level and glyph confidence level already determined for each group of candidate characters, combined with the corresponding semantic character set weight and glyph character set weight, a candidate character set including the glyph confidence level and semantic confidence level can be obtained.
[0082] In an exemplary embodiment of the present disclosure, the following method is used to determine the candidate character set based on the obtained set of semantically similar characters and the set of glyph-similar characters:
[0083] Among them, the weight of the set of semantically similar characters is 0.5, and the weight of the set of glyph-similar characters is 0.5.
[0084] The set of semantically similar characters is: A (confidence level is 0.8), B (confidence level is 0.7), C (confidence level is 0.6);
[0085] The set of glyph-similar characters is: B (confidence level is 0.8), A (confidence level is 0.6), C (confidence level is 0.5);
[0086] Then, based on the weight of the set of semantically similar characters and the weight of the set of glyph-similar characters, combined with the semantic confidence level and glyph confidence level of each candidate character, the determined candidate character set is: B (confidence level is 0.7 * 0.5 + 0.8 * 0.5 = 0.75), A (confidence level is 0.8 * 0.5 + 0.6 * 0.5 = 0.7), C (confidence level is 0.6 * 0.5 + 0.5 * 0.5 = 0.55).
[0087] In the embodiments of the present disclosure, based on the confidence level already determined for each candidate character, the candidate characters are arranged according to the confidence level and displayed on the input interface. As Fig.10 shown in the schematic diagram of displaying the candidate character set, candidate character sets such as "Shan Ruo, Xi Ruo, Pu Ruo, Pu Yan" are displayed on the input interface. Among them, the number of candidate characters in the candidate character set is not a fixed value. The candidate character set can display a part of the candidate characters on the bar display interface and switch to the next line by handwriting click, or display all the candidate characters in the entire candidate character set in matrix form.
[0088] In the embodiments of the present disclosure, the user can select a target character from a set of candidate characters and replace the character to be corrected, where the target character has the same font and size as the character to be corrected. For example, Fig.11 As shown, the user selects the target character "Shan Ruo" from the set of candidate characters and replaces the character to be corrected "Xi Yan".
[0089] In the embodiments of the present disclosure, the character to be corrected is selected based on the correction pen gesture. Based on the glyph of the character to be corrected, characters with a confidence greater than a certain threshold between the glyph matching degree and the glyph of the character to be corrected are determined as the set of similar-shaped characters of the character to be corrected, and based on the context information of the character to be corrected and the context of the character to be corrected, characters that match the context semantics and have a confidence greater than a certain threshold are determined as the set of characters that conform to the semantics of the context before and after. Furthermore, based on the set of similar-shaped characters and the set of characters that conform to the semantics of the context before and after, a set of candidate characters including the target character is determined. In response to selecting the target character from the set of candidate characters, the character to be corrected is replaced with the target character. Through the present disclosure, the character to be corrected is determined based on the correction pen gesture, and the set of candidate characters is displayed. After the target character in the selected set is selected, the character to be corrected is replaced to complete text correction, avoiding the need for the user to delete the misspelled character and repeat the handwriting input or call out the virtual keyboard for input when the handwriting is incorrect. The complexity of error correction in the case of handwriting recognition errors is simplified, making it simpler and saving the user's energy.
[0090] Based on the same concept, the embodiments of the present disclosure also provide a handwriting input device 100.
[0091] It can be understood that in order to implement the above functions, the handwriting input device 100 provided by the embodiments of the present disclosure includes the corresponding hardware structures and / or software modules for executing each function. Combining the units and algorithm steps of the various examples disclosed in the embodiments of the present disclosure, the embodiments of the present disclosure can be implemented in the form of hardware or a combination of hardware and computer software. Whether a certain function is executed in the way of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the technical solution of the embodiments of the present disclosure.
[0092] Fig.12 is a block diagram of a handwriting input device 100 shown according to an exemplary embodiment. Referring to Fig.12 , the device includes a determination unit 101, a processing unit 102, and a replacement unit 103.
[0093] The determination unit 101 is configured to determine the character to be corrected based on the correction pen gesture in response to receiving the correction pen gesture in the handwriting input interface.
[0094] The processing unit 102 is configured to determine and display a set of characters to be selected based on the shape and / or context information of the characters to be corrected.
[0095] The replacement unit 103 is configured to replace the to-be-corrected character with the target character in response to the target character in the to-be-selected character set being selected.
[0096] In one embodiment, the error correction gesture includes a graphic that frames the text.
[0097] In one implementation, the error correction gesture includes a triangle, and the base of the triangle is parallel to a text line corresponding to the input text in the handwriting input interface.
[0098] In one embodiment, the determination unit 101 determines the text to be corrected based on the correction gesture in the following manner: determine that the area selected by the correction gesture includes text information, and determine the text to be corrected based on the text information; the text information includes one or more of the starting point of the text line, the text size, and the text font.
[0099] In one embodiment, the processing unit 102 determines and displays a set of characters to be selected based on the glyph and / or context information of the characters to be corrected in the following manner: based on the glyph of the characters to be corrected, characters whose glyph matching degree and confidence between the glyph of the characters to be corrected are greater than a first confidence threshold are determined as the first set of characters to be selected, and the first set of characters to be selected is determined as the set of characters to be selected and displayed; or based on the context of the characters to be corrected, characters whose semantics match the context semantics and whose confidence is greater than a second confidence threshold are determined as the second set of characters to be selected, and the first set of characters to be selected is determined as the set of characters to be selected and displayed; or based on the glyph of the characters to be corrected, characters whose glyph matching degree and confidence between the glyph of the characters to be corrected are determined as the first set of characters to be selected, and based on the context of the characters to be corrected, characters whose semantics match the context semantics and whose confidence is greater than the second confidence threshold are determined as the second set of characters to be selected, and the union of the first set of characters to be selected and the second set of characters to be selected is determined as the set of characters to be selected and displayed.
[0100] In one implementation, the candidate character set is displayed by the processing unit 102 in the following manner: determining the confidence level corresponding to each candidate character in the candidate character set; and displaying all candidate characters in the candidate character set in descending order of confidence level on the handwriting input interface.
[0101] Regarding the device in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.
[0102] Fig.132 is a block diagram of a device 200 for handwriting input according to an exemplary embodiment. The device 200 may be provided as a terminal. For example, the device 200 may be a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, etc.
[0103] , device 200 may include one or more of the following components: a processing component 202 , a memory 204 , a power component 206 , a multimedia component 208 , an audio component 210 , an input / output (I / O) interface 212 , a sensor component 214 , and a communication component 216 .
[0104] The processing component 202 generally controls the overall operation of the device 200, such as operations associated with display, phone calls, data communications, camera operations, and recording operations. The processing component 202 may include one or more processors 220 to execute instructions to perform all or part of the steps of the above-described method. In addition, the processing component 202 may include one or more modules to facilitate interaction between the processing component 202 and other components. For example, the processing component 202 may include a multimedia module to facilitate interaction between the multimedia component 208 and the processing component 202.
[0105] The memory 204 is configured to store various types of data to support operations on the device 200. Examples of such data include instructions for any application or method operating on the device 200, contact data, phone book data, messages, pictures, videos, etc. The memory 204 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.
[0106] The power component 206 provides power to the various components of the device 200. The power component 206 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power for the device 200.
[0107] The multimedia component 208 includes a screen that provides an output interface between the device 200 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundaries of the touch or slide action, but also detect the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 208 includes a front camera and / or a rear camera. When the device 200 is in an operating mode, such as a shooting mode or a video mode, the front camera and / or the rear camera may receive external multimedia data. Each front camera and rear camera may be a fixed optical lens system or have a focal length and optical zoom capability.
[0108] The audio component 210 is configured to output and / or input audio signals. For example, the audio component 210 includes a microphone (MIC), and when the device 200 is in an operation mode, such as a call mode, a recording mode, and a speech recognition mode, the microphone is configured to receive an external audio signal. The received audio signal can be further stored in the memory 204 or sent via the communication component 216. In some embodiments, the audio component 210 also includes a speaker for outputting audio signals.
[0109] I / O interface 212 provides an interface between processing component 202 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include but are not limited to: a home button, a volume button, a start button, and a lock button.
[0110] The sensor assembly 214 includes one or more sensors for providing various aspects of the status assessment of the device 200. For example, the sensor assembly 214 can detect the open / closed state of the device 200, the relative positioning of components, such as the display and keypad of the device 200, the sensor assembly 214 can also detect the position change of the device 200 or a component of the device 200, the presence or absence of user contact with the device 200, the orientation or acceleration / deceleration of the device 200 and the temperature change of the device 200. The sensor assembly 214 can include a proximity sensor configured to detect the presence of a nearby object without any physical contact. The sensor assembly 214 can also include an optical sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 214 can also include an accelerometer, a gyroscope sensor, a magnetic sensor, a pressure sensor or a temperature sensor.
[0111] The communication component 216 is configured to facilitate wired or wireless communication between the device 200 and other devices. The device 200 can access a wireless network based on a communication standard, such as WiFi, 2G or 3G, or a combination thereof. In an exemplary embodiment, the communication component 216 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 216 also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.
[0112] In an exemplary embodiment, the apparatus 200 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors or other electronic components to perform the above method.
[0113] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 204 including instructions, and the instructions can be executed by the processor 220 of the device 200 to perform the above method. For example, the non-transitory computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.
[0114] It is to be understood that in the present disclosure, "plurality" refers to two or more than two, and other quantifiers are similar. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the associated objects before and after are in an "or" relationship. The singular forms "a", "the" and "the" are also intended to include plural forms, unless the context clearly indicates other meanings.
[0115] It is further understood that the terms "first", "second", etc. are used to describe various information, but such information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other, and do not indicate a specific order or degree of importance. In fact, the expressions "first", "second", etc. can be used interchangeably. For example, without departing from the scope of the present disclosure, the first information can also be referred to as the second information, and similarly, the second information can also be referred to as the first information.
[0116] It can be further understood that, unless otherwise specified, “connection” includes a direct connection without other components between the two, and also includes an indirect connection with other components between the two.
[0117] It is further understood that, although the operations are described in a specific order in the drawings in the embodiments of the present disclosure, it should not be understood as requiring that the operations be performed in the specific order shown or in a serial order, or requiring that all the operations shown be performed to obtain the desired results. In certain environments, multiple people and parallel processing may be advantageous.
[0118] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. This application is intended to cover any modifications, uses or adaptations of the present disclosure, which follow the general principles of the present disclosure and include common knowledge or customary technical means in the art that are not disclosed in the present disclosure.
[0119] It should be understood that the present disclosure is not limited to the precise structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the scope of the appended claims.
Claims
1. A handwriting input method, characterized in that: include: In response to receiving a correction gesture in the handwriting input interface, determining a character to be corrected based on the correction gesture; Determine and display a set of characters to be selected based on the character shape and / or context information of the character to be corrected; In response to a target character in the to-be-selected character set being selected, the to-be-corrected character is replaced with the target character.
2. The method according to claim 1, characterized in that The error correction gesture includes a graphic that frames and selects text.
3. The method according to claim 2, characterized in that The error correction gesture includes a triangle, and the base of the triangle is parallel to the text line corresponding to the input text in the handwriting input interface.
4. The method according to claim 2 or 3, characterized in that: The determining of the text to be corrected based on the correction gesture comprises: Determine that the area selected by the error correction gesture includes text information, and determine the text to be corrected based on the text information; The text information includes one or more of the text line starting point, text font size, and text font.
5. The method according to claim 1, characterized in that The step of determining and displaying a set of characters to be selected based on the character shape and / or context information of the character to be corrected comprises: Based on the glyphs of the characters to be corrected, determine the characters whose glyph matching degree and the confidence between the glyphs of the characters to be corrected and the characters is greater than a first confidence threshold, as the first set of characters to be selected, determine the first set of characters to be selected as the set of characters to be selected and display it; or Based on the context of the text to be corrected, determine the text whose semantics matches the context semantics and whose confidence is greater than a second confidence threshold as the second set of characters to be selected, determine the first set of characters to be selected as the set of characters to be selected and display it; or Based on the glyph of the text to be corrected, characters whose glyph matching degree and confidence between the glyph of the text to be corrected is greater than a first confidence threshold are determined as a first set of characters to be selected, and based on the context of the text to be corrected, characters whose semantics match the context semantics and whose confidence is greater than a second confidence threshold are determined as a second set of characters to be selected, and the union of the first set of characters to be selected and the second set of characters to be selected is determined as the set of characters to be selected and displayed.
6. The method according to claim 5, characterized in that The set of characters to be selected is displayed in the following manner: Determining the confidence level corresponding to each character to be selected in the set of characters to be selected; All the characters to be selected in the set of characters to be selected are displayed on the handwriting input interface in descending order of confidence.
7. A handwriting input device, characterized in that: include: A determination unit, configured to, in response to receiving a correction gesture in the handwriting input interface, determine a character to be corrected based on the correction gesture; A processing unit, configured to determine and display a set of characters to be selected based on the character shape and / or context information of the character to be corrected; A replacement unit is used to replace the to-be-corrected characters with the target characters in response to the target characters in the to-be-selected character set being selected.
8. The device according to claim 7, characterized in that The error correction gesture includes a graphic that frames and selects text.
9. The device according to claim 8, characterized in that The error correction gesture includes a triangle, and the base of the triangle is parallel to the text line corresponding to the input text in the handwriting input interface.
10. The device according to claim 8 or 9, characterized in that The determining unit determines the characters to be corrected based on the correcting gestures in the following manner: Determine that the area selected by the error correction gesture includes text information, and determine the text to be corrected based on the text information; The text information includes one or more of the text line starting point, text font size, and text font.
11. The device according to claim 7, characterized in that The processing unit determines and displays a set of characters to be selected based on the character shape and / or context information of the character to be corrected in the following manner: Based on the glyphs of the characters to be corrected, determine the characters whose glyph matching degree and the confidence between the glyphs of the characters to be corrected are greater than a first confidence threshold, as the first set of characters to be selected, determine the first set of characters to be selected as the set of characters to be selected and display it; or Based on the context of the text to be corrected, determine the text whose semantics matches the context semantics and whose confidence is greater than a second confidence threshold as the second set of characters to be selected, determine the first set of characters to be selected as the set of characters to be selected and display it; or Based on the glyph of the text to be corrected, characters whose glyph matching degree and confidence between the glyph of the text to be corrected is greater than a first confidence threshold are determined as a first set of characters to be selected, and based on the context of the text to be corrected, characters whose semantics match the context semantics and whose confidence is greater than a second confidence threshold are determined as a second set of characters to be selected, and the union of the first set of characters to be selected and the second set of characters to be selected is determined as the set of characters to be selected and displayed.
12. The device according to claim 11, characterized in that The set of characters to be selected is displayed by the processing unit in the following manner: Determining the confidence level corresponding to each character to be selected in the set of characters to be selected; All the characters to be selected in the set of characters to be selected are displayed on the handwriting input interface in descending order of confidence.
13. A handwriting input device, characterized in that: include: processor: a memory for storing processor-executable instructions; Wherein, the processor is configured to: execute the handwriting input method according to any one of claims 1 to 6.
14. A storage medium, characterized in that: The storage medium stores instructions, and when the instructions in the storage medium are executed by a processor, the processor is enabled to execute the handwriting input method according to any one of claims 1 to 6.
