Error character recognition method and device, equipment and storage medium

Abstract

The invention discloses a wrong character recognition method and device, equipment and a storage medium. The method comprises the steps of obtaining a first stroke sequence of a to-be-recognized character and a second stroke sequence of a correct character corresponding to the to-be-recognized character; performing stroke matching on the first stroke sequence and the second stroke sequence, and respectively determining a first difference stroke which is not successfully matched in the to-be-recognized character and a second difference stroke which is not successfully matched in the correct character; and performing wrong character recognition based on the first difference strokes and the second difference strokes by using a wrong character recognition model, and determining a target recognition result of the to-be-recognized character, the target recognition result being used for representing whether the to-be-recognized character is a wrong character. In this way, the accuracy of wrong character recognition can be improved.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a method, device, equipment and storage medium for identifying wrong characters. Background Art

[0002] With the rapid development of electronic engineering, people are increasingly using devices based on stylus. However, due to the diversity and irregularity of online handwriting, the recognition of writing errors in the online writing process is a major difficulty in current research. Especially in the field of education, such as essay correction and dictation correction, there are special high-precision requirements for the correctness and standardization of Chinese character writing. How to accurately detect wrong characters in handwritten Chinese characters is of great significance. Summary of the invention

[0003] The main technical problem solved by the present application is to provide a method, device, equipment and storage medium for misprint recognition, which can improve the accuracy of misprint recognition.

[0004] In order to solve the above technical problems, a technical solution adopted in the present application is: to provide a method for identifying wrong characters, the method comprising: obtaining a first stroke sequence of a character to be identified, and a second stroke sequence of a correct character corresponding to the character to be identified; performing stroke matching on the first stroke sequence and the second stroke sequence, and respectively determining a first difference stroke that has not been successfully matched in the character to be identified and a second difference stroke that has not been successfully matched in the correct character; using a wrong character recognition model to perform wrong character recognition based on the first difference stroke and the second difference stroke, and determining a target recognition result of the character to be identified, and the target recognition result is used to characterize whether the character to be identified is a wrong character.

[0005] In order to solve the above technical problems, another technical solution adopted in the present application is: to provide a wrong character recognition device, including: an acquisition module, a stroke matching module and a recognition module; the acquisition module is used to obtain a first stroke sequence of the text to be recognized, and a second stroke sequence of the correct text corresponding to the text to be recognized, and the stroke matching module is used to perform stroke matching on the first stroke sequence and the second stroke sequence, and respectively determine the first difference stroke that has not been successfully matched in the text to be recognized and the second difference stroke that has not been successfully matched in the correct text; the recognition module is used to use a wrong character recognition model to perform wrong character recognition based on the first difference stroke and the second difference stroke, and determine the target recognition result of the text to be recognized, and the target recognition result is used to characterize whether the text to be recognized is a wrong character.

[0006] To solve the above technical problems, another technical solution adopted in the present application is: to provide an electronic device, comprising a memory and a processor coupled to each other, the memory storing program instructions; the processor is used to execute the program instructions stored in the memory to implement the above method.

[0007] In order to solve the above technical problem, another technical solution adopted by the present application is: providing a computer-readable storage medium for storing program instructions, which can be executed to implement the above method.

[0008] The above scheme can respectively determine the first difference stroke that has not been successfully matched in the character to be recognized and the second difference stroke that has not been successfully matched in the correct character by performing stroke matching on the first stroke sequence of the character to be recognized and the second difference stroke that has not been successfully matched in the correct character. The character is composed of a stroke sequence, and the first difference stroke and the second difference stroke represent the difference strokes between the character to be recognized and the correct character, that is, the wrong character recognition model of the present application is recognized on the basis of the difference strokes. Compared with the wrong character recognition model that recognizes on the basis of the entire character to be recognized and the corresponding correct character, and needs to extract the difference strokes by the model itself, the above-mentioned method of directly providing the difference strokes to the wrong character recognition model of the present application is conducive to the wrong character recognition model to focus more on the difference in strokes between the character to be recognized and the correct character for wrong character recognition, which is conducive to improving the accuracy of wrong character recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] Figure 1 This is an example of handwritten Chinese characters written incorrectly provided by this application;

[0010] Figure 2 It is a flowchart of an embodiment of a method for identifying wrong characters provided by the present application;

[0011] Figure 3 yes Figure 2 The flowchart of step S22 of an embodiment is shown;

[0012] Figure 4 yes Figure 2 The flowchart of step S23 is shown as an embodiment;

[0013] Figure 5 yes Figure 3 The flowchart of step S33 is shown as an embodiment;

[0014] Figure 6 yes Figure 3 The flowchart of an embodiment before step S33 is shown;

[0015] Figure 7 It is a flowchart of another embodiment of the method for identifying wrong characters provided by the present application;

[0016] Figure 8 It is a schematic diagram of a framework of an embodiment of a wrong character recognition device provided by the present application;

[0017] Fig. 9 It is a schematic diagram of the framework of an embodiment of an electronic device provided by the present application;

[0018] Fig.10 It is a schematic diagram of the framework of the computer-readable storage medium provided by this application. DETAILED DESCRIPTION

[0019] In order to make the purpose, technical solution and effect of the present application clearer and more specific, the present application is further described in detail below with reference to the accompanying drawings and examples.

[0020] In addition, if there are descriptions involving "first", "second", etc. in the embodiments of the present application, the descriptions of "first", "second", etc. are only used for descriptive purposes and cannot be understood as indicating or suggesting their relative importance or implicitly indicating the number of technical features indicated. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In addition, the technical solutions between the various embodiments can be combined with each other, but they must be based on the ability of ordinary technicians in the field to implement them. When the combination of technical solutions is contradictory or cannot be implemented, it should be deemed that such combination of technical solutions does not exist and is not within the scope of protection required by this application.

[0021] The method for identifying wrong characters provided in the present application can be used in any scenario where wrong characters are required, such as in essay marking or dictation marking in the field of education, to provide a judgment result on whether the student's writing is correct or incorrect. Another example is in the recognition of text written by a user online. When the text written by the user is judged to be a wrong character, the correct text corresponding to the wrong character is output.

[0022] The text described in this application can be Chinese characters or other text, such as English words, etc., which is not specifically limited here. However, in order to facilitate the explanation of the wrong character recognition method of this application, the wrong character recognition method of this application will be described later using Chinese characters as an example.

[0023] See also Figure 1 , Figure 1 is an example of handwritten Chinese characters written incorrectly provided by this application. Figure 1 As shown in the figure, handwritten Chinese characters are highly diverse and complex, including different writing styles, stroke variations, and glyph changes such as more strokes, fewer strokes, and glyph structure errors, making the handwritten Chinese character typo recognition task more challenging.

[0024] The characters to be recognized in this application are handwritten characters to be recognized for wrong characters, and the correct characters can be understood as the correct answers corresponding to the characters to be recognized. The correct characters can be correct Chinese characters handwritten according to the standard writing specifications of Chinese characters, or they can be printed characters corresponding to the correct characters. Among them, for dictation task scenarios, the correct characters corresponding to each character to be recognized can be stored in advance. In other task scenarios, such as essay correction scenarios, after obtaining each character to be recognized, the relevant recognition model can be used to identify the correct characters corresponding to the characters to be recognized, and then the method of this application can be used to detect whether the characters to be recognized are written incorrectly based on the correct characters and the characters to be recognized.

[0025] It should be noted that the method provided in this application includes at least one or more of the following highlights:

[0026] First, the present application can first match the first stroke sequence of the character to be recognized with the second stroke sequence of the corresponding correct character, and respectively determine the first difference strokes that are not successfully matched in the character to be recognized and the second difference strokes that are not successfully matched in the correct character, so as to determine the difference strokes between the character to be recognized and the correct character, and then use the difference strokes to identify the wrong character. The method of directly providing the difference strokes to the wrong character recognition model is conducive to the wrong character recognition model to focus more on the difference in strokes between the character to be recognized and the correct character for wrong character recognition, which is conducive to improving the accuracy of wrong character recognition.

[0027] The first difference stroke that is not successfully matched in the character to be recognized refers to the difference stroke in the character to be recognized that is not matched to the corresponding stroke in the correct character after stroke matching. Similarly, the second difference stroke that is not successfully matched in the correct character refers to the difference stroke in the correct character that is not matched to the stroke in the character to be recognized after stroke matching.

[0028] Second, in the process of stroke matching, in order to determine more accurate difference strokes through stroke matching, multi-level stroke matching can be performed. For example, firstly, single strokes are matched to obtain the matching result of single strokes, and then based on the single stroke matching result, multiple strokes are matched as a whole (for example, radical matching and / or connected stroke matching), and then after multiple levels of stroke matching, more accurate difference strokes are obtained.

[0029] Third, the wrong character recognition model of the present application includes two task branches, and the final wrong character recognition result is determined by combining the recognition results of the two task branches. Among them, the first branch (corresponding to the stroke category identifier below) is used to recognize the text image corresponding to the text to be recognized, and after obtaining the first stroke category sequence of the text to be recognized, it is compared with the second stroke category sequence corresponding to the correct text to determine a recognition result, and the second branch (corresponding to the wrong character identifier below) is used to recognize the rendered images corresponding to the first difference strokes of the text to be recognized and the second difference strokes of the correct text, respectively, to determine another recognition result.

[0030] Fourth, during the training process of the typo recognition model of the present application, the first loss determined by the first branch and the second loss determined by the second branch are integrated to obtain a comprehensive loss, and the comprehensive loss is used to adjust the network parameters of each branch of the typo recognition model, that is, the typo recognition model is obtained by jointly training the first branch and the second branch.

[0031] Fifth, in the process of using the second branch of the wrong character recognition model to identify wrong characters, the correct text corresponding to the text to be recognized will be added as an answer prompt, so that the sub-model of the second branch knows what Chinese characters correspond to the different strokes, which is more conducive to the sub-model of the second branch to automatically focus on the wrong area written by the user.

[0032] See also Figure 2 , Figure 2 1 is a flow chart of an embodiment of a method for identifying wrong characters provided by the present application. It should be noted that if there are substantially the same results, this embodiment does not Figure 1 The process sequence shown is limited. Figure 2 As shown, this embodiment includes:

[0033] S21: Acquire a first stroke sequence of a character to be recognized and a second stroke sequence of a correct character corresponding to the character to be recognized.

[0034] This embodiment is used to use a wrong character recognition model to perform wrong character recognition based on the first difference strokes and the second difference strokes after determining the first difference strokes between the text to be recognized and the correct text, and determining the second difference strokes between the correct text and the text to be recognized, so that the wrong character recognition model can focus more on the stroke differences between the text to be recognized and the correct text to perform wrong character recognition, thereby obtaining a more accurate wrong character recognition result.

[0035] In one embodiment, the text to be recognized is the text written (handwritten) by the user online, wherein the online handwritten text has timing information, and the track points and the corresponding writing time in the writing process can be stored as the track point data of the text. Therefore, it can be understood that the first stroke sequence of the text to be recognized is determined by using several groups of track point sequences in the process of writing Chinese characters. Among them, each group of track point sequences is used to determine a stroke in the text to be recognized. Specifically, when the user writes each stroke of a Chinese character, there will be an initial track point and an end track point. Each group of track point sequences can be determined using each initial track point and the corresponding end track point. After one stroke is written, the next stroke is written. After completion, the track point sequences of all strokes are combined to form a Chinese character.

[0036] The correct text is the answer word corresponding to the text to be recognized. Regardless of whether the correct text is handwritten or printed, the strokes of the correct text can be regarded as being written in accordance with the standard writing specifications of Chinese characters. The correct text can be used as a standard character to determine whether the text to be recognized is correct or incorrect. In order to facilitate the subsequent distinction between the strokes in the text to be recognized and the correct text, the strokes used to write the text to be recognized are called first strokes, and the strokes used to write the correct text are called second strokes. In other words, the first stroke sequence includes a number of first strokes used to form the text to be recognized, and the second stroke sequence includes a number of second strokes used to form the correct text.

[0037] S22: Perform stroke matching on the first stroke sequence and the second stroke sequence, and respectively determine the first difference strokes that have not been successfully matched in the character to be recognized and the second difference strokes that have not been successfully matched in the correct character.

[0038] In this embodiment, single stroke matching may be performed on the first stroke sequence and the second stroke sequence to respectively determine the first difference stroke that failed to match in the character to be recognized and the second difference stroke that failed to match in the correct character based on the single stroke matching result.

[0039] Specifically, see Figure 3 , Figure 3 yes Figure 2 The flowchart of an embodiment of step S22 is shown. In this embodiment, step S22 further includes:

[0040] S31: For each second stroke, find a first target stroke matching the second stroke from the plurality of first strokes, and use the second stroke matching each first target stroke from the plurality of second strokes as the second target stroke.

[0041] This embodiment is used to perform single stroke matching on the first stroke sequence and the second stroke sequence, and then determine the first difference stroke that failed to match in the character to be recognized and the second difference stroke that failed to match in the correct character based on the single stroke matching result.

[0042] Since the correct text is written in accordance with the standard writing specifications of Chinese characters, and the purpose of wrong character recognition is to determine whether the handwritten text of the user is a wrong character, in the process of single stroke matching, each second stroke in the correct text can be preferentially used as a reference stroke for stroke matching. For example, the stroke features of each first stroke and each second stroke are first extracted, and then for each second stroke, the first stroke that best matches the stroke feature of the second stroke is found from a number of first strokes in the first stroke sequence as the first target stroke. Specifically, for each second stroke, first find the first stroke that meets the preset difference threshold for the second stroke based on the difference in stroke features between each first stroke and the second stroke, and then find the first stroke with the smallest difference from each first stroke that meets the preset difference threshold as the first target stroke that matches the second stroke. The specific preset difference threshold can be determined based on experience.

[0043] S32: taking each first stroke except each first target stroke among the plurality of first strokes as a third target stroke, and taking each second stroke except each second target stroke among the plurality of second strokes as a fourth target stroke.

[0044] It can be understood that the first target stroke is the stroke that matches the corresponding second stroke, so the third target stroke other than the first target strokes among the plurality of first strokes is the stroke that does not match the corresponding second stroke, and for the convenience of subsequent distinction, the third target stroke is referred to as the unsuccessfully matched stroke in the text to be recognized. Similarly, the second target stroke is the stroke that matches the first target stroke in the correct text, so the fourth target stroke other than the second target strokes among the plurality of second strokes can be referred to as the unsuccessfully matched stroke in the correct text.

[0045] In summary, the third target stroke is a stroke in the character to be recognized that has not been successfully matched with the correct character, and the fourth target stroke is a second stroke in the correct character that has no corresponding matching stroke.

[0046] S33: Determine a first difference stroke and a second difference stroke based on each third target stroke and each fourth target stroke.

[0047] In one embodiment, each third target stroke in the character to be recognized can be directly used as the first difference stroke, and each fourth target stroke in the correct character can be directly used as the second difference stroke for subsequent wrong character recognition.

[0048] In another embodiment, considering the randomness of the text to be recognized written by the user, it may occur that multiple standard and regular second strokes are written in a connected way as one stroke. Since the above single-stroke matching is to match each first stroke and each second stroke, when the first stroke is formed by connecting multiple standard second strokes, it is very difficult to successfully match the connected stroke with a certain second stroke.

[0049] Exemplarily, the standard stroke order of the standard Chinese character "口" is "vertical, horizontal fold, horizontal", that is, the standard writing of the Chinese character "口" requires 3 strokes to be written. However, some users may write it in one connected stroke. But in single-stroke matching, the "口" in the text to be recognized is one stroke. Matching any one of the second strokes in the correct "口" (i.e., "vertical, horizontal fold, horizontal") with the "口" formed by one connected stroke may not be successful. In fact, the "口" written by the user in one connected stroke is correct and is the same Chinese character as the standard "口" formed by three second strokes. It's just that the user's writing is not standard. If the 3 standard second strokes of "口" are combined into one stroke as a whole and matched with the "口" written by the user in one stroke, it is very likely to be successfully matched. Therefore, using the above method of "directly taking each third target stroke in the text to be recognized as the first difference stroke and directly taking each fourth target stroke in the correct text as the second difference stroke for subsequent misspelling recognition", it may also consider the strokes that are actually written correctly as difference strokes for subsequent matching.

[0050] To solve this problem, based on the determination of each third target stroke and each fourth target stroke that have no corresponding matching strokes according to single-stroke matching, first perform a connected-stroke judgment on each third target stroke based on each fourth target stroke in the correct text to determine whether each third target stroke is a connected stroke, and then determine the corresponding first difference stroke and second difference stroke according to the connected-stroke judgment result. For the specific implementation method, refer to the relevant descriptions of the embodiments shown below. Figure 5 The relevant descriptions of the embodiments shown below.

[0051] It should be noted that in combination with Figure 5 the content of the embodiments shown below, the purpose of the connected-stroke judgment in this application is: to find out the connected strokes that are actually formed by connecting multiple second strokes in the correct text from the third target strokes that have not been successfully matched with the second strokes (wherein, if the third target stroke is successfully matched with the corresponding multiple second strokes, it is considered that the third target stroke is a connected stroke formed by connecting the multiple second strokes), so as to delete the third target stroke that is a connected stroke and the multiple second strokes that are matched with the connected stroke from the third target strokes and fourth target strokes that have not been successfully matched respectively. Finally, the third target strokes and fourth target strokes that still have not been successfully matched with the corresponding strokes after the connected-stroke judgment are used as the corresponding first difference strokes and second difference strokes to participate in the misspelling recognition of the subsequent misspelling recognition model.

[0052] At the same time, it should be noted that, considering that only one stroke in a Chinese character is matched during single stroke matching, the feature differences between single strokes are not obvious, and it is easy to successfully match one stroke of two or more unmatched strokes with a stroke in another character, that is, single stroke matching is prone to mismatching. Furthermore, the subsequent mismatching is based on the unmatched difference strokes. Therefore, if the unmatched strokes are determined as matching strokes, it is easy to make the number of unmatched difference strokes screened out less than the number of real difference strokes, which will cause subsequent misrecognition.

[0053] Therefore, in order to solve this problem, in one embodiment, after single stroke matching, radical-level stroke matching can be performed based on the single stroke matching results, so as to use the radical-level matching results between Chinese characters to adjust the first target strokes and the corresponding second target strokes that match each other in the single stroke matching results, so that the strokes in the radical that are not matched in the single stroke matching but are not matched in the radical matching results are deleted from the first target strokes and the corresponding second target strokes that are determined to be matched before the radical matching, and the adjusted first target strokes and the corresponding second target strokes are obtained, and then the corresponding third target strokes and the fourth target strokes are re-determined based on the adjusted first target strokes and the second target strokes. The specific implementation method of radical matching can be referred to below Figure 6 Description of the illustrated embodiment.

[0054] Further, radical matching may be performed after single stroke matching and before connected stroke determination, that is, connected stroke determination is performed on the basis of the adjusted third target stroke and fourth target stroke.

[0055] It should be noted that, unlike connected-stroke judgment, the purpose of radical matching is to delete strokes that are essentially unmatched (or wrong) from the first target strokes and the second target strokes that are considered to be matched before radical matching, while connected-stroke judgment is to delete strokes that are essentially matched (correct) from the third target strokes and the fourth target strokes that are considered to be unmatched before connected-stroke judgment.

[0056] S23: Using the wrong character recognition model to perform wrong character recognition based on the first difference stroke and the second difference stroke, and determining a target recognition result of the character to be recognized, wherein the target recognition result is used to indicate whether the character to be recognized is a wrong character.

[0057] In one embodiment, the wrong character recognition model includes an encoder and a wrong character recognizer (corresponding to the sub-model of the second branch in the previous text). In this embodiment, the encoder can be used to extract the first feature of the first difference image and the second feature of the second difference image respectively, and then the wrong character recognizer can be used to perform wrong character recognition based on the first feature and the second feature to determine the first recognition result of the text to be recognized; finally, based on the first recognition result, the target recognition result of the text to be recognized is determined. The first difference image is a stroke image formed by rendering the first difference strokes, and the second difference image is a stroke image formed by rendering the second difference strokes.

[0058] Specifically, see Figure 4 , Figure 4 yes Figure 2 The flowchart of step S23 of an embodiment is shown. This embodiment mainly includes:

[0059] S41: using an encoder to respectively extract first features of a first difference image and second features of a second difference image; the first difference image is a stroke image formed by rendering the first difference strokes, and the second difference image is a stroke image formed by rendering the second difference strokes.

[0060] In this embodiment, the input image of the wrong character recognition model is a first difference image formed by rendering the first difference stroke and a second difference image formed by rendering the second difference stroke. After obtaining the first difference image and the second difference image, the encoder in the wrong character recognition model is used to extract the first feature of the first difference image and the second feature of the second difference image respectively.

[0061] S42: Using a wrong character identifier to identify wrong characters based on the first feature and the second feature, and determining a first recognition result of the text to be recognized.

[0062] In this embodiment, the wrong character identifier can be used to perform attention processing on the first feature and the second feature to obtain an attention weight matrix so that the error area of ​​the text to be recognized can be focused on according to the attention weight matrix; then the wrong character recognition is performed based on the attention weight matrix to determine the first recognition result of the text to be recognized.

[0063] In one embodiment, considering that when only the difference image is used for wrong character recognition, the wrong character recognition model does not know what Chinese character corresponds to the difference image, in order to make the wrong character recognition model know what character is in the difference image, the wrong character recognition can be performed based on the attention weight matrix to determine the first recognition result of the character to be recognized. The encoding vector of the correct character corresponding to the character to be recognized is obtained so that the model knows what strokes in the Chinese character the difference strokes in the difference image are, and then the wrong character recognizer is used to perform attention processing on the first feature and the second feature based on the encoding vector to obtain the attention weight matrix, and then the wrong character recognition is performed based on the attention weight matrix to determine the first recognition result of the character to be recognized.

[0064] In another embodiment, after using the wrong character identifier to perform attention processing on the first feature and the second feature to obtain the attention weight matrix, the wrong character recognition can also be performed based on the attention weight matrix and the encoding vector to determine the first recognition result of the text to be recognized.

[0065] The purpose of the attention processing is to enable the wrong character recognition model to automatically focus on the error area of ​​the text to be recognized, so that the model can perform recognition based on the error area and obtain more accurate recognition results.

[0066] S43: Based on the first recognition result, determine a target recognition result of the text to be recognized.

[0067] In an optional implementation, the first recognition result can be directly used as the target recognition result of the character to be recognized. For example, the first recognition result is the probability of the character to be recognized being a correct character and the probability of being a wrong character output by the wrong character recognizer, and whether the character to be recognized is a wrong character is determined based on the probability of the character to be recognized being a correct character and the probability of being a wrong character.

[0068] In another optional embodiment, the wrong character recognition model also includes a stroke category identifier (corresponding to the sub-model of the first branch above), which is used to recognize the text image corresponding to the text to be recognized and determine the first stroke category sequence of the text to be recognized; then, based on the first difference between the first stroke category sequence of the text to be recognized and the second stroke category sequence of the correct text, determine the second recognition result of the text to be recognized, and then combine the first recognition result and the second recognition result to obtain the target recognition result of the text to be recognized.

[0069] Among them, the stroke category refers to what kind of stroke each stroke is, for example, which one is dot, horizontal, vertical, left-falling stroke, right-falling stroke, right-falling stroke, right-falling stroke, and hook. It should be noted that the second stroke in the correct text is a standardized stroke, so the stroke category sequence of the correct text is determined and known, and through the writing sequence of the stroke category of a character, it is possible to determine what specific character is written. Therefore, after identifying the first stroke category sequence of the text to be recognized, the first stroke category sequence of the text to be recognized and the second stroke category sequence of the correct text can be compared, and the second recognition result of the text to be recognized can be determined based on the first difference between the two sequences. It can be understood that the smaller the first difference, the greater the probability that the second recognition result of the text to be recognized is the correct Chinese character, and vice versa.

[0070] In one embodiment, corresponding weights may be set in advance for the first recognition result and the second recognition result. After obtaining the first recognition result and the second recognition result, the two recognition results are weighted according to the weights corresponding to each recognition result to obtain the final target recognition result.

[0071] It should be noted that the present embodiment takes into account the method of determining the second recognition result by utilizing the first difference between the first stroke category sequence and the second stroke category sequence corresponding to the correct text, and is more inclined to give a correct result. For example, in a scenario where the text to be recognized and the correct text differ by only one stroke (for example, one stroke “,”), since the stroke category recognizer is based on the entire image for recognition, it is difficult to extract local information in the image. Therefore, when there is a very small error area in the text to be recognized, the stroke category recognizer will ignore the error, and therefore it is easier to give a second recognition result that the text to be recognized is the correct character. However, the first recognition result output by the wrong character recognizer based on the difference image and the corresponding correct text, since the difference image represents the difference area between the text to be recognized and the correct text, the model is likely to believe that the text to be recognized corresponding to the input difference image is an incorrect character, that is, compared with the stroke category recognizer, the wrong character recognizer is more inclined to give an incorrect result. Therefore, in order to make the final result more accurate, the results output by the two recognizers can be combined to determine the final target recognition result.

[0072] In some embodiments, before step S23, the following step of training a wrong character recognition model is also included:

[0073] First, obtain a number of sample data; each sample data includes a sample image of the sample text to be recognized, a first sample difference image, and a second sample difference image; the first sample difference image is a stroke image corresponding to the sample difference strokes in the sample text to be recognized, and the second sample difference image is a stroke image corresponding to the sample difference strokes in the correct sample text corresponding to the sample text to be recognized, and each sample difference stroke is a stroke that has not been successfully matched between the sample text to be recognized and the correct sample text.

[0074] The specific first sample difference image and the second sample difference image can refer to the first difference image and the second difference image above respectively. The method for obtaining the sample difference strokes in the sample text to be recognized and the sample difference strokes in the correct sample text corresponding to the sample text to be recognized can refer to the method for obtaining the first difference strokes and the second difference strokes above, and no further details will be given here.

[0075] Second, for each sample data, the stroke category identifier in the wrong character recognition model (corresponding to the first task branch above) is used to process the sample image to obtain a third stroke category sequence of the sample text to be recognized, and the wrong character recognition model (corresponding to the second task branch above) is used to process the first sample difference image and the second sample difference image to obtain a sample recognition result of the sample text to be recognized;

[0076] Third, determining a second difference between the third stroke category sequence and the fourth stroke category sequence of the correct sample text, and determining a third difference between the sample recognition result and the true result;

[0077] Fourth, using the comprehensive difference determined based on the first difference and the second difference, the network parameters of the wrong character recognition model are adjusted.

[0078] In summary, this embodiment adjusts the network parameters of the wrong character recognition model by combining the second difference corresponding to the first task branch and the third difference corresponding to the second task branch, so as to jointly train the wrong character recognition model and thereby improve the ability of the wrong character recognition model to capture wrong character features.

[0079] In some embodiments, before step S21, the method further includes:

[0080] First, a text image to be recognized is obtained, where the text image to be recognized includes a text image of the text to be recognized.

[0081] In one embodiment, the image of the text to be recognized only includes the image of the text to be recognized; in another embodiment, the image of the text to be recognized also includes the image of the correct text corresponding to the text to be recognized.

[0082] Second, the image of the text to be recognized is processed using the wrong character recognition model to obtain a third recognition result of the text to be recognized.

[0083] Third, in response to the third recognition result indicating that the character to be recognized is incorrect, obtaining a first stroke sequence of the character to be recognized, a second stroke sequence of a correct character corresponding to the character to be recognized, and subsequent steps.

[0084] In a specific embodiment, the third recognition result of the text to be recognized is obtained by processing the image of the text to be recognized using the wrong character recognizer in the wrong character recognition model. Further, if the third recognition result is that the probability that the text to be recognized is a correct character is greater than a preset probability threshold, the text to be recognized is determined to be a correct character, otherwise, it is determined that the third recognition result indicates that the text to be recognized is not a correct character (i.e., incorrect), and step S21 is performed when the text to be recognized is incorrect.

[0085] This embodiment takes into account that the wrong character identifier is more inclined to give the character to be recognized as a wrong character. Therefore, when the wrong character identifier gives the character to be recognized as a correct character, it means that the character to be recognized is a correct character. Therefore, step S21 and subsequent steps may not be executed in this case.

[0086] The above scheme can respectively determine the first difference stroke that has not been successfully matched in the character to be recognized and the second difference stroke that has not been successfully matched in the correct character by performing stroke matching on the first stroke sequence of the character to be recognized and the second difference stroke that has not been successfully matched in the correct character. The character is composed of a stroke sequence, and the first difference stroke and the second difference stroke represent the difference strokes between the character to be recognized and the correct character, that is, the wrong character recognition model of the present application is recognized on the basis of the difference strokes. Compared with the wrong character recognition model that recognizes on the basis of the entire character to be recognized and the corresponding correct character, and needs to extract the difference strokes by the model itself, the above-mentioned method of directly providing the difference strokes to the wrong character recognition model of the present application is conducive to the wrong character recognition model to focus more on the difference in strokes between the character to be recognized and the correct character for wrong character recognition, which is conducive to improving the accuracy of wrong character recognition.

[0087] See also Figure 5 , Figure 5 yes Figure 3 The flowchart of step S33 of an embodiment is shown. In this embodiment, based on each third target stroke and each fourth target stroke, determining the first difference stroke and the second difference stroke includes:

[0088] S51: performing connected stroke judgment on each third target stroke based on each fourth target stroke in the correct text, and determining a connected stroke judgment result of each third target stroke; the connected stroke judgment result is used to indicate whether the third target stroke is a connected stroke.

[0089] In this embodiment, considering that each second stroke in the correct text is a standard stroke, each fourth target stroke in the correct text for which no corresponding matching stroke is found can be taken as the current stroke in turn, and a stroke whose distance to the current stroke is less than or equal to a preset threshold is selected from the remaining fourth target strokes or several second strokes, and the stroke less than or equal to the preset threshold and the current stroke are taken as a stroke as a whole; then, each third target stroke is stroke-matched with each stroke as a whole to determine whether there is a third target stroke that matches each stroke as a whole; finally, the third target stroke that matches the stroke as a whole is determined as a connected stroke.

[0090] This embodiment takes into account that when multiple second strokes in the correct text are relatively close, the user may connect several second strokes that are relatively close into one stroke when writing the corresponding text to be recognized, so a connected stroke judgment method is set to find the connected strokes in the text to be recognized. Specifically, each third target stroke in the text to be recognized that has no corresponding matching stroke is matched with the stroke composed of several second strokes in the correct text. If the match is successful, it is determined that the third target stroke in the text to be recognized that has successfully matched is the first stroke formed by connecting multiple second strokes in the correct text.

[0091] It should be noted that the user may choose to connect two or more second strokes into one stroke, so when combining into a stroke as a whole, one or more second strokes among the strokes that are closer (strokes whose distance to the current stroke is less than or equal to a preset threshold) can be combined with the current stroke into a stroke as a whole to perform stroke matching. For example, the matching can be performed from less to more, or from more to less, for example, in a matching method from more to less, first the three strokes are matched as a stroke as a whole with the third target stroke, if the match is successful, the matching is stopped, if the match is not successful, the two strokes are matched as a stroke as a whole with the third target stroke.

[0092] It can be understood that if the third target stroke and the corresponding stroke composed of multiple second strokes are successfully matched as a whole, it means that the third target stroke is written correctly, but it is not written according to the standard strokes, but several standard second strokes are written in a connected manner.

[0093] S52: Determine a first difference stroke and a second difference stroke based on the result of the connected stroke judgment of each third target stroke.

[0094] In this embodiment, the non-connected strokes in each third target stroke are determined as the first difference strokes in the text to be recognized, and the fourth target stroke in the stroke group that does not match each connected stroke is determined as the second difference stroke in the correct text.

[0095] In summary, the method of connecting stroke judgment set in this embodiment is to determine the third target stroke that is not successfully matched before the connected stroke judgment but is formed by connecting multiple second strokes as a correct (successfully matched) written stroke. The third target stroke determined as a connected stroke will not be input into the subsequent wrong character recognition model and will not participate in the subsequent wrong character recognition process. The strokes that subsequently participate in the wrong character recognition model are the non-connected strokes in each third target stroke and the fourth target stroke in the stroke group that does not match each connected stroke.

[0096] See also Figure 6 , Figure 6 yes Figure 3 The flowchart of an embodiment before step S33 is shown. This embodiment mainly includes:

[0097] S61: Selecting the second target stroke that is the first radical of the correct character from the second target strokes in the correct character and using it as the first radical stroke.

[0098] S62: Selecting first target strokes that match first radical strokes from first target strokes in the character to be recognized as second radical strokes.

[0099] S63: Perform radical matching based on each first radical stroke and each second radical stroke to determine a radical matching result.

[0100] S64: Based on the radical matching result, adjust the first target stroke in the character to be recognized and the second target stroke in the correct character.

[0101] It should be noted that the correct character as the answer character has a known radical composed of several second strokes, so the second target stroke of the correct character radical (called the first radical) can be selected from the second target strokes that have been matched in the correct character before the radical matching, and used as the first radical stroke.

[0102] In addition, the strokes in the character to be recognized that are the same radical as the correct character radical must be strokes that have a high degree of matching with the first radical stroke. Therefore, after determining the first radical stroke, the first target strokes that match the first radical strokes can be selected from the first target strokes in the character to be recognized as the second radical stroke in the character to be recognized that is the same radical as the first radical, and then radical matching is performed based on the first radical stroke in the correct character and the second radical stroke in the character to be recognized to determine the radical matching result.

[0103] Among them, based on the difference in stroke features between all first radical strokes in the first radical and all second radical strokes in the second radical, a radical matching result for characterizing whether the first radical and the second radical match can be determined, and then based on the radical matching result, the first target stroke in the text to be recognized and the second target stroke in the correct text can be adjusted.

[0104] In this embodiment, after determining the radical matching result, if the first radical does not match the second radical, the second radical stroke in each first target stroke (the first stroke considered to be matched before the radical matching) is adjusted to the third target stroke (the first stroke considered not to be matched), and the first radical stroke in each second target stroke (the second stroke considered to be matched before the radical matching) is adjusted to the fourth target stroke (the second stroke considered not to be matched). That is, in this embodiment, only the strokes that are considered to be matched by both the single stroke matching and the radical matching are considered to be strokes that are truly matched successfully, otherwise, they are considered to be strokes that are not successfully matched.

[0105] In one specific embodiment, see Figure 7 , Figure 7 1 is a flow chart of another embodiment of the method for identifying wrong characters provided by the present application. This embodiment includes the following steps:

[0106] First, a first stroke sequence of the character to be recognized and a second stroke sequence of the correct character corresponding to the character to be recognized are obtained.

[0107] Second, multi-level stroke matching is performed on the first stroke sequence and the second stroke sequence to respectively determine the first difference strokes that have not been successfully matched in the character to be recognized and the second difference strokes that have not been successfully matched in the correct character.

[0108] Among them, the stroke matching of multiple levels is arranged in the order of stroke matching as follows: single stroke matching, radical matching and connected stroke judgment. That is, firstly, single stroke matching is performed to determine the matching strokes and unmatched strokes (first unmatched strokes) corresponding to the character to be recognized and the correct character respectively, and then radical matching is performed based on the matching strokes corresponding to the character to be recognized and the correct character respectively. If the radical matching is successful, it means that the matching strokes corresponding to the character to be recognized and the correct character respectively are real matching strokes. If the radical matching is unsuccessful, it means that the matching strokes corresponding to the recognized character and the correct character respectively are not real matching strokes. At this time, the strokes whose radical matching is unsuccessful in the matching strokes need to be adjusted to unmatched strokes (second unmatched strokes); then, connected stroke judgment is performed based on the first unmatched stroke and the second unmatched stroke, and the matching strokes (i.e. connected strokes and strokes matching the connected strokes) are filtered out from the first unmatched stroke and the second unmatched stroke according to the connected stroke judgment result to obtain the final first difference stroke and second difference stroke.

[0109] The first difference stroke is rendered to form a first difference image, and the second difference stroke is rendered to form a second difference image. In addition, the single stroke matching, radical matching and connected stroke judgment methods can refer to the relevant description of the corresponding embodiment above, and will not be repeated here.

[0110] Third, the encoder in the wrong character recognition model is used to extract image features of the text image of the text to be recognized, and the encoder is used to respectively extract the first feature of the first difference image and the second feature of the second difference image.

[0111] Fourth, use the stroke category identifier in the wrong character recognition model to decode based on the image features to obtain the first stroke category sequence of the text to be recognized, and then determine the second recognition result of the text to be recognized based on the difference between the first stroke category sequence and the second stroke category sequence corresponding to the correct text; and use the wrong character identifier in the wrong character recognition model to obtain the encoding vector corresponding to the correct text, and perform cross-attention processing on the encoding vector, the first feature and the second feature to obtain the attention weight matrix, and then recognize the first recognition result of the text to be recognized.

[0112] Fifth, the first recognition result and the second recognition result are combined to obtain a target recognition result of the character to be recognized; the target recognition result of the character to be recognized is used to indicate whether the character to be recognized is a wrong character.

[0113] It should be noted that, in one embodiment, the encoder is a shared encoder, and the training of the wrong character recognition model is carried out by jointly training and adjusting parameters using the recognition results of the stroke category identifier and the wrong character identifier. During the parameter adjustment process, the network parameters of the encoder, stroke category identifier and wrong character identifier will be adjusted.

[0114] See also Figure 8 , Figure 8 It is a framework diagram of an embodiment of a wrong character recognition device provided by the present application. In this embodiment, the wrong character recognition device 80 includes an acquisition module 81, a stroke matching module 82 and a recognition module 83. The acquisition module 81 is used to acquire a first stroke sequence of the character to be recognized, and a second stroke sequence of the correct character corresponding to the character to be recognized; the stroke matching module 82 is used to perform stroke matching on the first stroke sequence and the second stroke sequence, and respectively determine the first difference stroke that has not been successfully matched in the character to be recognized and the second difference stroke that has not been successfully matched in the correct character; the recognition module 83 is used to perform wrong character recognition based on the first difference stroke and the second difference stroke using the wrong character recognition model, and determine the target recognition result of the character to be recognized, and the target recognition result is used to characterize whether the character to be recognized is a wrong character.

[0115] In some embodiments, the first stroke sequence acquired by the acquisition module 81 is determined based on a plurality of groups of trajectory point sequences of the to-be-recognized characters written online by the user, and each group of trajectory point sequences is used to determine a stroke in the to-be-recognized characters.

[0116] In some embodiments, the first stroke sequence acquired by the acquisition module 81 includes several first strokes used to form the text to be recognized, and the second stroke sequence includes several second strokes used to form the correct text; the stroke matching module 82 performs stroke matching on the first stroke sequence and the second stroke sequence, and respectively determines the first difference strokes that are not successfully matched in the text to be recognized and the second difference strokes that are not successfully matched in the correct text, including: for each second stroke, find a first target stroke that matches the second stroke from a number of first strokes, and use the second stroke that matches each first target stroke in the number of second strokes as the second target stroke; use each first stroke except each first target stroke in the number of first strokes as the third target stroke, and use each second stroke except each second target stroke in the number of second strokes as the fourth target stroke; determine the first difference stroke and the second difference stroke based on each third target stroke and each fourth target stroke.

[0117] In some embodiments, based on each third target stroke and each fourth target stroke, a first difference stroke and a second difference stroke are determined, including: based on each fourth target stroke in the correct text, each third target stroke is judged to be connected, and a connected stroke judgment result of each third target stroke is determined; the connected stroke judgment result is used to characterize whether the third target stroke is a connected stroke; based on the connected stroke judgment result of each third target stroke, the first difference stroke and the second difference stroke are determined.

[0118] In some embodiments, each second target stroke is judged to be connected based on each fourth target stroke in the correct text to determine the result of the connected stroke judgment of each second target stroke, including: taking each fourth target stroke in the correct text as the current stroke in turn, selecting a stroke whose distance to the current stroke is less than or equal to a preset threshold from the remaining fourth target strokes or several second strokes, and treating the stroke less than or equal to the preset threshold and the current stroke as a whole stroke; performing stroke matching on each third target stroke with each stroke as a whole to determine whether there is a third target stroke that matches each stroke as a whole; and determining the third target stroke that matches the stroke as a whole as a connected stroke.

[0119] In some embodiments, based on the connected stroke judgment results of each third target stroke, the first difference stroke and the second difference stroke are determined, including: determining the non-connected strokes in each third target stroke as the first difference stroke, and determining the fourth target stroke in the stroke as a whole that does not match each connected stroke as the second difference stroke.

[0120] In some embodiments, before determining the first difference stroke and the second difference stroke based on each third target stroke and each fourth target stroke, it also includes: selecting the second target stroke that is the first radical of the correct character from each second target stroke in the correct character, and using it as the first radical stroke; selecting each first target stroke that matches each first radical stroke from each first target stroke in the character to be recognized as the second radical stroke; performing radical matching based on each first radical stroke and each second radical stroke to determine the radical matching result; and adjusting the first target stroke in the character to be recognized and the second target stroke in the correct character based on the radical matching result.

[0121] In some embodiments, radical matching is performed based on each first radical stroke and each second radical stroke to determine the radical matching result, including: determining the radical matching result based on the stroke feature difference between all first radical strokes in the first radical and all second radical strokes in the second radical, the radical matching result is used to characterize whether the first radical matches the second radical; based on the radical matching result, adjusting the first target stroke in the to-be-recognized text and the second target stroke in the correct text, including: in response to the first radical not matching the second radical, adjusting the second radical stroke in each first target stroke to a third target stroke, and adjusting the first radical stroke in each second target stroke to a fourth target stroke.

[0122] In some embodiments, the wrong character recognition model includes an encoder and a wrong character recognizer; the recognition module 83 uses the wrong character recognition model to perform wrong character recognition based on the first difference stroke and the second difference stroke to determine the target recognition result of the text to be recognized, including: using the encoder to respectively extract the first feature of the first difference image and the second feature of the second difference image; the first difference image is a stroke image formed by rendering the first difference stroke, and the second difference image is a stroke image formed by rendering the second difference stroke; using the wrong character recognizer to perform wrong character recognition based on the first feature and the second feature to determine the first recognition result of the text to be recognized; based on the first recognition result, determine the target recognition result of the text to be recognized.

[0123] In some embodiments, the wrong character recognition model also includes a stroke category identifier, which is used to recognize the text image corresponding to the text to be recognized and determine the first stroke category sequence of the text to be recognized; before determining the target recognition result of the text to be recognized based on the first recognition result, the method also includes: determining the second recognition result of the text to be recognized based on the first difference between the first stroke category sequence of the text to be recognized and the second stroke category sequence of the correct text; based on the first recognition result, determining the target recognition result of the text to be recognized, including: combining the first recognition result and the second recognition result to obtain the target recognition result.

[0124] In some embodiments, a wrong character identifier is used to perform wrong character recognition based on a first feature and a second feature to determine a first recognition result of the text to be recognized, including: using the wrong character identifier to perform attention processing on the first feature and the second feature to obtain an attention weight matrix; performing wrong character recognition based on the attention weight matrix to determine a first recognition result of the text to be recognized.

[0125] In some embodiments, before performing misprint recognition based on the attention weight matrix to determine the first recognition result of the text to be recognized, the method also includes: obtaining an encoding vector of the correct text; using a misprint recognizer to perform attention processing on the first feature and the second feature based on the encoding vector to obtain an attention weight matrix; or, performing misprint recognition based on the attention weight matrix and the encoding vector to determine the first recognition result of the text to be recognized.

[0126] In some embodiments, before the recognition module 83 uses the wrong character recognition model to perform wrong character recognition based on the first difference stroke and the second difference stroke, it also includes the following training steps for the wrong character recognition model: obtaining a number of sample data; each sample data includes a sample image of the sample text to be recognized, a first sample difference image and a second sample difference image; the first sample difference image is a stroke image corresponding to the sample difference stroke in the sample text to be recognized, and the second sample difference image is a stroke image corresponding to the sample difference stroke in the correct sample text corresponding to the sample text to be recognized, and each sample difference stroke is a stroke that is not successfully matched between the sample text to be recognized and the correct sample text; for each sample data, the sample image is processed by the stroke category identifier in the wrong character recognition model to obtain a third stroke category sequence of the sample text to be recognized, and the first sample difference image and the second sample difference image are processed by the wrong character recognizer in the wrong character recognition model to obtain a sample recognition result of the sample text to be recognized; determine the second difference between the third stroke category sequence and the fourth stroke category sequence of the correct sample text, and determine the third difference between the sample recognition result and the true result; use the comprehensive difference determined based on the first difference and the second difference to adjust the network parameters of the wrong character recognition model.

[0127] In some embodiments, before the acquisition module 81 acquires the first stroke sequence of the text to be recognized and the second stroke sequence of the correct text corresponding to the text to be recognized, the method also includes: acquiring an image of the text to be recognized, the image of the text to be recognized including a text image of the text to be recognized; processing the image of the text to be recognized using a wrong character recognition model to obtain a third recognition result of the text to be recognized; in response to the third recognition result indicating that the text to be recognized is incorrect, executing the steps of acquiring the first stroke sequence of the text to be recognized and the second stroke sequence of the correct text corresponding to the text to be recognized and subsequent steps.

[0128] See also Fig. 9 , Fig. 91 is a schematic diagram of a framework of an electronic device according to an embodiment of the present application. In this embodiment, the electronic device 90 includes a memory 91 and a processor 92 coupled to each other.

[0129] The memory 91 stores program instructions, and the processor 92 is used to execute the program instructions stored in the memory 91 to implement the steps of any of the above-mentioned method implementation methods. In a specific implementation scenario, the electronic device 90 may include, but is not limited to: a microcomputer, a server, and in addition, the electronic device 90 may also include a mobile device such as a laptop computer and a tablet computer, which is not limited here.

[0130] Specifically, the processor 92 is used to control itself and the memory 91 to implement the steps of any of the above-mentioned embodiments. The processor 92 can also be called a CPU (Central Processing Unit). The processor 92 may be an integrated circuit chip with signal processing capabilities. The processor 92 can also be a general-purpose processor, a digital signal processor (Digital Signal Processor, DSP), an application-specific integrated circuit (Application Specific Integrated Circuit, ASIC), a field-programmable gate array (Field-Programmable Gate Array, FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc. In addition, the processor 92 can be implemented by an integrated circuit chip.

[0131] See also Fig.10 , Fig.10 It is a schematic diagram of the framework of the computer-readable storage medium provided by the present application. The computer-readable storage medium 100 of the embodiment of the present application stores a program instruction 101, and the program instruction 101 is executed to implement the method provided by any embodiment of the above method and any non-conflicting combination. Among them, the program instruction 101 can form a program file and be stored in the above-mentioned computer-readable storage medium 100 in the form of a software product, so that a computer device (which can be a personal computer, a server, or a network device, etc.) executes all or part of the steps of each implementation method of the present application. The aforementioned computer-readable storage medium 100 includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a disk or an optical disk, or a terminal device such as a computer, a server, a mobile phone, and a tablet.

[0132] The above scheme can respectively determine the first difference stroke that has not been successfully matched in the character to be recognized and the second difference stroke that has not been successfully matched in the correct character by performing stroke matching on the first stroke sequence of the character to be recognized and the second difference stroke that has not been successfully matched in the correct character. The character is composed of a stroke sequence, and the first difference stroke and the second difference stroke represent the difference strokes between the character to be recognized and the correct character, that is, the wrong character recognition model of the present application is recognized on the basis of the difference strokes. Compared with the wrong character recognition model that recognizes on the basis of the entire character to be recognized and the corresponding correct character, and needs to extract the difference strokes by the model itself, the above-mentioned method of directly providing the difference strokes to the wrong character recognition model of the present application is conducive to the wrong character recognition model to focus more on the difference in strokes between the character to be recognized and the correct character for wrong character recognition, which is conducive to improving the accuracy of wrong character recognition.

[0133] In some embodiments, the functions or modules included in the device provided by the embodiments of the present disclosure can be used to execute the method described in the above method embodiments. The specific implementation can refer to the description of the above method embodiments, and for the sake of brevity, it will not be repeated here.

[0134] The above description of various embodiments tends to emphasize the differences between the various embodiments. The same or similar aspects can be referenced to each other, and for the sake of brevity, they will not be repeated herein.

[0135] In the several embodiments provided in the present application, it should be understood that the disclosed methods and devices can be implemented in other ways. For example, the device implementation described above is only schematic. For example, the division of modules or units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, and the indirect coupling or communication connection of devices or units can be electrical, mechanical or other forms.

[0136] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0137] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0138] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) or a processor (processor) to perform all or part of the steps of each implementation method of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program code.

[0139] The above description is only an implementation method of the present application, and does not limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly used in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A method for identifying wrong characters, characterized in that: The method comprises: Acquire a first stroke sequence of a character to be recognized and a second stroke sequence of a correct character corresponding to the character to be recognized; Performing stroke matching on the first stroke sequence and the second stroke sequence, and respectively determining a first difference stroke that is not successfully matched in the character to be recognized and a second difference stroke that is not successfully matched in the correct character; The wrong character recognition model is used to perform wrong character recognition based on the first difference strokes and the second difference strokes to determine a target recognition result of the character to be recognized, wherein the target recognition result is used to indicate whether the character to be recognized is a wrong character.

2. The method according to claim 1, characterized in that: The first stroke sequence is determined based on a plurality of groups of trajectory point sequences of the to-be-recognized characters written online by a user, and each group of trajectory point sequences is used to determine a stroke in the to-be-recognized characters.

3. The method according to claim 1 or 2, characterized in that: The first stroke sequence includes a plurality of first strokes used to form the character to be recognized, and the second stroke sequence includes a plurality of second strokes used to form the correct character; The performing stroke matching on the first stroke sequence and the second stroke sequence to respectively determine a first difference stroke that is not successfully matched in the character to be recognized and a second difference stroke that is not successfully matched in the correct character, comprises: For each second stroke, find a first target stroke matching the second stroke from the plurality of first strokes, and use the second stroke matching each first target stroke from the plurality of second strokes as the second target stroke; taking each first stroke among the plurality of first strokes except each first target stroke as a third target stroke, and taking each second stroke among the plurality of second strokes except each second target stroke as a fourth target stroke; Based on each third target stroke and each fourth target stroke, the first difference stroke and the second difference stroke are determined.

4. The method according to claim 3, characterized in that The determining the first difference strokes and the second difference strokes based on each third target stroke and each fourth target stroke includes: Performing connected stroke judgment on each of the third target strokes based on each of the fourth target strokes in the correct text, and determining a connected stroke judgment result of each of the third target strokes; the connected stroke judgment result is used to indicate whether the third target stroke is a connected stroke; Based on the connected stroke judgment result of each of the third target strokes, the first difference stroke and the second difference stroke are determined.

5. The method according to claim 4, characterized in that The performing connected stroke judgment on each of the second target strokes based on each of the fourth target strokes in the correct text to determine the connected stroke judgment result of each of the second target strokes includes: sequentially taking each of the fourth target strokes in the correct character as a current stroke, selecting a stroke whose distance to the current stroke is less than or equal to a preset threshold from the remaining fourth target strokes or a plurality of second strokes, and taking the stroke whose distance to the current stroke is less than or equal to the preset threshold and the current stroke as a whole stroke; Perform stroke matching on each of the third target strokes and each of the strokes as a whole to determine whether there is a third target stroke that matches each of the strokes as a whole; The third target stroke that matches the stroke as a whole is determined as a connected stroke.

6. The method according to claim 4, characterized in that The determining the first difference stroke and the second difference stroke based on the result of connecting stroke judgment of each of the third target strokes comprises: The non-connected strokes in each of the third target strokes are determined as the first difference strokes, and the fourth target strokes in the stroke group that do not match each of the connected strokes are determined as the second difference strokes.

7. The method according to claim 3, characterized in that Before determining the first difference strokes and the second difference strokes based on each third target stroke and each fourth target stroke, the method further includes: From each of the second target strokes in the correct character, select a second target stroke that is the first radical of the correct character and use it as the first radical stroke; Selecting first target strokes matching first radical strokes from first target strokes in the character to be recognized as second radical strokes; Perform radical matching based on each of the first radical strokes and each of the second radical strokes to determine a radical matching result; Based on the radical matching result, the first target stroke in the to-be-recognized character and the second target stroke in the correct character are adjusted.

8. The method according to claim 7, characterized in that The performing radical matching based on each of the first radical strokes and each of the second radical strokes to determine the radical matching result includes: Determine the radical matching result based on the stroke feature difference between all first radical strokes in the first radical and all second radical strokes in the second radical, wherein the radical matching result is used to indicate whether the first radical matches the second radical; The adjusting the first target strokes in the to-be-recognized character and the second target strokes in the correct character based on the radical matching result includes: In response to the first radical not matching the second radical, each of the second radical strokes in the first target strokes is adjusted to the third target stroke, and each of the first radical strokes in the second target strokes is adjusted to the fourth target stroke.

9. The method according to claim 1, characterized in that: The wrong character recognition model includes an encoder and a wrong character recognizer; The step of using the wrong character recognition model to perform wrong character recognition based on the first difference strokes and the second difference strokes to determine a target recognition result of the character to be recognized includes: Extracting a first feature of a first difference image and a second feature of a second difference image respectively by using the encoder; the first difference image is a stroke image formed by rendering the first difference strokes, and the second difference image is a stroke image formed by rendering the second difference strokes; Using the wrong character identifier to identify the wrong character based on the first feature and the second feature, and determining a first recognition result of the character to be identified; Based on the first recognition result, a target recognition result of the to-be-recognized text is determined.

10. The method according to claim 9, characterized in that The wrong character recognition model further includes a stroke category identifier, which is used to recognize a character image corresponding to the character to be recognized and determine a first stroke category sequence of the character to be recognized; Before determining a target recognition result of the to-be-recognized character based on the first recognition result, the method further includes: Determining a second recognition result of the character to be recognized based on a first difference between the first stroke category sequence of the character to be recognized and the second stroke category sequence of the correct character; The step of determining a target recognition result of the to-be-recognized text based on the first recognition result includes: The target recognition result is obtained by combining the first recognition result and the second recognition result.

11. The method according to claim 9, characterized in that The step of using the wrong character identifier to identify the wrong character based on the first feature and the second feature to determine a first recognition result of the character to be identified includes: Using the wrong character identifier to perform attention processing on the first feature and the second feature to obtain an attention weight matrix; Wrong characters are identified based on the attention weight matrix to determine a first recognition result of the text to be identified.

12. The method according to claim 11, characterized in that Before performing wrong character recognition based on the attention weight matrix and determining a first recognition result of the to-be-recognized character, the method further includes: Obtaining an encoding vector of the correct text; The method comprises: using the wrong character identifier to perform attention processing on the first feature and the second feature based on the encoding vector to obtain the attention weight matrix; or performing wrong character recognition based on the attention weight matrix and the encoding vector to determine a first recognition result of the text to be recognized.

13. The method according to claim 1, characterized in that Before using the wrong character recognition model to recognize the wrong character based on the first difference strokes and the second difference strokes, the method further includes the following training steps for the wrong character recognition model: Acquire a number of sample data; each of the sample data includes a sample image of a sample text to be recognized, a first sample difference image, and a second sample difference image; the first sample difference image is a stroke image corresponding to a sample difference stroke in the sample text to be recognized, and the second sample difference image is a stroke image corresponding to a sample difference stroke in a correct sample text corresponding to the sample text to be recognized, and each of the sample difference strokes is a stroke that is not successfully matched between the sample text to be recognized and the correct sample text; For each sample data, the stroke category identifier in the wrong character recognition model is used to process the sample image to obtain a third stroke category sequence of the sample text to be recognized, and the wrong character recognizer in the wrong character recognition model is used to process the first sample difference image and the second sample difference image to obtain a sample recognition result of the sample text to be recognized; Determining a second difference between the third stroke category sequence and a fourth stroke category sequence of the correct sample character, and determining a third difference between the sample recognition result and a true result; The network parameters of the wrong character recognition model are adjusted using the comprehensive difference determined based on the first difference and the second difference.

14. The method according to claim 1, characterized in that Before obtaining the first stroke sequence of the character to be recognized and the second stroke sequence of the correct character corresponding to the character to be recognized, the method further includes: Acquire a text image to be recognized, wherein the text image to be recognized includes a text image of the text to be recognized; Processing the image of the text to be recognized by using the wrong character recognition model to obtain a third recognition result of the text to be recognized; In response to the third recognition result indicating that the character to be recognized is incorrect, the steps of acquiring a first stroke sequence of the character to be recognized, a second stroke sequence of a correct character corresponding to the character to be recognized, and subsequent steps are performed.

15. A device for identifying wrong characters, characterized in that: The device comprises: An acquisition module, used for acquiring a first stroke sequence of a character to be recognized and a second stroke sequence of a correct character corresponding to the character to be recognized; A stroke matching module, used for performing stroke matching on the first stroke sequence and the second stroke sequence, and respectively determining a first difference stroke that is not successfully matched in the character to be recognized and a second difference stroke that is not successfully matched in the correct character; The recognition module is used to use the wrong character recognition model to perform wrong character recognition based on the first difference stroke and the second difference stroke, and determine the target recognition result of the character to be recognized, wherein the target recognition result is used to indicate whether the character to be recognized is a wrong character.

16. An electronic device, characterized in that: comprising a memory and a processor coupled to each other, The memory stores program instructions; The processor is used to execute the program instructions stored in the memory to implement the method according to any one of claims 1 to 14.

17. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores program instructions that can be run by a processor, and the program instructions can be executed by the processor to implement the method according to any one of claims 1 to 14.

Images