Error character recognition method and device

Abstract

The invention relates to the technical field of computers, and provides an erroneous character recognition method and device.The number of various element prototypes contained in a target character is determined based on the matching degree between image features of the target character and the element prototypes, and the element prototypes refer to basic units forming the character; based on the image features and the number of various element prototypes, element decomposition is carried out on the target character to obtain an element sequence of the target character, and the element sequence refers to a basic unit sequence forming the target character; and determining a wrong character recognition result of the target character based on the element sequence. According to the method, the determination of the number of various element prototypes is not interfered by language information, so that the problem of relatively low erroneous character recognition precision possibly caused by excessive dependence or language information learned by a model in a training process in related technologies can be avoided.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a method and device for identifying wrong characters. Background Art

[0002] Handwritten typos are incorrect glyphs that occur during the handwriting process. Such errors may manifest as confusion of strokes, structures, or glyphs, which makes the originally intended word inconsistent with the final glyph, leading to misunderstanding or confusion of information. Therefore, it is necessary to identify handwritten typos.

[0003] Taking handwritten Chinese characters as an example, the current model mainly decomposes Chinese characters by radicals to obtain radical sequences, and then uses the radical sequences to identify whether the Chinese character is a wrong character. However, when the model performs radical decomposition reasoning, it tends to use the language information that has been learned to output the radical sequence of the wrong character corresponding to the correct character, which makes it easy to misjudge the wrong character as the correct character. Summary of the invention

[0004] The invention provides a method and device for identifying wrong characters, which are used to solve the defects in the prior art.

[0005] The present invention provides a method for identifying wrong characters, comprising the following steps: Based on the matching degree between the image features of the target text and each element prototype, the number of each element prototype contained in the target text is determined, where the element prototype refers to the basic unit constituting the text; Based on the image features and the number of prototypes of various elements, the target text is decomposed into elements to obtain an element sequence of the target text, where the element sequence refers to a basic unit sequence constituting the target text; Based on the element sequence, a wrong character recognition result of the target text is determined.

[0006] According to a method for identifying wrong characters provided by the present invention, the number of each type of element prototypes contained in the target character is determined based on the matching degree between the image features of the target character and each element prototype, including: Based on the matching degree between the image features and each element prototype, a heat map corresponding to each element prototype is generated, and each heat map is used to represent the probability of the corresponding element prototype existing in the target text; Based on the heat maps corresponding to all element prototypes, the number of each type of element prototype contained in the target text is obtained.

[0007] According to a method for identifying wrong characters provided by the present invention, the number of each type of element prototypes contained in the target text is obtained based on the heat map corresponding to all element prototypes, including: Group the heatmaps corresponding to all element prototypes and perform convolution operations on the heatmaps in each group independently; Based on each group of heat maps after convolution, the number of each type of element prototypes contained in the target text is determined.

[0008] According to a method for identifying wrong characters provided by the present invention, the heat maps corresponding to all element prototypes are grouped, and a convolution operation is performed independently on the heat map in each group, including: The heat maps corresponding to each element prototype are grouped separately, and the convolution operation is performed independently on the heat maps in each group.

[0009] According to a method for identifying wrong characters provided by the present invention, the element decomposition of the target text is performed based on the image features and the number of each type of element prototypes to obtain the element sequence of the target text, including: Based on the image features, predicting an initial element sequence of the target text; Based on the number of each type of element prototypes, the initial element sequence is modified to obtain the element sequence of the target text.

[0010] According to a method for identifying wrong characters provided by the present invention, the initial element sequence is corrected based on the number of each type of element prototypes to obtain the element sequence of the target text, including: Based on the number of each element prototype, the probability of each element prototype in the initial element sequence is modified; Based on the corrected probabilities of the prototypes of each element, the element sequence of the target text is obtained.

[0011] According to a method for identifying wrong characters provided by the present invention, the image feature of the target text is obtained by encoding the target text image, and the initial element sequence is obtained by decoding the image feature.

[0012] The present invention also provides a wrong character recognition device, comprising the following modules: A determination unit, configured to determine the number of each type of element prototypes contained in the target text based on the matching degree between the image features of the target text and each element prototype, wherein the element prototype refers to a basic unit constituting the text; A decomposition unit, configured to decompose the target text into elements based on the image features and the number of each type of element prototypes to obtain an element sequence of the target text, wherein the element sequence refers to a basic unit sequence constituting the target text; The recognition unit is used to determine the wrong character recognition result of the target text based on the element sequence.

[0013] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, any one of the above-mentioned methods for identifying wrong characters is implemented.

[0014] The present invention also provides a non-transitory computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the method for identifying wrong characters as described above is implemented.

[0015] The present invention also provides a computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the computer program implements any of the above-mentioned methods for identifying wrong characters.

[0016] The method and device for identifying wrong characters provided by the present invention are independent of language information in terms of the number of element prototypes of each type contained in the target text, because the number of element prototypes of each type contained in the target text is based on the result of the matching degree between the image features and the element prototypes. In other words, the number of element prototypes of each type is determined without interference from language information, thereby avoiding the problem of low wrong character recognition accuracy caused by excessive reliance on or learning of language information by the model in the training process in the related art. By combining the number of element prototypes of each type obtained independently of language information, the element prototypes predicted based on the image features are verified and corrected to ensure that the element sequence finally obtained is accurate, and then the wrong character recognition result can be accurately obtained based on the element sequence, thereby improving the accuracy of the wrong character recognition result. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0018] Figure 1 It is a schematic diagram of model radical decomposition in the related technology.

[0019] Figure 2 It is a flow chart of the wrong character recognition method provided by the present invention.

[0020] Figure 3 It is a flow chart of another wrong character recognition method provided by the present invention.

[0021] Figure 4 It is a schematic diagram of the working principle of the counter provided by the present invention.

[0022] Figure 5 It is a structural schematic diagram of the wrong character recognition device provided by the present invention.

[0023] Figure 6 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0024] To make the objectives, technical solutions and advantages of the present invention clearer, the technical solutions in the present invention will be clearly and completely described below with reference to the accompanying drawings in the present invention. Obviously, the described embodiments are some but not all of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments in the present invention without making creative efforts fall within the scope of protection of the present invention.

[0025] Handwritten Chinese character error correction mainly includes two major stages: "evaluation" and "correction". When a user submits a handwritten Chinese character sample, in the "evaluation" stage, the model needs to determine whether the sample is a misspelled character; if it is determined to be a misspelled character, then in the "correction" stage, the model needs to predict and give the correct Chinese character corresponding to the misspelled character to achieve the purpose of error correction.

[0026] Currently, handwritten Chinese character error correction models generally adopt a method based on radical modeling. The overall process is as follows: First, the model decomposes the handwritten Chinese character input by the user into a radical sequence. In the "evaluation" stage, based on the decomposed radical sequence, the model will query whether it belongs to the set of radical sequences of correct Chinese characters. If it does not belong, it is determined to be a misspelled character. Finally, combined with the decomposed radical sequence, the model will predict and give the correct Chinese character corresponding to the misspelled character.

[0027] Among them, Figure 1 is a schematic diagram of the model radical decomposition in the related art. As Figure 1 shown, when decomposing the radical sequence, the above model often has "bias", that is, when a misspelled character is input, although the image is clear, the model accuracy is not high, and it often outputs the radical sequence of the correct Chinese character corresponding to the misspelled character. The reason is that misspelled characters are unpredictable and infinite. Therefore, the model may not have seen some misspelled characters during training, but only the correct Chinese characters corresponding to them. The model will "memorize" the radical sequences of these correct Chinese characters, such as "⿱⿰屰月土", "⿱大⿱冖牛", that is, the model has learned the language information contained in these radical sequences. However, misspelled characters are contrary to the language information. Therefore, in the inference stage, if a misspelled character not seen during training is input, the model will tend to use the language information and output the radical sequence of the correct Chinese character corresponding to the misspelled character. Taking the misspelled character "塑" in Figure 1 as an example, the model has seen the correctly written "塑" during training, so it has learned the radical sequence of "⿱⿰屰月土"; in the test stage, when encountering the misspelled character "塑", the model still outputs "⿱⿰屰月土", although it is visually clear that the bottom of the character is "王" instead of "土".

[0028] In this regard, in order to solve the "bias" problem of the above-mentioned model, the present invention provides a method for identifying wrong characters, which can be applied to scenarios such as intelligent dictation and intelligent marking. In addition, the method is not limited to the identification of wrong characters in Chinese characters, but can also be expanded to the identification of wrong characters in other languages ​​that have a similar structural system (such as radical structure) to Chinese characters. For example, Japanese and Korean have a radical structure similar to that of Chinese characters, that is, the present invention can also be used for the identification of wrong characters in Japanese and Korean. In order to facilitate the understanding of the technical solution of the present invention, the following embodiments are all explained by taking the application to Chinese characters as an example.

[0029] in, Figure 2 It is a flow chart of the method for identifying wrong characters provided by the present invention, such as Figure 2 As shown, the method includes step 210, step 220 and step 230.

[0030] Step 210: Based on the matching degree between the image features of the target text and each element prototype, the number of each element prototype contained in the target text is determined, where the element prototype refers to the basic unit constituting the text.

[0031] Here, the target text can be understood as the handwritten Chinese characters to be recognized. The image features of the target text refer to a series of characteristics or attributes of the target text that are visually recognizable and used to distinguish and recognize the text. The image features of the target text can be extracted from a text image containing the target text, such as encoding the text image to obtain the image features of the target text.

[0032] In addition, the element prototype refers to the basic unit that constitutes a character. The element prototype can be a learnable prototype in the dictionary, which is learned from a large number of character samples through machine learning or deep learning algorithms and can represent common structures or components in characters. For Chinese characters, the element prototype can be understood as the radical prototype.

[0033] The matching degree between the image features of the target text and each element prototype is used to characterize the similarity between the target text and each element prototype. The higher the matching degree, the closer the target text is to the corresponding element prototype in terms of visual features, that is, the target text may contain the element prototype.

[0034] Since different element prototypes play different roles in the composition of text, and the target text may contain multiple different element prototypes, the number of each type of element prototype contained in the target text can be determined based on the matching degree between the image features of the target text and each element prototype. For example, if the matching degree between the image features of the target text and a radical prototype is greater than a preset threshold, it can be considered that the target text contains the corresponding radical prototype.

[0035] Among them, the target text here refers to a single handwritten Chinese character. If it is necessary to identify misspelled characters in multiple handwritten Chinese characters, the image containing multiple handwritten Chinese characters can be segmented, and each sub-image obtained only includes one handwritten Chinese character, that is, the handwritten Chinese character in each sub-image can be regarded as the handwritten Chinese character in step 210 above.

[0036] Since the number of various element prototypes contained in the target text is based on the statistical result of the matching degree between the image features and the element prototypes, the number of various element prototypes contained in the target text is independent of the specific text content. Furthermore, the number of various element prototypes contained in the target text is independent of the language information. That is to say, the determination of the number of various element prototypes is not interfered by the language information, thereby avoiding the problem of misspelled character recognition accuracy that may be caused by the model relying too much on or learning the language information during the training process in the related technology. That is to say, step 210 directly determines the number of elements based on the image features and the matching degree of the element prototypes, which can more objectively and accurately reflect the actual element prototype composition of the target text and improve the accuracy and reliability of misspelled character recognition.

[0037] Step 220: Based on the image features and the number of various element prototypes, decompose the target text into an element sequence of the target text, where the element sequence refers to the sequence of basic units that make up the target text.

[0038] Specifically, element decomposition means decomposing the target text into multiple basic units, that is, decomposing the target text into multiple element prototypes. The element sequence refers to the sequence of basic units that make up the target text. For handwritten Chinese characters, the element sequence refers to the radical sequence that makes up the handwritten Chinese character. For example, for the character "塑", its corresponding element sequence is "⿱⿰屰月土", where "⿱" represents the combination method of the upper and lower structures, that is, combining "屰月" and "土" in the upper and lower structures, and "⿰" represents the combination method of the left and right structures, that is, combining "屰" and "月" in the left and right structures.

[0039] If the target text is a misspelled character, there must be a radical error in the radical prototypes that make up the target text. As Figure 1 shown, the radicals of the correct character "塑" include "屰月土", but the radicals of the misspelled character include "屰月王", that is, in the misspelled character, "土" is wrongly written as "王".

[0040] When decomposing the target text into elements based on the image features, usually the language information learned by the model is used to decompose the target text. When the model encounters a misspelled character that it has not seen during the training process, it will introduce the learned language information, that is, the model has a "bias" and is likely to recognize the misspelled character as a correct character. For example, when encountering Figure 1 the misspelled character "塑" shown, it is likely to recognize the incorrect radical "王" as the correct radical "土".

[0041] To avoid the "bias" problem brought by the above model, in the embodiment of the present invention, when performing element decomposition, the number of various element prototypes independent of language information acquisition is introduced. This number is used to represent the number of element prototypes actually included in the target text. Based on the number of various element prototypes, the element prototypes predicted based on image features can be corrected, and then the element sequence of the target text can be accurately obtained. For example, when performing element decomposition based on the Figure 1 image feature of the misspelled character "塑" in the middle, the predicted element sequence of the target text is "⿱⿰屰月土". However, if the number of the element prototype "土" determined based on step 210 is 0, it can be determined that when performing element decomposition based on image features, the misspelled character "塑" is misrecognized as a correct character, and then its elements are wrongly decomposed into "⿱⿰屰月土".

[0042] It can be seen that introducing the number of various element prototypes is to solve the "bias" problem that may be introduced in the recognition process of the model, that is, the model may tend to recognize unseen misspelled characters as correct characters seen in the training process. By combining the number of various element prototypes independent of language information acquisition, the element prototypes predicted based on image features can be verified and corrected to ensure that the finally obtained element sequence is accurate.

[0043] Step 230: Determine the misspelling recognition result of the target text based on the element sequence.

[0044] Specifically, the element sequence is the basic unit sequence that constitutes the target text. This sequence is used to represent the element prototypes included in the target text and the combination method of each element prototype (the combination method includes upper and lower structure combinations, left and right structure combinations, etc.). If any element prototype is incorrect and / or the combination method of the element prototypes is incorrect, it indicates that the target text is written incorrectly, that is, the misspelling recognition result is that the target text is a misspelled word.

[0045] Based on this, the element sequence of the target text can be compared with the element sequences corresponding to each correct text stored in the sequence library. If the element sequence of the target text matches the element sequence corresponding to any correct text in the sequence library, it indicates that the target text is written correctly, that is, the misspelling recognition result is that the target text is a correct word; if the element sequence of the target text does not match the element sequences corresponding to all correct texts in the sequence library, it indicates that the target text is written incorrectly, that is, the misspelling recognition result is that the target text is a misspelled word. Here, the match means that the element prototypes included in the target text are the same as the element prototypes included in the correct text, and the combination method of each element prototype in the target text is the same as the combination method of each element prototype in the correct text.

[0046] In addition, the wrong character recognition result can also be the probability that the target character is a wrong character or a correct character. Exemplarily, the similarity between the element sequence of the target character and the element sequence corresponding to each correct character stored in the sequence library can be determined, and the wrong character recognition result can be determined based on the maximum similarity. For example, if the maximum similarity is greater than a threshold, it indicates that the target character is likely to be a correct character, and the value corresponding to the maximum similarity can be used as the probability that the target character is a correct character; if the maximum similarity is less than or equal to the threshold, it indicates that the target character is likely to be a wrong character, and the value corresponding to (1-maximum similarity) can be used as the probability that the target character is a wrong character.

[0047] In the method for identifying wrong characters provided by the embodiment of the present invention, since the number of each type of element prototypes contained in the target text is based on the result of the matching degree between the image features and the element prototypes, the number of each type of element prototypes contained in the target text is independent of the language information, that is, the determination of the number of each type of element prototypes is not interfered by the language information, thereby avoiding the problem of low wrong character recognition accuracy caused by the model in the related art over-relying on or learning the language information during the training process. By combining the number of each type of element prototypes obtained independently of the language information, the element prototypes predicted based on the image features are verified and corrected to ensure that the final element sequence is accurate, and then the wrong character recognition result can be accurately obtained based on the element sequence, thereby improving the accuracy of the wrong character recognition result.

[0048] Based on the above embodiment, the number of each type of element prototypes contained in the target text is determined based on the matching degree between the image features of the target text and each element prototype, including: Based on the matching degree between the image features and each element prototype, a heat map corresponding to each element prototype is generated. Each heat map is used to represent the probability of the corresponding element prototype existing in the target text. Based on the heat maps corresponding to all element prototypes, the number of each type of element prototypes contained in the target text is obtained.

[0049] Specifically, considering that in a text image with a complex background, the target text may be confused with the background elements, resulting in a large amount of noise in the extracted image features, or the illumination change may cause part of the target text area to be too dark or too bright, affecting the quality of image feature extraction. If the element prototypes contained in the target text in the text image are directly identified based on image features containing a large amount of noise or image features of poor quality, it may interfere with the recognition accuracy, thereby affecting the number of various element prototypes contained in the identified target text.

[0050] Furthermore, considering that the heat map provides rich spatial distribution information by showing the probability of the existence of a specific element prototype in each pixel or image area, the spatial distribution information helps to distinguish the target text from the background elements and reduce the interference of the background elements on the identification of the element prototype. In addition, when there are lighting changes in the text image, some areas of the target text may become too bright or too dark, making it difficult to extract image features. However, the heat map can reduce the impact of lighting changes on recognition to a certain extent by calculating the feature matching degree of the local area. Even if the lighting conditions in some areas are not good, the heat map can still infer the feature information of the lighting change area based on the features of other areas.

[0051] Based on this, when determining the number of each type of element prototype contained in the target text, the embodiment of the present invention first generates a heat map corresponding to each element prototype based on the matching degree between the image feature and each element prototype. The heat map can characterize the probability of the corresponding element prototype existing in the target text. For example, a high-brightness (or warm-toned) area indicates that the corresponding element prototype is likely to exist in the area, while a low-brightness (or cold-toned) area indicates that the corresponding element prototype is likely not to exist in the area. Exemplarily, the matching degree between the image feature and the feature vector corresponding to each element prototype can be determined, and a heat map of the corresponding element prototype can be generated based on the matching degree between the two. The value range of the heat map is [0,1]. The larger the value, the higher the probability that the corresponding element prototype exists in the target text.

[0052] Since the heat map of each element prototype is used to characterize the probability of the corresponding element prototype existing in the target text, it is possible to determine whether the corresponding element prototype exists in the target text based on the heat map of each element prototype. For example, a corresponding threshold can be set, and when the heat map value is greater than the threshold, it is considered that the corresponding element prototype exists in the target text, otherwise, the corresponding element prototype does not exist.

[0053] After identifying whether the target text has each element prototype, the number of each element prototype existing is counted to obtain the number of each element prototype contained in the target text.

[0054] Based on any of the above embodiments, based on the heat maps corresponding to all element prototypes, the number of each type of element prototypes contained in the target text is obtained, including: Group the heatmaps corresponding to all element prototypes and perform convolution operations on the heatmaps in each group independently; Based on the heat maps after each group of convolution, the number of prototypes of each type of element contained in the target text is determined.

[0055] Specifically, considering that the heatmap generation algorithm may produce inaccurate results when dealing with complex data or boundary conditions, and these results are manifested as noise in the heatmap, which may affect the subsequent determination of the number of element prototypes. Therefore, the embodiments of the present invention introduce a convolution operation to smooth the heatmap, remove this noise, enhance the spatial continuity of the heatmap, and make the representation of the heatmap more accurate and reliable.

[0056] In addition, considering that if the convolution operation is performed on all heatmaps in one channel, the convolution kernel will act on the heatmaps of all element prototypes simultaneously, extracting their common or unique language features. Since the convolution kernel is reused throughout the input data, it can learn the common features between different element prototypes, and these features may reflect a certain structure or pattern in the language. Furthermore, the convolution kernel can learn the language information between different element prototypes. In this way, the same problem of the above-mentioned model "bias" will exist.

[0057] Based on this, the embodiments of the present invention group the heatmaps corresponding to all element prototypes and independently perform convolution operations on the heatmaps in each group. Thus, the heatmaps in each group remain independent during the convolution process, that is, when the heatmaps in a certain group are subjected to the convolution operation, they will not learn the language information of the element prototypes in other groups, ensuring that the feature extraction of the heatmaps in each group is independent of the language information of other groups. Furthermore, based on the heatmaps after convolution in each group, the number of various element prototypes contained in the target text can be analyzed and determined more precisely, avoiding the "bias" caused by the model learning the common features between different element prototypes, and realizing the accurate statistics of the number of element prototypes of the target text.

[0058] Exemplarily, when grouping the heatmaps corresponding to all element prototypes, the heatmaps corresponding to the element prototypes with similar or related semantic categories can be grouped into one group, so that when grouping the heatmaps in the same group, the mutual interference of feature information in different groups can be avoided. For example, "wood" is usually related to trees and plants, and "氵" is usually related to water and liquids. Since "wood" and "氵" have a certain correlation in semantic categories, the heatmaps corresponding to "wood" and "氵" can be divided into the same group.

[0059] Based on any of the above embodiments, grouping the heatmaps corresponding to all element prototypes and independently performing convolution operations on the heatmaps in each group includes: Grouping the heatmaps corresponding to each element prototype separately and independently performing convolution operations on the heatmaps in each group.

[0060] Specifically, the heatmaps corresponding to each element prototype are grouped separately, which means creating a separate channel for each heatmap and performing convolution operations on the corresponding heatmaps in the separate channels. In this way, the heatmaps in each channel will not affect each other, and the convolution kernel will not learn the language information in different element prototypes, ensuring that the feature extraction of each group of heatmaps is independent of the feature information of other groups, avoiding the "bias" caused by learning the common features between different element prototypes, and ensuring that the convolution operation can focus on the unique features of each element prototype, so as to more accurately count the number of various element prototypes contained in the target text.

[0061] Based on any of the above embodiments, the target text is decomposed into elements based on the image features and the number of prototypes of various elements to obtain an element sequence of the target text, including: Based on the image features, the initial element sequence of the target text is predicted; Based on the number of prototypes of each type of element, the initial element sequence is modified to obtain the element sequence of the target text.

[0062] Specifically, when the target text is decomposed into elements based on image features, the language information learned by the model is usually used to decompose the target text into elements. When the model encounters a wrong character that has not been seen during the training process, it will introduce the learned language information, that is, the model has a "bias" and is prone to identifying the wrong character as the correct character. In other words, the initial element sequence predicted based on image features may contain incorrect element prototype recognition results.

[0063] In order to avoid the "bias" problem brought by the above-mentioned model, the embodiment of the present invention introduces the number of various element prototypes independent of the acquisition of language information when performing element decomposition. This number is used to characterize the number of element prototypes actually contained in the target text. Based on the number of various element prototypes, the initial element sequence obtained based on the prediction of image features can be corrected, so as to accurately obtain the element sequence of the target text.

[0064] Exemplarily, the initial element sequence may include the predicted probability of each element prototype existing in the target text. If the number of any element prototype obtained above is large, it indicates that the probability of the corresponding element prototype existing in the target text is high. In this case, the predicted probability of the corresponding element prototype in the initial element sequence can be increased. Similarly, if the number of any element prototype is small, it indicates that the probability of the corresponding element prototype existing in the target text is low. In this case, the predicted probability of the corresponding element prototype in the initial element sequence can be reduced, thereby correcting the initial element sequence and obtaining an accurate element sequence of the target text.

[0065] Based on any of the above embodiments, based on the number of each type of element prototypes, the initial element sequence is corrected to obtain the element sequence of the target text, including: Based on the number of element prototypes of each type, the probability of each element prototype in the initial element sequence is corrected; Based on the corrected probabilities of each element prototype, the element sequence of the target text is obtained.

[0066] Specifically, the probability of each element prototype in the initial element sequence is used to reflect the possibility that each element prototype is contained in the target text. Given that the initial element sequence is obtained based on the prediction of image features, and the language information learned by the model during the training phase is incorporated into this prediction process, this may lead to incorrect characters being misjudged as correct characters. In other words, the initial element sequence generated based on image feature prediction may contain incorrect element prototype recognition results, that is, the probability values ​​of each element prototype in the initial element sequence may have errors.

[0067] To address this problem, the embodiment of the present invention introduces a number of element prototypes that is independent of the language information acquisition process, and the number is used to represent the number of element prototypes actually contained in the target text. By using this number, the probability value of each element prototype in the initial element sequence can be corrected. Based on the corrected probability value of each element prototype, the element sequence of the target text can be accurately determined.

[0068] Specifically, if the number of an element prototype is large, it means that the element prototype is more likely to appear in the target text. At this time, the prediction probability of the corresponding element prototype in the initial element sequence can be appropriately increased. On the contrary, if the number of an element prototype is small, it means that the element prototype is less likely to appear in the target text. At this time, the prediction probability of the corresponding element prototype in the initial element sequence can be appropriately reduced. In this way, the initial element sequence can be corrected to obtain an accurate target text element sequence.

[0069] Based on any of the above embodiments, the image feature of the target text is obtained by encoding the target text image, and the initial element sequence is obtained by decoding the image feature.

[0070] Specifically, the target text image refers to an image containing the target text. Encoding refers to the process of converting the target text image into a format or data structure that can represent the key information of the image. This format or data structure can be understood as "image features". The purpose of encoding is to remove redundant information from the target text image and extract features that are useful for wrong character recognition for subsequent processing. Decoding refers to the process of converting the encoded image features into the initial element sequence.

[0071] Exemplarily, the target text image can be input into an encoder, which encodes the target text image to obtain the image features of the target text. The image features of the target text are input into a decoder, which decodes the image features to obtain an initial element sequence. In addition, the decoder can also correct the initial element sequence based on the number of various element prototypes to obtain the element sequence of the target text.

[0072] Based on any of the above embodiments, Figure 3 is a schematic flowchart of another typo recognition method provided by the present invention. As Figure 3 shown, this method performs typo recognition on the target text through a typo recognition model, and the typo recognition model includes an encoder, a counter, and a decoder. Among them, Figure 4 is a schematic diagram of the working principle of the counter provided by the present invention. There is a dictionary inside the counter, and different radical prototypes (Learnable Prototype) are stored in this dictionary.

[0073] Among them, the encoder is used to encode the input target text image to obtain the image features F of the target text, and input the image features F of the target text into the counter. The counter first performs a linear transformation (Linear) on the image features F, and then performs a dot product (Dot Product) with the radical prototype (Learnable Prototype), and then passes through an activation function (such as the Sigmoid activation function) to obtain the heat map corresponding to the radical prototype. As Figure 4 shown, for the radicals existing in the image, such as "female" and "strength", the maximum value of the heat map is nearly 1, while the heat map of the non-existing radical "again" is almost all close to 0. Therefore, by taking the global maximum value (Global Max Pooling) of the heat map, a probability vector can be obtained, and each value of this probability vector represents the probability of the corresponding radical existing in the target text. As Figure 4 the 0.9 in the probability vector in represents that the probability of "female" existing in the target text is 0.9.

[0074] In order to obtain the number of each radical, perform a group convolution on the heat map, that is, use a separate channel for each heat map for group convolution, avoiding the mutual influence of different heat maps, resulting in the counter learning the language knowledge in the radical sequence. Finally, use global average pooling (Global Average Pooling) to obtain a counting vector. Among them, each value in the counting vector represents the number of each radical in the target text. As Figure 4 the "0.0" in the counting vector in represents that there are 0 "again" in the target text, that is, "again" does not exist in the target text.

[0075] After that, the count vector is linearly transformed and input into the decoder. Based on the image feature F, the decoder obtains the radicals of each position step by step, obtains the radical sequence of the target text, and finally determines the wrong character recognition result of the target text based on the radical sequence of the target text. The radicals at each position are obtained based on the following steps: based on the image feature F, the predicted probability P of the radical at the current position is obtained, and the predicted probability P is multiplied element by element by Tanh (v) (v is the count vector) to obtain the pseudo probability Q corresponding to each radical, and the radical corresponding to the maximum pseudo probability is taken as the radical at the current position. That is, if the counter predicts a radical with a low probability, the decoder also reduces its predicted probability; conversely, if the counter predicts a radical with a high probability, the decoder also increases its predicted probability.

[0076] Among them, the counter is different from the decoder. The counter does not output in a step-by-step manner, but independently outputs the number of all radicals, thereby avoiding the interference of language information on the counter, and then assisting the decoder to accurately decompose the radicals of the target text and improve the accuracy of mischaracter recognition of the target text.

[0077] The following is a description of the wrong character recognition device provided by the present invention. The wrong character recognition device described below and the wrong character recognition method described above can be referenced to each other.

[0078] Based on any of the above embodiments, Figure 5 Schematic diagram of the structure of the wrong character recognition device provided by the present invention, such as Figure 5 As shown, the device comprises: A determination unit 510 is used to determine the number of each type of element prototypes contained in the target text based on the matching degree between the image features of the target text and each element prototype, where the element prototype refers to a basic unit constituting the text; A decomposition unit 520 is used to decompose the target text into elements based on the image features and the number of each element prototype to obtain an element sequence of the target text, where the element sequence refers to a basic unit sequence constituting the target text; The recognition unit 530 is used to determine the wrong character recognition result of the target text based on the element sequence.

[0079] Based on any of the above embodiments, the number of each type of element prototypes contained in the target text is determined based on the matching degree between the image feature of the target text and each element prototype, including: Based on the matching degree between the image features and each element prototype, a heat map corresponding to each element prototype is generated. Each heat map is used to represent the probability of the corresponding element prototype existing in the target text. Based on the heat maps corresponding to all element prototypes, the number of each type of element prototypes contained in the target text is obtained.

[0080] Based on any of the above embodiments, based on the heat maps corresponding to all element prototypes, the number of each type of element prototypes contained in the target text is obtained, including: Group the heatmaps corresponding to all element prototypes and perform convolution operations on the heatmaps in each group independently; Based on the heat maps after each group of convolution, the number of prototypes of each type of element contained in the target text is determined.

[0081] Based on any of the above embodiments, the heat maps corresponding to all element prototypes are grouped, and a convolution operation is performed independently on the heat maps in each group, including: The heat maps corresponding to each element prototype are grouped separately, and the convolution operation is performed independently on the heat maps in each group.

[0082] Based on any of the above embodiments, the target text is decomposed into elements based on the image features and the number of prototypes of various elements to obtain an element sequence of the target text, including: Based on the image features, the initial element sequence of the target text is predicted; Based on the number of prototypes of each type of element, the initial element sequence is modified to obtain the element sequence of the target text.

[0083] Based on any of the above embodiments, based on the number of each type of element prototypes, the initial element sequence is corrected to obtain the element sequence of the target text, including: Based on the number of element prototypes of each type, the probability of each element prototype in the initial element sequence is corrected; Based on the corrected probabilities of each element prototype, the element sequence of the target text is obtained.

[0084] Based on any of the above embodiments, the image feature of the target text is obtained by encoding the target text image, and the initial element sequence is obtained by decoding the image feature.

[0085] Figure 6 is a schematic diagram of the structure of the electronic device provided by the present invention, such as Figure 6As shown, the electronic device may include: a processor 610, a communication interface 620, a memory 630 and a communication bus 640, wherein the processor 610, the communication interface 620 and the memory 630 communicate with each other through the communication bus 640. The processor 610 may call the logic instructions in the memory 630 to execute the wrong character recognition method, which includes: determining the number of various element prototypes contained in the target text based on the matching degree between the image feature of the target text and each element prototype, the element prototype refers to the basic unit constituting the text; based on the image feature and the number of various element prototypes, performing element decomposition on the target text to obtain the element sequence of the target text, the element sequence refers to the basic unit sequence constituting the target text; based on the element sequence, determining the wrong character recognition result of the target text.

[0086] In addition, the logic instructions in the above-mentioned memory 630 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0087] On the other hand, the present invention also provides a computer program product, which includes a computer program. The computer program can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the wrong character recognition method provided by the above-mentioned methods, and the method includes: based on the matching degree between the image features of the target text and each element prototype, determining the number of each type of element prototype contained in the target text, and the element prototype refers to the basic unit that constitutes the text; based on the image features and the number of each type of element prototype, performing element decomposition on the target text to obtain the element sequence of the target text, and the element sequence refers to the basic unit sequence that constitutes the target text; based on the element sequence, determining the wrong character recognition result of the target text.

[0088] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to execute the method for identifying wrong characters provided by the above-mentioned methods, the method comprising: determining the number of each type of element prototype contained in the target text based on the matching degree between the image features of the target text and each element prototype, the element prototype referring to the basic unit constituting the text; performing element decomposition on the target text based on the image features and the number of each type of element prototype to obtain an element sequence of the target text, the element sequence referring to the basic unit sequence constituting the target text; and determining the wrong character recognition result of the target text based on the element sequence.

[0089] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0090] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0091] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for identifying wrong characters, characterized in that: include: Based on the matching degree between the image features of the target text and each element prototype, the number of each element prototype contained in the target text is determined, where the element prototype refers to the basic unit constituting the text; Based on the image features and the number of prototypes of various elements, the target text is decomposed into elements to obtain an element sequence of the target text, where the element sequence refers to a basic unit sequence constituting the target text; Based on the element sequence, a wrong character recognition result of the target text is determined.

2. The method for identifying wrong characters according to claim 1, characterized in that: The method of determining the number of each type of element prototypes contained in the target text based on the matching degree between the image features of the target text and each element prototype includes: Based on the matching degree between the image features and each element prototype, a heat map corresponding to each element prototype is generated, and each heat map is used to represent the probability of the corresponding element prototype existing in the target text; Based on the heat maps corresponding to all element prototypes, the number of each type of element prototype contained in the target text is obtained.

3. The method for identifying wrong characters according to claim 2, characterized in that: The heat map corresponding to all element prototypes is used to obtain the number of each type of element prototype contained in the target text, including: Group the heatmaps corresponding to all element prototypes and perform convolution operations on the heatmaps in each group independently; Based on each group of heat maps after convolution, the number of each type of element prototypes contained in the target text is determined.

4. The method for identifying wrong characters according to claim 3, characterized in that: The heat maps corresponding to all element prototypes are grouped, and the heat maps in each group are independently convolved, including: The heat maps corresponding to each element prototype are grouped separately, and the convolution operation is performed independently on the heat maps in each group.

5. The method for identifying wrong characters according to any one of claims 1 to 4, characterized in that: The step of performing element decomposition on the target text based on the image features and the number of each type of element prototypes to obtain an element sequence of the target text includes: Based on the image features, predicting an initial element sequence of the target text; Based on the number of each type of element prototypes, the initial element sequence is modified to obtain the element sequence of the target text.

6. The method for identifying wrong characters according to claim 5, characterized in that: The step of correcting the initial element sequence based on the number of each type of element prototypes to obtain the element sequence of the target text includes: Based on the number of each element prototype, the probability of each element prototype in the initial element sequence is modified; Based on the corrected probabilities of the prototypes of each element, the element sequence of the target text is obtained.

7. The method for identifying wrong characters according to claim 5, characterized in that: The image feature of the target text is obtained by encoding the target text image, and the initial element sequence is obtained by decoding the image feature.

8. A device for identifying wrong characters, characterized in that: include: A determination unit, configured to determine the number of each type of element prototypes contained in the target text based on the matching degree between the image features of the target text and each element prototype, wherein the element prototype refers to a basic unit constituting the text; A decomposition unit, configured to decompose the target text into elements based on the image features and the number of each type of element prototypes to obtain an element sequence of the target text, wherein the element sequence refers to a basic unit sequence constituting the target text; The recognition unit is used to determine the wrong character recognition result of the target text based on the element sequence.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that: When the processor executes the computer program, the method for identifying wrong characters according to any one of claims 1 to 7 is implemented.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method for identifying wrong characters according to any one of claims 1 to 7 is implemented.

11. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the method for identifying wrong characters according to any one of claims 1 to 7 is implemented.

Images