A text recognition method and device

Abstract

The present invention discloses a text recognition method and device, and relates to the field of computer technology. A specific implementation of the method includes acquiring a text image for recognition, obtaining a candidate character set and a corresponding character recognition probability; using a list of similar characters to expand the candidate character set, based on a character co-occurrence matrix, the expanded candidate character set, according to all character recognition probabilities and transition probabilities in each text line, to obtain a calculation result set, and using the text line corresponding to the maximum value in the calculation result set as the output text; generating a semantically similar word list from the output text through a word vector model, traversing the list of semantically similar words, and using the list of similar characters to filter out words including similar characters, so as to calculate the co-occurrence probability of the original word and the semantically similar and similar words, and outputting the similar words if the co-occurrence probability is less than a preset first probability threshold, so as to obtain the recognized text. Thus, the implementation of the present invention can solve the problems of low accuracy of existing text extraction and poor recognition effect.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a text recognition method and device. Background Art

[0002] At present, extraction technology has been pursuing high accuracy in the recognition stage and has been applied in various fields. In the existing text extraction, language models are used for recognition and selection is made in the recognition result candidate list.

[0003] In the process of implementing the present invention, the inventors found that there are at least the following problems in the prior art:

[0004] Generally speaking, in order to ensure that all texts are fed into the recognition framework, the recall rate is appropriately sacrificed in the detection stage, which results in a large amount of incorrect and unnecessary information in the text extraction results. In addition, the language model needs to combine text probability and transition probability to calculate the text string with the highest score through dynamic programming, which does not fully utilize semantic information. In addition, for specific applications, a set of overall frameworks is built, the degree of componentization is reduced, and customization is more difficult.

[0005] In addition, due to the complexity of the background, non-text decorative lines or textures may be recognized as characters in the text extraction results, which will reduce the proportion of correct characters in the recognition results. For some text extraction by line recognition, special text (such as vertical text) is not specially processed. In addition, whether it is English or Chinese text, a large number of similar characters make it difficult to make significant improvements in text extraction technology. Summary of the invention

[0006] In view of this, an embodiment of the present invention provides a text recognition method and device, which can solve the problems of low accuracy of existing text extraction and poor recognition effect.

[0007] To achieve the above object, according to one aspect of an embodiment of the present invention, a text recognition method is provided, comprising acquiring a text image for recognition, obtaining a candidate character set and corresponding character recognition probabilities;

[0008] The candidate character set is expanded by using the list of similar characters, and the expanded candidate character set is calculated based on the character co-occurrence matrix, according to the recognition probability and transition probability of all characters in each text line, and the text line corresponding to the maximum value in the calculation result set is used as the output text;

[0009] The output text is generated into a list of semantically similar words through a word vector model, the list of semantically similar words is traversed, and words including similar characters are screened out using a list of similar characters to calculate the co-occurrence probability of the original word and the semantically similar and similar words, and if the co-occurrence probability is less than a preset first probability threshold, the similar word is output to obtain the recognized text.

[0010] Optionally, after obtaining the text image for recognition, the following steps are included:

[0011] The text box position information is obtained by identification, and the proportion of characters in the text box whose character recognition probability is less than a preset second probability threshold is calculated. If the proportion is greater than the preset proportion threshold, the text box is deleted.

[0012] Optionally, after obtaining the candidate character set and the corresponding character recognition probability, the following steps are performed:

[0013] If the second character in the candidate character set is a first-category target character or symbol and the corresponding character recognition probability is greater than a preset first probability threshold, and the first character is a second-category target character and the corresponding character recognition probability is less than a preset second probability threshold, then delete the first character; or

[0014] If the second character in the candidate character set is a third-category target character and the corresponding character recognition probability is greater than a preset first probability threshold, and the first character is a preset character or a preset symbol in the second-category target character, then the first character is corrected to a preset character in the third-category target character; or

[0015] If the second character in the candidate character set is a first-category target character and the corresponding character recognition probability is greater than a preset first probability threshold, and the first character is a preset character or a preset symbol in the second-category target character, then delete the first character; or

[0016] If the first character in the candidate character set is a symbol and is not the left symbol of a preset paired symbol or a preset special symbol, the first character is deleted.

[0017] Optionally, after obtaining the candidate character set and the corresponding character recognition probability, the following steps are performed:

[0018] If the second character in reverse order of the last character in the candidate character set is a first-category target character and the corresponding character recognition probability is greater than a preset first probability threshold, and the last character is a second-category target character and the corresponding character recognition probability is less than the preset first probability threshold, then delete the last character; or

[0019] If the second character in reverse order of the last character in the candidate character set is a third-category target character and the corresponding character recognition probability is greater than a preset first probability threshold, and the last character is a preset character or a preset symbol in the second-category target character, then the last character is corrected to a preset character in the first-category target character; or

[0020] If the second character in reverse order of the last character in the candidate character set is a first-category target character and the corresponding character recognition probability is greater than a preset first probability threshold, and the last character is a preset character or a preset symbol in the second-category target character, then delete the last character; or

[0021] If the tail character in the candidate character set is a symbol and is not the right symbol of a preset paired symbol or a preset sentence or phrase connecting symbol, the tail character is deleted.

[0022] Optionally, after obtaining the candidate character set and the corresponding character recognition probability, the following steps are performed:

[0023] A text line containing a preset number of characters and symbols in the candidate character set is obtained, and if the character recognition probability corresponding to the text line is less than a preset third probability threshold, the text line is deleted.

[0024] Optionally, the candidate character set is expanded using a list of similar characters, including:

[0025] If there are characters in the candidate character set whose character recognition probability is less than the preset fourth probability threshold, multiple characters in the candidate character set are obtained through the TOPN function or the character recognition probability is greater than the preset fifth probability threshold, so as to add all the characters in the list of similar characters corresponding to the multiple characters to the candidate character set, and assign the character recognition probabilities of the multiple characters to the corresponding similar characters in the candidate character set respectively, and then normalize the similar characters after removing duplicates.

[0026] Optionally, it also includes:

[0027] The position information of multiple text boxes is obtained, and it is determined that there are multiple text boxes in a certain direction in which the continuous candidate character set is one character, and then the multiple text boxes are merged.

[0028] In addition, the present invention also provides a text recognition device, including a priori rule component, for acquiring a text image for recognition, and obtaining a candidate character set and a corresponding character recognition probability;

[0029] A language model component is used to expand the candidate character set using the list of similar characters, and to obtain a calculation result set based on the character co-occurrence matrix according to the recognition probability and transition probability of all characters in each text line after the expansion of the candidate character set, and to use the text line corresponding to the maximum value in the calculation result set as the output text;

[0030] A semantic model component is used to generate a list of semantically similar words from the output text through a word vector model, traverse the list of semantically similar words, and use a list of similar characters to filter out words including similar characters to calculate the co-occurrence probability between the original word and the semantically similar and similar words, and output the similar words if the co-occurrence probability is less than a preset first probability threshold to obtain the recognized text.

[0031] An embodiment of the above invention has the following advantages or beneficial effects: The present invention improves the accuracy of text extraction results by expanding the candidate character set space and making full use of semantic information, introducing word vector feature similarity measurement. In addition, the influence of word order on text extraction results is taken into account, and the word order problem of vertical text is better solved. At the same time, the present invention forms a highly componentized text extraction post-processing method, which can build customized applications for personalized applications.

[0032] The further effects of the above-mentioned non-conventional optional manner will be described below in conjunction with the specific implementation manner. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] The accompanying drawings are used to better understand the present invention and do not constitute an improper limitation of the present invention.

[0034] Figure 1 is a schematic diagram of the main process of the text recognition method according to the first embodiment of the present invention;

[0035] Figure 2 is a schematic diagram of the main process of the text recognition method according to the second embodiment of the present invention;

[0036] Figure 3 is a schematic diagram of the main process of the text recognition method according to the third embodiment of the present invention;

[0037] Figure 4 is a schematic diagram of main modules of a text recognition device according to an embodiment of the present invention;

[0038] Figure 5 is an exemplary system architecture diagram to which embodiments of the present invention may be applied;

[0039] Figure 6 It is a schematic diagram of the structure of a computer system of a terminal device or a server suitable for implementing an embodiment of the present invention. DETAILED DESCRIPTION

[0040] The following is a description of exemplary embodiments of the present invention in conjunction with the accompanying drawings, including various details of the embodiments of the present invention to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present invention. Similarly, for clarity and conciseness, the description of well-known functions and structures is omitted in the following description.

[0041] Figure 1 is a schematic diagram of the main process of the text recognition method according to the first embodiment of the present invention. Figure 1 As shown, the text recognition method includes:

[0042] Step S101, obtaining a text image for recognition, and obtaining a candidate character set and corresponding character recognition probabilities.

[0043] In a preferred embodiment, the acquired text image can be recognized by an OCR model to obtain text box position information, candidate character set and corresponding character recognition probability. OCR (Optical Character Recognition) refers to the process in which an electronic device (such as a scanner or a digital camera) examines characters printed on paper, determines their shapes by detecting dark and light patterns, and then translates the shapes into computer text using a character recognition method.

[0044] In some embodiments, after acquiring a text image for recognition, the text box position information, the candidate character set and the corresponding character recognition probability can be identified, and then the proportion of characters in the text box whose character recognition probability is less than a preset second probability threshold is calculated. If the proportion is greater than the preset proportion threshold, the text box is deleted.

[0045] In a further embodiment, the misdetected text boxes can be removed. In fact, there is no text in such text boxes, and the character probability in the OCR model output result is generally low. Therefore, the second probability threshold is set to determine whether it is an illegal character, and the illegal character ratio is counted to determine the legitimacy of the recalled text box. The formula for counting the illegal character ratio is as follows:

[0046]

[0047]

[0048] Among them, x i represents the probability of character recognition of the ith character, where T 1 It represents the second probability threshold, which and the proportion threshold for determining legality can be customized. Preferably, the second probability threshold is 0.8.

[0049] For example: the text box does not contain any text but is recalled and then recognized. The recognition result may be "!~~", and the corresponding text probability is "0.3, 0.6, 0.3". According to T1=0.8, the rate is calculated by the formula at this time=0, and the proportion threshold is set to 0.5. Since the rate is less than the proportion threshold, the text line is deleted.

[0050] In other embodiments, after obtaining the candidate character set and the corresponding character recognition probability, the text box can also be processed for some special application scenarios, that is, the processing of the text recognition application scenario is realized. The specific implementation process includes: determining the application scenario corresponding to the text in the image based on the preset text feature information, and then merging the corresponding text box according to the text box position information.

[0051] In a preferred embodiment, based on preset text feature information (for example, sentences of ancient poetry are "five-character ancient poetry" in separate lines, i.e., text features unique to poetry), it is determined whether the text in the image is in separate lines. If so, the application scenario corresponding to the text in the image is determined, and then the text boxes with the same application scenario are merged according to the text box position information, that is, the contents in text lines with close distances are spliced ​​together, and multiple lines of text become one paragraph of text. Otherwise, the text box is not processed.

[0052] In a further embodiment, after the text boxes in the image are merged for the application scenario, it is possible to detect whether the characters that appear in pairs are legal. The specific implementation process includes: calling the paired symbol detection component in the prior rule component to determine whether the left symbol or right symbol of the preset paired symbol exists in the corresponding candidate character set in the text box, and then detecting whether the right symbol or left symbol of the preset paired symbol exists. If it exists, no processing is performed, otherwise the left symbol or right symbol of the preset paired symbol that is determined to exist is deleted, that is, the paired symbol detection component removes unpaired symbols. For example: the preset paired symbols include: "" () "》 <> and other symbols.

[0053] As a preferred embodiment, after obtaining the candidate character set and the corresponding character recognition probability, the following steps are specifically included:

[0054] If the second character in the candidate character set is a first-category target character or symbol and the corresponding character recognition probability is greater than the preset first probability threshold, and the first character is a second-category target character and the corresponding character recognition probability is less than the preset second probability threshold, then delete the first character. For example: if the second character is a Chinese character or symbol and the character recognition probability is greater than the first probability threshold of 0.9, and the first character is an English letter and the character recognition probability is less than the second probability threshold of 0.6, then delete the first character.

[0055] Alternatively, if the second character in the candidate character set is a third-category target character and the corresponding character recognition probability is greater than a preset first probability threshold, and the first character is a preset character or a preset symbol in the second-category target character, the first character is corrected to a preset character in the third-category target character. For example: if the second character is a number and the character recognition probability is greater than the first probability threshold of 0.9, and the first character is the letter "l" or "|", the first character is corrected to the number "1".

[0056] Alternatively, if the second character in the candidate character set is a first-category target character and the corresponding character recognition probability is greater than a preset first probability threshold, and the first character is a preset character or a preset symbol in the second-category target character, then the first character is deleted. For example: if the second character is a Chinese character and the character recognition probability is greater than the first probability threshold 0.9, and the first character is the letter "l" or "|", then the first character is deleted.

[0057] Alternatively, if the first character in the candidate character set is a symbol and is not the left symbol of a preset paired symbol or a preset special symbol, the first character is deleted. For example, if the first character is a symbol and is not the left symbol of a paired symbol (such as "'"'((《[

【{, etc.) or a special symbol (such as ￥$, etc.), the first character is deleted.

[0058] For example: Figure 2 For the text box shown, the OCR model output is |THECOREFINCTION, and the output result after processing by the first character legality judgment component is THECOREFINCTION:

[0059]

[0060]

[0061] Among them, [] represents the coordinates of the four corners of the text box and the confidence value.

[0062] As a preferred embodiment, after obtaining the candidate character set and the corresponding character recognition probability, the following steps are specifically included:

[0063] If the second character in reverse order of the last character in the candidate character set is a first-category target character and the corresponding character recognition probability is greater than the preset first probability threshold, and the last character is a second-category target character and the corresponding character recognition probability is less than the preset first probability threshold, then delete the last character. For example: if the second character in reverse order is a Chinese character and the character recognition probability is greater than the first probability threshold of 0.9, and the last character is an English letter and the character recognition probability is less than the first probability threshold of 0.9, then delete the last character.

[0064] Alternatively, if the second character in reverse order of the last character in the candidate character set is a third-category target character and the corresponding character recognition probability is greater than a preset first probability threshold, and the last character is a preset character or a preset symbol in the second-category target character, then the last character is corrected to a preset character in the first-category target character. For example: if the second character in reverse order is a number and the character recognition probability is greater than the first probability threshold 0.9, and the last character is the letter "l" or "|", then the last character is corrected to the number "1".

[0065] Alternatively, if the second character in reverse order of the last character in the candidate character set is a first-category target character and the corresponding character recognition probability is greater than a preset first probability threshold, and the last character is a preset character or a preset symbol in the second-category target character, then the last character is deleted. For example, if the second character in reverse order is a Chinese character and the character recognition probability is greater than the first probability threshold 0.9, and the last character is the letter "l" or "|", then the last character is deleted.

[0066] Alternatively, if the last character in the candidate character set is a symbol and is not the right symbol of a preset paired symbol or a preset sentence or short sentence connecting symbol, the last character is deleted. For example, if the last character is a symbol and is not the right symbol of a paired symbol (such as "'"'))》】}], etc.), or a sentence or short sentence connecting symbol (such as,.;?!,.?!, etc.), the last character is deleted.

[0067] As some other embodiments of the present invention, it is also possible to correct letters in English words that are easily confused with numbers or symbols. Specifically, when the easily confused letter correction component recognizes that the characters before and after the current character are both letters, the current character is corrected by running the following rules in sequence:

[0068] 1 / |=l,8=B,9=g,6=b,0=o,D=T,￥=Y,$=S

[0069] As a preferred embodiment, after obtaining the candidate character set and the corresponding character recognition probability, the following steps are specifically included:

[0070] Obtain a text line that contains a preset number of characters and is a symbol in the candidate character set, and if the character recognition probability corresponding to the text line is less than a preset third probability threshold, delete the text line. Preferably, the secondary legitimacy judgment component can be used for the final test of preprocessing. For example: if there is only one character left in the text line and it is a symbol, and the character recognition probability is less than the third probability threshold of 0.8, delete the text line.

[0071] Step S102, using the list of similar characters to expand the candidate character set, based on the character co-occurrence matrix, the expanded candidate character set is calculated according to all character recognition probabilities and transition probabilities in each text line, and the text line corresponding to the maximum value in the calculation result set is used as the output text.

[0072] In some embodiments, the candidate character set is expanded using a list of similar characters, including:

[0073] If there are characters in the candidate character set whose character recognition probability is less than the preset fourth probability threshold, multiple characters in the candidate character set are obtained through the TOPN function or the character recognition probability is greater than the preset fifth probability threshold, so as to add all the characters in the list of similar characters corresponding to the multiple characters to the candidate character set, and assign the character recognition probabilities of the multiple characters to the corresponding similar characters in the candidate character set respectively, and then normalize the similar characters after removing duplicates.

[0074] It can be seen that not only the TOPN characters in the text recognition output result are taken as candidates, but also the candidate character set is expanded using the list of similar-looking characters. Because the TOPN characters may still not include the correct character, and even if the subsequent processing accuracy is very high, the correct error correction result cannot be obtained based on this. The expansion of the candidate character set can alleviate this problem to a certain extent. Among them, TOPN refers to sorting by the recognition probability value from large to small and taking the first N. In the field of machine learning, it is called TOPN. Among them, TOP1 means only taking the first one after sorting, that is, the one with the largest probability value. Preferably, the candidate character set component in the language model component can be called to execute the above process.

[0075] For example: In the recognition result of a certain text line, there is a character "无" (wu), and the recognition probability of the TOP1 character in its recognition result is 0.4. Since it is less than the fourth probability threshold of 0.99, the TOPN (N = 2) results ("羌" 0.4, "夫" 0.3) need to be taken, and then the characters in the list of similar-looking characters of these two characters (羌: 无; 夫: 天) are added to the candidate set. The recognition probabilities of these similar-looking characters correspond to their original characters. At this time, the candidate set includes ("羌" 0.4, "无" 0.4, "夫" 0.3, "天" 0.3). It should be noted that in fact, the number of characters in the list of similar-looking characters is more than one, and this is only an example here. The probabilities of the characters in the candidate set are (0.4, 0.4, 0.3, 0.3). After normalization, 0.4 / (0.4 + 0.4 + 0.3 + 0.3) = 0.286, 0.3 / (0.4 + 0.4 + 0.3 + 0.3) = 0.214, so the normalized result is (0.286, 0.286, 0.214, 0.214).

[0076] In a further embodiment, the final output text is determined by calculating the maximum value of the product of the recognition probabilities and transition probabilities of all characters in the text line. Preferably, the language model can select bigram / trigram according to requirements, and the dynamic programming is the classic Viterbi algorithm. Among them, the character co-occurrence matrix is obtained by corpus statistics, which represents the probability that a certain character appears after another character. When using it, the displayed value is divided by the total number of each line to obtain the probability value). It should be noted that bigram is selected when pursuing speed, and trigram is selected when pursuing accuracy. When applying, bigram calculates two characters each time, and trigram calculates three characters each time.

[0077] For example: As Figure 3 shown in the extracted text box, the input text result is "防止电池过功睾放电" (Prevent the battery from discharging with excessive power), and the error correction result output by the language model component is "防止电池过功率放电" (Prevent the battery from discharging with excessive power):

[0078]

[0079] Among them, [] represents the coordinate position and confidence value of the four corners of the text box.

[0080] Step S103, generates a list of semantically similar words from the output text through a word vector model, then traverses the list of semantically similar words, and uses a list of similar characters to filter out words including similar characters to calculate the co-occurrence probability between the original word and the semantically similar and similar words. If the co-occurrence probability is less than a preset first probability threshold, the similar word is output to obtain the recognized text.

[0081] In some embodiments, the result outputted from step 102 needs to be used as corpus to train the word vector model. This is a method based on statistical theory. In a large amount of text corpus, there are similar or identical descriptions. For example, in N identical or similar sentences, each word may be recognized correctly or incorrectly. According to the semantic similarity (context), the incorrectly recognized words can be matched with the correctly recognized words. Using this correspondence and other statistical information, the incorrectly recognized situations can be corrected.

[0082] In a further embodiment, when executing step S103, a word vector model based on the recognized text can be obtained, and the text can be vectorized for measuring the semantic similarity of the word. It is worth noting that the output result of step 102 must be used, and the public corpus cannot be used directly for training, otherwise the purpose of error correction cannot be achieved. Preferably, the word2vec model is used as the word vector model, and of course other models with stronger representation capabilities can be used as a replacement according to application requirements.

[0083] Then, the words that need to be corrected using semantics are determined, and the principle of determination is that the TOP1 character recognition probability of the characters in the word segmentation result is less than the first probability threshold (for example, 0.99 is taken here).

[0084] Afterwards, in addition to the language model, the present invention proposes an algorithm for text post-processing that makes full use of semantic information, and determines whether to perform text error correction processing based on semantic similarity, thereby improving the accuracy of text extraction.

[0085] To sum up, step S103 uses the word vector model to calculate the TOPN semantically similar word list of the word to be corrected, and uses the similar character list to determine whether the included similar characters can be screened out. If not, continue to traverse the correction word sequence until the traversal is completed and stop outputting the result. If so, calculate the co-occurrence probability of the original word and the semantically similar and similar words. If the co-occurrence probability of the original word is greater than the first probability threshold, the original word is output without correction. If the co-occurrence probability of the original word is less than or equal to the first probability threshold, the similar word with the largest co-occurrence probability is output.

[0086] Among them, when using the word vector model to calculate the TOPN semantically similar word list of the word to be corrected, it includes training to obtain the word vector model, and then using the development toolkit to load the word vector model. The toolkit will provide an interface: input the correction word and return the word with higher semantic similarity. Of course, you can also develop a toolkit by yourself, calculate the similarity between the correction word and all the words in the word vector model (for example, you can use the cosine distance), and select the TOPN similar results, which are the words with higher semantic similarity.

[0087] For example, the extracted text box below has the input "Warm Tips: To choose the right size, it is best to measure your own clothes before comparing and purchasing", and the processed result is "Warm Tips: To choose the right size, it is best to measure your own clothes before comparing and purchasing", where "appropriate" is corrected to "appropriate", but "wash" is not corrected for some reason. To this end, the output of the language model component is input into the semantic model component, and the output result is "Warm Tips: To choose the right size, it is best to measure your own clothes before comparing and purchasing", where "wash" is corrected to "select" to obtain the final correct text extraction result.

[0088]

[0089] Language model component input:

[0090]

[0091] Language model component results:

[0092]

[0093] Semantic model component results:

[0094]

[0095] As another embodiment, if there is a large amount of vertical text in the application, this component can better solve the word order problem during output. The specific implementation process includes:

[0096] The position information of multiple text boxes is obtained, and it is determined that there are multiple text boxes in a certain direction in which the continuous candidate character set is one character, and then the multiple text boxes are merged.

[0097] In a further embodiment, the processing comprises:

[0098] Step 1: Arrange all text boxes in ascending order with the upper left corner as the origin and the left border as the y-axis (vertical direction), that is, the text boxes are arranged from top to bottom. Among them, the upper left corner of the text box is the origin, the left border of the text box is the y-axis, and the upper border of the text box is the x-axis.

[0099] Step 2: All text boxes form a set A, and the number of text boxes is N.

[0100] Step 3: Select the first text box i=1.

[0101] Step 4: Determine whether i is less than N. If not, proceed to step 10. If it is less than N, proceed to step 5 (i.e., whether there are text lines that need to be merged).

[0102] Step 5: Determine whether the recognized text is a single character. If it is multiple characters, i=i+1, go to step 4; if it is a single character, take the text box to be merged j=i+1 and proceed to step 6.

[0103] Step 6: Determine whether j is less than or equal to N.

[0104] Step 7: If not, save the merged text line (if there is no merge, no need to save), then update the number of text boxes N, take i=i+1 and return to step 4; if yes, proceed to step 8.

[0105] Step 8: Determine whether the recognized text j is a single character. If not, take the next text line, i.e., j=j+1, and return to step 6; if yes, proceed to step 9.

[0106] Step nine: Determine whether the preset merging condition is met based on the text box position information. If so, after merging the text boxes, take the next text line j=j+1 and return to step six. If not, directly take the next text line j=j+1 and return to step six.

[0107] The preset merging conditions are:

[0108]

[0109]

[0110] w and h refer to the width and height of the text box respectively.

[0111] Among them, after merging the text boxes, the coordinates of the i text row need to be updated and j is removed. The coordinates of the i text row after the update are:

[0112] Upper left corner: (x i1 ,y i1 )

[0113] Lower right corner: (x j4 ,y j4 )

[0114] Step 10: Output all merged text boxes and text lines that are not involved in the merge.

[0115] For example: In the row-based OCR model, vertical text is often detected as multiple text boxes:

[0116] Typically, the output of raw text extraction looks like this:

[0117]

[0118] After processing, the text extraction results are as follows:

[0119]

[0120] In summary, the text recognition method described in the present invention can not only process pure Chinese text, but also solve the problem of post-processing of Chinese and English mixed text extraction. The post-processing method based on the language model is improved to improve the accuracy of the results. At the same time, the influence of word order on text extraction results is considered, forming a highly componentized text extraction post-processing method.

[0121] Figure 2 is a schematic diagram of the main process of a text recognition method according to a second embodiment of the present invention. The text recognition method may include:

[0122] Step S201, obtaining a text image for recognition, and obtaining text box position information, a candidate character set and corresponding character recognition probability.

[0123] Step S202, calculating the proportion of characters in the text box whose character recognition probability is less than a preset second probability threshold, and deleting the text box if the proportion is greater than the preset proportion threshold.

[0124] Step S203, obtaining position information of multiple text boxes, determining that there are multiple text boxes with a continuous candidate character set as one character in a certain direction, and then merging the multiple text boxes.

[0125] Step S204: determining the application scenario corresponding to the text in the image based on the preset text feature information, and then merging the corresponding text boxes according to the text box position information.

[0126] Step S205, using the list of similar characters to expand the candidate character set, and calculating the product of all character recognition probabilities and transition probabilities in each text line for the expanded candidate character set based on the character co-occurrence matrix, and then taking the text line corresponding to the maximum product as the output text.

[0127] Step S206, generates a list of semantically similar words from the output text through a word vector model, then traverses the list of semantically similar words, and uses a list of similar characters to filter out words including similar characters to calculate the co-occurrence probability between the original word and the semantically similar and similar words, and if the co-occurrence probability is less than a preset first probability threshold, output the similar word to obtain the recognized text.

[0128] Figure 3 is a schematic diagram of the main modules of the text recognition device according to an embodiment of the present invention. Figure 3 As shown, the text recognition device 300 includes a priori rule component 301, a language model component 302 and a semantic model component 303. Among them, the priori rule component 301 obtains a text image for recognition, and obtains a candidate character set and a corresponding character recognition probability; the language model component 302 uses a list of similar characters to expand the candidate character set, and obtains a calculation result set based on the character co-occurrence matrix for the expanded candidate character set according to all character recognition probabilities and transition probabilities in each text line, and takes the text line corresponding to the maximum value in the calculation result set as the output text; the semantic model component 303 generates a semantically similar word list from the output text through a word vector model, traverses the semantically similar word list, and uses the list of similar characters to filter out words including similar characters, so as to calculate the co-occurrence probability between the original word and a word that is similar in meaning and shape, and outputs a similar word if the co-occurrence probability is less than a preset first probability threshold, so as to obtain a recognized text.

[0129] In some embodiments, after the prior rule component 301 obtains the text image for recognition, it includes:

[0130] The text box position information is obtained by identifying, and the recall judgment component in the prior rule component is called to calculate the proportion of characters in the text box whose character recognition probability is less than a preset second probability threshold. If the proportion is greater than the preset proportion threshold, the text box is deleted.

[0131] In some embodiments, after obtaining the candidate character set and the corresponding character recognition probability, the a priori rule component 301 includes:

[0132] Call the first character legitimacy judgment component in the prior rule component, and if the second character in the candidate character set is a first-category target character or symbol and the corresponding character recognition probability is greater than a preset first probability threshold, and the first character is a second-category target character and the corresponding character recognition probability is less than a preset second probability threshold, then delete the first character; or

[0133] If the second character in the candidate character set is a third-category target character and the corresponding character recognition probability is greater than a preset first probability threshold, and the first character is a preset character or a preset symbol in the second-category target character, then the first character is corrected to a preset character in the third-category target character; or

[0134] If the second character in the candidate character set is a first-category target character and the corresponding character recognition probability is greater than a preset first probability threshold, and the first character is a preset character or a preset symbol in the second-category target character, then delete the first character; or

[0135] If the first character is a symbol and is not the left symbol of a preset paired symbol or a preset special symbol, the first character is deleted.

[0136] In some embodiments, after obtaining the candidate character set and the corresponding character recognition probability, the a priori rule component 301 includes:

[0137] Call the tail character legitimacy judgment component in the prior rule component, and if the second character in reverse order of the tail character in the candidate character set is a first-category target character and the corresponding character recognition probability is greater than a preset first probability threshold, and the tail character is a second-category target character and the corresponding character recognition probability is less than the preset first probability threshold, then delete the tail character; or

[0138] If the second character in reverse order of the last character in the candidate character set is a third-category target character and the corresponding character recognition probability is greater than a preset first probability threshold, and the last character is a preset character or a preset symbol in the second-category target character, then the last character is corrected to a preset character in the first-category target character; or

[0139] If the second character in reverse order of the last character in the candidate character set is a first-category target character and the corresponding character recognition probability is greater than a preset first probability threshold, and the last character is a preset character or a preset symbol in the second-category target character, then delete the last character; or

[0140] If the tail character in the candidate character set is a symbol and is not the right symbol of a preset paired symbol or a preset sentence or phrase connecting symbol, the tail character is deleted.

[0141] In some embodiments, after obtaining the candidate character set and the corresponding character recognition probability, the a priori rule component 301 includes:

[0142] The secondary legitimacy judgment component in the prior rule component is called to obtain a text line that contains a preset number of characters and symbols in the candidate character set. If the character recognition probability corresponding to the text line is less than a preset third probability threshold, the text line is deleted.

[0143] In some embodiments, the language model component 302 expands the candidate character set using a list of similar characters, including:

[0144] If there are characters in the candidate character set whose character recognition probability is less than the preset fourth probability threshold, multiple characters in the candidate character set are obtained through the TOPN function or the character recognition probability is greater than the preset fifth probability threshold, so as to add all the characters in the list of similar characters corresponding to the multiple characters to the candidate character set, and assign the character recognition probabilities of the multiple characters to the corresponding similar characters in the candidate character set respectively, and then normalize the similar characters after removing duplicates.

[0145] In some embodiments, the a priori rule component 301 is further configured to:

[0146] The position information of multiple text boxes is obtained, and it is determined that there are multiple text boxes in a certain direction in which the continuous candidate character set is one character, and then the multiple text boxes are merged.

[0147] Figure 4 is a schematic diagram of main modules of a text recognition device according to another embodiment of the present invention, which may include:

[0148] Get an image containing text, and obtain text box position information, candidate character set and corresponding character recognition probability through OCR model. Then, based on the candidate character set and the corresponding character recognition probability, perform preprocessing through the recall judgment component, first character legality judgment component, last character legality judgment component, paired symbol detection component, easily confused letter correction component, and legality secondary judgment component in the prior rule component. After that, input the prior rule result (i.e. the preprocessed result) into the language model component, and then obtain the speech model result through the candidate character set selection component (need to use the list of similar characters) and the dynamic programming algorithm component (need to use the character co-occurrence matrix generated based on the obtained multi-source corpus).

[0149] The speech model result is input into the semantic model component for word segmentation processing, and a preliminary judgment is made based on the character recognition probability, so as to use the trained word vector model to screen similar words to obtain the semantic model result.

[0150] Finally, based on the obtained text box position, the vertical text component is called to merge the text of the semantic model results and then output the final result.

[0151] In some embodiments, the recall judgment component can remove misdetected text boxes, in which there is actually no text and the character probability in the OCR model output result is generally low.

[0152] In other embodiments, after the a priori rule component obtains the candidate character set and the corresponding character recognition probability, it can also process the text box for some special application scenarios, that is, it realizes the processing of text recognition application scenarios. The specific implementation process includes: determining the application scenario corresponding to the text in the image based on the preset text feature information, and then merging the corresponding text box according to the text box position information.

[0153] In a further embodiment, after the text boxes in the image are merged for the application scenario, the paired symbol detection component in the prior rule component is called to detect whether the characters that appear in pairs are legal.

[0154] As some further embodiments of the present invention, the prior rule component is also used to correct letters in English words that are easily confused with numbers or symbols.

[0155] In a further embodiment, a dynamic programming algorithm component in the language model component is called, which determines the final output text by calculating the maximum value of the product of all character recognition probabilities and transition probabilities in the text line based on the candidate character set selection component and the character co-occurrence matrix.

[0156] In some embodiments, the semantic model component needs to use the output of the preceding component as corpus to train the word vector model. This is a method based on statistical theory. In a large amount of text corpus, there are similar or identical descriptions.

[0157] In a further embodiment, the word vector model algorithm component in the semantic model component is called to obtain a word vector model based on the recognized text, and the text is vectorized for the semantic similarity measurement of the word. It is worth noting that the output result of the pre-component must be used, and the public corpus cannot be used directly for training, otherwise the purpose of error correction cannot be achieved. Then, the probability pre-determination component in the semantic model component is called, and the component performs preliminary screening based on the character recognition probability to determine the words that need to be corrected using semantics. The principle of judgment is that the TOP1 character recognition probability of the character in the word segmentation result is less than the first probability threshold (for example: 0.99 is taken here). Afterwards, the semantic similarity judgment component in the semantic model component is called. In addition to the language model, the present invention proposes an algorithm that makes full use of semantic information for text post-processing, and determines whether to perform text error correction processing based on semantic similarity, thereby improving the accuracy of text extraction.

[0158] It should be noted that the text recognition method and the text recognition device of the present invention have a corresponding relationship in terms of specific implementation contents, so the repeated contents will not be described again.

[0159] Figure 5 An exemplary system architecture 500 is shown to which the text recognition method or text recognition apparatus according to the embodiment of the present invention can be applied.

[0160] like Figure 5 As shown, system architecture 500 may include terminal devices 501, 502, 503, a network 504 and a server 505. Network 504 is used to provide a medium for communication links between terminal devices 501, 502, 503 and server 505. Network 504 may include various connection types, such as wired, wireless communication links or optical fiber cables, etc.

[0161] Users can use terminal devices 501, 502, 503 to interact with server 505 through network 504 to receive or send messages, etc. Various communication client applications can be installed on terminal devices 501, 502, 503, such as shopping applications, web browser applications, search applications, instant messaging tools, email clients, social platform software, etc. (only as examples).

[0162] The terminal devices 501 , 502 , and 503 may be various electronic devices having a text recognition screen and supporting web browsing, including but not limited to smart phones, tablet computers, laptop computers, desktop computers, and the like.

[0163] The server 505 may be a server that provides various services, such as a backend management server (only an example) that provides support for shopping websites browsed by users using the terminal devices 501, 502, and 503. The backend management server may analyze and process the received data such as product information query requests, and feed back the processing results (such as target push information, product information - only an example) to the terminal device.

[0164] It should be noted that the text recognition method provided in the embodiment of the present invention is generally executed by the server 505 , and accordingly, the computing device is generally disposed in the server 505 .

[0165] It should be understood that Figure 5 The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided according to implementation requirements.

[0166] Reference below Figure 6 , which shows a schematic diagram of the structure of a computer system 600 of a terminal device suitable for implementing an embodiment of the present invention. Figure 6 The terminal device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0167] like Figure 6 As shown, the computer system 600 includes a central processing unit (CPU) 601, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 602 or a program loaded from a storage part 608 into a random access memory (RAM) 603. In the RAM 603, various programs and data required for the operation of the computer system 600 are also stored. The CPU 601, the ROM 602, and the RAM 603 are connected to each other via a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.

[0168] The following components are connected to the I / O interface 605: an input section 606 including a keyboard, a mouse, etc.; an output section 607 including a cathode ray tube (CRT), a liquid crystal text recognizer (LCD), etc., and a speaker, etc.; a storage section 608 including a hard disk, etc.; and a communication section 609 including a network interface card such as a LAN card, a modem, etc. The communication section 609 performs communication processing via a network such as the Internet. A drive 610 is also connected to the I / O interface 605 as needed. A removable medium 611, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 610 as needed, so that a computer program read therefrom is installed into the storage section 608 as needed.

[0169] In particular, according to the embodiments disclosed in the present invention, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present invention include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 609, and / or installed from the removable medium 611. When the computer program is executed by the central processing unit (CPU) 601, the above-mentioned functions defined in the system of the present invention are executed.

[0170] It should be noted that the computer-readable medium shown in the present invention may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present invention, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device. In the present invention, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer-readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, optical cable, RF, etc., or any suitable combination of the above.

[0171] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present invention. In this regard, each box in the flow chart or block diagram can represent a module, a program segment, or a part of a code, and the above-mentioned module, program segment, or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flow chart, and the combination of the boxes in the block diagram or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0172] The modules involved in the embodiments of the present invention may be implemented in software or hardware. The modules described may also be set in a processor, for example, may be described as: a processor including a priori rule component, a language model component, and a semantic model component. The names of these modules do not, in some cases, constitute limitations on the modules themselves.

[0173] As another aspect, the present invention also provides a computer-readable medium, which may be included in the device described in the above embodiment; or it may exist independently and not be assembled into the device. The above computer-readable medium carries one or more programs. When the above one or more programs are executed by a device, the device includes acquiring a text image for recognition, obtaining a candidate character set and a corresponding character recognition probability; using a list of similar characters to expand the candidate character set, and calculating the product of all character recognition probabilities and transition probabilities in each text line for the expanded candidate character set based on a character co-occurrence matrix, and then using the text line corresponding to the maximum product as output text; generating a list of semantically similar words from the output text through a word vector model, traversing the list of semantically similar words, and using the list of similar characters to filter out words including similar characters, so as to calculate the co-occurrence probability of the original word and the semantically similar and similar words, and outputting the similar words if the co-occurrence probability is less than a preset first probability threshold.

[0174] The technical solution according to the embodiment of the present invention can solve the problems of low accuracy in existing text extraction and poor recognition effect.

[0175] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions may occur depending on design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A text recognition method, characterized in that: include: Obtain text images for recognition, and obtain candidate character sets and corresponding character recognition probabilities; The candidate character set is expanded by using the list of similar characters, and the expanded candidate character set is calculated based on the character co-occurrence matrix, according to the recognition probability and transition probability of all characters in each text line, and the text line corresponding to the maximum value in the calculation result set is used as the output text; The output text is used to generate a list of semantically similar words through a word vector model, the list of semantically similar words is traversed, and words including similar characters are screened out using a list of similar characters to calculate the co-occurrence probability of the original word and the semantically similar and similar words, and if the co-occurrence probability is less than a preset first probability threshold, the similar words are output to obtain the recognized text; wherein the output text is used as corpus to train the word vector model.

2. The method according to claim 1, characterized in that After obtaining the text image for recognition, it includes: The text box position information is obtained by identification, and the proportion of characters in the text box whose character recognition probability is less than a preset second probability threshold is calculated. If the proportion is greater than the preset proportion threshold, the text box is deleted.

3. The method according to claim 1, characterized in that After obtaining the candidate character set and the corresponding character recognition probability, it includes: If the second character in the candidate character set is a first-category target character or symbol and the corresponding character recognition probability is greater than a preset first probability threshold, and the first character is a second-category target character and the corresponding character recognition probability is less than a preset second probability threshold, then delete the first character; or If the second character in the candidate character set is a third-category target character and the corresponding character recognition probability is greater than a preset first probability threshold, and the first character is a preset character or a preset symbol in the second-category target character, then the first character is corrected to a preset character in the third-category target character; or If the second character in the candidate character set is a first-category target character and the corresponding character recognition probability is greater than a preset first probability threshold, and the first character is a preset character or a preset symbol in the second-category target character, then delete the first character; or If the first character in the candidate character set is a symbol and is not the left symbol of a preset paired symbol or a preset special symbol, the first character is deleted.

4. The method according to claim 1, characterized in that: After obtaining the candidate character set and the corresponding character recognition probability, it includes: If the second character in reverse order of the last character in the candidate character set is a first-category target character and the corresponding character recognition probability is greater than a preset first probability threshold, and the last character is a second-category target character and the corresponding character recognition probability is less than the preset first probability threshold, then delete the last character; or If the second character in reverse order of the last character in the candidate character set is a third-category target character and the corresponding character recognition probability is greater than a preset first probability threshold, and the last character is a preset character or a preset symbol in the second-category target character, then the last character is corrected to a preset character in the first-category target character; or If the second character in reverse order of the last character in the candidate character set is a first-category target character and the corresponding character recognition probability is greater than a preset first probability threshold, and the last character is a preset character or a preset symbol in the second-category target character, then delete the last character; or If the tail character in the candidate character set is a symbol and is not the right symbol of a preset paired symbol or a preset sentence or phrase connecting symbol, the tail character is deleted.

5. The method according to claim 1, characterized in that After obtaining the candidate character set and the corresponding character recognition probability, it includes: A text line containing a preset number of characters and symbols in the candidate character set is obtained, and if the character recognition probability corresponding to the text line is less than a preset third probability threshold, the text line is deleted.

6. The method according to claim 1, characterized in that Expand the candidate character set using a list of similar characters, including: If there are characters in the candidate character set whose character recognition probability is less than the preset fourth probability threshold, multiple characters in the candidate character set are obtained through the TOPN function or the character recognition probability is greater than the preset fifth probability threshold, so as to add all the characters in the list of similar characters corresponding to the multiple characters to the candidate character set, and assign the character recognition probabilities of the multiple characters to the corresponding similar characters in the candidate character set respectively, and then normalize the similar characters after removing duplicates.

7. The method according to any one of claims 1 to 6, characterized in that: Also includes: The position information of multiple text boxes is obtained, and it is determined that there are multiple text boxes in a certain direction in which the continuous candidate character set is one character, and then the multiple text boxes are merged.

8. A text recognition device, characterized in that: include: A priori rule component is used to obtain text images for recognition and obtain candidate character sets and corresponding character recognition probabilities; A language model component is used to expand the preprocessed candidate character set using the similar character list, to obtain a calculation result set based on the character co-occurrence matrix according to all character recognition probabilities and transition probabilities in each text line, and to use the text line corresponding to the maximum value in the calculation result set as the output text; A semantic model component is used to generate a semantically similar word list from the output text through a word vector model, traverse the semantically similar word list, and use a list of similar characters to filter out words including similar characters to calculate the co-occurrence probability between the original word and the semantically similar and similar words, and output the similar words if the co-occurrence probability is less than a preset first probability threshold to obtain the recognized text; wherein the output text is used as corpus to train the word vector model.

9. An electronic device, characterized in that: include: one or more processors; a storage device for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 7.

10. A computer readable medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images