A text error correction method, system and storage medium based on OCR

Abstract

The present invention discloses a text error correction method, system and storage medium based on optical character recognition (OCR). The method comprises: using a probability matrix obtained in a text recognition process to calculate a candidate text matrix, combining the candidate text matrix with a dictionary of similar words as a candidate set for replacing wrong characters; if a word is not in the dictionary of commonly used words and the probability of any word in the word is less than a threshold, cyclically replacing the word with the candidate set; calculating a pixel spacing vector between two words, finding positions where wrong characters or missing characters may exist through the probability vectors and pixel spacing vectors of the words, cyclically deleting or adding words, and realizing error correction for wrong characters, multiple characters and missing characters. The present invention improves the generalization ability of the dictionary, has a low error correction rate, and can solve OCR text recognition errors in various situations.

Description

Technical Field

[0001] The present invention relates to a text error correction method, in particular to a text error correction method based on OCR. Background Art

[0002] In office work, paper documents often need to be entered into systems. Manual entry is time-consuming and labor-intensive, especially for repetitive document entry. There is an urgent need to automate this process, and Optical Character Recognition (OCR) is a technology that can achieve this. Text recognition, commonly known as optical character recognition (OCR), uses optical and computer technologies to read and recognize printed or handwritten text, converting it into a format that can be read by both computers and humans.

[0003] The OCR recognition process is generally divided into the following steps: text detection, text recognition, and post-processing. Post-processing, which includes text merging and error correction, is crucial for improving the accuracy of OCR recognition results. There are three main types of errors involved in OCR text correction: typos, missing characters, and extra characters. Currently, OCR text correction primarily targets the first type of error, namely, correcting similar characters. Two main methods are used: text correction based on a dictionary of similar characters, and text correction based on the Bert model. Correction based on a dictionary of similar characters requires a relatively good and comprehensive dictionary of similar characters. Correcting characters that do not exist in the dictionary is difficult, resulting in insufficient generalization performance. Correction based solely on the Bert model offers high generalization performance but is prone to correcting correct characters. Even if an incorrect character is found, the text may become more fluent after correction, but replacing it with a similarly meaningless word will not correct the error. Furthermore, existing methods fail to take the OCR text recognition process into account. Summary of the Invention

[0004] Purpose of the invention: The purpose of the present invention is to provide a method for text error correction combined with OCR text recognition results, reduce the error correction rate, improve the generalization performance of the vocabulary, and solve the three errors of wrong characters, missing characters and extra characters.

[0005] Technical solution: The text error correction method based on OCR described in the present invention includes the following steps:

[0006] (1) Input the image to be recognized, perform text detection and CRNN+CTC text recognition, and obtain the probability matrix, the probability vector of each word, the text content, and the pixel spacing vector between two words;

[0007] (2) Obtain the other k characters corresponding to the first k+1 probability values ​​in each column in the probability matrix to obtain the candidate text matrix;

[0008] (3) Segment the text content. If the word is not in the common word dictionary and the probability of any character in the word is less than the threshold, the word candidate set is obtained by using the similar word dictionary and the candidate text matrix, and cyclic replacement is performed. Otherwise, the original word is retained.

[0009] (4) If the probability vector of the word is less than the threshold ppl_thre2 and the pixel spacing vector is less than the threshold d_thre1, then the word is deleted cyclically, otherwise the original word is retained;

[0010] (5) If the pixel spacing vector of the word is greater than the threshold d_thre2, then the word is added cyclically, otherwise the original word is retained;

[0011] (6) Output the OCR recognized text after error correction.

[0012] Wherein, step (3) also includes: if the minimum value ppl of the perplexity vector of the sentence after the loop replacement m Satisfy the inequality ppl m ≤ppl_thre*ppl o , then replace it, otherwise keep the original word; where ppl_thre is the perplexity threshold, ppl o is the perplexity of the sentence before replacement.

[0013] Step (4) also includes: if the confusion vector of the sentence after cyclic deletion is Satisfy the inequality Then delete i j Otherwise, keep the word.

[0014] Step (5) also includes: if the minimum value ppl of the confusion vector of the sentence after the loop is added m′ Satisfy the inequality ppl m′ ≤ppl_thre*ppl o , then add a word, otherwise keep the original word.

[0015] The OCR-based text error correction system of the present invention includes the following modules:

[0016] (1) Text detection and recognition module, which performs text detection and CRNN+CTC text recognition on the input image, and obtains the probability matrix, the probability vector of each word, the text content, and the pixel spacing vector between two words;

[0017] (2) a wrong character correction module, the wrong character correction module also includes a segmentation submodule for segmenting text content; a candidate set for cyclically replacing wrong characters, the candidate set including a dictionary of similar words and a candidate text matrix, the candidate text matrix being the other k characters corresponding to the first k+1 probability values ​​in each column of the probability matrix, the wrong characters being words that are not in the common word dictionary after segmentation and the probability of any character being less than a threshold;

[0018] (3) a multi-word correction module for cyclically deleting words at positions where the probability vector is less than a threshold value ppl_thre2 and the pixel spacing vector is less than a threshold value d_thre1;

[0019] (4) A missing word correction module is used to cyclically add words at positions where the pixel spacing vector is greater than a threshold value d_thre2.

[0020] The computer-readable storage medium of the present invention stores a computer program thereon, and when the computer program is executed by a processor, the steps of the text error correction method based on OCR of the present invention are implemented.

[0021] Beneficial effects: Compared with the prior art, the present invention has the following advantages: (1) The vocabulary has strong generalization performance. In the error correction process, the probability matrix obtained by the OCR text recognition process is used to calculate the candidate text matrix, and the candidate text matrix is ​​combined with the similar word dictionary as a candidate set for error correction, thereby improving the generalization ability of the model; (2) The error correction accuracy is high. Words that are not in the common word dictionary and whose text probability is less than the threshold are regarded as a word set that may contain errors, thereby reducing the error correction rate and avoiding replacing the correct word with the wrong word; (3) It can correct multiple words and fewer words: The pixel spacing vector between each two words is obtained by segmenting the text line during the text recognition process. The pixel spacing vector is used to judge the position where the multiple words or fewer words may exist, and the error correction of multiple words and fewer words can be performed. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 This is a flowchart of text error correction of the present invention. DETAILED DESCRIPTION

[0023] The technical solution of the present invention will be further described below with reference to the accompanying drawings.

[0024] like Figure 1 As shown, the text error correction method based on OCR of the present invention includes the following steps:

[0025] (1) Text detection and text recognition

[0026] For the input image, text detection algorithms such as DBnet, PSEnet, Craft, etc. are used to detect text lines in the image.

[0027] The detected text position is cropped and binarized, and then fed into the text recognition algorithm CTPN to realize the text recognition process.

[0028] The text recognition process is mainly divided into CRNN+CTC process; specifically, the image is cut with a fixed width of 16 pixels to obtain the cut area A=(A1, A2, ..., A n ). Then predict and classify each area to obtain the probability matrix P of each area belonging to each character (including blank characters) aw =(p ij ) m×n , where m is the total number of characters + 1, i.e., one space character is added, and n is the number of cut regions. At this time, the character corresponding to the maximum probability is selected for each region, and the CTC transcription process is performed to obtain the predicted text content of the text line C = (c1, c2, ..., c l ), the probability vector of the text P=(p1,p2,......,p l ) and calculate the pixel spacing vector between two characters as D = (D 1,2 ,D 2,3 ,......,D l-1,l ). At the same time, we can also use the above probability matrix to obtain the other k characters corresponding to the first k+1 probability values ​​in each column, that is, to obtain the candidate text matrix C cand =(c ij ) k×l , where c ij is the i-th candidate word corresponding to the j-th word.

[0029] Prepare a dictionary of similar words (stroke), a dictionary of common words (word_freq), and train an N-Gram language model. Next, perform text correction for misspellings, extra characters, and missing characters.

[0030] (2) Typo correction

[0031] For the above identified text content C=(c1,c2,......,c l ), use jieba to perform word segmentation, and if the word is not in the common word dictionary word_freq, add the word set that may have errors:

[0032] May_errs=(May_err1,...,May_err M )

[0033] For the May_errs obtained above, loop to obtain its candidate set. iFirst, according to the probability vector P corresponding to the recognized text, l ), get May_err i The probability of each word. For each word, if the predicted probability is greater than or equal to the threshold pre_thre1 (for example: 0.99), then the word will not be corrected. Otherwise, according to the dictionary of similar words, all similar words are found to replace the word, and the candidate set of similar words is obtained. 1,i In addition to finding candidate sets from the dictionary of similar words, it is also necessary to calculate the candidate text matrix C obtained above. cand =(c ij ) k×l Similarly, for the text whose predicted probability is less than the threshold pre_thre1, the k words corresponding to the candidate text matrix are used to replace them to obtain the probability candidate set Candidate 2,i , remove and merge the above similar candidate sets and probability candidate sets to obtain May_err i Candidate set i :

[0034]

[0035] Finally, the total candidate set of May_errs can be obtained:

[0036] Candidate=(Can1,Can2,...,Can N )

[0037] For each word that may be an error, May_err i Candidate set i , cyclically replace the words corresponding to the original text, calculate the confusion of the replaced sentence according to the language model N-Gram, and obtain the confusion vector And calculate the perplexity ppl of the sentence without replacement o , set the threshold ppl_thre, if the minimum value in the confusion vector satisfies the inequality: ppl m ≤ppl_thre*ppl o , then replace it, otherwise keep the original word,

[0038] (3) Multi-word error correction processing

[0039] For the recognized text C=(c1,c2,......,c l ), probability vector P=(p1,p2,......,p l ) and pixel spacing vector D = (D 1,2 ,D2,3 ,......,D l-1,l ), set the threshold and d_thre1. If and If i is 1 or l, then only D needs to be satisfied. i-1,i <d_thre1 or D i,i+1 <d_thre1, then the position is a possible multi-word position, and the candidate set of multi-word positions is obtained:

[0040] Candidate3=(i1,i2,......,i in )

[0041] For the above-mentioned multi-word position candidate set, the original text is looped, and one word is removed each time. For the text after removing the words, the language model N-Gram is used to calculate the sentence perplexity to obtain the perplexity vector

[0042]

[0043] Same confusion as the original sentence ppl o Compare and if the inequality is satisfied: Then remove i j The word at the position is retained otherwise.

[0044] (4) Missing word correction

[0045] For the recognized text C=(c1,c2,......,c l ) and pixel spacing D=(D 1,2 ,D 2,3 ,......,D l-1,l ), set the threshold d_thre2. If D i-1,i > d_thre2, then the positions between i-1 and i are the possible missing word positions. At the same time, since the probability of missing the beginning and end of the text detection is relatively high, the beginning and end positions should also be added to form the candidate set of missing word positions:

[0046] Candidate4=(0,j1,j2,...j jn ,l+1)

[0047] For the candidate set of missing word positions, the Bert model is used to predict the possible words at the position to obtain jn+2 sentences. The language model N-Gram is used to calculate the perplexity PPL4 of these jn+2 sentences respectively. j,n+2 ), take the smallest perplexity and compare it with the original sentence, if it satisfies ppl m ≤ppl_thre*pplo , then replace it, otherwise keep the original sentence unchanged.

[0048] (5) Output the OCR recognized text after error correction.

Claims

1. A text error correction method based on OCR, comprising the following steps: (1) Input the image to be recognized, perform text detection and CRNN+CTC text recognition, and obtain the probability matrix, recognized text content, probability vector and pixel interval vector; The probability vector is composed of the probability value of each word in the text content, and the pixel interval vector is composed of the pixel interval between every two adjacent words in the text content; (2) Get the number of columns before each column in the probability matrix k +1 probability value corresponding to other k characters, and obtain the candidate text matrix; (3) Segment the text content. If the word is not in the common word dictionary and the probability value of any character in the word is less than the first threshold, the candidate set of the character is obtained by using the dictionary of similar words and the candidate text matrix, and cyclic replacement is performed. Otherwise, the original word is retained. (4) If the probability value of the text is less than the second threshold And the pixel interval between the character and its adjacent characters is less than the third threshold , then the probability value is less than the second threshold The position of the text is the position of possible multiple words, and the words are deleted cyclically for all possible multiple words positions, otherwise the original text is retained; (5) If the pixel distance between two adjacent characters is greater than the fourth threshold , then the position between the two adjacent characters is the possible missing character position, and the Bert model is used to predict the possible characters at the corresponding positions of all possible missing characters, and characters are added cyclically, otherwise the original characters are retained; (6) Output the OCR recognized text after error correction.

2. The text error correction method based on OCR according to claim 1, characterized in that: The positions where words may be missed in step (5) also include the beginning and the end of the text.

3. The text error correction method based on OCR according to claim 1, characterized in that: The method for obtaining the candidate set using the similar-looking word dictionary and the candidate text matrix in step (3) is as follows: finding similar-looking words of the word in the similar-looking word dictionary to obtain a similar-looking candidate set; selecting k words corresponding to the word in the candidate text matrix to generate a probabilistic candidate set; The similar candidate set and the probability candidate set are removed and merged to obtain the candidate set of the character.

4. An OCR-based text error correction system based on the method of claim 1, characterized in that: Includes the following modules: The text detection and recognition module is used to perform text detection and CRNN+CTC text recognition on the input image, obtaining the probability matrix, recognized text content, probability vector, and pixel interval vector; A wrong character correction module, the wrong character correction module including a word segmentation submodule for performing word segmentation on text content; The multi-word correction module is used to correct the word when the probability value is less than the second threshold And the pixel interval between each of its adjacent characters is less than the third threshold Delete words cyclically at the position of the text; The missing word correction module is used to correct the missing word when the pixel interval is greater than the fourth threshold The position between two adjacent characters is cyclically added.

5. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the OCR-based text error correction method according to any one of claims 1 to 3 are implemented.

Images