OCR post-processing method and device for Chinese document based on n-gram model and readable medium

Abstract

The invention discloses a Chinese document OCR post-processing method and device based on an n-gram model and a readable medium, and the method comprises the steps: firstly, obtaining the average recognition confidence and position of each textbox through OCR recognition, and judging whether textbox merging needs to be carried out or not through the score of the n-gram model, and if the textbox merging needs to be carried out, carrying out the OCR post-processing on the Chinese document through the n-gram model; and if the n-gram model of the text box is not changed, further judging whether the text result of the combined text box needs to be replaced by the characters with the similar shapes in the character library with the similar shapes, and if the score of the n-gram model of the replaced text result is remarkably improved, outputting the replaced text result as an OCR recognition result, so that the optimization of the recognition result is realized, and the recognition accuracy is improved.

Description

Technical Field

[0001] The present invention relates to the field of OCR recognition, and in particular to an OCR post-processing method, device and readable medium for Chinese documents based on an n-gram model. Background Art

[0002] As a necessary carrier and medium for communication and information exchange, text plays an important role in people's daily life, scientific research and learning. In the electronic age, it is very common to convert paper documents into electronic documents. Not only can important documents be effectively saved, but also the efficiency of document retrieval can be greatly improved. All these operations require the use of OCR (Optical Character Recognition) technology. There are many types of OCR scenarios, such as license plate recognition, business card recognition, document recognition, etc. The interference caused by the environment in different scenarios is different, and the requirements for data accuracy are also different. The common point is that OCR recognition is greatly affected by the environment, and there will always be some errors. Therefore, it is hoped that some methods can be used to optimize the recognition results to improve the accuracy. Common post-processing methods include increasing the connection between contexts and replacing idioms through corpora. Most of these methods use the method of accessing large models. Although this method sometimes works well, it is too divergent and often deviates from the original text. This is an inevitable result caused by the derivative nature of large models. Summary of the invention

[0003] The purpose of this application is to propose an OCR post-processing method, device and readable medium for Chinese documents based on the n-gram model in response to the above-mentioned technical problems.

[0004] In a first aspect, the present invention provides an OCR post-processing method for Chinese documents based on an n-gram model, characterized in that it comprises the following steps:

[0005] Obtain a Chinese document to be recognized and perform OCR recognition to obtain recognition results of several text boxes, wherein the recognition result of each text box includes the position of the text box, the average recognition confidence of the text box, the text result of the text box, and the recognition confidence of each word in the text result, and perform word segmentation on the text result in the recognition result of each text box to obtain the word segmentation result corresponding to each text box;

[0006] In response to determining that the average recognition confidence of a first text box among the plurality of text boxes is less than a first threshold, the word segmentation result corresponding to the first text box is input into an n-gram model for scoring to obtain a score corresponding to the first text box; in response to determining that the score corresponding to the first text box is less than a second threshold, a second text box located near the first text box is determined according to the position of the first text box and the positions of the remaining text boxes among the plurality of text boxes, the word segmentation result corresponding to the second text box is input into an n-gram model for scoring to obtain a score corresponding to the second text box, and whether the first text box and the second text box need to be merged is determined according to the score corresponding to the first text box and the score corresponding to the second text box; in response to determining that the first text box and the second text box need to be merged, a merged text box, a text result of the merged text box, and a recognition confidence of each text in the text result of the merged text box are obtained;

[0007] A similar character library is constructed, and the text result of the merged text box is segmented to obtain the word segmentation result corresponding to the merged text box. The word segmentation result corresponding to the merged text box is input into an n-gram model for scoring to obtain a score corresponding to the merged text box; in response to the score corresponding to the merged text box being greater than or equal to a third threshold, the text result of the merged text box is used as an OCR recognition result; in response to the score corresponding to the merged text box being less than the third threshold, the text in the text result of the merged text box is processed based on the similar character library to obtain an OCR recognition result.

[0008] 2. The OCR post-processing method for Chinese documents based on the n-gram model according to claim 1, characterized in that it also includes:

[0009] In response to determining that an average recognition confidence of a first text box among the plurality of text boxes is greater than or equal to a first threshold, directly taking a text result of the first text box as an OCR recognition result;

[0010] In response to determining that the first text box and the second text box do not need to be merged, the text in the text result of the first text box is processed based on the similar character library to obtain an OCR recognition result.

[0011] 3. The OCR post-processing method for Chinese documents based on the n-gram model according to claim 2, characterized in that the process of obtaining the OCR recognition result specifically includes:

[0012] Determine whether there is a character whose recognition confidence is less than a fourth threshold in the character result of the merged text box or the character result of the first text box and whether a similar character corresponding to the character can be found in the similar character library;

[0013] In response to determining that there is a character whose recognition confidence is less than a fourth threshold value in the text result of the merged text box or the text result of the first text box and the similar character corresponding to the character can be found in the similar character library, the corresponding similar characters are used to replace the characters whose recognition confidence is less than the fourth threshold value in the text result of the merged text box or the text result of the first text box one by one to obtain a replaced text result; the replaced text result is segmented to obtain a segmentation result corresponding to the replaced text result, and the segmentation result corresponding to the replaced text result is input into an n-gram model for scoring to obtain a score corresponding to the replaced text result;

[0014] In response to determining that there is no character with a recognition confidence less than a fourth threshold in the character result of the merged text box or the character result of the first text box, or that a similar character corresponding to the character cannot be found in the similar character library, directly using the character result of the merged text box or the character result of the first text box as the OCR recognition result;

[0015] In response to determining that the score corresponding to the replaced text result is greater than the fifth threshold or greater than m times the score corresponding to the merged text box or the first text box, where m is greater than 1, taking the replaced text result as the OCR recognition result;

[0016] In response to determining that the score corresponding to the replaced text result is less than or equal to the fifth threshold or less than and equal to m times the score corresponding to the merged text box or the first text box, the above steps are repeated.

[0017] 4. The OCR post-processing method for Chinese documents based on the n-gram model according to claim 3 is characterized in that the second text box located near the first text box is determined according to the position of the first text box and the positions of the remaining text boxes among the plurality of text boxes, specifically comprising:

[0018] The position of the first text box includes the lower left vertex (x 1a ,y 1a ), the lower right vertex (x 2a ,y 2a ), the upper left vertex (x 3a ,y 3a ) and the upper right vertex (x 4a ,y 4a ) four coordinates, the position of another text box in the plurality of text boxes except the first text box includes the lower left vertex (x 1b ,y 1b ), the lower right vertex (x 2b ,y 2b ), the upper left vertex (x 3b ,y3b ) and the upper right vertex (x 4b ,y 4b ) four coordinates;

[0019] If (y 1a -y 3b )<0.01(y 1a -y 2b ) and (x 1a -x 1b )(x 2a -x 2b )<0.01or(x 1b -x 2a )<0.01(y 1a -y 2b ), then the other text box is the second text box.

[0020] 5. The OCR post-processing method for Chinese documents based on the n-gram model according to claim 1, characterized in that judging whether to merge the first text box and the second text box according to the score corresponding to the first text box and the score corresponding to the second text box specifically comprises:

[0021] Merge the first text box and the second text box to obtain a merged text result, perform word segmentation, obtain a word segmentation result corresponding to the merged text result, input the word segmentation result corresponding to the merged text result into an n-gram model for scoring, and obtain a score corresponding to the merged text result;

[0022] In response to determining that the score corresponding to the merged text result is greater than s times the product of the score corresponding to the first text box and the score corresponding to the second text box, and s is greater than or equal to 1, determining that the first text box and the second text box need to be merged;

[0023] In response to determining that the score corresponding to the merged text result is less than or equal to s times the product of the score corresponding to the first text box and the score corresponding to the second text box, and s is greater than or equal to 1, it is determined that the first text box and the second text box do not need to be merged.

[0024] As a preferred method, constructing a similar character library specifically includes:

[0025] S41, obtaining all characters in the OCR dictionary and forming a character set;

[0026] S42, using the pygame.font tool to generate an image corresponding to each character in the character set;

[0027] S43, performing maximum pooling on the image corresponding to each character to convert it into a tensor corresponding to each character, and normalizing the tensor corresponding to each character to obtain preprocessed data corresponding to each word;

[0028] S44, inputting the preprocessed data corresponding to each character into a convolutional neural network, wherein the convolutional neural network includes a first convolutional layer, a second convolutional layer, and a third convolutional layer, wherein the first convolutional layer, the second convolutional layer, and the third convolutional layer output a first feature, a second feature, and a third feature, respectively, and calculating the total vector corresponding to each character using the following formula:

[0029] v=p 1 *t 1 +p 2 *t 2 +p 3 *t 3 +p 4 *k 1 ;

[0030] Among them, v represents the total vector corresponding to each word, t 1 ,t 2 and t 3 Represent the first feature, the second feature and the third feature respectively, k 1 represents preprocessed data, p 1 、p 2 、p 3 and p 4 are the corresponding weights respectively;

[0031] S45, by comparing the similarities of the total vectors of different characters, finding g groups of characters with the highest similarity of the total vectors;

[0032] S46, comparing the g group of characters with the highest similarity with the G group of similar characters searched online, and calculating the inclusion rate of the G group of similar characters searched online by the similar character library according to the following formula:

[0033]

[0034] Wherein, f represents the inclusion rate, c represents the number of groups of characters in the g group with the highest similarity that contain the G group of similar characters searched online;

[0035] S47, find the p in step S44 by simulated annealing algorithm 1 、p 2 、p 3 and p 4 The p corresponding to the maximum value of the inclusion rate calculated in steps S45 and S46 after adjustment 1 、p 2 、p 3 and p4 , according to the p corresponding to the maximum value of the inclusion rate 1 、p 2 、p 3 and p 4 After step S45, the g group of characters with the highest total vector similarity are classified into the similar character library.

[0036] In a second aspect, the present invention provides an OCR post-processing device for Chinese documents based on an n-gram model, comprising:

[0037] The recognition module is configured to obtain a Chinese document to be recognized and perform OCR recognition to obtain recognition results of several text boxes, wherein the recognition result of each text box includes the position of the text box, the average recognition confidence of the text box, the text results in the text box, and the recognition confidence of each word in the text results, and perform word segmentation on the text results in the recognition results of each text box to obtain the word segmentation results corresponding to each text box;

[0038] A merging module is configured to, in response to determining that an average recognition confidence of a first text box among a plurality of text boxes is less than a first threshold, input a word segmentation result corresponding to the first text box into an n-gram model for scoring to obtain a score corresponding to the first text box; in response to determining that the score corresponding to the first text box is less than a second threshold, determine a second text box located near the first text box according to a position of the first text box and positions of the remaining text boxes among the plurality of text boxes, input a word segmentation result corresponding to the second text box into an n-gram model for scoring to obtain a score corresponding to the second text box, and determine whether it is necessary to merge the first text box and the second text box according to the score corresponding to the first text box and the score corresponding to the second text box; in response to determining that it is necessary to merge the first text box and the second text box, obtain a merged text box, a text result of the merged text box, and a recognition confidence of each text in the text result of the merged text box;

[0039] The output module is configured to construct a similar character library, segment the text results of the merged text box, obtain the segmentation results corresponding to the merged text box, input the segmentation results corresponding to the merged text box into the n-gram model for scoring, and obtain the score corresponding to the merged text box; the module is configured to take the text results of the merged text box as the OCR recognition result in response to the score corresponding to the merged text box being greater than or equal to a third threshold; in response to the score corresponding to the merged text box being less than the third threshold, process the text in the text results of the merged text box based on the similar character library to obtain the OCR recognition result.

[0040] In a third aspect, the present invention provides an electronic device comprising one or more processors; a storage device for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the method described in any implementation manner in the first aspect.

[0041] In a fourth aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method described in any implementation manner in the first aspect.

[0042] In a fifth aspect, the present invention provides a computer program product, comprising a computer program, which, when executed by a processor, implements the method described in any implementation manner in the first aspect.

[0043] Compared with the prior art, the present invention has the following beneficial effects:

[0044] (1) The OCR post-processing method for Chinese documents based on the n-gram model proposed in the present invention can accurately solve specific problems in recognition and effectively improve recognition accuracy in specific situations by focusing on optimizing text results with abnormal scores of the n-gram model.

[0045] (2) The OCR post-processing method for Chinese documents based on the n-gram model proposed in the present invention can specifically process characters with similar shapes by constructing a similar character library. This method can effectively correct recognition errors caused by similar character shapes.

[0046] (3) The OCR post-processing method for Chinese documents based on the n-gram model proposed in the present invention is relatively simple in processing flow. Post-processing is performed through the n-gram model and a similar character library. The steps are clear and targeted, which reduces the complex processing of the entire text. The recognition confidence and n-gram score are used to determine whether to perform post-processing, making the optimization process dynamically adaptable and avoiding a one-size-fits-all treatment of all recognition results. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0048] Figure 1 A schematic flow chart of an OCR post-processing method for Chinese documents based on an n-gram model according to an embodiment of the present application;

[0049] Figure 2A flowchart of an OCR post-processing method for Chinese documents based on an n-gram model according to an embodiment of the present application;

[0050] Figure 3 A schematic diagram of an OCR post-processing device for Chinese documents based on an n-gram model according to an embodiment of the present application;

[0051] Figure 4 A schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0052] In order to make the purpose, technical scheme and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0053] Figure 1 An OCR post-processing method for Chinese documents based on an n-gram model provided in an embodiment of the present application is shown, comprising the following steps:

[0054] S1, obtain the Chinese document to be recognized and perform OCR recognition to obtain the recognition results of several text boxes. The recognition result of each text box includes the position of the text box, the average recognition confidence of the text box, the text results in the text box, and the recognition confidence of each word in the text results. The text results in the recognition results of each text box are segmented to obtain the word segmentation results corresponding to each text box.

[0055] Specifically, refer to Figure 2 By scanning the Chinese document, the recognition result of each text box is output. The recognition result of each text box includes four parameters: the position of the text box, the average recognition confidence of the text box, the text result in the text box, and the recognition confidence of each word in the text result. The output format of the text box position is: [(x 1 ,y 1 )、(x 2 ,y 2 )、(x 3 ,y 3 )、(x 4 ,y 4 )] The four coordinates represent the lower left vertex, lower right vertex, upper left vertex, and upper right vertex of the rectangular text box. The output form of all confidences in the text box is Among them, P 1 ,P 2 ,P 3 ,P 4,…,P d Indicates the recognition confidence of the first d characters in the text box sorted from left to right. Represents the average recognition confidence.

[0056] The text result of OCR output is segmented using mainstream word segmentation tools to obtain the word segmentation result corresponding to each text box. In an embodiment of the present application, jieba word segmentation tool is used for word segmentation. Jieba word segmentation tool uses prefix dictionary to build an efficient word graph scanning mechanism to form a directed acyclic graph (DAG) of words that all Chinese characters in a sentence may form. Then, the maximum probability path is searched by a dynamic programming algorithm, and the optimal phrase segmentation method is found based on word frequency. For words that do not exist in the vocabulary, jieba word segmentation tool introduces a hidden Markov model (HMM) based on the ability of Chinese characters to form words, and uses the Viterbi algorithm to determine the most likely hidden state sequence, effectively reducing the interference of sequence noise. The combination of these technologies improves the accuracy and efficiency of word segmentation.

[0057] S2, in response to determining that the average recognition confidence of a first text box among several text boxes is less than a first threshold, the word segmentation result corresponding to the first text box is input into the n-gram model for scoring to obtain a score corresponding to the first text box; in response to determining that the score corresponding to the first text box is less than a second threshold, a second text box located near the first text box is determined according to the position of the first text box and the positions of the remaining text boxes among the several text boxes, the word segmentation result corresponding to the second text box is input into the n-gram model for scoring to obtain a score corresponding to the second text box, and whether the first text box and the second text box need to be merged is determined according to the score corresponding to the first text box and the score corresponding to the second text box; in response to determining that the first text box and the second text box need to be merged, the merged text box, the text result of the merged text box, and the recognition confidence of each text in the text result of the merged text box are obtained.

[0058] Specifically, the embodiment of the present application constructs an n-gram model, where n is 2 or 3, and the n-gram model is comprehensively scored through a 3-gram model and a 2-gram model to obtain a score.

[0059] The calculation process of the n-gram model before optimization used in the embodiment of the present application is shown in the following formula:

[0060]

[0061] Among them, P(w i-n+1 |w i-n+2 |…|w i-1 ) represents the scoring result, w i Indicates the current detection word, wi-n+1 represents the first n-1 words of the current detection word, C(w i-n+1 ,…,w i ) represents the number of times the phrase consisting of all words from the first n-1 words of the current detection word to the current detection word appears in the n-gram vocabulary, C(w i-n+1 ,...,w i-1 ) represents the number of times the phrase consisting of all words from the first n-1 words of the current detection word to the first word of the current detection word appears in the (n-1)-gram vocabulary;

[0062] The absolute smoothing technique is used to optimize the scoring results calculated by the above n-gram model, as shown in the following formula:

[0063] If C(w i-n+1 ,...,w i-1 )≠0, the following formula is used to calculate the score output by the n-gram model:

[0064]

[0065] If C(w i-n+1 ,…,w i-1 )=0, the following formula is used to calculate the score output by the n-gram model:

[0066]

[0067] Here, d represents the discount constant.

[0068] If the score output by the n-gram model corresponding to a text box is too low, the text boxes near the text box are detected according to the position of the text box to determine whether the two text boxes need to be merged.

[0069] In a specific embodiment, determining a second text box located near the first text box according to the position of the first text box and the positions of the remaining text boxes among the plurality of text boxes specifically includes:

[0070] The position of the first text box includes the lower left vertex (x 1a ,y 1a ), the lower right vertex (x 2a ,y 2a ), the upper left vertex (x 3a ,y 3a ) and the upper right vertex (x 4a ,y 4a ) four coordinates, the position of another text box in the plurality of text boxes except the first text box includes the lower left vertex (x 1b ,y 1b ), the lower right vertex (x2b ,y 2b ), the upper left vertex (x 3b ,y 3b ) and the upper right vertex (x 4b ,y 4b ) four coordinates;

[0071] If (y 1a -y 3b )<0.01(y 1a -y 2b ) and (x 1a -x 1b )(x 2a -x 2b )<0.01or(x 1b -x 2a )<0.01(y 1a -y 2b ), the other text box is the second text box.

[0072] In a specific embodiment, judging whether to merge the first text box and the second text box according to the score corresponding to the first text box and the score corresponding to the second text box specifically includes:

[0073] The first text box and the second text box are combined to obtain a combined text result, and then segmented to obtain a segmentation result corresponding to the combined text result. The segmentation result corresponding to the combined text result is input into an n-gram model for scoring to obtain a score corresponding to the combined text result.

[0074] In response to determining that the score corresponding to the merged text result is greater than s times the product of the score corresponding to the first text box and the score corresponding to the second text box, and s is greater than or equal to 1, determining that the first text box and the second text box need to be merged;

[0075] In response to determining that the score corresponding to the merged text result is less than or equal to s times the product of the score corresponding to the first text box and the score corresponding to the second text box, and s is greater than or equal to 1, it is determined that there is no need to merge the first text box and the second text box.

[0076] In a specific embodiment, it also includes:

[0077] In response to determining that an average recognition confidence of a first text box among the plurality of text boxes is greater than or equal to a first threshold, directly taking a text result of the first text box as an OCR recognition result;

[0078] In response to determining that the first text box and the second text box do not need to be merged, the text in the text result of the first text box is processed based on a similar character library to obtain an OCR recognition result.

[0079] Specifically, if the average recognition confidence of the first text box is greater than or equal to the first threshold, no processing will be performed, and the text result of the first text box will be directly output as the OCR recognition result. If the average recognition confidence of the first text box is less than the first threshold, it is further determined whether the first text box needs to be merged with the text boxes nearby. There are two conditions for determining whether the first text box and the second text box need to be merged. One is that a second text box that satisfies the positional relationship can be found, and the score corresponding to the merged text result is greater than s times the product of the score corresponding to the first text box and the score corresponding to the second text box. If the above two conditions cannot be met at the same time, the text in the text result of the first text box is processed based on the similar character library to obtain the OCR recognition result.

[0080] Specifically, assuming that there are two text boxes, a first text box and a second text box, corresponding to symbols a and b respectively, the position of the first text box includes the lower left vertex (x 1a ,y 1a ), the lower right vertex (x 2a ,y 2a ), the upper left vertex (x 3a ,y 3a ) and the upper right vertex (x 4a ,y 4a ), the position of the second text box includes the lower left vertex (x 1b ,y 1b ), the lower right vertex (x 2b ,y 2b ), the upper left vertex (x 3b ,y 3b ) and the upper right vertex (x 4b ,y 4b ), and calculate the score P corresponding to the first text box respectively a The score P corresponding to the second text box b , if the following two situations exist:

[0081] Case 1: If (y 1a -y 3b )<0.01(y 1a -y 2b ) and (x 1a -x 1b )(x 2a -x 2b)<0.01, which means that the second text box is below the first text box. Then the second text box is merged to the right of the first text box. At the same time, the text result of the first text box is merged with the text result of the second text box to obtain a merged text result. The merged text result is segmented to obtain the word segmentation result corresponding to the merged text result. The word segmentation result corresponding to the merged text result is input into the n-gram model for scoring to obtain the score h corresponding to the merged text result. If h>s×P a ×P b , then the first text box is merged with the second text box to obtain a merged text box, and the merged text result is used as the text result of the merged text box.

[0082] Case 2: If (x 1b -x 2a )<0.01(y 1a -y 2b ), which means that the second text box is to the right of the first text box, so the second text box is merged to the right of the first text box, and the text result of the first text box is merged with the text result of the second text box to obtain a merged text result, and the merged text result is segmented to obtain the word segmentation result corresponding to the merged text result, and the word segmentation result corresponding to the merged text result is input into the n-gram model for scoring to obtain the score h corresponding to the merged text result. If h>s×P a ×P b , then the first text box is merged with the second text box to obtain a merged text box, and the merged text result is used as the text result of the merged text box.

[0083] If the second text box that meets the above coordinate requirements cannot be found, the text in the text result of the first text box is directly replaced based on the similar character library to obtain the OCR recognition result.

[0084] If there exists h≤s×P a ×P b , it means that the first text box and the second text box do not meet the requirements of merging, and the text in the text result of the first text box is directly replaced based on the similar character library to obtain the OCR recognition result.

[0085] S3, constructing a library of similar characters, segmenting the text results of the merged text box, obtaining the word segmentation results corresponding to the merged text box, inputting the word segmentation results corresponding to the merged text box into the n-gram model for scoring, and obtaining the score corresponding to the merged text box; in response to the score corresponding to the merged text box being greater than or equal to a third threshold, taking the text results of the merged text box as the OCR recognition result; in response to the score corresponding to the merged text box being less than the third threshold, processing the text in the text results of the merged text box based on the library of similar characters to obtain the OCR recognition result.

[0086] In a specific embodiment, constructing a similar character library specifically includes:

[0087] S41, obtaining all characters in the OCR dictionary and forming a character set;

[0088] S42, using the pygame.font tool to generate an image corresponding to each character in the character set;

[0089] S43, performing maximum pooling on the image corresponding to each character to convert it into a tensor corresponding to each character, and performing normalization processing on the tensor corresponding to each character to obtain preprocessed data corresponding to each word;

[0090] S44, input the preprocessed data corresponding to each character into a convolutional neural network, the convolutional neural network includes a first convolutional layer, a second convolutional layer and a third convolutional layer, the first convolutional layer, the second convolutional layer and the third convolutional layer output a first feature, a second feature and a third feature respectively, and the total vector corresponding to each character is calculated using the following formula:

[0091] v=p 1 *t 1 +p 2 *t 2 +p 3 *t 3 +p 4 *k 1 ;

[0092] Among them, v represents the total vector corresponding to each word, t 1 ,t 2 and t 3 Represent the first feature, the second feature and the third feature respectively, k 1 represents preprocessed data, p 1 、p 2 、p 3 and p 4 are the corresponding weights respectively;

[0093] S45, by comparing the similarities of total vectors of different characters, find g groups of characters with the highest total vector similarity;

[0094] S46, comparing the g group of characters with the highest similarity with the G group of similar characters searched online, and calculating the inclusion rate of the similar character library to the G group of similar characters searched online according to the following formula:

[0095]

[0096] Among them, f represents the inclusion rate, c is the number of groups of characters with the highest similarity that contain the G group of similar characters searched online;

[0097] S47, find the p in step S44 by simulated annealing algorithm 1 、p 2 、p 3 and p 4 The p corresponding to the maximum value of the inclusion rate calculated in steps S45 and S46 after adjustment 1 、p 2 、p 3 and p 4 , according to the p corresponding to the maximum value of the inclusion rate 1 、p 2 、p 3 and p 4 After step S45, the g group of characters with the highest total vector similarity are classified into the similar character library.

[0098] Specifically, after the merged text box is obtained, some text in the text result of the merged text box may cause abnormal scores output by the n-gram model. Therefore, it is necessary to further use a similar character library to replace some text in the text result of the merged text box to obtain the OCR recognition result.

[0099] In a specific embodiment, the process of obtaining the OCR recognition result specifically includes:

[0100] Traversing each character in the text result of the merged text box or the text result of the first text box, determining whether there is a character whose recognition confidence is less than a fourth threshold in the text result of the merged text box or the text result of the first text box and a similar character corresponding to the character can be found in the similar character library;

[0101] In response to determining that there is a character whose recognition confidence is less than a fourth threshold value in the text result of the merged text box or the text result of the first text box and a similar character corresponding to the character can be found in a similar character library, the corresponding similar characters are used to replace the characters whose recognition confidence is less than the fourth threshold value in the text result of the merged text box or the text result of the first text box one by one to obtain a replaced text result; segmenting the replaced text result to obtain a segmentation result corresponding to the replaced text result, and inputting the segmentation result corresponding to the replaced text result into an n-gram model for scoring to obtain a score corresponding to the replaced text result;

[0102] In response to determining that there is no character with a recognition confidence less than a fourth threshold in the character result of the merged text box or the character result of the first text box, or a similar character corresponding to the character cannot be found in the similar character library, directly using the character result of the merged text box or the character result of the first text box as the OCR recognition result;

[0103] In response to determining that the score corresponding to the replaced text result is greater than the fifth threshold or greater than m times the score corresponding to the merged text box or the first text box, where m is greater than 1, obtaining the replaced text result;

[0104] In response to determining that the score corresponding to the replaced text result is less than or equal to the fifth threshold or less than and equal to m times the score corresponding to the merged text box or the first text box, the above steps are repeated until all replaced text results are obtained and used as the OCR recognition result.

[0105] Specifically, first calculate the score corresponding to the merged text box, and determine whether the score corresponding to the merged text box is less than the third threshold. If so, further compare the recognition confidence of each character in the text result of the merged text box with the fourth threshold. If there is a character with a recognition confidence less than the fourth threshold in the text result of the merged text box, match the character with the corresponding similar character in the similar character library. If there is a corresponding similar character, replace the character with the similar character, and recalculate the score corresponding to the replaced text result. If the score corresponding to the replaced text result is greater than the fifth threshold or significantly improved, output the replaced text result, and give priority to replacing characters with lower recognition confidence.

[0106] The process of processing the text in the text result of the first text box based on the similar character library is the same as the above-mentioned process of processing the text in the text result of the merged text box based on the similar character library.

[0107] The Chinese document to be recognized, the OCR recognition result and whether it is effective (0 or 1) are taken as a piece of data in the database. The database can be used as an OCR training set after manual correction to further improve the accuracy of OCR recognition.

[0108] Further references Figure 3 As an implementation of the methods shown in the above figures, the present application provides an embodiment of an OCR post-processing device for Chinese documents based on an n-gram model. Figure 1 Corresponding to the method embodiment shown, the device can be specifically applied to various electronic devices.

[0109] The embodiment of the present application provides an OCR post-processing device for Chinese documents based on an n-gram model, comprising:

[0110] The recognition module 1 is configured to obtain a Chinese document to be recognized and perform OCR recognition to obtain recognition results of several text boxes, wherein the recognition result of each text box includes a position, an average recognition confidence, a text result, and a recognition confidence of each text in the text result, and segment the text results in the recognition result of each text box to obtain a segmentation result corresponding to each text box;

[0111] The merging module 2 is configured to, in response to determining that the average recognition confidence of a first text box among the plurality of text boxes is less than a first threshold, input the word segmentation result corresponding to the first text box into the n-gram model for scoring, and obtain the score corresponding to the first text box; in response to determining that the score corresponding to the first text box is less than a second threshold, determine a second text box located near the first text box according to the position of the first text box and the positions of the remaining text boxes among the plurality of text boxes, input the word segmentation result corresponding to the second text box into the n-gram model for scoring, and obtain the score corresponding to the second text box; determine whether the first text box and the second text box need to be merged according to the score corresponding to the first text box and the score corresponding to the second text box; in response to determining that the first text box and the second text box need to be merged, obtain the merged text box, the text result of the merged text box, and the recognition confidence of each text in the text result of the merged text box;

[0112] The output module 3 is configured to construct a similar character library, segment the text results of the merged text box, obtain the segmentation results corresponding to the merged text box, input the segmentation results corresponding to the merged text box into the n-gram model for scoring, and obtain the score corresponding to the merged text box; the module is configured to respond to the score corresponding to the merged text box being greater than or equal to the third threshold, then use the text results of the merged text box as the OCR recognition result; respond to the score corresponding to the merged text box being less than the third threshold, then process the text in the text results of the merged text box based on the similar character library to obtain the OCR recognition result.

[0113] Figure 4 Schematic diagram of the hardware structure of the electronic device provided by the embodiment of the present invention. Figure 4 As shown, the electronic device of this embodiment includes: a processor 401 and a memory 402; wherein the memory 402 is used to store computer-executable instructions; the processor 401 is used to execute the computer-executable instructions stored in the memory to implement the various steps performed by the electronic device in the above embodiment. For details, please refer to the relevant description in the above method embodiment.

[0114] Optionally, the memory 402 may be independent or integrated with the processor 401 .

[0115] When the memory 402 is independently provided, the electronic device further includes a bus 403 for connecting the memory 402 and the processor 401 .

[0116] The embodiment of the present invention further provides a computer storage medium, in which computer execution instructions are stored. When the processor 401 executes the computer execution instructions, the above method is implemented.

[0117] The embodiment of the present invention further provides a computer program product, including a computer program. When the computer program is executed by the processor 401, the above method is implemented.

[0118] In the embodiments provided by the present invention, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of modules is only a logical function division, and there may be other division methods in actual implementation, such as multiple modules can be combined or integrated into another system, or some features can be ignored or not executed. Another point, the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or modules, which can be electrical, mechanical or other forms.

[0119] The modules described as separate components may or may not be physically separated, and the components shown as modules may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to implement the solution of this embodiment.

[0120] In addition, each functional module in each embodiment of the present invention may be integrated into one processing unit, each module may exist physically separately, or two or more modules may be integrated into one unit. The unit formed by the above modules may be implemented in the form of hardware or in the form of hardware plus software functional units.

[0121] The above-mentioned integrated module implemented in the form of a software function module can be stored in a computer-readable storage medium. The above-mentioned software function module is stored in a storage medium, including a number of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor 401 to perform some steps of the methods of various embodiments of the present application.

[0122] It should be understood that the processor 401 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), etc. A general-purpose processor may be a microprocessor or the processor 401 may be any conventional processor 401, etc. The steps of the method disclosed in the invention may be directly embodied in the hardware processor 401 for execution, or may be executed by a combination of hardware and software modules in the processor 401.

[0123] The memory 402 may include a high-speed RAM memory, and may also include a non-volatile storage NVM, such as at least one disk memory, and may also be a USB flash drive, a mobile hard disk, a read-only memory, a magnetic disk, or an optical disk.

[0124] The bus 403 may be an Industry Standard Architecture (ISA), a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. The bus 403 may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, the bus 403 in the drawings of the present application is not limited to only one bus 403 or one type of bus 403.

[0125] The above storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk. The storage medium can be any available medium that can be accessed by a general or special purpose computer.

[0126] An exemplary storage medium is coupled to the processor 401, so that the processor 401 can read information from the storage medium and write information to the storage medium. Of course, the storage medium can also be a component of the processor 401. The processor 401 and the storage medium can be located in an application specific integrated circuit (ASIC). Of course, the processor 401 and the storage medium can also exist as discrete components in an electronic device or a main control device.

[0127] Those skilled in the art can understand that all or part of the steps of implementing the above-mentioned method embodiments can be completed by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, the steps of the above-mentioned method embodiments are executed; and the aforementioned storage medium includes: ROM, RAM, disk or optical disk and other media that can store program codes.

[0128] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A Chinese document OCR post-processing method based on n-gram model, characterized in that: The following steps are involved: Obtain a Chinese document to be recognized and perform OCR recognition to obtain recognition results of several text boxes, wherein the recognition result of each text box includes the position of the text box, the average recognition confidence of the text box, the text result of the text box, and the recognition confidence of each word in the text result, and perform word segmentation on the text result in the recognition result of each text box to obtain the word segmentation result corresponding to each text box; In response to determining that an average recognition confidence of a first text box among the plurality of text boxes is less than a first threshold, inputting a word segmentation result corresponding to the first text box into an n-gram model for scoring to obtain a score corresponding to the first text box; In response to determining that the score corresponding to the first text box is less than a second threshold, determining a second text box located near the first text box according to the position of the first text box and the positions of the remaining text boxes among the plurality of text boxes, inputting the word segmentation result corresponding to the second text box into the n-gram model for scoring to obtain a score corresponding to the second text box, and determining whether the first text box and the second text box need to be merged according to the score corresponding to the first text box and the score corresponding to the second text box; In response to determining that the first text box and the second text box need to be merged, obtaining a merged text box, a text result of the merged text box, and a recognition confidence of each character in the text result of the merged text box; Constructing a similar character library, performing word segmentation on the text results of the merged text box to obtain a word segmentation result corresponding to the merged text box, inputting the word segmentation result corresponding to the merged text box into an n-gram model for scoring, and obtaining a score corresponding to the merged text box; In response to the score corresponding to the merged text box being greater than or equal to a third threshold, the text result of the merged text box is used as the OCR recognition result; in response to the score corresponding to the merged text box being less than the third threshold, the text in the text result of the merged text box is processed based on the similar character library to obtain the OCR recognition result.

2. The OCR post-processing method for Chinese documents based on the n-gram model according to claim 1, characterized in that: Also includes: In response to determining that an average recognition confidence of a first text box among the plurality of text boxes is greater than or equal to a first threshold, directly taking a text result of the first text box as an OCR recognition result; In response to determining that the first text box and the second text box do not need to be merged, the text in the text result of the first text box is processed based on the similar character library to obtain an OCR recognition result.

3. The OCR post-processing method for Chinese documents based on the n-gram model according to claim 2, characterized in that: The process of obtaining the OCR recognition result specifically includes: Determine whether there is a character whose recognition confidence is less than a fourth threshold in the character result of the merged text box or the character result of the first text box and whether a similar character corresponding to the character can be found in the similar character library; In response to determining that there is a character whose recognition confidence is less than a fourth threshold value in the text result of the merged text box or the text result of the first text box and the similar character corresponding to the character can be found in the similar character library, the corresponding similar characters are used to replace the characters whose recognition confidence is less than the fourth threshold value in the text result of the merged text box or the text result of the first text box one by one to obtain a replaced text result; the replaced text result is segmented to obtain a segmentation result corresponding to the replaced text result, and the segmentation result corresponding to the replaced text result is input into an n-gram model for scoring to obtain a score corresponding to the replaced text result; In response to determining that there is no character with a recognition confidence less than a fourth threshold in the character result of the merged text box or the character result of the first text box, or that a similar character corresponding to the character cannot be found in the similar character library, directly using the character result of the merged text box or the character result of the first text box as the OCR recognition result; In response to determining that the score corresponding to the replaced text result is greater than the fifth threshold or greater than m times the score corresponding to the merged text box or the first text box, where m is greater than 1, taking the replaced text result as the OCR recognition result; In response to determining that the score corresponding to the replaced text result is less than or equal to the fifth threshold or less than and equal to m times the score corresponding to the merged text box or the first text box, the above steps are repeated.

4. The OCR post-processing method for Chinese documents based on the n-gram model according to claim 3, characterized in that: Determining a second text box located near the first text box according to the position of the first text box and the positions of the remaining text boxes among the plurality of text boxes specifically includes: The position of the first text box includes the lower left vertex (x 1a ,y 1a ), the lower right vertex (x 2a ,y 2a ), the upper left vertex (x 3a ,y 3a ) and the upper right vertex (x 4a ,y 4a ) four coordinates, the position of another text box in the plurality of text boxes except the first text box includes the lower left vertex (x 1b ,y 1b ), the lower right vertex (x 2b ,y 2b ), the upper left vertex (x 3b ,y 3b ) and the upper right vertex (x 4b ,y 4b ) four coordinates; If (y 1a -y 3b )<0.01(y 1a -y 2b ) and (x 1a -x 1b )(x 2a -x 2b )<0.01or(x 1b -x 2a )<0.01(y 1a -y 2b ), then the other text box is the second text box.

5. The OCR post-processing method for Chinese documents based on the n-gram model according to claim 1, characterized in that: Determining whether to merge the first text box and the second text box according to the score corresponding to the first text box and the score corresponding to the second text box specifically includes: Merge the first text box and the second text box to obtain a merged text result, perform word segmentation, obtain a word segmentation result corresponding to the merged text result, input the word segmentation result corresponding to the merged text result into an n-gram model for scoring, and obtain a score corresponding to the merged text result; In response to determining that the score corresponding to the merged text result is greater than s times the product of the score corresponding to the first text box and the score corresponding to the second text box, and s is greater than or equal to 1, determining that the first text box and the second text box need to be merged; In response to determining that the score corresponding to the merged text result is less than or equal to s times the product of the score corresponding to the first text box and the score corresponding to the second text box, and s is greater than or equal to 1, it is determined that the first text box and the second text box do not need to be merged.

6. The OCR post-processing method for Chinese documents based on the n-gram model according to claim 1, characterized in that: Construct a similar character library, including: S41, obtaining all characters in the OCR dictionary and forming a character set; S42, using the pygame.font tool to generate an image corresponding to each character in the character set; S43, performing maximum pooling on the image corresponding to each character to convert it into a tensor corresponding to each character, and normalizing the tensor corresponding to each character to obtain preprocessed data corresponding to each word; S44, inputting the preprocessed data corresponding to each character into a convolutional neural network, wherein the convolutional neural network includes a first convolutional layer, a second convolutional layer, and a third convolutional layer, wherein the first convolutional layer, the second convolutional layer, and the third convolutional layer output a first feature, a second feature, and a third feature, respectively, and calculating the total vector corresponding to each character using the following formula: v=p1*t1+p2*t2+p3*t3+p4*k1; Where v represents the total vector corresponding to each word, t1, t2 and t3 represent the first feature, the second feature and the third feature respectively, k1 represents the preprocessed data, and p1, p2, p3 and p4 are the corresponding weights respectively; S45, by comparing the similarities of the total vectors of different characters, finding g groups of characters with the highest similarity of the total vectors; S46, comparing the g group of characters with the highest similarity with the G group of similar characters searched online, and calculating the inclusion rate of the G group of similar characters searched online by the similar character library according to the following formula: Wherein, f represents the inclusion rate, c represents the number of groups of characters in the g group with the highest similarity that contain the G group of similar characters searched online; S47, through the simulated annealing algorithm, find p1, p2, p3 and p4 corresponding to the maximum value of the inclusion rate calculated by steps S45 and S46 after the adjustment of p1, p2, p3 and p4 in step S44, and determine the g groups of characters with the highest similarity of the total vector through step S45 according to the p1, p2, p3 and p4 corresponding to the maximum value of the inclusion rate and include them in the similar character library.

7. An OCR post-processing device for Chinese documents based on the n-gram model, characterized in that: include: The recognition module is configured to obtain a Chinese document to be recognized and perform OCR recognition to obtain recognition results of several text boxes, wherein the recognition result of each text box includes the position of the text box, the average recognition confidence of the text box, the text results in the text box, and the recognition confidence of each word in the text results, and perform word segmentation on the text results in the recognition results of each text box to obtain the word segmentation results corresponding to each text box; a merging module configured to, in response to determining that an average recognition confidence of a first text box among the plurality of text boxes is less than a first threshold, input a word segmentation result corresponding to the first text box into an n-gram model for scoring, and obtain a score corresponding to the first text box; In response to determining that the score corresponding to the first text box is less than a second threshold, determining a second text box located near the first text box according to the position of the first text box and the positions of the remaining text boxes among the plurality of text boxes, inputting the word segmentation result corresponding to the second text box into the n-gram model for scoring to obtain a score corresponding to the second text box, and determining whether the first text box and the second text box need to be merged according to the score corresponding to the first text box and the score corresponding to the second text box; In response to determining that the first text box and the second text box need to be merged, obtaining a merged text box, a text result of the merged text box, and a recognition confidence of each character in the text result of the merged text box; an output module configured to construct a similar character library, segment the text results of the merged text box to obtain a segmentation result corresponding to the merged text box, input the segmentation result corresponding to the merged text box into an n-gram model for scoring, and obtain a score corresponding to the merged text box; The module is configured to, in response to the score corresponding to the merged text box being greater than or equal to a third threshold, use the text result of the merged text box as the OCR recognition result; in response to the score corresponding to the merged text box being less than the third threshold, process the text in the text result of the merged text box based on the similar character library to obtain the OCR recognition result.

8. An electronic device comprising: one or more processors; a storage device for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

Images