Optical character recognition method and system

Abstract

The application provides an optical character recognition method and system, comprising: pre-training a detection error correction hybrid model together with a text recognition model to be trained to obtain a trained text recognition model; inputting a text image to be recognized into the trained text recognition model to obtain a text recognition result output by the text recognition model. The application considers the performance problem in the actual operation process of the model, the detection error correction hybrid model does not participate in the calculation in the actual operation stage, and only participates in the pre-training in the process of pre-training the text recognition model to improve the text recognition result of the text recognition model, and then correct the model parameters of the text recognition model to improve the overall recognition performance of the text recognition model, solving the defect that the text recognition is not accurate in the prior art, effectively improving the recognition accuracy without affecting the recognition speed of the text recognition model.

Description

Technical Field

[0001] This invention relates to the field of computer technology, and in particular to an optical character recognition method and system. Background Technology

[0002] Optical Character Recognition (OCR) refers to the process of analyzing and recognizing image files containing text data to obtain text and layout information; that is, recognizing the text in an image and returning it in the form of text.

[0003] Currently, there are two main technical solutions in the OCR field. The first is a two-stage approach based on text line detection and character recognition, while the second is a character recognition solution based on an end-to-end model. The first approach involves first locating the text lines in the image file, and then performing character recognition on the content of those located lines. The end-to-end approach integrates text line detection and character recognition into a single model for recognition.

[0004] Currently, the first approach mentioned above is still the most widely used in industry, mainly for two reasons: (1) Traditional OCR technology incorporates many image processing and machine learning techniques, such as image processing techniques including binarization, connected component analysis, and projection analysis, and statistical machine learning techniques such as Support Vector Machine (SVM) and Adaboost; (2) It avoids error propagation. When errors occur in text line positioning, they will greatly affect subsequent text recognition, meaning that errors from the previous stage will propagate to the next stage and generate even larger errors. In addition, although the end-to-end OCR model is simpler to build than the two-stage model, its recognition performance is still somewhat inferior to that of the two-stage model.

[0005] In practice, the two methods mentioned above often have shortcomings, such as misrecognizing similar-looking Chinese characters (especially handwritten characters), which can completely change the meaning of the entire text sequence, or misrecognizing specific characters, which can change the meaning of the entire text sequence. Summary of the Invention

[0006] To address the problems existing in the text recognition using OCR technology in the prior art, this invention provides an optical character recognition method and system.

[0007] The present invention provides an optical character recognition method, comprising: pre-training a detection and error correction hybrid model together with a text recognition model to be trained to obtain a trained text recognition model; and inputting a text image to be recognized into the trained text recognition model to obtain a text recognition result output by the text recognition model.

[0008] According to an optical character recognition method provided by the present invention, the detection and error correction hybrid model includes a detection sub-model and an error correction sub-model; the detection sub-model is composed of a Bi-LSTM network model or a Bi-GRU network model; the error correction sub-model is composed of a BERT network model or an XLNet network model.

[0009] According to an optical character recognition method provided by the present invention, the step of pre-training a text recognition model together with a detection and error correction hybrid model to obtain a trained text recognition model includes: acquiring multiple text image samples and text labels for each text image sample; for any one of the text image samples, inputting the text image sample into the text recognition model to be trained to obtain a recognized text sequence output by the text recognition model to be trained; comparing the recognized text sequence with the text label corresponding to the text image sample; if the comparison result is a recognition error, obtaining a recognition error set; forming a correct-error sample pair by combining any recognized text in the recognition error set with the corresponding text label; inputting the correct-error sample pair into the detection and error correction hybrid model to obtain a training recognition result output by the detection and error correction hybrid model; calculating the classification cross-entropy loss on the recognition error set according to the training recognition result to update the model parameters of the text recognition model to be trained; and traversing the multiple text image samples until a trained text recognition model is obtained.

[0010] According to an optical character recognition method provided by the present invention, the step of inputting the correct and incorrect sample pairs into the detection and error correction hybrid model to obtain the training recognition result output by the detection and error correction hybrid model includes: inputting any recognized text in the recognition error set into the detection sub-model to obtain a probability sequence output by the detection sub-model, wherein each element in the probability sequence is in the range [0,1]; obtaining a linear combination of input embedding and mask embedding composed of the probability values ​​of each element in the probability sequence as weights; and inputting the linear combination into the error correction sub-model to obtain the training recognition result output by the error correction sub-model.

[0011] According to an optical character recognition method provided by the present invention, the expression for the linear combination is:

[0012] bi =p i *e mask +(1-p i )*e i

[0013] Among them, b i For the linear combination, p i Let e ​​be the probability value of the i-th element in the probability sequence. mask For Maskembedding, e i This is the input embedding corresponding to the i-th element in the probability sequence.

[0014] According to an optical character recognition method provided by the present invention, the formula for calculating the classification cross-entropy loss on the recognition error set based on the training recognition results is as follows:

[0015]

[0016] Where Loss is the classification cross-entropy loss on the misclassification set; N is the cardinality of the misclassification set E, i.e., N = |E|; L i The cross-entropy loss is used to identify the i-th identified text in the error set; y ij The indicator variable for identifying the i-th element of the identified text in the error set is set to 1 if it matches the truth value corresponding to the text label, and 0 otherwise; p ij For y ij The probability of predicting a value of 1; the formula for updating the model parameters of the text recognition model to be trained is:

[0017] L = Loss + CTCLoss;

[0018]

[0019] Wherein, CTCLoss is the original loss function value of the text recognition model; L is the loss function value obtained by adding the original loss function value to the classification cross-entropy loss based on the principle of residual linking, and then enhancing the model parameters to focus on error-prone recognition; μ is the step size hyperparameter; w is the text recognition model parameter; S is the softmax function in the text recognition model; X is the parameter value of the text recognition model before the update; and X′ is the parameter value of the text recognition model after the update.

[0020] According to an optical character recognition method provided by the present invention, a residual linking mechanism is used to enhance the expressive power of the input embedding, specifically expressed as follows:

[0021] h i ′=h i +ei ;

[0022] Among them, e i h is the input embedding corresponding to the i-th element in the probability sequence. i The training and recognition results output by the error correction sub-model; h i ′ represents the enhanced training and recognition results.

[0023] According to an optical character recognition method provided by the present invention, after acquiring multiple text image samples and text labels for each text image sample, the method further includes: performing data augmentation processing on each text image sample, including at least one of the following methods: random translation transformation, random reflection transformation, and random illumination and color transformation; creating new text image samples by data synthesis and acquiring text labels corresponding to each new text image sample; constructing a text image training set from the new text image samples, the text image samples, and their respective corresponding text labels; wherein the ratio of the number of text image samples to the number of new text image samples is between 1:1 and 1:3.

[0024] According to an optical character recognition method provided by the present invention, before inputting the correct and incorrect sample pairs into the detection and error correction hybrid model, the method further includes: pre-training the error correction sub-model using general text and the corresponding labels of the general text, and fine-tuning the model parameters of the error correction sub-model.

[0025] The present invention also provides an optical character recognition system, comprising: a pre-training unit for pre-training a text recognition model to be trained together with a detection and error correction hybrid model to obtain a trained text recognition model; and a recognition unit for inputting a text image to be recognized into the trained text recognition model to obtain a text recognition result output by the text recognition model.

[0026] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of any of the optical character recognition methods described above.

[0027] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of any of the optical character recognition methods described above.

[0028] The optical character recognition method and system provided by this invention takes into account the performance issues in the actual operation of the model. The detection and error correction hybrid model does not participate in the calculation during the actual operation stage. It is only introduced into the pre-training process of the text recognition model to improve the text recognition results of the text recognition model, thereby correcting the model parameters of the text recognition model and improving the overall recognition performance of the text recognition model. This solves the defect of inaccurate text recognition using network models in the prior art. It effectively improves the recognition accuracy without affecting the recognition speed of the text recognition model. Attached Figure Description

[0029] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0030] Figure 1 This is a flowchart illustrating the optical character recognition method provided by the present invention;

[0031] Figure 2 This is a schematic diagram of the process of pre-training the detection and error correction hybrid model and the text recognition model to be trained together, as provided by the present invention.

[0032] Figure 3 This is a schematic diagram of the optical character recognition system provided by the present invention;

[0033] Figure 4 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0034] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0035] It should be noted that in the description of the embodiments of the present invention, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element. The terms "upper," "lower," etc., indicating orientation or positional relationships are based on the orientation or positional relationships shown in the accompanying drawings and are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of the present invention. Unless otherwise expressly specified and limited, the terms "installed," "connected," and "linked" should be interpreted broadly. Those skilled in the art can understand the specific meaning of the above terms in the present invention according to the specific circumstances.

[0036] The following is combined Figures 1-4 The optical character recognition method and system provided in the embodiments of the present invention are described.

[0037] Figure 1 This is a flowchart illustrating the optical character recognition method provided by the present invention, as shown below. Figure 1 As shown, including but not limited to the following steps:

[0038] Step 11: Pre-train the text recognition model with the Detection And Correction Net (DACNet) to obtain a trained text recognition model.

[0039] OCR refers to the process by which electronic devices (such as scanners or digital cameras) examine printed characters on paper, determine their shapes by detecting dark and light patterns, and then translate those shapes into computer text using character recognition methods. In other words, it uses optical methods to convert printed characters in paper documents into black-and-white dot matrix image files, and then uses recognition software to convert the text in the image into text format for further editing by word processing software. The main indicators for evaluating the performance of an OCR core model include: rejection rate, false recognition rate, recognition speed, user interface friendliness, product stability, ease of use, and feasibility. Therefore, how to eliminate errors or utilize auxiliary information to improve the recognition accuracy of the core model is the most important issue in OCR.

[0040] The text recognition model, as the core model of the OCR in the optical character recognition method provided by this invention, can be a neural network model composed of YOLOv4+CRNN. ​​For ease of description, in the following embodiments, the text recognition model will be uniformly referred to as the CRNN model, which is not considered as a specific limitation on the scope of protection of this invention.

[0041] The DACNet model mainly consists of a detection sub-model and an error correction sub-model, similar to the generative and discriminative models in Generative Adversarial Networks (GANs).

[0042] It should be noted that in the optical character recognition method provided by this invention, the DACNet model does not participate in the calculation during the recognition of the input text image to be recognized. Instead, it is used to pre-train the CRNN model to improve the model parameters of the CRNN model and thus enhance the overall recognition performance of the CRNN model.

[0043] Specifically, in the actual training process, the DACNet model is used to detect errors in the output of the CRNN model (obtaining the error set), and the identified erroneous texts are corrected at the same time. Finally, the classification cross-entropy loss on the entire error set is calculated to adjust the model parameters of the CRNN model, which can effectively improve the detection and recognition accuracy of the CRNN model and reduce the probability of recognition errors.

[0044] Step 12: Input the text image to be recognized into the trained text recognition model to obtain the text recognition result output by the text recognition model.

[0045] After pre-training the CRNN model using the DACNet model in conjunction with the CRNN model, a trained text recognition model is obtained.

[0046] In the actual detection process, after inputting the text image to be recognized into the trained CRNN model, the text recognition result output by it can be obtained.

[0047] Optionally, before inputting the text image to be recognized into the trained CRNN model, the text image can be preprocessed, such as sharpening, binarization, grayscale conversion, or noise reduction, to further improve the accuracy of the text recognition results.

[0048] It should be noted that the optical character recognition method provided by this invention is consistent with existing mainstream solutions in the OCR recognition stage described in step 11, namely, acquiring the text image to be recognized and inputting the text image into a trained text recognition model to obtain the text recognition result. However, the difference from existing mainstream solutions lies in the fact that this invention introduces the DACNet model with detection and error correction capabilities into the pre-training process of the CRNN model to correct the OCR recognition result, updating only the parameters of the CRNN model. In addition to improving the detection speed of the CRNN model, it also focuses more on the recognition accuracy of the CRNN model.

[0049] Since the DACNet model does not participate in the calculation during the actual detection process, it will not reduce the speed of the entire OCR recognition.

[0050] The optical character recognition method provided by this invention can effectively solve the problem of inaccurate text recognition in existing technical solutions, and therefore can be applied to complex scenarios such as e-commerce store photo recognition and case file document recognition with handwriting.

[0051] The optical character recognition method provided by this invention takes into account the performance issues in the actual operation of the model. The detection and error correction hybrid model does not participate in the calculation during the actual operation stage. It is only introduced into the pre-training process of the text recognition model to improve the text recognition results of the text recognition model, thereby correcting the model parameters of the text recognition model and improving the overall recognition performance of the text recognition model. This solves the defect of inaccurate text recognition using network models in the existing technical solutions, and effectively improves the recognition accuracy without affecting the recognition speed of the text recognition model.

[0052] Based on the above embodiments, as an optional embodiment, the detection and error correction hybrid model includes a detection sub-model and an error correction sub-model; the detection sub-model is composed of a Bi-LSTM network model or a Bi-GRU network model; the error correction sub-model is composed of a BERT network model or an XLNet network model.

[0053] The DACNet model provided by this invention mainly consists of a detection sub-model and an error correction sub-model, which are similar to the generative model and the discriminative model in the GAN model, respectively.

[0054] The detection sub-model can be composed of a Bi-LSTM network model or a Bi-GRU network model, etc. Its input is the text recognition result of the CRNN model mentioned above, and its output is a probability sequence. Each element in the probability sequence is in the range [0,1]. The larger the probability value of an element, the more likely the character at that position is incorrect, and vice versa.

[0055] The error correction sub-model can be composed of a BERT network model or an XLNet network model, etc. The input of the error correction sub-model can be: the input embedding (e) which uses the probabilities in the probability sequence output by the detection sub-model as weights. i ) and Maskembedding(e mask A linear combination of ); the output b of the error correction sub-model i After its own attention module and feedforward neural network output the training and recognition result h i .

[0056] Furthermore, h can be i The input is fed into the softmax module to obtain the probability distribution over the entire dictionary. The candidate character with the highest probability is then selected as the output and processed into a text sequence. This text sequence is the result after correcting the text recognition errors detected by the detection sub-model.

[0057] The optical character recognition method provided by this invention introduces a DACNet model consisting of a detection sub-model and an error correction sub-model during the pre-training process of the CRNN model to detect and correct the output results of the CRNN model. Then, the classification cross-entropy loss is calculated on the recognition error set of the CRNN model to complete the parameter update of the CRNN model, which can effectively improve the recognition accuracy of the CRNN model.

[0058] Figure 2 This is a schematic diagram illustrating the process of pre-training the detection and error correction hybrid model provided by this invention together with the text recognition model to be trained. As an optional embodiment, such as... Figure 2 As shown, the step of pre-training the text recognition model together with the detection and error correction hybrid model to obtain a trained text recognition model includes:

[0059] Multiple text image samples and text labels for each text image sample are obtained. For any one of the text image samples, the text image sample is input into the text recognition model to be trained to obtain a recognized text sequence output by the text recognition model to be trained. The recognized text sequence is compared with the text label corresponding to the text image sample. If the comparison result is a recognition error, a recognition error set is obtained. Any recognized text in the recognition error set is combined with its corresponding text label to form a correct-error sample pair. The correct-error sample pair is input into the detection-correction hybrid model to obtain the training recognition result output by the detection-correction hybrid model. Based on the training recognition result, the classification cross-entropy loss on the recognition error set is calculated to update the model parameters of the text recognition model to be trained. The multiple text image samples are traversed until a trained text recognition model is obtained.

[0060] Specifically, the optical character recognition method provided by this invention introduces a DACNet model with detection and error correction capabilities to correct the recognition results of the CRNN model during the pre-training process, so as to update the parameters of the CRNN model, including but not limited to the following steps:

[0061] Step (1): First, retrieve the training sample set consisting of multiple text image samples and their corresponding text labels.

[0062] Take any text image sample from the training sample set, pre-train it, and then input it into the CRNN model to be trained. First, obtain the logit probability matrix output by the model. The shape of the logit probability matrix is ​​N*T, where N is the dictionary size and T is the number of time steps.

[0063] After the above logit probability matrix is ​​decoded by the Connectionist Temporal Classification (CTC) algorithm, the CRNN model finally outputs its prediction result, which is a recognized text sequence (represented by Pred).

[0064] Furthermore, the identified text sequence is compared with the text label (GroundTruth, GT) corresponding to the input text image sample. The comparison result is divided into two parts: correct recognition and incorrect recognition.

[0065] Step (2): For correctly identified results, save them directly.

[0066] For the parts that are incorrectly identified, an incorrect identification set E can be constructed, and any Pred in it can be... j With GTj (j∈E) form correct and incorrect sample pairs, which are used as training data and input into the DACNet model to obtain the corresponding training recognition results output by the DACNet model.

[0067] As an optional embodiment, the above-described input of the correct and incorrect sample pairs to the detection and error correction hybrid model to obtain the training recognition results output by the detection and error correction hybrid model includes, but is not limited to:

[0068] Any recognized text in the recognition error set is input into the detection sub-model to obtain a probability sequence output by the detection sub-model, wherein each element in the probability sequence is in the range of [0,1]. A linear combination of input embedding and mask embedding is obtained, which is composed of the probability values ​​of each element in the probability sequence as weights. The linear combination is input into the error correction sub-model to obtain the training recognition result output by the error correction sub-model.

[0069] Specifically, first, the results of the above-mentioned identification errors are Pred j (j∈E) is input to the detection sub-model, and its output is a probability sequence, in which each element ranges from [0,1]. The larger the probability value of each element in the probability sequence, the more likely the character at that position is incorrect; the smaller the probability value of each element, the more likely the character at that position is correct.

[0070] Furthermore, the input to the error correction sub-model is: embedding(e) which uses the probability of each element in the probability sequence output by the detection sub-model as the input weights. i ) and Mask embedding(e mask A linear combination of ).

[0071] As an optional embodiment, the expression for the linear combination is:

[0072] b i =p i *e mask +(1-p i )*e i ;

[0073] Among them, b i For the linear combination, p i Let e ​​be the probability value of the i-th element in the probability sequence. mask For Maskembedding, e i This is the input embedding corresponding to the i-th element in the probability sequence.

[0074] It should be noted that when p i The closer the value is to 1, the greater the probability that the character at position i is incorrect, and the closer the input of the error correction sub-model is to e. mask In other words, we want to predict the character at that position; otherwise, we will directly use the output e of the detection sub-model. i As the output value at that location, it is sent to step (iii).

[0075] Step (3): The output of step (2) is b i The training and recognition result h is obtained after passing through the attention module and the feedforward neural network. i .

[0076] As an optional embodiment, after obtaining the training recognition result output by the error correction sub-model, the method further includes:

[0077] A residual linking mechanism is used to enhance the expressive power of the input embedding, specifically:

[0078] h i ′=h i +e i ;

[0079] Among them, e i h is the input embedding corresponding to the i-th element in the probability sequence. i The training and recognition results output by the error correction sub-model; h i ′ represents the enhanced training and recognition results.

[0080] Finally, h i The probability distribution over the entire dictionary is obtained by inputting the '' into the softmax module, and then organized into the final output recognition result.

[0081] Step (4): Finally, calculate the classification cross-entropy loss over the entire recognition error set E, and update the parameters of YOLO, CRNN, detection model and error correction model based on the classification cross-entropy loss.

[0082] It should be noted that in the optical character recognition method provided by this invention, only the parameters of the CRNN model can be updated to improve the recognition speed of the CRNN model and also pay more attention to the recognition accuracy of the CRNN model.

[0083] As an optional embodiment, the formula for calculating the classification cross-entropy loss on the recognition error set based on the training recognition results is as follows:

[0084]

[0085] Where Loss is the classification cross-entropy loss on the misclassification set; N is the cardinality of the misclassification set E, i.e., N = |E|; L i The cross-entropy loss is used to identify the i-th identified text in the error set; y ij The indicator variable for identifying the i-th element of the identified text in the error set is set to 1 if it matches the truth value corresponding to the text label, and 0 otherwise; p ij For y ij The probability of predicting a value of 1;

[0086] The formula for updating the model parameters of the text recognition model to be trained is as follows:

[0087] L = Loss + CTCLoss;

[0088]

[0089] Wherein, CTCLoss is the original loss function value of the text recognition model; L is the loss function value obtained by adding the original loss function value to the classification cross-entropy loss based on the principle of residual linking, and then enhancing the model parameters to focus on error-prone recognition; μ is the step size hyperparameter; w is the text recognition model parameter; S is the softmax function in the text recognition model; X is the parameter value of the text recognition model before the update; and X′ is the parameter value of the text recognition model after the update.

[0090] The optical character recognition method provided by this invention takes into account the performance issues in the actual operation of the model. The detection and error correction hybrid model does not participate in the calculation during the actual operation stage. It is only introduced into the pre-training process of the text recognition model to improve the text recognition results of the text recognition model, thereby correcting the model parameters of the text recognition model and improving the overall recognition performance of the text recognition model. This solves the defect of inaccurate text recognition using network models in the prior art. It effectively improves the recognition accuracy without affecting the recognition speed of the text recognition model.

[0091] Based on the above embodiments, as an optional embodiment, after obtaining multiple text image samples and the text label of each text image sample, the method further includes:

[0092] Data augmentation processing is performed on each of the text image samples, including at least one of the following methods: random translation transformation, random reflection transformation, and random illumination and color transformation; new text image samples are created by data synthesis, and text labels corresponding to each new text image sample are obtained; a text image training set is constructed from the new text image samples, the text image samples, and their respective text labels; the ratio of the number of text image samples to the number of new text image samples is between 1:1 and 1:3.

[0093] Specifically, in step (a) above, after obtaining the text image sample, the present invention performs preprocessing operations on the text image sample, which may include, but is not limited to, the following steps:

[0094] (1) Data augmentation of text image samples. In order to enhance the robustness of the CRNN model, the input text image samples are subjected to a certain degree of random translation transformation, random reflection transformation, and random illumination and color transformation to achieve image data augmentation.

[0095] (2) Sample Balance. In specific scenarios, if it is difficult to collect a large number of text image samples, synthetic data can be considered. Generally, it is ideal to ensure that the ratio of real text image samples to synthetic new text image samples in a batch is about 1:1 to 1:3. If the synthetic new text image samples are too large, it will overfit to the synthetic data, and the prediction effect will often be poor.

[0096] The optical character recognition method provided by this invention can effectively improve the robustness of the CRNN model by pre-training it using text image samples before pre-training it.

[0097] Based on the above embodiments, as an optional embodiment, before inputting the correct and incorrect sample pairs into the detection and error correction hybrid model, the method further includes:

[0098] The error correction sub-model is pre-trained using general text and the corresponding tags, and the model parameters for implementing the error correction sub-model are fine-tuned.

[0099] Furthermore, the pre-training of the error correction sub-model in the DACNet module in step (ii) above is as follows:

[0100] In the initial stage, the error correction sub-model can be pre-trained based on large-scale general text, and then fine-tuned using the labels of the pre-training data (general text) to obtain the final error correction sub-model.

[0101] Furthermore, the attention mechanism in step (iii) is described in detail below:

[0102] Considering the contextual dependencies of text sequences, this application employs a multi-head self-attention mechanism. Because each character in the text sequence has a different level of importance to the predicted character, the multi-head self-attention mechanism calculates weights for each character in the sequence based on their relative relationship with the predicted character. This effectively measures the relative importance in the representation while improving computational efficiency.

[0103] Figure 3This is a schematic diagram of the optical character recognition system provided by the present invention, as shown below. Figure 3 As shown, including but not limited to: a pre-training unit 31 and a recognition unit 32; wherein, the pre-training unit 31 is mainly used to pre-train the text recognition model to be trained together with the detection and error correction hybrid model to obtain a trained text recognition model; the recognition unit 32 is mainly used to input the text image to be recognized into the trained text recognition model to obtain the text recognition result output by the text recognition model.

[0104] It should be noted that the optical character recognition system provided in this embodiment of the invention can be implemented based on the optical character recognition method described in any of the above embodiments, and this embodiment will not elaborate on this.

[0105] The optical character recognition system provided by this invention takes into account the performance issues in the actual operation of the model. The detection and error correction hybrid model does not participate in the calculation during the actual operation stage. It is only introduced into the pre-training process of the text recognition model to improve the text recognition results of the text recognition model, thereby correcting the model parameters of the text recognition model and improving the overall recognition performance of the text recognition model. This solves the defect of inaccurate text recognition using network models in the prior art. It effectively improves the recognition accuracy without affecting the recognition speed of the text recognition model.

[0106] Figure 4 This is a schematic diagram of the structure of the electronic device provided by the present invention, such as... Figure 4 As shown, the electronic device may include a processor 410, a communication interface 420, a memory 430, and a communication bus 440, wherein the processor 410, the communication interface 420, and the memory 430 communicate with each other via the communication bus 440. The processor 410 can call logical instructions in the memory 430 to execute an optical character recognition method, which includes: pre-training a text recognition model using a detection and error correction hybrid model and a text recognition model to be trained to obtain a trained text recognition model; and inputting a text image to be recognized into the trained text recognition model to obtain a text recognition result output by the text recognition model.

[0107] Furthermore, the logical instructions in the aforementioned memory 430 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, essentially, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0108] On the other hand, the present invention also provides a computer program product, the computer program product comprising a computer program stored on a non-transitory computer-readable storage medium, the computer program comprising program instructions, wherein when the program instructions are executed by a computer, the computer is able to execute the optical character recognition method provided by the above methods, the method comprising: pre-training a detection error correction hybrid model together with a text recognition model to be trained to obtain a trained text recognition model; and inputting a text image to be recognized into the trained text recognition model to obtain a text recognition result output by the text recognition model.

[0109] In another aspect, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon. When executed by a processor, the computer program is implemented to perform the optical character recognition method provided in the above embodiments. The method includes: pre-training a detection and error correction hybrid model together with a text recognition model to be trained to obtain a trained text recognition model; and inputting a text image to be recognized into the trained text recognition model to obtain a text recognition result output by the text recognition model.

[0110] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0111] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0112] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. An optical character recognition method, characterized in that, include: The detection and error correction hybrid model is pre-trained together with the text recognition model to be trained to obtain a well-trained text recognition model. The text image to be recognized is input into the trained text recognition model to obtain the text recognition result output by the text recognition model; The step of pre-training the text recognition model together with the detection and error correction hybrid model to obtain a trained text recognition model includes: Obtain multiple text image samples and the text label for each text image sample; For any given text image sample, the text image sample is input into the text recognition model to be trained to obtain the recognized text sequence output by the text recognition model to be trained; the recognized text sequence is obtained by decoding the logistic regression probability matrix using a connection-time classification algorithm; the logistic regression probability matrix is ​​output by the text recognition model to be trained; The identified text sequence is compared with the text labels corresponding to the text image samples; If the comparison result indicates an identification error, obtain the identification error set; Combine any identified text in the error set with its corresponding text label to form a correct-error sample pair; The correct and incorrect sample pairs are input into the detection and error correction hybrid model to obtain the training recognition results output by the detection and error correction hybrid model; Based on the training recognition results, the classification cross-entropy loss on the recognition error set is calculated to update the model parameters of the text recognition model to be trained; The process involves iterating through the multiple text image samples until a trained text recognition model is obtained. The detection and error correction hybrid model includes a detection sub-model and an error correction sub-model; The detection sub-model is composed of a Bi-LSTM network model or a Bi-GRU network model. The error correction sub-model consists of either a BERT network model or an XLNet network model.

2. The optical character recognition method according to claim 1, characterized in that, The step of inputting the correct and incorrect sample pairs into the detection and error correction hybrid model to obtain the training recognition result output by the detection and error correction hybrid model includes: Input any recognized text from the set of recognition errors into the detection sub-model to obtain a probability sequence output by the detection sub-model, wherein each element in the probability sequence is in the range of [0,1]. Obtain a linear combination of input embedding and mask embedding, weighted by the probability values ​​of each element in the probability sequence; The linear combination is input into the error correction sub-model to obtain the training recognition result output by the error correction sub-model.

3. The optical character recognition method according to claim 2, characterized in that, The expression for the linear combination is: ； in, For the linear combination, Let be the probability value of the i-th element in the probability sequence. For Maskembedding, This is the input embedding corresponding to the i-th element in the probability sequence.

4. The optical character recognition method according to claim 1, characterized in that, The formula for calculating the classification cross-entropy loss on the recognition error set based on the training recognition results is as follows: in, Let N be the classification cross-entropy loss on the misclassification set; N is the cardinality of the misclassification set E, i.e. ; The cross-entropy loss is used to identify the i-th identified text in the error set; The indicator variable is used to identify the i-th element of the text in the error set. If it is the same as the truth value corresponding to the text label, it is 1; otherwise, it is 0. for The probability of predicting a value of 1; The formula for updating the model parameters of the text recognition model to be trained is as follows: ； ； in, L represents the original loss function value of the text recognition model; L is the loss function value obtained by adding the original loss function value to the classification cross-entropy loss based on the principle of residual linking, and then enhancing the model parameters to focus on error-prone recognition. It is the step size hyperparameter; For text recognition model parameters; This refers to the softmax function in text recognition models. These are the parameter values ​​before the text recognition model was updated. These are the updated parameter values ​​for the text recognition model.

5. The optical character recognition method according to claim 3, characterized in that, After obtaining the training recognition results output by the error correction sub-model, the method further includes: A residual linking mechanism is used to enhance the expressive power of the input embedding, specifically: ； in, Let be the input embedding corresponding to the i-th element in the probability sequence. The training and recognition results output by the error correction sub-model; To enhance the training and recognition results after processing.

6. The optical character recognition method according to claim 1, characterized in that, After obtaining multiple text image samples and the text label for each text image sample, the process also includes: Each of the text image samples is subjected to data augmentation processing, including at least one of the following: random translation transformation, random reflection transformation, and random illumination and color transformation; New text image samples are created through data synthesis, and the text label corresponding to each new text image sample is obtained; A text image training set is constructed from the new text image samples, the text image samples, and their respective text labels; The ratio of the number of text image samples to the number of new text image samples is between 1:1 and 1:

3.

7. The optical character recognition method according to claim 1, characterized in that, Before inputting the correct and incorrect sample pairs into the detection and error correction hybrid model, the method further includes: The error correction sub-model is pre-trained using general text and the corresponding tags, and the model parameters for implementing the error correction sub-model are fine-tuned.

8. An optical character recognition system, characterized in that, include: The pre-training unit is used to pre-train the text recognition model together with the detection and error correction hybrid model to obtain a trained text recognition model. The recognition unit is used to input the text image to be recognized into the trained text recognition model in order to obtain the text recognition result output by the text recognition model; The step of pre-training the text recognition model together with the detection and error correction hybrid model to obtain a trained text recognition model includes: Obtain multiple text image samples and the text label for each text image sample; For any given text image sample, the text image sample is input into the text recognition model to be trained to obtain the recognized text sequence output by the text recognition model to be trained; the recognized text sequence is obtained by decoding the logistic regression probability matrix using a connection-time classification algorithm; the logistic regression probability matrix is ​​output by the text recognition model to be trained; The identified text sequence is compared with the text labels corresponding to the text image samples; If the comparison result indicates an identification error, obtain the identification error set; Combine any identified text in the error set with its corresponding text label to form a correct-error sample pair; The correct and incorrect sample pairs are input into the detection and error correction hybrid model to obtain the training recognition results output by the detection and error correction hybrid model; Based on the training recognition results, the classification cross-entropy loss on the recognition error set is calculated to update the model parameters of the text recognition model to be trained; The process involves iterating through the multiple text image samples until a trained text recognition model is obtained. The detection and error correction hybrid model includes a detection sub-model and an error correction sub-model; The detection sub-model is composed of a Bi-LSTM network model or a Bi-GRU network model. The error correction sub-model consists of either a BERT network model or an XLNet network model.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the optical character recognition method as described in any one of claims 1 to 7.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the optical character recognition method as described in any one of claims 1 to 7.

Images