Method and system for text recovery in character recognition - Patents.com

Abstract

To provide a method and apparatus for training a machine learning system configured to recognize a damaged and / or degraded character.SOLUTION: A method includes: receiving an input text and / or image data; modifying the input text and / or the image data to generate degraded data; training a machine learning system using the degraded data; receiving the degraded data to the machine learning system; correcting the degraded data using the machine learning system to generate corrected data; in response to detecting that adjustment of the machine learning system is required after reading the corrected data, adjusting one or more weights of nodes in the machine learning system after reading the corrected data; and repeating the correction and adjustment until it is determined that adjustment no longer is required.SELECTED DRAWING: None

Description

[Technical field]

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS This application is related to U.S. Application No. 17 / 491,122, filed September 30, 2021, entitled "Method and Apparatus for Customized Deep Learning-Based Text Correction," the entire contents of which are incorporated by reference herein.

[0002] Aspects of the present invention relate to image processing, and more particularly, to character recognition. Summary of the Invention [Problem to be solved by the invention]

[0003] Printed end-user data may contain damaged and / or degraded characters (e.g., letters or numbers that are smudged, smudged, blurred, scratched, or partially missing). Such characters may arise for a variety of reasons, ranging from scanning errors to moisture or other foreign objects on the document. All of these issues pose challenges for optical character recognition (OCR) or image character recognition (ICR). Retraining or fine-tuning a character recognition system to address such errors may be difficult. For example, the number of examples on which to perform retraining or fine-tuning may be limited. There may also be concerns that the training data may be biased, resulting in poor performance and / or drift after retraining or fine-tuning. To avoid such issues, larger training sets can be applied, but this may be time-consuming and expensive, not only in terms of running the training set, but also in terms of the time to generate and check the training set. Expanding an existing data set may introduce drift and / or bias. [Means for solving the problem]

[0004] In view of the above, in one aspect, an embodiment of the present invention provides an approach to data generation and recovery, including character data. In an embodiment, the problem of generating and recovering Japanese or more generally East Asian character data is addressed by creating data for training a machine learning (ML) system that can operate in conjunction with an optical / image character recognition (OICR) engine as part of an overall OICR system, where the OICR engine reads a particular type of data and the ML system reads data that may be more specialized. Operating the ML system and OICR engine together eliminates the need to perform significant retraining or fine-tuning of the OICR engine.

[0005] In an embodiment, the ML system includes a convolutional neural network (CNN) for generating characters with defects and a neural network for identifying and correcting defective characters. Regression type In an embodiment, the entire CRNN system comprises a generative network model that learns and recognizes low-level noisy patterns and distinguishes them from high-level content. In an embodiment, the characters and fonts on which the CRNN system operates may be specific to an end user, while the characters and fonts on which the OICR engine operates may be more general. In an embodiment, the OICR engine may be referred to as a base model, and the CRNN system may be referred to as an end-user model. In an embodiment, the OICR engine may be an ML system. In an embodiment, the OICR engine may operate with multiple CRNN systems for different end users.

[0006] This generative network model has two main application areas in the context of optical character recognition and image character recognition. One is in data generation, generating training sets for CRNN systems. For data generation, the generative network is applied to generate characters with various defects, such as characters or parts of characters missing, or characters that are blurred. Another is in image restoration, cleaning data and reducing noise in the data. For image restoration, the generative network is applied to restore severely damaged characters in a particular font, such as characters with merged or missing strokes, blurred or smudged characters, and to remove background noise. In an embodiment, the font and background noise may be specific to an end user. Thus, the CRNN may be trained based on data specific to the end user. By training the CRNN using this data instead of the OICR engine, it is possible to avoid distortions and biases of the OICR engine with end user data, which may be extreme or very different from other training data used for the OICR engine.

[0007] Another application of generative network models is to generate end-user models that allow the recognition of severely damaged text, for which the trained discriminator can also be used as a sub-model and applied to recognize different types of degradation in the end-user samples.

[0008] Aspects of the present invention will now be described with reference to embodiments illustrated in the accompanying drawings. [Brief description of the drawings]

[0009] [Figure 1A] FIG. 2 is a high-level diagram of the flow within the different neural networks that make up the CRNN system according to an embodiment. [Figure 1B]FIG. 2 is a high-level diagram of the flow within the different neural networks that make up the CRNN system according to an embodiment. [Figure 2A] FIG. 13 illustrates an example of data generation according to an embodiment. [Figure 2B] FIG. 13 illustrates an example of data generation according to an embodiment. [Figure 2C] FIG. 1 illustrates an example of data restoration according to an embodiment. [Figure 2D] FIG. 1 illustrates an example of data restoration according to an embodiment. [Figure 3A] FIG. 13 illustrates a further example of data recovery according to an embodiment. [Figure 3B] FIG. 13 illustrates a further example of data recovery according to an embodiment. [Figure 3C] FIG. 13 illustrates a further example of data recovery according to an embodiment. [Figure 3D] FIG. 13 illustrates a further example of data recovery according to an embodiment. [Figure 4A] FIG. 13 illustrates another example of data restoration according to an embodiment. [Figure 4B] FIG. 13 illustrates another example of data restoration according to an embodiment. [Figure 4C] FIG. 13 illustrates another example of data restoration according to an embodiment. [Figure 5A] FIG. 13 illustrates another example of data restoration according to an embodiment. [Figure 5B] FIG. 13 illustrates another example of data restoration according to an embodiment. [Figure 5C] FIG. 13 illustrates another example of data restoration according to an embodiment. [Figure 6A] FIG. 13 illustrates another example of data restoration according to an embodiment. [Figure 6B] FIG. 13 illustrates another example of data restoration according to an embodiment. [Figure 6C] FIG. 13 illustrates another example of data restoration according to an embodiment. [Figure 7A] FIG. 13 illustrates another example of data restoration according to an embodiment. [Figure 7B] FIG. 13 illustrates another example of data restoration according to an embodiment. [Figure 7C] FIG. 13 illustrates another example of data restoration according to an embodiment. [Figure 8A] FIG. 13 illustrates another example of data restoration according to an embodiment. [Figure 8B] FIG. 13 illustrates another example of data restoration according to an embodiment. [Figure 8C] FIG. 13 illustrates another example of data restoration according to an embodiment. [Figure 9A] FIG. 13 illustrates another example of data restoration according to an embodiment. [Figure 9B] FIG. 13 illustrates another example of data restoration according to an embodiment. [Figure 9C] FIG. 13 illustrates another example of data restoration according to an embodiment. [Figure 10] 1 is a flowchart of the general operation of a CRNN system according to an embodiment. [Figure 11] FIG. 1 illustrates a high-level example of a system for receiving input data and training either an OICR engine or an end-user model according to an embodiment. [Figure 12] FIG. 1 illustrates a high-level diagram of a portion of a deep learning system for model training according to an embodiment. [Figure 13] FIG. 1 illustrates a high-level example of a deep learning system for model training according to an embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] An aspect of the present invention provides a method of training the machine learning system in an optical / image character recognition (OICR) system comprising an OICR engine and a machine learning system, the method comprising: receiving input text and / or image data; modifying the input text and / or image data to generate degraded data; training the machine learning system using the degraded data; receiving the degraded data to the machine learning system; correcting the degraded data with the machine learning system to generate corrected data; after reading the corrected data, in response to detecting that adjustment of the machine learning system is required, adjusting weights of one or more nodes in the machine learning system; and repeating the correction and adjustment until it is determined that adjustment is no longer required, wherein the adjustment is performed without requiring improvements or other changes to the OICR engine.

[0011] In a further aspect, the machine learning system uses additional data in addition to the degraded data for training. In yet another aspect, the machine learning system is a convolutional recurrent neural network (CRNN), the CRNN comprising a convolutional neural network (CNN) and a recurrent neural network (RNN). In yet another further aspect, the CNN generates the degraded data and the RNN generates the corrected data. The CNN can be trained using a generative adversarial network (GAN) loss, and the RNN can be trained using a connectionist temporal categorical (CTC) loss.

[0012] Another aspect of the present invention provides a method of restoring degraded data in an optical / image character recognition (OICR) system comprising an OICR engine and a machine learning system, the method comprising: receiving degraded data in the machine learning system; correcting the degraded data with the machine learning system to generate corrected data; adjusting weights of one or more nodes in the machine learning system in response to detecting that adjustment of the machine learning system is required after reading the corrected data; and repeating the correction and adjustment until it is determined that adjustment is no longer required. In a further aspect, the degraded data comprises characters having one or more of combined or missing strokes and background noise. In yet another aspect, the method comprises making changes to the OICR system to incorporate one or more aspects of the machine learning system in response to determining that the contents of the machine learning system warrant incorporation of one or more aspects of the machine learning system into the OICR system. In yet another further aspect, the machine learning system is a convolutional recurrent neural network (CRNN), the CRNN comprising a convolutional neural network (CNN) and a recurrent neural network (RNN). In an embodiment, the CNN generates the degraded data and the RNN generates the corrected data.

[0013] Yet another aspect of the present invention provides an optical / image character recognition (OICR) system comprising an OICR engine and a machine learning system, the machine learning system being programmed to perform a method comprising: receiving input text and / or image data; modifying the input text and / or image data to generate degraded data; training the machine learning system using the degraded data; receiving the degraded data into the machine learning system; correcting the degraded data with the machine learning system to generate corrected data; and adjusting weights of one or more nodes in the machine learning system in response to detecting that the machine learning system needs to be adjusted after reading the corrected data; and repeating the correction and adjustment until it is determined that the adjustment is no longer necessary, the adjustment being performed without requiring improvements or other changes to the OICR engine. In a further aspect, the machine learning system uses additional data in addition to the degraded data for training. In yet another further aspect, the machine learning system is a convolutional recurrent neural network (CRNN), the CRNN comprising a convolutional neural network (CNN) and a recurrent neural network (RNN). In an embodiment, the CNN generates the degraded data and the RNN generates the corrected data. In yet another further aspect, the CNN is trained with a generative adversarial network (GAN) loss and the RNN is trained with a connectionist temporal categorical (CTC) loss.

[0014] Yet another aspect of the present invention provides a method of restoring degraded data in a machine learning system, the method comprising: receiving degraded end-user data; correcting the degraded end-user data with a machine learning system to generate corrected end-user data; adjusting weights of one or more nodes in the machine learning system in response to detecting a need for adjustment of the machine learning system after reading the corrected end-user data; repeating the correction and adjustment until it is determined that adjustment is no longer necessary; and outputting the corrected end-user data. End User Data comprises characters having one or more of: joined or missing strokes, and background noise. According to yet an additional aspect, in response to a determination that the contents of the machine learning system warrant incorporation of one or more aspects of the machine learning system into the OICR system, modifications are made to the OICR system to incorporate the one or more aspects. According to yet another additional aspect, the machine learning system is a convolutional recurrent neural network (CRNN), the CRNN comprising a convolutional neural network (CNN) and a recurrent neural network (RNN). In an embodiment, the CNN generates the degraded data and the RNN generates the corrected data.

[0015] As mentioned above, according to an embodiment, the method and system of the present invention described herein is particularly applicable to the generation and restoration of oriental language characters. In an embodiment, the method and system of the present invention performs character generation and restoration in Japanese. In an embodiment, a convolutional recurrent neural network (CRNN) learns and separates two levels of information: (1) noise or other impairments in printed text, and (2) high-level font or handwritten content.

[0016] In an embodiment, the convolutional neural network (CNN) deals with noise at an earlier stage. In a later stage, the recurrent neural network (RNN) learns high-level font information. In an embodiment, the entire convolutional recurrent neural network (CRNN) is trained with a specific loss function, including the relevant generative adversarial network (GAN) loss for the CNN and the connectionist temporal categorical (CTC) loss for the RNN. As those skilled in the art will understand, each loss function is associated with a specific instance.

[0017] According to different embodiments, there are several aspects of the CRNN. ​​In one aspect, the CRNN generates degraded text and / or images to provide degraded images similar to images in the end user data. A font may be selected to generate the degraded images. The degradation may have different effects depending on the font.

[0018] In another embodiment, the CRNN recovers text and / or images from an input degraded image. The degraded images generated in the first described embodiment can be used to train the CRNN to recognize what is missing or missing in the input degraded image.

[0019] In an embodiment, the CRNN receives noisy samples or image templates to train the network to recognize characters, and the network outputs new images with preserved content from the input images.

[0020] 1A and 1B illustrate a CRNN-based generative network according to an embodiment. In FIG. 1A, a generation module 110 generates degraded text in a manner described. In the generation module 110, a labeling module 112 describes the content being input. In 114, appropriate text or image data is generated based on the output of the labeling module 112. A CNN feature encoder 116 takes the data and generates an appropriate feature set. The CNN learns low-level features such as noise patterns or style. For text, in an embodiment, the CNN may learn the shape of the font (e.g., sharpness, roundness).

[0021] The feature set from the CNN feature encoder 116 is then combined with various types of data that degrade the feature set in a combiner 125. In an embodiment, a random noise block 120 provides the degraded features. An RNN 130 receives the results of the combiner 125 as degraded text and preserves the content in the generated image / degraded text.

[0022] The generative network 140 receives the output of the RNN 130 and provides an output 150 comprising degraded characters or text. As FIG. 1B illustrates, the generative network 140 includes a deconvolutional decoder 146, an upsampling module 142, and a projection module 144 (modules 142 and 144 together are referred to as an upsampling projector). The deconvolutional decoder 146 may create high-level information about the data received as input. The upsampling projector 142 / 144 may preserve some characteristics of the data and some basic content style.

[0023] In an embodiment, within the generative network 140, the deconvolutional decoder 146 preserves the basic content of the characters. For example, in an embodiment, the deconvolutional decoder 146 may respond to a degraded character or text input by providing a corresponding generic character that may have the basic shape but may lack appearance details such as font. The upsampling projector 142 / 144 may add specific features such as font sharpness and roundness. As a result of the operation of the generative network 140, the generated / restored image will resemble the target image, i.e., the ground truth.

[0024] FIG. 1B illustrates a portion of the discriminator module that learns to distinguish between generated and real data. In FIG. 1B, text or images of characters to be restored are input at 160. In an embodiment, the output 150 in FIG. 1A may be directly fed to the image input 160. The data from the image input 160 is sent to the CNN feature encoder 170, which generates an appropriate feature set and passes it to the content preservation module 180. The discriminator module may identify that the data being input at the image input 160 is generated data rather than real data. The determination of generated data rather than real data is reflected as a GAN loss, which is then used to train the CNN at the encoder 172. The content preservation module 180 includes an RNN 182 that preserves the content in the restored image, and a CTC block 184. The CTC loss resulting from the CTC block 184 is used to train the RNN.

[0025] In an embodiment, the network of FIG. 1A and FIG. 1B may be a competitive poverty optimization (CPRO) based deep learning model.

[0026] In an embodiment, degraded samples that may appear in an end user's dataset are generated. If necessary, these degraded samples can be used to fine-tune the OICR engine to improve performance. Such synthetic data generation is in contrast to current synthetic data generation, which requires identification of noise patterns and application of existing fonts. Thus, the process is simpler and provides end-to-end learning for synthetic data generation and augmentation. In an embodiment, the process can be automated and integrated into the OICR engine fine-tuning and retraining process itself to support specific end user data and requirements. Such integration may be done for specific end users where there is a significant overlap between the specific requirements met by the CRNN system and the OICR engine itself.

[0027] In CRNN, a CRNN encoder / decoder is used to maintain content information throughout the training process. The GAN loss learns low-level noise patterns so that it can separate noise from content. As a result, the degradation level in the generated samples can be better controlled. This improved control avoids misdistribution of the training data and possible drift that could impair the overall functionality of the OICR engine.

[0028] 2A and 2B illustrate examples of data augmentation according to an embodiment. Fig. 2A is an input image that is electronically generated using templates of related fonts. Fig. 2B is an output with character degradation provided by the system. In Fig. 2B, various smudges, blurring, and even large missing characters may be seen compared to Fig. 2A.

[0029] 2C and 2D illustrate examples of data restoration according to embodiments. Fig. 2C is an input image with corrupted characters obtained from real data. Fig. 2D is an image with characters restored from the input image.

[0030] 3A and 3B, as well as 3C and 3D, are examples of image restoration. FIG. 3A is an input text, including a word with smudged or blurred characters in many lines, and missing characters in some lines. FIG. 3B is the text restored by the CRNN system. It is easy to see that the missing or significantly incomplete characters are filled in, and the smudged or blurred characters appear much clearer. Thus, according to the embodiment, missing parts of the image data can be added, or irrelevant background or surrounding data can be removed.

[0031] Similar to Figures 3A and 3B, Figure 3C is the input text and Figure 3D is the text recovered by the engine. In embodiments, longer data strings can provide context to the CRNN and provide more information to recover missing, smudged, or blurry image data.

[0032] 4A-4C illustrate input samples, model inference, and ground truth according to an embodiment. In FIG. 4A, the first character on the left is very badly smudged. FIG. 4B illustrates the result of applying the CRNN system according to an embodiment. All characters are clearer, especially the first character on the left. FIG. 4B is favorably compared to FIG. 4C, which is the ground truth, i.e., the way the image containing the font actually appears.

[0033] 5A-5C illustrate further samples, model inferences, and ground truths according to an embodiment. In FIG. 5A, all the characters are very blurred, especially the two characters on the left. FIG. 5B illustrates a clearer version of the characters in FIG. 5A.

[0034] 6A-6C illustrate further samples, model inferences, and ground truth according to an embodiment. In Fig. 6A, there is a mark that appears above or at the top of the input image data, and in Fig. 6B, it is removed. Similar to Fig. 4C and Fig. 5C, Fig. 6C represents the ground truth.

[0035] 7A-7C illustrate further samples, model inferences, and ground truth according to embodiments. These figures illustrate an example of the ability to replace a character (here, the "F" in "FAX") that is almost entirely missing in FIG. 7A but largely replaced in FIG. 7B. The "F" is clearly readable. Again, FIG. 7C now represents the ground truth.

[0036] 8A-8C illustrate further input samples, model inferences, and ground truth according to an embodiment. FIG. 8A illustrates a very bad background where the characters do not appear easily distinguishable. FIG. 8B illustrates the background being removed and the characters underneath being reproduced. Without being bound by any theory, it is possible that once the CRNN system has learned how to distinguish the background from the characters and is able to remove the background, reproduction of the characters themselves may become easier. It is also possible that the context in which the characters appear can inform the correct identification of the characters. A comparison of FIG. 8B with the ground truth of FIG. 8C illustrates good reproduction, but not perfect reproduction, especially with respect to the leftmost character.

[0037] 9A-9C illustrate further samples, model inferences, and ground truth according to embodiments. In FIG. 9A, the character on the left is faded, and dark marks are seen on almost all characters. Again, without being bound to a particular theory, it may be possible to tell what the characters are by the order in which they appear, even if they are blurred. Knowing how the characters are displayed may make it easier to remove the dark marks that are displayed. Alternatively, removing some of the annotations may make it easier to decipher the characters. FIG. 9B illustrates a very good match with the ground truth in FIG. 9C.

[0038] In an embodiment, depending on the location of the data in a particular field or the characters immediately surrounding the corrupted character, the surrounding data may convey the correct identity of the corrupted character to the CRNN.

[0039] Figure 10 provides a high level overview of the flow in a CRNN system according to an embodiment. At 1005, content to which degradation is applied is identified, and at 1010, text or images corresponding to the identified content are output. At 1015, an appropriate feature set is generated based on the output text / image. At 1020, the feature set is combined with other data (e.g., random noise, background color, etc.), and at 1025, the content with degradation features (degraded text) is output. This content is saved. The feature set may be retained for use in text restoration.

[0040] In an embodiment, at 1030, the degraded text (whether resulting from 1005-1025 or from actual degraded text to be corrected) is provided to an encoder (such as the CNN feature encoder 170 in FIG. 1B). At 1035, the comparison of the degraded text (the generated data) and the actual data, reflected as the GAN loss, can be used to train the CNN feature encoder. At 1040, data restoration is provided by considering the preservation of content in the restored image. In restoring the data, dirty or blurry data can be cleaned up as in FIG. 4A and FIG. 5A, incomplete data can be completed as in FIG. 7A, background color or other background noise can be removed as in FIG. 8A, and / or irrelevant marks in the middle of the data can be removed as in FIG. 9A. The filling of missing data and removal of irrelevant background or foreground during the process of content preservation (such as the content preservation module in FIG. 1B) can be part of the analysis of the input image text.

[0041] At 1045, a check is made to determine if the image has been restored. If it is determined that the image has not been restored, flow returns to 1040 for further modification. Once the image has been restored, at 1050, the restored image is output.

[0042] FIG. 11 is a high-level diagram of a CRNN system according to an embodiment. In FIG. 11, input text 1110 may be provided to an ICR module 1150 or an OCR module 1170 via a scanner or other input device 1120 depending on the text recognition process to be performed. A processing system 1140 may include a processing module 1190 that may communicate with the ICR module 1150 or the OCR module 1170 as needed to provide the necessary processing power via one or a central processing unit (CPU) or a graphics processing unit (GPU) and associated non-transitory storage and / or non-transitory memory. In an embodiment, the processing system 1140 may communicate directly with either the computer 1130 or the scanner / input device 1120. The processing system 1140 may be self-contained or have its various elements connected via a network or cloud 1160. Either or both of the ICR module 1150 or the OCR module 1170 may each have their own CPU and / or GPU and may communicate with the processing module 1190 via a network or cloud 1160.

[0043] Various elements in the processing system may be in communication with a CRNN 1200, which is described in more detail below with reference to FIGS.

[0044] FIG. 12 is a top-level diagram of node layers in a CRNN 1200 according to an embodiment. In FIG. 12, the CRNN 1200 has an input layer 1210 with multiple nodes. Depending on the system, there may be one or more hidden layers 1220-1240 (FIG. 12 shows P such layers, where P is a positive integer), each with multiple nodes. The final layer 1250 provides an output. In an embodiment, as one skilled in the art will appreciate, training the CRNN 1200 includes assigning weights to various nodes in various layers based on input training data for a particular end user.

[0045] Figure 13 illustrates the CRNN 1200 in a little more detail. The model database 1260 stores the weights and data. Node Weighting Module 1280 calculates weights for various nodes in various layers based on, among other things, the comparison of the results and assigns those weights to layers 1270-1, 1270-2, ... 1270-N-2, 1270-N-1, 1270-N accordingly.

[0046] In an embodiment, as described above, the CRNN 1200 and the OICR engine may both be deep learning systems. In an embodiment, the CRNN 1200 and the OICR engine may share one or more layers of nodes in FIG. 12. In such an embodiment, the CRNN system manages the layers of nodes specific to the CRNN 1200 and does not affect any of the node layers of the OICR engine.

[0047] Although the embodiments according to the aspects of the present invention have been described above, the present invention should not be considered limited to those embodiments or aspects. Those skilled in the art will understand modifications of the present invention within the scope and spirit of the appended claims.

Claims

1. 1. A method for training an optical / image character recognition (OICR) system comprising an OICR engine and a machine learning system, the method comprising: receiving input text and / or image data; modifying the input text and / or image data to generate degraded data; training the machine learning system using the degraded data; and receiving the degraded data into the machine learning system; correcting the degraded data using the machine learning system to generate corrected data; and adjusting weights of one or more nodes in the machine learning system in response to detecting that the machine learning system needs to be adjusted after reading the modified data; and repeating said modifying and adjusting until it is determined that said adjustment is no longer necessary; A method, wherein the tuning is performed without requiring modifications or other changes to the OICR engine.

2. The method of claim 1 , wherein the machine learning system uses additional data in addition to the degraded data for training.

3. 3. The method of claim 1 or 2, wherein the machine learning system is a Convolutional Recurrent Neural Network (CRNN), the CRNN comprising a Convolutional Neural Network (CNN) and a Recurrent Neural Network (RNN).

4. The method of claim 3 , wherein the CNN generates the degraded data and the RNN generates the corrected data.

5. 4. The method of claim 3, wherein the CNN is trained with a generative adversarial network (GAN) loss and the RNN is trained with a connectionist temporal categorical (CTC) loss.

6. 1. A method for restoring degraded data in an optical / image character recognition (OICR) system having an OICR engine and a machine learning system, the method comprising: receiving the corrupted data; and correcting the degraded data using the machine learning system to generate corrected data; and adjusting weights of one or more nodes in the machine learning system in response to detecting that the machine learning system needs to be adjusted after reading the modified data; and and repeating the modification and adjustment until it is determined that the adjustment is no longer necessary.

7. The method of claim 6 , wherein the degraded data comprises characters having one or more of: combined or missing strokes, and background noise.

8. 8. The method of claim 6 or 7, comprising, in response to a determination that content of the machine learning system warrants incorporation of one or more aspects of the machine learning system into the OICR system, making modifications to the OICR system to incorporate the one or more aspects.

9. 8. The method of claim 6 or 7, wherein the machine learning system is a Convolutional Recurrent Neural Network (CRNN), the CRNN comprising a Convolutional Neural Network (CNN) and a Recurrent Neural Network (RNN).

10. 10. The method of claim 9, wherein the CNN generates the degraded data and the RNN generates the corrected data.

11. 1. An optical / image character recognition (OICR) system comprising an OICR engine and a machine learning system, the machine learning system comprising: receiving input text and / or image data; modifying the input text and / or image data to generate degraded data; training the machine learning system using the degraded data; and receiving the degraded data into the machine learning system; correcting the degraded data using the machine learning system to generate corrected data; and adjusting weights of one or more nodes in the machine learning system in response to detecting that the machine learning system needs to be adjusted after reading the modified data; and repeating said modification and adjustment until it is determined that said adjustment is no longer necessary; A system programmed to execute a method, wherein the adjustment is performed without requiring modifications or other changes to the OICR engine.

12. The system of claim 11 , wherein the machine learning system uses additional data in addition to the degraded data for training.

13. 13. The system of claim 11 or 12, wherein the machine learning system is a Convolutional Recurrent Neural Network (CRNN), the CRNN comprising a Convolutional Neural Network (CNN) and a Recurrent Neural Network (RNN).

14. 14. The system of claim 13, wherein the CNN generates the degraded data and the RNN generates the corrected data.

15. 14. The system of claim 13, wherein the CNN is trained with a generative adversarial network (GAN) loss and the RNN is trained with a connectionist temporal categorical (CTC) loss.

16. The method further comprises a method for restoring degraded data in the machine learning system, the method comprising: receiving degraded end user data; correcting the degraded end-user data with the machine learning system to generate corrected end-user data; adjusting weights of one or more nodes in the machine learning system in response to detecting that adjustment of the machine learning system is required after reading the modified end user data; and repeating said modification and adjustment until it is determined that said adjustment is no longer necessary; and and outputting the modified end-user data.

17. 20. The system of claim 16, wherein the degraded end-user data comprises characters having one or more of combined or missing strokes, and background noise.

18. 17. The system of claim 16, further comprising, in response to a determination that content of the machine learning system warrants incorporation of one or more aspects of the machine learning system into the OICR system, making changes to the OICR system to incorporate the one or more aspects.

19. 17. The system of claim 16, wherein the machine learning system is a Convolutional Recurrent Neural Network (CRNN), the CRNN comprising a Convolutional Neural Network (CNN) and a Recurrent Neural Network (RNN).

20. 20. The system of claim 19, wherein the CNN generates the degraded data and the RNN generates the corrected data.

Images