Deep learning-based text correction method and apparatus

Abstract

To provide a more efficient computer-implemented text correction method and device for developing a deep learning-based spell checker.SOLUTION: In a text correction method for taking advantage of a greatly reduced number of error-ground truth pairs to train a deep learning model, to generate the error-ground truth pair, different characters in a ground truth word are replaced with symbols without appearing in any ground truth word, an error word to be paired with the ground truth word is generated, a pair of characters in the ground truth word are replaced with symbols to generate the error word to be paired with the ground truth word, and the error-ground truth pair is provided.SELECTED DRAWING: Figure 10

Description

CROSS-REFERENCE TO RELATED APPLICATIONS

[0001] The present invention relates to U.S. Patent Application No. 17 / 491,122, entitled "METHOD AND APPARATUS FOR CUSTOMIZED DEEP-LEACHING BASED TEXT CORRECTION," filed on September 30, 2021, the entire contents of which are incorporated by reference in their entirety. [Background technology]

[0002] Spell checking systems can automatically detect typos and spelling errors when a user enters a sequence of characters. Traditionally, such systems use search algorithms of a fixed dictionary created by a predefined vocabulary. Error words are pre-calculated and use edited distances. In optical character recognition (OCR) and image character recognition (ICR), there are two types of typos that often have different causes. OCR errors tend to occur due to typos, incorrect pronunciations, and poor image quality. ICR errors tend to occur due to misreading different handwriting styles of different users. Summary of the Invention [Problem to be solved by the invention]

[0003] Figure 1 shows the basic concept of a traditional dictionary-based method of spell corrector. If the user's input is "Figere", the algorithm generates modifications to this input using the edited distance. The algorithm may generate candidates including but not limited to "Figude", "Figede", "Figuco", "Figgice", and "Figuac". By searching these generated candidates in the dictionary, the correct possible word "Figure" can be found. If there are multiple "correct" candidates in the dictionary, the most probable word is selected from a frequency table that indicates the probability of the word occurring. This method works well for small dictionary lookups, but requires that the frequency table can be obtained from a sample. However, when the vocabulary is large, computing the edited candidates and searching the dictionary for those edited candidates one by one can be very costly. It is therefore desirable to have a high-performance and accurate technique that leverages the quality of OCR and / or ICR engines.

[0004] Deep Learning (DL) based techniques are becoming increasingly essential for the development of large-scale state-of-the-art models deployed in many domains, including digital documentation, natural language, medical imaging, social media, autopilots, sports, and gaming. DL based methods have many advantages over traditional approaches in many aspects, including generality, quality performance, and speed efficiency. The trainable network weights can adapt to any non-parametric distribution and recurring patterns from the dataset. During training, the weights store important information, preserving the probability of each error word and its mapping to the correct word. DL models can be utilized to adapt the mapping from error words to ground truth words in the dataset. The mapping or relationship between error and ground truth can be highly non-linear and cannot be programmed by simple rules.

[0005] For various reasons, it can be complicated to select the right network as well as generate training and test datasets for developing high-performing DL models. For example, the size of the training dataset can pose a significant problem due to the very large size of the vocabulary. For example, for the word “Figure”, we can create training word pairs (error, ground truth): (Figuda, Figure), (Figeda, Figure), (Figuro, Figure), (Figuac, Figure), etc., and anchor pairs (ground truth, ground truth): (Figure, Figure). To generate all possible error-ground truth pairs, we need to randomly replace each string (character) with a candidate character from among 26 characters and 10 digits (ignoring capital letters). The word “Figure” has 6 characters. Using only one edited distance per character, we have (26+10)×6 error-ground truth pairs. For a vocabulary of 1 million words, the size of the training dataset can easily scale to 100 million. Therefore, we cannot train a model against this existing approach. Furthermore, in languages ​​with a large number of characters or character sets, such as Japanese, candidate strings often contain several thousand characters, making enumeration even more difficult than developing an engine for English.

[0006] Additionally, there may be inconsistencies in the training dataset or mismatches between the training and test datasets. For example, Figure 2 shows that overlaps and inconsistencies can occur between error and ground truth pairs if strings within words are randomly replaced. For example, the error words "seej" and "seqk" may map to the same or different ground truths, such as "seed" and "seek". In the training dataset, it is easy to generate both (seej, seek) and (seej, seed) cases in the training dataset without any control. This can cause the engine to stop learning and may also saturate after many training epochs.

[0007] It would be desirable to provide more efficient approaches and techniques for developing DL-based spell checkers. [Means for solving the problem]

[0008] In view of the above, embodiments according to the present invention provide a text correction method and apparatus that utilizes a significantly reduced number of error-ground truth pairs to train a deep learning model. To generate these error-ground truth pairs, various characters of a ground truth word are replaced with symbols that are not present in any ground truth word to generate an error word that is paired with the ground truth word to provide an error-ground truth word pair. This process may be repeated for all ground truth words for which training is performed. In embodiments, pairs of characters in a ground truth word may be replaced with symbols to generate an error word that is paired with the ground truth word to provide an error-ground truth word pair. Again, this process is repeated for all ground truth words for which training is performed. In embodiments, this technique may be applied to training base models and end user models of the type discussed in the referenced patent application. [Brief description of the drawings]

[0009] Various aspects according to embodiments of the present invention will now be described in detail with reference to the following drawings, which are described as follows: [Figure 1] FIG. 1 is a diagram illustrating dictionary-based correction. [Diagram 2] FIG. 2 is a diagram showing word substitution. [Diagram 3] FIG. 3 is a diagram showing an example of mask levels according to the embodiment. [Figure 4] FIG. 4 is a high-level conceptual diagram showing a network according to an embodiment. [Diagram 5] FIG. 5 is a diagram illustrating examples of relevant masking positions according to an embodiment. [Figure 6] FIG. 6 is a table showing the strength of relationships between characters according to the embodiment. [Figure 7] FIG. 7 is a table showing the strength of relationships between characters according to the embodiment. [Figure 8] FIG. 8 is a diagram illustrating an example of identifying text to be modified. [Figure 9] Figure 9A is a flowchart illustrating a high-level example of DL model training and text correction according to an embodiment, and Figures 9B-9D are diagrams illustrating variations of the flowchart of Figure 9A according to various embodiments. [Figure 10] FIG. 10 illustrates a high-level example of a system for receiving input data and training a DL model, according to an embodiment. [Figure 11] FIG. 11 is a diagram illustrating an example of a higher level of nodes in the DL model according to the embodiment. [Figure 12] FIG. 12 is a diagram illustrating an example of a higher level DL system for model learning according to the embodiment. Detailed Description of the Invention

[0010] The present application provides a method for generating, with a processor, a training data set comprising a plurality of error-ground truth word pairs for a plurality of ground truth words, wherein one or more of the plurality of error-ground truth word pairs include a ground truth word including a plurality of characters and an error word including one or more symbols substituted for one or more of the plurality of characters in the ground truth word, the one or more symbols being characters that are not present in any of the plurality of ground truth words; Organizing, at the processor, the plurality of error-ground truth word pairs into one or more groups of one or more error-ground truth pairs, each group comprising: one or more of the characters in the ground truth word; having an error word including one or more symbols replaced in a predetermined pattern; and selecting, by a processor, one or more from the one or more groups for training a deep learning (DL) model on a training dataset to perform text correction.

[0011] The present application relates to a processor and and a non-transitory memory containing instructions that, when executed by a processor, enable operation of a method, comprising: The method is generating a training data set comprising a plurality of error-ground truth word pairs for a plurality of ground truth words, wherein one or more of the plurality of error-ground truth word pairs include a ground truth word including a plurality of characters and an error word including one or more symbols replaced with one or more of the plurality of characters in the ground truth word, the one or more symbols being characters that are not present in any of the plurality of ground truth words; Organizing the plurality of error-ground truth word pairs into one or more groups of one or more error-ground truth pairs, each group comprising: one or more of the characters in the ground truth word; having an error word including one or more symbols replaced in a predetermined pattern; and selecting one or more from the one or more groups to train a deep learning (DL) model on a training dataset to perform text correction.

[0012] In one embodiment, the one or more groups comprise a first group of a plurality of pairs of error-ground truth words; For each ground truth word in the plurality of ground truth words in the first group of the plurality of error-ground truth word pairs, each error-ground truth word pair comprises the ground truth word and an error word including symbols replaced with different characters in the ground truth word, and for each ground truth word, there are a number of error words equal to the number of characters in the ground truth word.

[0013] In one embodiment, the one or more groups comprises a second group of a plurality of pairs of error-ground truth words; For each ground truth word in the plurality of ground truth words in the second group of the plurality of error-ground truth word pairs, each error-ground truth word pair comprises the ground truth word and an error word including a symbol replaced with two different consecutive characters in the ground truth word, and for each ground truth word, there are one less error words than the number of characters in the ground truth word.

[0014] In one embodiment, the one or more groups comprise a third group of a plurality of pairs of error-ground truth words; For each ground truth word in the plurality of ground truth words in the third group of the plurality of error-ground truth word pairs, each error-ground truth word pair comprises the ground truth word and an error word including symbols replaced with every other character in the ground truth word, and for each ground truth word, there are two error words.

[0015] In one embodiment, in the third group of the plurality of error-ground truth word pairs, each ground truth word has one error word that includes even-numbered characters replaced with a symbol and one error word that includes odd-numbered characters replaced with a symbol.

[0016] In one embodiment, the one or more groups comprise a first group of a plurality of pairs of error-ground truth words and a second group of a plurality of pairs of error-ground truth words; For each ground truth word in the plurality of ground truth words in the first group of the plurality of error-ground truth word pairs, each error-ground truth word pair comprises the ground truth word and an error word including a symbol substituted for a different character in the ground truth word; For each ground truth word in the plurality of ground truth words in the second group of the plurality of error-ground truth word pairs, each error-ground truth word pair comprises the ground truth word and an error word including a symbol replaced with two consecutive characters that are different in the ground truth word.

[0017] In one embodiment, the one or more groups comprise a first group of a plurality of pairs of error-ground truth words, a second group of a plurality of pairs of error-ground truth words, and a third group of a plurality of pairs of error-ground truth words; For each ground truth word in the plurality of ground truth words in the first group of the plurality of error-ground truth word pairs, each error-ground truth word pair comprises the ground truth word and an error word including a symbol substituted for a different character in the ground truth word; for each ground truth word in the plurality of ground truth words in the second group of the plurality of error-ground truth word pairs, each error-ground truth word pair comprises the ground truth word and an error word including a symbol replaced with two consecutive characters that are different in the ground truth word; For each ground truth word in the plurality of ground truth words in the third group of the plurality of error-ground truth word pairs, each error-ground truth word pair comprises the ground truth word and an error word including symbols replaced with every other character in the ground truth word.

[0018] In one embodiment, the text correction comprises one or more of image character recognition (ICR) and optical character recognition (OCR).

[0019] In one embodiment, the method also includes, after selection, training the DL model with the training dataset.

[0020] In one embodiment, the method also includes, after training, correcting the input text using the trained DL model.

[0021] In the following description, various embodiments use various of the following levels of error-ground truth pairs, referred to herein as Levels 1 through 3. Additionally, embodiments may use what is referred to as "Level 0," which is a source text input.

[0022] (Level 1) In one embodiment, during the generation of the training dataset, instead of randomly replacing strings (characters) to provide a very large set of error words, strings of one or more characters can be replaced with one or more symbols that have never been used in the ground truth. For example, if the word "Figure" is used to generate the training dataset, instead of 26 x 6 error-ground truth pairs that would result from using all possible characters at each position in the word, only 6 error-ground truth pairs can be generated by replacing an unused symbol with each character in "Figure". The resulting error-ground truth pairs look like this: (Figure) (F%ure, Figure) (Fi%ure, Figure) (Fig. (Fig%e, Figure) (Figure%)

[0023] These 6 error-ground truth pairs can then be used to train a DL model. This approach significantly reduces the error training space, making it easier for the engine to train. Specifically, the number of error-ground truth pairs removed is (26×6-6) / 26×6=.962, which means that the error training space is reduced by 96.2%.

[0024] (Level 2) In one embodiment, five error-ground truth pairs can be generated by replacing pairs of two consecutive character symbols in “Figure”. The resulting error-ground truth pairs are: (%%gure, Figure) (F%%ure, Figure) (Fi%%re, Figure) (Fig. (Fig%%, Figure)

[0025] These 5 error-ground truth pairs can then be used to train a DL model. This approach further reduces the error training space, making it easier for the engine to train. Specifically, the number of error-ground truth pairs removed is (26×26×5-5) / 26×26×5=.998, which means the error training space is reduced by 99.8%.

[0026] (Level 3) In one embodiment, to further leverage learning efficiency and reduce the number of inconsistencies, a symmetric deletion method can be used where all odd characters in a word or all even characters in a word can be replaced with a symbol. This technique produces two error-ground truth pairs. Looking again at the ground truth word "Figure", in the first pair, the characters in odd positions in the string are replaced with a symbol. In the second pair, the characters in even positions are replaced with a symbol. The resulting error-ground truth pairs look like this: (Figure) (F%g%r%, Figure)

[0027] In this embodiment, it is possible to avoid most of the inconsistencies in the training set and reduce the amount of training data by up to 90%. Those skilled in the art will appreciate that this symmetry removal method is particularly effective for longer words.

[0028] In this embodiment, at inference, two masked versions of the error word are input. The correction can then be verified using symmetry matches. If it is found that the inferences "%i%u%e" and "F%g%r%" recover each other's missing letters and map to the same word "Figure", the correction word "Figure" is considered final and output to the user accordingly. Otherwise, one of the most likely words is output using a two-stage inference providing two inferences for each masked word. (e.g. "%i%u%e" to "Figure" and "F%g%r%" to "Figure"). If there is symmetry or reflection of the two inferences, meaning that the mapping is unique and has a high probability, the result is considered final and output to the user accordingly. Otherwise, two final candidate words are output from which the user can choose the correct one.

[0029] During training, these error pairs can be grouped into a batch and fed to the network with that batch of words, allowing the engine to focus on learning the global relationships between characters within a word while ignoring the effects of individual characters and any inserted or substituted error characters.

[0030] FIG. 3 illustrates four levels of deletion and replacement for training a DL model according to one embodiment, where levels 1 to 3 are as described and level 0 is the original text input as above.

[0031] Those skilled in the art will appreciate that replacing too many consecutive characters in a ground truth word with a symbol may produce a set of error-ground truth pairs that are ineffective or inaccurate. Those skilled in the art will also appreciate that for longer ground truth words, replacing three consecutive characters with a symbol, or even more characters with a symbol, may be useful. In this situation, for longer words, one approach is to provide another level of replacing three consecutive characters with a symbol. For different languages, or different sequences of characters or words, especially long word sequences, additional levels may be useful. On the other hand, for such long words, word sequences, or strings, level 3 alone may be sufficient, even without levels 0 to 2.

[0032] In embodiments, depending on the input word, less than all of levels 0 through 3 may be required. For example, levels 1 through 3, or levels 1 and 2 may be sufficient. For short words or strings, all of levels 0 through 3 may be required.

[0033] In an embodiment, any of levels 1 to 3 can be used individually or in any combination to train a DL model.

[0034] Those skilled in the art will also appreciate that the various levels of symbolic substitution described above facilitate the generation of synthetic training data sets.

[0035] In general, from the foregoing, we can see that context is important when trying to translate an input word into a correct output. One way to provide that context could be a situation such as a customer correcting a character in a word. That character in that position in the word is found to be correct, providing context for future learning purposes. That corrected character can then be used in place of the symbol in the error-ground truth pair.

[0036] Context can also occur in the context of a word sequence. For example, there may be common word sequences where knowing what one of the words is provides context to limit the range of error words, and therefore the number of error-ground truth pairs. An example of such a sequence includes the word "cell." Often, there are many words that go along with the word "cell," especially those that precede it. Examples of a non-exhaustive list include "stem," "epithelial," "cancer," "animal," "human," "plant," "nerve," "spreadsheet," "solar," "fuel," "storage," "detention center," and "prison."

[0037] There are also many words that form word pairs following the word "cell." A non-exhaustive list of examples includes: "block," "telephone," "number," "biology," "division," "culture," "membrane," "division," "wall," and "structure."

[0038] Word combinations that contain roots such as "cell" appear in a variety of contexts. In the above word pairs, biology, communication, criminology, accounting or word processing are examples of areas of learning or knowledge, providing different contexts for the word pairing with "cell."

[0039] The following description of Figures 4 and 5 refers to masking positions.

[0040] FIG. 4 shows an OCR and ICR network with details of the internal modules of the masking network (left) and the correction network (right). On the left side of FIG. 4, an attention masking network 410 masks all positions associated with incorrect characters. As described herein, according to various embodiments, the mask can include one or more symbols that are arranged in a specific manner depending on the level of error-ground truth pairing applied. In one embodiment, the attention masking network 410 applies a deep learning algorithm 412 to create a masked feature vector to be incorporated into the input vector for the correction network to decode for the final output. In one embodiment, the deep learning algorithm 412 is a bidirectional long short-term memory (Bi-LSTM).

[0041] On the right side of Figure 4, a correction network 420 outputs a final sequence for the input sequence. In one embodiment, the correction network 420 includes a deep learning network 422. In one embodiment, the deep learning network 422 may be a Bidirectional Encoding Representation by Transformer (BERT) network. In another embodiment, the network 422 may be a Bi-LSTM network. In another embodiment, the network 422 may be a Neural Machine Translation (NMT) network.

[0042] In one embodiment, the output of the output decoder 424 of the error correction network 420 passes to a loss block 430. In one embodiment, the loss block 430 computes a connectionist temporal classification (CTC) loss. In another embodiment, the loss block 430 can compute a multi-layer perceptron (MLP) loss accompanied by cross-entropy (CE).

[0043] Figure 5 shows an example of relevant masked positions. Figure 5 shows vector composition (error masked embedding), where the masked relevant positions are labeled with black and grey cubes (top array), and grey indicates the original embedding vector of the input word (bottom array). The masked embedding vector is the input to the error correction network 420 on the right side of Figure 4.

[0044] A self-trained attention matrix is ​​a matrix that contains correlations between individual characters. Figure 6 shows an example of a trained self-trained attention matrix between characters for the word "FIGURE". In this example, darker tones represent weaker relationships and lighter tones represent stronger relationships. Figure 6 shows a sequence of numbers representing strong and weak relationships between characters. This attention matrix can be trained and included in a model.

[0045] The attention mask can be learned by supervised learning and is strongly linked to the error modes and patterns of the input and ground truth text. The training dataset can be generated using the edit distance (Levenshtein distance) between two sequences. Levenshtein distance can measure the alignment between two related sequences. The minimum edit distance is calculated through an iterative process in the algorithm. Common edits used in Levenshtein distance include (1) substitutions, (2) insertions, and (3) deletions.

[0046] Figure 7 shows an example of the Levenshtein distance between the error word "Lavenshtien" and the ground truth word "Levenshtein". The table in Figure 7 shows the distance between each alignable subsequence.

[0047] The results are transformed from the Levenshtein distance to generate training pairs for training a mask network. All edited positions are marked and a binary mask is applied representing correct and incorrect positions. Thus, for the “Lavenshtien” and “Levenshtein” example in Figure 7, the labeling vector corresponding to the three modifications “replace a with e”, “insert e after t”, and “delete e after i” is “0100001100”.

[0048] In one embodiment, the input and ground truth sequences can have different lengths. No extra digits are added during insertion and deletion to match the ground truth sequence. For example, if the input error word of "Lavenshtien" has a length of "11", then the length of the ground truth vector should be "11" even after insertion and deletion operations. Deletion and insertion change the mapping of the length of the input and target sequences. In one embodiment, the maximum number of deletion and insertion operations should not be less than half the original characters of the ground truth word.

[0049] FIG. 8 shows the results of the masking technique according to one embodiment. The results show examples of learned character relationships and inferred words considering the masked positions. These results show that DL spell checking models can be trained in this highly efficient and robust manner to learn spatial relationships between characters and masked positions. Our technique does not require the creation of all possible error words for ground truth words. This results in significant savings in both training time and training corpus size. Also, the training can be easily adapted to self-attention networks such as Transformer networks, which are state-of-the-art networks that can provide excellent performance for many natural language processing (NLP) tasks.

[0050] 9A is a flow chart showing a sequence of operations of the system of the present invention according to one embodiment. At 910, an error word is generated by replacing characters of the ground truth word with one or more symbols, where the symbols do not appear as characters in any of the ground truth words. In one embodiment, only one type of replacement is made at this point. In other embodiments, more than one type of replacement can be made.

[0051] At 920, the generated error words are matched against their ground truth words to generate a training data set consisting of error-ground truth word pairs. At 930, the error-ground truth word pairs are organized into groups, each group having the same predefined pattern of symbols for each ground truth word. The resulting groups are pairs that contain ground truth words in the data set and match error words that have the same predefined pattern of symbols for the corresponding ground truth word.

[0052] At 940, the generated group is selected as a dataset for training the DL. At 950, it is determined whether there are multiple groups in the dataset, each with a different pattern. If so, at 955, another generated group is added. At 960, it is determined whether there are more groups in the dataset. If so, the flow returns to 955. If not, the dataset is complete and at 970, a DL model is trained on the dataset. At 980, the input text is modified using the trained DL model.

[0053] FIG. 9B shows a variation 930′ of FIG. 9A in which a group of error-ground truth word pairs is generated by substituting a symbol for each of the corresponding characters in a ground truth word to generate a set of error-ground truth word pairs for that ground truth word. The number of word pairs in the set is equal to the number of characters in that ground truth word. The groups then constitute one of these sets for each ground truth word.

[0054] FIG. 9C shows a variation 930″ of 930 in FIG. 9A in which a group of error-ground truth word pairs is generated by substituting two consecutive symbols for two corresponding consecutive characters of a ground truth word to generate a set of error-ground truth word pairs for that ground truth word. The number of word pairs in that set is one less than the number of characters in that ground truth word. The groups then constitute one of these sets for each ground truth word.

[0055] FIG. 9D shows a variation 930′″′ of 930 in FIG. 9A in which a group of error-ground truth word pairs is generated by substituting a symbol for each of the alternative letters of a ground truth word to generate a set of error-ground truth word pairs for that ground truth word. The number of word pairs in the set is two, one with even numbered letters replaced with a symbol and one with odd numbered letters replaced with a symbol. The group then constitutes one of these sets for each ground truth word.

[0056] It should be noted that the order of operations of at least some of 910-980 is not important. For example, any of 930', 930", or 930'''' can be performed in place of 910 to generate only one group at a time. Additionally or alternatively, any two or more of 930', 930", or 930'''' can be performed in place of 910 to generate two or more groups at a time. Furthermore, all of 930', 930", or 930'''' can be performed in place of 910 to generate three groups at a time.

[0057] FIG. 10 illustrates a high-level example of a system for training and implementing DL models, according to one embodiment. In FIG. 10, input text 1010 may be provided via a scanner or other input device 1020 to an ICR module 1050 or an OCR module 1070 depending on the text recognition process to be performed. The processing system 1040 may include a processing module 1090, which may communicate with either the ICR module 1050 or the OCR module 1070 as appropriate or necessary and provide the necessary processing power via one or more central processing units (CPUs) or graphics processing units (GPUs) and associated non-transitory storage and / or non-transitory memory. In one embodiment, the processing system 1040 may communicate directly with either the computer 1030 or the scanner / input device 1020. The processing system 1040 may be self-contained or its various elements may be connected via a network or cloud 1060. Either or both of the ICR module 1050 or the OCR module 1070 may each have their own CPU and / or GPU and may communicate with the processing module 1090 via a network or cloud 1060.

[0058] A deep learning system 1200, described in more detail below with reference to FIG. 12, processes changes to the DL model.

[0059] 11 is a diagram illustrating a high-level example of a DL model according to an embodiment. The DL model 1200 has an input layer 1210 consisting of multiple nodes. In some systems, there may be one or more hidden layers 1220-1240 (FIG. 11B shows P such layers, where P is a positive integer), each of which in turn consists of multiple nodes. There may also be an output layer 1250.

[0060] 12 shows the deep learning system 1200 in a bit more detail. A database 1260 stores weights and data for the model. A node weighting module 1270 calculates weights for various nodes in various layers based, among other things, on comparison with the results and assigns those weights accordingly to layers 1270-1, 1270-2, ... 1270-N-2, 1270-N-1, and 1270-N.

[0061] The techniques described herein are applicable to base model deep learning systems and / or end user model deep learning systems, as described in the above-referenced U.S. patent applications. Its application is, among other things, to user-customizable DL models that work from a base model trained in a more general way. For example, the base model may have an input layer and a certain number of hidden layers. Depending on the embodiment, the last hidden layer may be an output layer or may be a layer prior to further layers in the end user model. In any case, the error-ground truth pairing techniques described herein may be used to train either the base model or the end user model. In an embodiment, the base model has a more general vocabulary, while different end user models may have different specific vocabularies associated with them.

[0062] In the above-referenced US patent application embodiment, the end user is not able to modify the base model itself. End user modifications focus on the needs of the particular end user with respect to the output layer of the overall DL model. In the deep learning model embodiment using backpropagation, the end user may modify the weightings of the nodes in one or more of the input or intermediate layers of the deep learning system of the model. Such modifications may be filtered back to the base model in some form, depending on how the datasets from different end users are presented to the deep learning system of the base model, how large those datasets are, etc. These and other possibilities are described in detail above. But essentially, even if there is an output from the end user model that modifies the weightings of the base model, the modification is made by the host that controls the base model.

[0063] Although embodiments of the present invention have been described in detail with reference to various drawing figures, those skilled in the art will recognize that numerous modifications may be made within the scope and spirit of the invention. Accordingly, the present invention is limited only by the following claims.

Claims

1. generating, with a processor, a training data set comprising a plurality of error ground truth word pairs for a plurality of ground truth words, wherein one or more of the plurality of error ground truth word pairs include a ground truth word including a plurality of characters and an error word including one or more symbols substituted for one or more of the plurality of characters in the ground truth word, the one or more symbols being characters not present in any of the plurality of ground truth words; Organizing, with a processor, the plurality of error-ground truth word pairs into one or more groups of one or more error-ground truth pairs, each group having an error word including one or more of the symbols replaced in a predetermined pattern with one or more of the plurality of characters in the ground truth word; and selecting, at a processor, one or more from the one or more groups for training a deep learning (DL) model on the training dataset to perform text correction.

2. the one or more groups comprising a first group of a plurality of pairs of error-ground truth words; 2. The computer-implemented method of claim 1, wherein for each ground truth word in the plurality of ground truth words in the first group of a plurality of error ground truth word pairs, each error ground truth word pair comprises the ground truth word and an error word including a symbol substituted for a different character in the ground truth word, and for each ground truth word, there are a number of error words equal to a number of characters in the ground truth word.

3. the one or more groups comprising a second group of a plurality of pairs of error ground truth words; 3. The method of claim 1, wherein for each ground truth word in the plurality of ground truth words in the second group of a plurality of error ground truth word pairs, each error ground truth word pair comprises a ground truth word and an error word including a symbol replaced with two consecutive characters that are different in the ground truth word, and for each ground truth word, there is one less error word than the number of characters in the ground truth word.

4. the one or more groups comprising a third group of a plurality of pairs of error ground truth words; 3. The computer-implemented method of claim 1, wherein for each ground truth word in the plurality of ground truth words in the third group of a plurality of error ground truth word pairs, each error ground truth word pair comprises a ground truth word and an error word including symbols replaced with every other character in the ground truth word, and for each ground truth word, there are two error words.

5. 5. The computer-implemented method of claim 4, wherein in the third group of error ground truth word pairs, each ground truth word has one error word that includes even-numbered characters replaced with a symbol and one error word that includes odd-numbered characters replaced with a symbol.

6. the one or more groups comprising a first group of a plurality of pairs of error-ground truth words and a second group of a plurality of pairs of error-ground truth words; for each ground truth word in the plurality of ground truth words in the first group of a plurality of error-ground truth word pairs, each error-ground truth word pair comprises the ground truth word and an error word comprising a symbol substituted for a different character in the ground truth word; 3. The computer-implemented text correction method of claim 1, wherein for each ground truth word in the plurality of ground truth words in the second group of a plurality of error ground truth word pairs, each error ground truth word pair comprises a ground truth word and an error word that includes a symbol replaced with two consecutive characters that are different in the ground truth word.

7. the one or more groups comprising a first group of pairs of error-ground truth words, a second group of pairs of error-ground truth words, and a third group of pairs of error-ground truth words; for each ground truth word in the plurality of ground truth words in the first group of a plurality of error-ground truth word pairs, each error-ground truth word pair comprises the ground truth word and an error word comprising a symbol substituted for a different character in the ground truth word; for each ground truth word in the plurality of ground truth words in the second group of a plurality of error-ground truth word pairs, each error-ground truth word pair comprises a ground truth word and an error word including a symbol replaced with two consecutive characters that are different in the ground truth word; 3. The computer-implemented method of claim 1, wherein for each ground truth word in the plurality of ground truth words in the third group of a plurality of error ground truth word pairs, each error ground truth word pair comprises a ground truth word and an error word including symbols replaced with every other character in the ground truth word.

8. 3. The computer-implemented method of text correction of claim 1 or 2, wherein the text correction comprises one or more of Image Character Recognition (ICR) and Optical Character Recognition (OCR).

9. The computer-implemented method of claim 1 or 2, further comprising, after said selection, training said DL model with said training data set.

10. 3. The computer-implemented method of claim 1 or 2, further comprising, after said training, correcting an input text using the trained DL model.

11. A processor; and a non-transitory memory containing instructions that, when executed by a processor, enable operation of a method, comprising: The method further comprising: generating a training data set comprising a plurality of error-ground truth word pairs for a plurality of ground truth words, wherein one or more of the plurality of error-ground truth word pairs include a ground truth word including a plurality of characters and an error word including one or more symbols substituted for one or more of the plurality of characters in the ground truth word, the one or more symbols being characters not present in any of the plurality of ground truth words; Organizing the plurality of error-ground truth word pairs into one or more groups of one or more error-ground truth pairs, each group having an error word including one or more of the symbols substituted in a predetermined pattern with one or more of the plurality of characters in the ground truth word; and selecting one or more from the one or more groups for training a deep learning (DL) model on the training dataset to perform text correction.

12. the one or more groups comprising a first group of a plurality of pairs of error-ground truth words; 12. The computer-implemented text correction apparatus of claim 11, wherein for each ground truth word in the plurality of ground truth words in the first group of a plurality of error ground truth word pairs, each error ground truth word pair comprises the ground truth word and an error word including a symbol substituted for a different character in the ground truth word, and for each ground truth word, there are a number of error words equal to the number of characters in the ground truth word.

13. the one or more groups comprising a second group of a plurality of pairs of error ground truth words; 13. The computer-implemented text correction apparatus of claim 11 or 12, wherein for each ground truth word in the plurality of ground truth words in the second group of a plurality of error ground truth word pairs, each error ground truth word pair comprises a ground truth word and an error word including a symbol replaced with two different consecutive characters in the ground truth word, and for each ground truth word, there is one less error word than the number of characters in the ground truth word.

14. the one or more groups comprising a third group of a plurality of pairs of error ground truth words; 13. The computer-implemented text correction apparatus of claim 11 or 12, wherein for each ground truth word in the plurality of ground truth words in the third group of a plurality of error ground truth word pairs, each error ground truth word pair comprises a ground truth word and an error word including symbols replaced with every other character in the ground truth word, and for each ground truth word there are two error words.

15. 15. The computer-implemented text correction apparatus of claim 14, wherein in the third group of error ground truth word pairs, each ground truth word has one error word that includes even-numbered characters replaced with a symbol and one error word that includes odd-numbered characters replaced with a symbol.

16. the one or more groups comprising a first group of a plurality of pairs of error-ground truth words and a second group of a plurality of pairs of error-ground truth words; for each ground truth word in the plurality of ground truth words in the first group of a plurality of error-ground truth word pairs, each error-ground truth pair comprises an error word including the ground truth word and a symbol replaced with a different character in the ground truth word; 13. The computer-implemented text correction apparatus of claim 11 or 12, wherein for each ground truth word in the plurality of ground truth words in the second group of a plurality of error ground truth word pairs, each error ground truth word pair comprises a ground truth word and an error word including a symbol replaced with two consecutive characters that are different in the ground truth word.

17. the one or more groups comprising a first group of pairs of error-ground truth words, a second group of pairs of error-ground truth words, and a third group of pairs of error-ground truth words; for each ground truth word in the plurality of ground truth words in the first group of a plurality of error-ground truth word pairs, each error-ground truth word pair comprises an error word including the ground truth word and a symbol substituted for a different character in the ground truth word; for each ground truth word in the plurality of ground truth words in the second group of a plurality of error-ground truth word pairs, each error-ground truth word pair comprises an error word including a ground truth word and a symbol replaced with two consecutive characters that are different in the ground truth word; 13. The computer-implemented text correction apparatus of claim 11 or 12, wherein for each ground truth word in the plurality of ground truth words in the third group of a plurality of error ground truth word pairs, each error ground truth pair comprises a ground truth word and an error word including symbols replaced with every other character in the ground truth word.

18. 13. A computer implemented text correction apparatus according to claim 11 or 12, wherein the text correction comprises one or more of Image Character Recognition (ICR) and Optical Character Recognition (OCR).

19. 13. A computer-implemented text correction apparatus according to claim 11 or 12, further comprising, after said selection, training said DL model with said training data set.

20. 13. A computer-implemented text correction apparatus according to claim 11 or 12, further comprising, after said training, correcting an input text using the trained DL model.

Images