Spelling check model generation method, device, computing device and storage medium

By constructing the second word confusion set and word confusion set, and combining the BERT model for spell checking model training, the problems of insufficient Chinese spell checking data and missing words with easy confusion are solved, and the accuracy of spell checking is significantly improved.

CN118468860BActive Publication Date: 2024-11-19PEOPLE CN CO LTD +1
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202410648420.4
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-05-23
Publication Date
2024-11-19
Estimated Expiration
2044-05-23

AI Technical Summary

Technical Problem

The existing Chinese spelling check model cannot effectively correct words with different sounds and shapes that do not appear when there is insufficient data and missing confusing words. It also relies too much on word confusing sets, ignoring the semantics of the sentence, resulting in error correction ability. limit.

Method used

By constructing the second word obfuscation set and the word obfuscation set, using the preset mixed mask strategy to mask the training text, generate enhanced training samples, and combine the BERT model for model training to improve the accuracy of the spell check model.

Benefits of technology

The generated spelling check model can be closer to the text of real misspellings, reduce the cost of manpower labeling, and significantly improve the accuracy of spelling checks.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118468860B_ABST
    Figure CN118468860B_ABST
Patent Text Reader

Abstract

The present application discloses a method, device, computing device and storage medium for generating a spelling check model, the method comprising: constructing a second character confusion set based on a first character confusion set and a first corpus data set disclosed; constructing a word confusion set based on a second corpus data set constructed by an input method and a first corpus data set; randomly sampling and determining a plurality of characters to be masked and at least one word to be masked from an original training text in a training text set; based on a preset mixed masking strategy, masking a plurality of characters to be masked using a second character confusion set and masking at least one word to be masked using a word confusion set to obtain an enhanced training text; and performing model training based on the original training text and the enhanced training text to generate a spelling check model. The present application realizes the automatic generation of enhanced training samples required for training, overcomes the problem of insufficient Chinese spelling check data, reduces the cost of human annotation, and also greatly improves the accuracy of the spelling check model.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of Internet technology, and in particular to a method, apparatus, computing device and storage medium for generating a spelling check model. Background Art

[0002] Chinese spelling check (CSC) is a basic natural language processing task. It detects and corrects potential character replacement errors in Chinese sentences by learning contextual information to ensure the accuracy and fluency of the article. Due to the lack of annotated corpus in early CSC research tasks, most of them adopt unsupervised N-gram statistical language model methods. Most existing CSC research tasks regard it as a sequence labeling task, construct pseudo data sentence pairs through word confusion sets, and use deep learning models such as LSTM, Transformer, and BERT to learn the semantic representation of the input sequence. Using word confusion sets to construct data sets for model training will cause the model to memorize the mapping between confusion set-related errors and correct characters. The final error correction will be overly restricted to the confusion set, unable to correct characters with dissimilar sounds and shapes that do not appear in the training set, and ignore the semantics of the entire sentence. In addition, after word segmentation, Chinese is usually composed of two or more words, phrases, or idioms, etc., ignoring the characteristics that Chinese sentences are usually words after word segmentation, and there is a lack of easily confused word data in the training set. Summary of the invention

[0003] In view of the above problems, the present application is proposed to provide a spelling check model generation method, apparatus, computing device, computer storage medium and computer program product that overcome the above problems or at least partially solve the above problems.

[0004] According to one aspect of the present application, a method for generating a spelling check model is provided, comprising:

[0005] Constructing a second character confusion set based on the first character confusion set and the first corpus data set;

[0006] Constructing a word confusion set according to the second corpus data set constructed by the input method and the first corpus data set;

[0007] Randomly sampling from the original training text in the training text set to determine a plurality of characters to be masked and at least one word to be masked;

[0008] Based on a preset mixed masking strategy, a plurality of characters to be masked are masked using a second character confusion set, and at least one word to be masked is masked using a word confusion set, to obtain an enhanced training text;

[0009] Model training is performed based on the original training text and the enhanced training text to generate a spelling check model.

[0010] Optionally, constructing the second character confusion set according to the disclosed first character confusion set and the first corpus data set further includes:

[0011] S1, performing word segmentation processing on each first corpus data in the first corpus data set, counting the occurrence frequency of each word, and retaining words whose occurrence frequency ranking is within a preset ratio;

[0012] S2, filtering the first character confusion set according to the characters reserved in S1, filtering out characters that do not appear in the characters reserved in S1 in the first character confusion set, and determining whether any first easily confused character pair in the first character confusion set after filtering out has a similar character shape structure and / or a similar character pronunciation; if both the character shape structure and the character pronunciation are not similar, filtering out the first easily confused character pair;

[0013] S3, generating a glyph structure and / or a pinyin sequence corresponding to each character in the characters reserved in S1, calculating an edit distance between each character in the characters reserved in S1 and any other character according to the glyph structure and / or the pinyin sequence, deleting easily confused character pairs whose edit distance is less than or equal to a first preset edit distance threshold, and forming second easily confused character pairs;

[0014] S4, merging the first easily confused character pair and the second easily confused character pair in the first character confusion set after being screened out to form a second character confusion set.

[0015] Optionally, generating a pinyin sequence corresponding to each character in the characters reserved by S1, and calculating the edit distance between each character in the characters reserved by S1 and any other character according to the glyph structure and / or the pinyin sequence further comprises:

[0016] If any character in the characters reserved by S1 is a polyphone, multiple pinyin sequences corresponding to the polyphone are generated, and the edit distance between each character in the characters reserved by S1 and any other character is calculated according to the glyph structure and / or any pinyin sequence.

[0017] Optionally, the method further includes: if any of the characters reserved in S1 contains a preset syllable, obtaining characters in the pinyin sequence that contain similar syllables that are paired with the preset syllable as easily confused characters to form a second easily confused character pair.

[0018] Optionally, the second corpus data set includes: a second corpus data pair;

[0019] Constructing a word confusion set based on the second corpus data set constructed by the input method and the first corpus data set further includes:

[0020] Performing word segmentation processing on two second corpus data in each second corpus data pair, determining whether two words corresponding to each word segmentation position are consistent, and if inconsistent, determining that the two words are an easily confused word pair;

[0021] Performing word segmentation processing on each first corpus data in the first corpus data set, and counting the word length of each word;

[0022] Two words with the same length are combined into a word pair, and it is determined whether the two characters corresponding to each character position in the word pair have similar glyph structures and / or similar pronunciations. If the two characters corresponding to each character position have similar glyph structures and / or similar pronunciations, the word pair is determined to be an easily confused word pair, and a word confusion set is constructed based on the easily confused word pairs.

[0023] Optionally, based on a preset hybrid masking strategy, masking a plurality of characters to be masked using a second character confusion set and masking at least one word to be masked using a word confusion set to obtain an enhanced training text further comprises:

[0024] For any word to be masked, randomly select the easily confused word corresponding to the word to be masked from the word confusion set, and use the selected easily confused word to replace the word to be masked;

[0025] Randomly select characters to be masked of a first replacement ratio, and for any character to be masked, randomly select an easily confused character corresponding to the character to be masked from the second character confusion set, and replace the character to be masked with the selected easily confused character;

[0026] Randomly select characters to be masked in the second replacement ratio, and replace the characters to be masked in the second replacement ratio with preset characters;

[0027] The characters to be masked in the third replacement ratio are randomly selected, and the characters to be masked in the third replacement ratio are replaced with any characters in the vocabulary.

[0028] Optionally, the method further includes: obtaining a text to be spell-checked;

[0029] The text to be spell-checked is input into the spell-checking model for spell-checking, and the text after spell-checking is output.

[0030] According to another aspect of the present application, a spelling check model generation device is provided, comprising:

[0031] A character confusion set construction module, adapted to construct a second character confusion set according to the first public character confusion set and the first corpus data set;

[0032] A word confusion set construction module, adapted to construct a word confusion set according to the second corpus data set constructed by the input method and the first corpus data set;

[0033] A determination module, adapted to randomly sample and determine a plurality of characters to be masked and at least one word to be masked from the original training text in the training text set;

[0034] A mask processing module, adapted to perform mask processing on a plurality of characters to be masked using a second character confusion set and perform mask processing on at least one word to be masked using a word confusion set based on a preset mixed masking strategy, to obtain an enhanced training text;

[0035] The generation module is suitable for training the model based on the original training text and the enhanced training text to generate a spelling check model.

[0036] According to another aspect of the present application, there is provided a computing device, comprising: a processor, a memory, a communication interface and a communication bus, wherein the processor, the memory and the communication interface communicate with each other via the communication bus;

[0037] The memory is used to store at least one executable instruction, and the executable instruction enables the processor to execute operations corresponding to the above-mentioned spelling check model generation method.

[0038] According to another aspect of the present application, a computer storage medium is provided, in which at least one executable instruction is stored, and the executable instruction enables a processor to perform operations corresponding to the above-mentioned spelling check model generation method.

[0039] According to another aspect of the present application, a computer program product is provided, comprising at least one executable instruction, wherein the executable instruction enables a processor to perform operations corresponding to the above-mentioned spelling check model generation method.

[0040] According to the spelling check model generation method, apparatus, computing device, computer storage medium and computer program product provided in the embodiments of the present application, enhanced training samples required for training are automatically generated. The enhanced training samples are closer to the text with actual spelling errors, overcoming the problem of insufficient Chinese spelling check data, reducing the cost of manual annotation, laying the foundation for the spelling check model, and also greatly improving the accuracy of the spelling check model.

[0041] The above description is only an overview of the technical solution of the present application. In order to more clearly understand the technical means of the present application, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the present application more obvious and easy to understand, the specific implementation methods of the present application are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Various other advantages and benefits will become apparent to those of ordinary skill in the art by reading the detailed description of the preferred embodiments below. The accompanying drawings are only for the purpose of illustrating the preferred embodiments and are not to be considered as limiting the present application. Also, the same reference symbols are used throughout the accompanying drawings to represent the same components. In the accompanying drawings:

[0043] Figure 1 A schematic diagram of a process for generating a spelling check model according to an embodiment of the present application is shown;

[0044] Figure 2 A schematic flow chart of a method for generating a spelling check model according to another embodiment of the present application is shown;

[0045] Figure 3 A schematic diagram of the structure of a spelling check model generating device according to an embodiment of the present application is shown;

[0046] Figure 4 A schematic diagram of the structure of a computing device according to an embodiment of the present application is shown. DETAILED DESCRIPTION

[0047] The exemplary embodiments of the present application will be described in more detail below with reference to the accompanying drawings. Although the exemplary embodiments of the present application are shown in the accompanying drawings, it should be understood that the present application can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided in order to enable a more thorough understanding of the present application and to fully convey the scope of the present application to those skilled in the art.

[0048] Figure 1 FIG. 2 shows a flow chart of a method for generating a spelling check model according to an embodiment of the present application. Figure 1 As shown, the method comprises the following steps:

[0049] Step S101, constructing a second character confusion set according to the first public character confusion set and the first corpus data set.

[0050] Specifically, in order to improve the accuracy of the spelling check model, the present application needs to construct a second character confusion set, and the second character confusion set is constructed based on the disclosed first character confusion set and the first corpus data set.

[0051] The first character confusion set is a publicly available machine-generated character confusion set, which counts characters that are easily misused together to form a first easily confused character pair. For example, the first character confusion set may be a character confusion set such as SIGHAN, NCU_NLPLab, etc. However, there may be a situation where the characters in the first easily confused character pair and the easily confused characters are not similar in pronunciation and glyph structure. Therefore, the publicly available first character confusion set may be screened to screen out the first easily confused character pairs with similar pronunciation and / or glyph structure, and to exclude the first easily confused character pairs in the first character confusion set with dissimilar pronunciation and glyph structure.

[0052] In addition, there may be a situation where the first character confusion set does not cover all aspects. Therefore, when constructing the second character confusion set, the first corpus data set is also considered, wherein the first corpus data set includes multiple first corpus data, such as news data, etc. Therefore, easily confused character pairs can be constructed based on the first corpus data set, thereby constructing the second character confusion set.

[0053] Step S102, constructing a word confusion set according to the second corpus data set constructed by the input method and the first corpus data set.

[0054] Specifically, easily confused words can be selected from the second corpus data set constructed by the input method and the first corpus data set, and a word confusion set can be constructed based on the selected easily confused words. The word confusion set is a set of easily confused word pairs, and easily confused words constitute easily confused word pairs. The second corpus data set is a set of second corpus data pairs, and the second corpus data pairs are data pairs that are used based on the easily confused words constructed by the input method.

[0055] Step S103: randomly sampling from the original training text in the training text set to determine a plurality of characters to be masked and at least one word to be masked.

[0056] The original texts in the training text set are texts without spelling errors. In order to train a more accurate spelling check model, enhanced training samples required for training the spelling check model are generated based on the original text. The enhanced training samples are texts with spelling errors.

[0057] When generating enhanced training samples, it is necessary to first randomly sample from the original text in the training text set to determine multiple characters to be masked and at least one word to be masked, that is, which characters and which words in the original text are masked are random.

[0058] Step S104, based on a preset mixed masking strategy, mask multiple characters to be masked using the second character confusion set and mask at least one word to be masked using the word confusion set to obtain enhanced training text.

[0059] After determining multiple characters to be masked and at least one word to be masked that need to be masked, based on a preset mixed masking strategy, the multiple characters to be masked can be masked using a second character confusion set and the at least one word to be masked can be masked using a word confusion set to obtain enhanced training text. The preset mixed masking strategy defines a specific masking method, for example, defines a replacement ratio, defines a masking method, etc.

[0060] Step S105: Perform model training based on the original training text and the enhanced training text to generate a spelling check model.

[0061] After obtaining the enhanced training text, model training is performed based on the original text and the enhanced training text to generate a spelling check model. Specifically, the enhanced training text is used as a task for fine-tuning the BERT model, and the entire sentence of the enhanced training text is input for mask prediction to obtain the predicted text. The prediction results of the mask positions in the predicted text are compared to see whether they are consistent with the data of the corresponding positions in the original text, and the accuracy of the mask positions is counted. When the accuracy of the mask positions does not meet the preset training end conditions, continue to perform mask prediction on the enhanced training text; when the accuracy of the mask positions meets the preset training end conditions, stop training the BERT model to obtain the spelling check model.

[0062] In summary, according to the method provided in this embodiment, enhanced training samples required for training are automatically generated. The samples are closer to real texts with spelling errors, which overcomes the problem of insufficient Chinese spelling check data, reduces the cost of manual annotation, lays the foundation for the spelling check model, and also greatly improves the accuracy of the spelling check model.

[0063] Figure 2 FIG. 2 shows a flow chart of a method for generating a spelling check model according to another embodiment of the present application. Figure 2 As shown, the method comprises the following steps:

[0064] Step S201 , performing word segmentation processing on each first corpus data set in the first corpus data set, counting the occurrence frequency of each word, and retaining words whose occurrence frequency ranking is within a preset ratio.

[0065] Specifically, the first corpus data set includes a plurality of first corpus data, each of which is divided into characters to form a character table, and the number of occurrences of each character is counted to obtain the occurrence frequency of each character, and the characters whose occurrence frequency is ranked before a preset ratio are determined as high-frequency characters, wherein the preset ratio can be flexibly set according to actual needs, for example, set to 40%, that is, retaining the characters whose occurrence frequency is ranked within 40%. The first corpus data set can be a news data set.

[0066] Step S202, filtering the first character confusion set according to the reserved characters in step S201, filtering out characters in the first character confusion set that do not appear in the reserved characters in step S201, and determining whether any first easily confused character pair in the first character confusion set after filtering out has similar glyph structures and / or similar pronunciations; if both the glyph structures and pronunciations are not similar, filtering out the first easily confused character pair.

[0067] The first character confusion set is a publicly available machine-generated character confusion set, which collects characters that are easily misused to form first easily confused character pairs. Table 1 briefly lists some of the first easily confused character pairs in the first character confusion set.

[0068] Table 1

[0069] Character Confusing words Character son, self, capital … … I Find, hold

[0070] However, some characters in the first character confusion set may have a low frequency of occurrence, a low probability of use, etc. Retaining these characters will take up storage space and also reduce the speed of subsequent searches. Therefore, the first character confusion set can be filtered according to the characters reserved in step S201 to filter out the characters in the first character confusion set that do not appear in the characters reserved in step S201. For example, it is determined whether each character in the first character confusion set can match the characters in the characters reserved in step S201. If they match, the character is retained; otherwise, the character is filtered out. In addition, there may be a situation where the characters in the first easily confused character pair in the first character confusion set are not similar to the easily confused characters in pronunciation and character shape structure. Therefore, it is necessary to determine whether any first easily confused character pair in the first character confusion set after being screened out has similar character shape structure and / or similar character pronunciation. If both the character shape structure and the character pronunciation are not similar, the first easily confused character pair is screened out. When determining whether the character shape structure of the characters in the first easily confused character pair is similar to the character shape structure of the easily confused characters corresponding to it, the character in the first easily confused character pair and the character shape structure of the easily confused characters corresponding to it are split to determine whether the character shape structures of the two characters are similar. When determining whether the pronunciation of the characters in the first easily confused character pair is similar to the pronunciation of the easily confused characters corresponding to it, the pinyin of the characters can be input in the input method, and the front and back nasal sounds or a wrong input of a letter can be input. It is determined whether the easily confused characters are in the preset number of characters prompted. If so, it is determined that the characters have similar pronunciations. If not, it is determined that the characters have different pronunciations.

[0071] If the character structure and pronunciation are not similar, the first easily confused character pairs are screened out. After the above process, the number of first easily confused character pairs remaining in the first character confusion set may be less than or equal to the number of characters ranked before the preset ratio in terms of frequency of occurrence.

[0072] Step S203, generating a glyph structure and / or pinyin sequence corresponding to each character in the characters reserved in step S201, calculating the edit distance between each character in the characters reserved in step S201 and any other character according to the glyph structure and / or pinyin sequence, deleting easily confused character pairs whose edit distance is less than or equal to a first preset edit distance threshold, and forming second easily confused character pairs.

[0073] Specifically, for each high-frequency character in the characters retained in step S201, its glyph structure and / or pinyin sequence is generated, any two characters in the high-frequency characters can form a confusing character pair, and the edit distance is used as a judgment method to calculate the edit distance between the character and any character in the remaining high-frequency characters, mainly based on the glyph structure and / or pinyin sequence. It should be noted that when generating the pinyin sequence, it is not necessary to mark the tone.

[0074] After calculating the edit distance, the edit distance can be compared with a first preset edit distance threshold. If the edit distance is less than or equal to the first preset edit distance threshold, it indicates that the two characters in the pair of easily confused characters are not easily confused, and the pair of easily confused characters is deleted. Among them, the edit distance is a quantitative measurement of the difference degree between two characters, and the measurement method is to see how many times of processing are required at least to change one character into another character. After all pairs of easily confused characters have been processed as described above, a second set of easily confused characters can be constructed based on the deleted pairs of easily confused characters.

[0075] Here, mainly the first edit distance between a character and any easily confused character in terms of the glyph structure and / or the second edit distance between them in terms of the pronunciation are calculated. Therefore, the first edit distance can be compared with the first preset edit distance threshold. If the first edit distance is less than or equal to the first preset edit distance threshold, the easily confused character or the pair of easily confused characters is deleted; and / or, the second edit distance is compared with the first preset edit distance threshold. If the second edit distance is less than or equal to the first preset edit distance threshold, the easily confused character or the pair of easily confused characters is deleted. Or, a comprehensive edit distance can be calculated based on the first edit distance and the second edit distance, and the comprehensive edit distance is compared with the first preset edit distance threshold. If the comprehensive edit distance is less than or equal to the first preset edit distance threshold, the pair of easily confused characters is deleted.

[0076] More preferably, if any character in the high-frequency characters is a polyphonic character, then multiple pinyin sequences will be generated for this polyphonic character. For each pinyin sequence, the edit distance between each character in the high-frequency characters and any other character will be calculated according to this pinyin sequence, and finally, the pairs of easily confused characters corresponding to each pinyin sequence of this polyphonic character will be generated. For example, "大" is a polyphonic character, and the pinyins without tone marks are "da" and "dai". Finally, the characters that are homophonic and near-homophonic to "da" and "dai" are obtained as the near-homophonic characters of "大", that is, multiple pronunciations of the polyphonic character are within the range of selecting near-homophonic characters.

[0077] In addition, the situation of fuzzy sounds also needs to be considered. For example, some dialect speakers cannot correctly distinguish similar syllables such as ("z", "zh"), ("l", "n"), ("r", "l") during the process of learning and using Mandarin. To overcome the above problems, if any character whose frequency ranking is before the preset ratio contains a preset syllable, for example, any syllable in the syllables such as "z", "zh", "l", "n", "r", "l", etc., then the characters in the pinyin sequence that contain the similar syllable paired with the preset syllable are obtained as the easily confused characters, and a second set of easily confused characters is formed.

[0078] Step S204, merge the first set of easily confused characters in the first character confusion set after screening and the second set of easily confused characters to form a second character confusion set.

[0079] There may be a situation where the characters of the first easily confused character pair in the first character confusion set that is screened out are the same as the characters in the second easily confused character pair that is formed, but the easily confused character parts are the same or completely different, and the first character confusion set after screening out contains a first easily confused character pair that is different from the second easily confused character pair. Therefore, the first easily confused character pair in the first character confusion set after screening out can be merged with the second easily confused character pair, wherein the easily confused characters of the same characters are merged, and the different characters are retained respectively, thereby obtaining the merged easily confused character pairs as the second character confusion set.

[0080] Step S205, performing word segmentation processing on the two second corpus data in each second corpus data pair, and determining whether the two words corresponding to each word segmentation position are consistent, and if inconsistent, determining that the two words are an easily confused word pair.

[0081] The second corpus data set includes: second corpus data pairs, two second corpus data in each second corpus data pair are segmented and divided into multiple words. It is determined whether the two words corresponding to each segmentation position are consistent, and if not, the two words are determined to be easily confused word pairs. The second corpus data pair is a data pair that depends on the input method to construct and use easily confused words.

[0082] Step S206, performing word segmentation processing on each first corpus data set in the first corpus data set, and counting the word length of each word.

[0083] Perform word segmentation processing on each first corpus data in the first corpus data set. For example, the word segmentation processing can be performed based on a string matching method or a rule-based method. Through the word segmentation processing, the first corpus data is divided into multiple words. The lengths of the multiple words may not be the same. Usually, the lengths of two words in an easily confused word pair are consistent. Therefore, it is necessary to count the word lengths of each word after word segmentation.

[0084] Step S207, forming a word pair with two words of the same length, determining whether the two characters corresponding to each character position in the word pair have similar glyph structures and / or similar pronunciations; if the two characters corresponding to each character position have similar glyph structures and / or similar pronunciations, the word pair is determined to be an easily confused word pair, and a word confusion set is constructed based on the easily confused word pairs.

[0085] Under normal circumstances, the lengths of the two words in an easily confused word pair are the same. Therefore, when constructing a word confusion set, first form word pairs by taking any two words with the same length. Then, determine whether the two characters corresponding to each character position in the word pair are similar in glyph structure and / or pronunciation. For example, the edit distance between the glyph structures of the two characters corresponding to each character position and / or the edit distance between the pronunciations can be calculated, and the edit distance is used to determine whether the two characters are similar in glyph structure and / or pronunciation. If the glyph structures and / or pronunciations of all corresponding position characters are similar, the word pair is determined to be an easily confused word pair, and a word confusion set is constructed based on the easily confused word pairs.

[0086] When judging the similarity of pronunciation, it can be determined whether the pronunciations of the two characters corresponding to each character position are similar syllables. If so, it is determined that the pronunciations of the two characters corresponding to each character position are similar. In this application, a list of similar syllables can also be preset, and the list of similar syllables is queried according to the pinyin sequences of the two characters corresponding to each character position to determine whether the pronunciations of the two characters corresponding to each character position are similar syllables.

[0087] For example, the lengths of the two words "like" and "love me, love my dog" are different, and no similarity judgment is required; "捉摸" and "琢磨" are easily confused words with each other, and the corresponding characters "捉" and "琢", "摸" and "磨" are all near-homophonic characters; "必须" is not an easily confused word of "必要", and "须" and "要" are neither near-homophonic nor similar in glyph.

[0088] Step S208, randomly sample from the original training text in the training text set to determine multiple characters to be masked and at least one word to be masked.

[0089] For example, a masking ratio can be preset. For example, the masking ratio of characters is set to 15%, and the number of words to be masked is 1. According to the length of the original training text and the masking ratio, multiple characters to be masked and at least one word to be masked are randomly selected from the original training text. Among them, the number of words to be masked can be set according to actual needs. For example, it can be set to 1, and of course it can also be multiple.

[0090] Step S209, for any word to be masked, randomly select an easily confused word corresponding to the word to be masked from the word confusion set, and replace the word to be masked with the selected easily confused word.

[0091] After determining at least one to-be-masked word that needs to be masked, for any to-be-masked word, a word that is easily confused with the to-be-masked word can be randomly selected from the word confusion set to replace it, that is, the to-be-masked word is replaced with the selected easily confused word. By using easily confused words for replacement, it can truly simulate the situation in actual input editing, where the misselected word and the target word are similar in pronunciation or appearance, and both are correct words, thereby making the trained spelling check model more accurate.

[0092] Step S210, randomly selecting characters to be masked in a first replacement ratio, for any character to be masked, randomly selecting an easily confused character corresponding to the character to be masked from the second character confusion set, and replacing the character to be masked with the selected easily confused character.

[0093] The reason for errors in most real scenarios is that the two characters are similar in pronunciation or shape. In order to simulate real errors, easily confused characters of the character are randomly selected from the second character confusion set to replace them to simulate real errors.

[0094] Here, characters to be masked of the first replacement ratio are randomly selected to replace easily confused characters, that is, characters to be masked of the first replacement ratio are randomly selected from a plurality of characters to be masked, and for any selected character to be masked, easily confused characters corresponding to the characters to be masked are randomly selected from the second character confusion set, and the characters to be masked are replaced with the selected easily confused characters. The first replacement ratio is set according to actual needs, for example, set to 60%.

[0095] Step S211, randomly selecting characters to be masked in a second replacement ratio, and replacing the characters to be masked in the second replacement ratio with preset characters.

[0096] In order to enable the model to fully learn contextual features and predict the word in the vacant position based on the contextual information, the characters to be masked with the second replacement ratio are randomly selected here to replace the preset characters. That is to say, the characters to be masked with the second replacement ratio are randomly selected from multiple characters to be masked, and the characters to be masked with the second replacement ratio are replaced with preset characters, for example, replaced with "[MASK]", wherein the second replacement ratio is set according to actual needs, for example, set to 10%.

[0097] Step S212: randomly select characters to be masked in the third replacement ratio, and replace the characters to be masked in the third replacement ratio with any characters in the character table to obtain enhanced training text.

[0098] To enable the model to learn more error cases that are irrelevant to pronunciation and shape similarity and improve the model's generalization ability, the masked characters with the third replacement ratio are randomly selected here and replaced with any character, that is, the masked characters with the third replacement ratio are randomly selected from multiple masked characters, and the masked characters with the third replacement ratio are replaced with any character in the character table, where the third replacement ratio is set according to actual needs. For example, it is set to 20%.

[0099] In the embodiment of the present application, the masked characters with the fourth replacement ratio are also randomly selected to remain unchanged to prevent over-correction problems in the CSC task. The sum of the first replacement ratio, the second replacement ratio, the third replacement ratio, and the fourth replacement ratio is 100%.

[0100] In traditional masked prediction tasks, the replacement method of the "[MASK]" token has the highest proportion. However, if the proportion of the "[MASK]" token is too large, the model will be more inclined to correct this token in the CSC task. Therefore, the proportion of this replacement method is reduced to achieve the purpose of narrowing the gap between the MLM task and the CSC task. Most Chinese spelling mistakes are character misuses caused by similar pronunciations or similar characters with similar pronunciations, and Chinese sentences are usually segmented into words of two characters or more. Therefore, a character-word confusion set is used as the replacement method to ensure that the constructed pseudo-data is more in line with the errors in actual applications. This masking strategy is used for all data to construct training data. Specifically, 15% of the characters in the input sequence are selected and replaced by combining multiple methods such as the second character confusion set, the word confusion set, and randomness. Given a sentence "Large language models play an important role in the field of natural language processing. They can learn the structure, grammar, semantics, and context relevance of language and perform language-related tasks.", the replaced characters are marked with underlines. The specific hybrid masking strategy is as follows:

[0101] a) 60% of the characters are replaced with the characters in the second character confusion set. In most real scenarios, the reason for the error is similar pronunciation or similar shape. Randomly select the easily confused characters of this character for replacement to simulate real errors. For example, select 5 characters for replacement, replace "型" with "形", "言" with "信", etc. The replaced sentence is "大 shape Language model in natural language letter Processing field Hui plays an important role, and they can learn the structure, language hair , semantics, and context relevance, and perform shape Language-related tasks."

[0102] 您的问题我无法回答。​​​​​​​​​​​​​​​​​​​​​​​​​After obtaining the enhanced training text, model training is performed based on the original training text and the enhanced training text to generate a spelling check model. Specifically, the enhanced training text is used as a task for fine-tuning the BERT model, and the entire sentence of the enhanced training text is input for mask prediction to obtain the predicted text. The prediction results of the mask positions in the predicted text are compared to see whether they are consistent with the data of the corresponding positions in the original training text, and the accuracy of the mask positions is counted. When the accuracy of the mask positions does not meet the preset training end conditions, continue to perform mask prediction on the enhanced training text; when the accuracy of the mask positions meets the preset training end conditions, stop training the BERT model to obtain the spelling check model.

[0109] Step S214, obtaining the text to be spell-checked, inputting the text to be spell-checked into the spell-checking model for spell-checking, and outputting the text after spell-checking.

[0110] When there is a need for spelling check, the user can upload the text to be spell-checked, obtain the text to be spell-checked uploaded by the user, input the obtained text to be spell-checked into the spell-checking model for spell-checking, and output the text after spell-checking.

[0111] For example, the spelling check model first uses the instantiated image representations of the three fonts of Fangsong, Xingkai, and Lishu as glyph features, and the pinyin sequence and tone representation as pinyin features. The above glyph features and pinyin features are used to simulate the unique semantic attributes of Chinese characters, similar to the Token Embedding in BERT, with glyphs and pinyin generating Embeddings separately. Secondly, the glyph embeddings, pinyin embeddings, and word embeddings obtained by BERT are concatenated together and input into a multi-layer bidirectional Transformer. This structure obtains the semantic information of the text by learning the information on the left and right sides of the words. Then, the word at each position is predicted based on the semantic information, and the predicted text is the text after spelling check. In addition, the prediction probability corresponding to each word can be output. If the prediction probability is less than the preset probability threshold, the word in the text to be spell-checked is retained without error correction. If there is a non-Chinese "*" in the text to be checked, this type of "*" may be corrected to "—", and the non-Chinese symbols in the text to be spell-checked are retained without error correction. Idioms or familiar expressions in the text to be spell-checked are not corrected to prevent the model from over-correcting, thereby improving the accuracy of the detection results. Finally, the processed text is fed back to the user.

[0112] In summary, according to the method provided in this embodiment, the spelling check data is enhanced by using multiple methods such as the second character confusion set, word confusion set, random replacement, etc., and the enhanced training samples required for training are automatically generated. The enhanced training samples are closer to the text with real spelling errors, which overcomes the problem of insufficient Chinese spelling check data, reduces the cost of manual annotation, lays the foundation for the spelling check model, and also greatly improves the accuracy of the spelling check model.

[0113] Figure 3 FIG. 1 is a schematic diagram showing the structure of a device for generating a spelling check model according to an embodiment of the present application. Figure 3 As shown, the device comprises:

[0114] A character confusion set construction module 301, adapted to construct a second character confusion set according to the first public character confusion set and the first corpus data set;

[0115] A word confusion set construction module 302, adapted to construct a word confusion set according to the second corpus data set constructed by the input method and the first corpus data set;

[0116] A determination module 303, adapted to randomly sample and determine a plurality of characters to be masked and at least one word to be masked from the original training text in the training text set;

[0117] The mask processing module 304 is adapted to perform mask processing on a plurality of characters to be masked using the second character confusion set and perform mask processing on at least one word to be masked using the word confusion set based on a preset mixed masking strategy, so as to obtain an enhanced training text;

[0118] The generation module 305 is adapted to perform model training based on the original training text and the enhanced training text to generate a spelling check model.

[0119] Optionally, the word confusion set construction module is further adapted to: S1, perform word segmentation processing on each first corpus data in the first corpus data set, count the occurrence frequency of each word, and retain words whose occurrence frequency ranking is within a preset ratio;

[0120] S2, filtering the first character confusion set according to the characters reserved in S1, filtering out characters that do not appear in the characters reserved in S1 in the first character confusion set, and determining whether any first easily confused character pair in the first character confusion set after filtering out has a similar character shape structure and / or a similar character pronunciation; if both the character shape structure and the character pronunciation are not similar, filtering out the first easily confused character pair;

[0121] S3, generating a glyph structure and / or a pinyin sequence corresponding to each character in the characters reserved in S1, calculating an edit distance between each character in the characters reserved in S1 and any other character according to the glyph structure and / or the pinyin sequence, deleting easily confused character pairs whose edit distance is less than or equal to a first preset edit distance threshold, and forming second easily confused character pairs;

[0122] S4, merging the first easily confused character pair and the second easily confused character pair in the first character confusion set after being screened out to form a second character confusion set.

[0123] Optionally, the character confusion set construction module is further adapted to: if any of the characters reserved by S1 is a polyphonetic character, generate multiple pinyin sequences corresponding to the polyphonetic characters, and calculate the edit distance between each character in the characters reserved by S1 and any other character according to the glyph structure and / or any pinyin sequence.

[0124] Optionally, the character confusion set construction module is further adapted to: if any character among the characters reserved in S1 contains a preset syllable, then obtaining characters in the pinyin sequence containing similar syllables paired with the preset syllable as easily confused characters to form a second easily confused character pair.

[0125] Optionally, the second corpus data set includes: a second corpus data pair;

[0126] The word confusion set construction module is further adapted to: perform word segmentation processing on two second corpus data in each second corpus data pair, determine whether the two words corresponding to each word segmentation position are consistent, and if not, determine that the two words are an easily confused word pair;

[0127] Performing word segmentation processing on each first corpus data set in the first corpus data set, and counting the word length of each word;

[0128] Two words with the same length are combined into a word pair, and it is determined whether the two characters corresponding to each character position in the word pair have similar glyph structures and / or similar pronunciations. If the two characters corresponding to each character position have similar glyph structures and / or similar pronunciations, the word pair is determined to be an easily confused word pair, and a word confusion set is constructed based on the easily confused word pairs.

[0129] Optionally, the mask processing module is further adapted to: for any word to be masked, randomly select a confusing word corresponding to the word to be masked from the word confusion set, and replace the word to be masked with the selected confusing word;

[0130] Randomly select characters to be masked of a first replacement ratio, and for any character to be masked, randomly select an easily confused character corresponding to the character to be masked from the second character confusion set, and replace the character to be masked with the selected easily confused character;

[0131] Randomly select characters to be masked in the second replacement ratio, and replace the characters to be masked in the second replacement ratio with preset characters;

[0132] The characters to be masked in the third replacement ratio are randomly selected, and the characters to be masked in the third replacement ratio are replaced with any characters in the vocabulary.

[0133] Optionally, the device further comprises: an acquisition module, adapted to acquire the text to be spell-checked;

[0134] The spell checking module is suitable for inputting the text to be spell checked into the spell checking model for spell checking, and outputting the text after spell checking.

[0135] In summary, according to the device provided in this embodiment, spelling check data is enhanced by using multiple methods such as second character confusion set, word confusion set, random replacement, etc., and enhanced training samples required for training are automatically generated. The enhanced training samples are closer to the text with real spelling errors, which overcomes the problem of insufficient Chinese spelling check data, reduces the cost of manual annotation, lays the foundation for the spelling check model, and also greatly improves the accuracy of the spelling check model.

[0136] An embodiment of the present application provides a non-volatile computer storage medium, which stores at least one executable instruction or computer program, which can enable a processor to perform operations corresponding to the spelling check model generation method in any of the above method embodiments.

[0137] An embodiment of the present application provides a computer program product, which includes at least one executable instruction or computer program, and the executable instruction or computer program can enable a processor to perform operations corresponding to the spelling check model generation method in any of the above method embodiments.

[0138] Figure 4 A schematic diagram of the structure of an embodiment of a computing device of the present application is shown, and the specific embodiment of the present application does not limit the specific implementation of the computing device.

[0139] like Figure 4 As shown, the computing device may include: a processor (processor) 402 , a communications interface (Communications Interface) 404 , a memory (memory) 406 , and a communication bus 408 .

[0140] The processor 402, the communication interface 404, and the memory 406 communicate with each other via the communication bus 408. The communication interface 404 is used to communicate with other devices such as a client or other server network elements. The processor 402 is used to execute the program 410, which can specifically execute the relevant steps in the above-mentioned embodiment of the spelling check model generation method for a computing device.

[0141] Specifically, the program 410 may include program codes, which include computer operation instructions.

[0142] The processor 402 may be a central processing unit (CPU), or an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present application. The one or more processors included in the computing device may be processors of the same type, such as one or more CPUs; or processors of different types, such as one or more CPUs and one or more ASICs.

[0143] The memory 406 is used to store the program 410. The memory 406 may include a high-speed RAM memory, and may also include a non-volatile memory (non-volatile memory), such as at least one disk memory.

[0144] The program 410 can be specifically used to enable the processor 402 to execute the spelling check model generation method in any of the above method embodiments. The specific implementation of each step in the program 410 can refer to the corresponding descriptions in the corresponding steps and units in the above spelling check model generation embodiment, which will not be repeated here. Those skilled in the art can clearly understand that for the convenience and simplicity of description, the specific working process of the above-described devices and modules can refer to the corresponding process description in the above-mentioned method embodiment, which will not be repeated here.

[0145] The algorithm or display provided here are not inherently related to any specific computer, virtual system or other equipment. Various general systems can also be used together with the teaching based on this. According to the above description, it is obvious to construct the structure required for this type of system. In addition, the present application embodiment is not directed to any specific programming language yet. It should be understood that various programming languages ​​can be utilized to realize the content of the present application described here, and the above description of specific languages ​​is to disclose the best mode of implementation of the present application.

[0146] In the description provided herein, a large number of specific details are described. However, it is understood that the embodiments of the present application can be practiced without these specific details. In some instances, well-known methods, structures and techniques are not shown in detail so as not to obscure the understanding of this description.

[0147] Similarly, it should be understood that in order to streamline the present application and help understand one or more of the various inventive aspects, in the above description of the exemplary embodiments of the present application, the various features of the embodiments of the present application are sometimes grouped together into a single embodiment, figure, or description thereof. However, the disclosed method should not be interpreted as reflecting the following intention: the claimed application requires more features than the features clearly stated in each claim. More specifically, as reflected in the claims below, the inventive aspects are less than all the features of the single embodiment disclosed above. Therefore, the claims following the specific embodiment are hereby expressly incorporated into the specific embodiment, wherein each claim itself serves as a separate embodiment of the present application.

[0148] Those skilled in the art will appreciate that the modules in the devices in the embodiments may be adaptively changed and arranged in one or more devices different from the embodiments. The modules or units or components in the embodiments may be combined into one module or unit or component, and in addition they may be divided into a plurality of submodules or subunits or subcomponents. Except that at least some of such features and / or processes or units are mutually exclusive, all features disclosed in this specification (including the accompanying claims, abstracts and drawings) and all processes or units of any method or device disclosed in this manner may be combined in any combination. Unless otherwise expressly stated, each feature disclosed in this specification (including the accompanying claims, abstracts and drawings) may be replaced by an alternative feature providing the same, equivalent or similar purpose.

[0149] In addition, those skilled in the art will appreciate that, although some embodiments herein include certain features included in other embodiments but not other features, the combination of features of different embodiments is meant to be within the scope of the present application and form different embodiments. For example, in the claims below, any one of the claimed embodiments may be used in any combination.

[0150] The various component embodiments of the present application can be implemented in hardware, or in software modules running on one or more processors, or in a combination thereof. It should be understood by those skilled in the art that a microprocessor or digital signal processor (DSP) can be used in practice to implement some or all functions of some or all components according to the embodiments of the present application. The application can also be implemented as a device or apparatus program (e.g., computer program and computer program product) for executing a part or all of the methods described herein. Such a program implementing the present application can be stored on a computer-readable medium, or can have the form of one or more signals. Such a signal can be downloaded from an Internet website, or provided on a carrier signal, or provided in any other form.

[0151] It should be noted that the above embodiments illustrate the present application rather than limit the present application, and that those skilled in the art may design alternative embodiments without departing from the scope of the appended claims. In the claims, any reference symbol between brackets shall not be constructed as a limitation on the claims. The word "comprising" does not exclude the presence of elements or steps not listed in the claims. The word "one" or "an" preceding an element does not exclude the presence of multiple such elements. The present application may be implemented by means of hardware including several different elements and by means of a suitably programmed computer. In a unit claim that lists several devices, several of these devices may be embodied by the same hardware item. The use of the words first, second, and third, etc. does not indicate any order. These words may be interpreted as names. The steps in the above embodiments, unless otherwise specified, should not be understood as limitations on the order of execution.

Claims

1. A method for generating a spelling check model, comprising: Constructing a second character confusion set based on the first character confusion set and the first corpus data set; Constructing a word confusion set according to the second corpus data set constructed by the input method and the first corpus data set; Randomly sampling from the original training text in the training text set to determine a plurality of characters to be masked and at least one word to be masked; Based on a preset mixed masking strategy, the second character confusion set is used to perform masking processing on a plurality of characters to be masked, and the word confusion set is used to perform masking processing on the at least one word to be masked, so as to obtain an enhanced training text; Perform model training based on the original training text and enhanced training text to generate a spelling check model; Wherein, constructing the second character confusion set according to the disclosed first character confusion set and the first corpus data set further comprises: S1, performing word segmentation processing on each first corpus data in the first corpus data set, counting the occurrence frequency of each word, and retaining words whose occurrence frequency ranking is within a preset ratio; S2, filtering the first character confusion set according to the characters reserved in S1, filtering out characters that do not appear in the characters reserved in S1 in the first character confusion set, and determining whether any first easily confused character pair in the first character confusion set after filtering out has a similar character shape structure and / or a similar character pronunciation; if both the character shape structure and the character pronunciation are not similar, filtering out the first easily confused character pair; S3, generating a glyph structure and / or pinyin sequence corresponding to each character in the characters reserved by S1, calculating the edit distance between each character in the characters reserved by S1 and any other character according to the glyph structure and / or pinyin sequence, deleting easily confused character pairs whose edit distance is less than or equal to a first preset edit distance threshold, and forming a second easily confused character pair, wherein, if any character in the characters reserved by S1 is a polyphonic character, generating a plurality of pinyin sequences corresponding to the polyphonic character, calculating the edit distance between each character in the characters reserved by S1 and any other character according to the glyph structure and / or any pinyin sequence, and if any character in the characters reserved by S1 contains a preset syllable, obtaining a character in the pinyin sequence containing a similar syllable paired with the preset syllable as an easily confused character, and forming a second easily confused character pair; S4, merging the first easily confused character pair and the second easily confused character pair in the first character confusion set after being screened out as a second character confusion set; The second corpus data set includes: a second corpus data pair; The second corpus data set constructed according to the input method and the first corpus data set construct a word confusion set further comprising: Performing word segmentation processing on two second corpus data in each second corpus data pair, determining whether two words corresponding to each word segmentation position are consistent, and if inconsistent, determining that the two words are an easily confused word pair; Performing word segmentation processing on each first corpus data set in the first corpus data set, and counting the word length of each word; Two words with the same word length are combined into a word pair, and it is determined whether the two characters corresponding to each character position in the word pair have similar glyph structures and / or similar pronunciations. If the two characters corresponding to each character position have similar glyph structures and / or similar pronunciations, the word pair is determined to be an easily confused word pair, and a word confusion set is constructed according to the easily confused word pairs; The step of masking a plurality of characters to be masked by using the second character confusion set and masking the at least one word to be masked by using the word confusion set based on the preset mixed masking strategy to obtain the enhanced training text further comprises: For any word to be masked, randomly select an easily confused word corresponding to the word to be masked from the word confusion set, and replace the word to be masked with the selected easily confused word; Randomly selecting characters to be masked in a first replacement ratio, for any character to be masked, randomly selecting an easily confused character corresponding to the character to be masked from the second character confusion set, and replacing the character to be masked with the selected easily confused character; Randomly selecting characters to be masked in a second replacement ratio, and replacing the characters to be masked in the second replacement ratio with preset characters; The characters to be masked in the third replacement ratio are randomly selected, and the characters to be masked in the third replacement ratio are replaced with any characters in the vocabulary.

2. The method according to claim 1, wherein: The method further comprises: obtaining a text to be spell-checked; The text to be spell-checked is input into the spell-checking model for spell-checking, and the text after spell-checking is output.

3. A spelling check model generating device, comprising: A character confusion set construction module, adapted to construct a second character confusion set according to the first public character confusion set and the first corpus data set; A word confusion set construction module, adapted to construct a word confusion set according to the second corpus data set constructed by the input method and the first corpus data set; A determination module, adapted to randomly sample and determine a plurality of characters to be masked and at least one word to be masked from the original training text in the training text set; A mask processing module, adapted to perform mask processing on a plurality of characters to be masked using the second character confusion set and perform mask processing on the at least one word to be masked using the word confusion set based on a preset mixed masking strategy, to obtain an enhanced training text; A generation module, suitable for training a model based on the original training text and the enhanced training text to generate a spelling check model; The character confusion set construction module is further adapted to: S1, perform character segmentation processing on each first corpus data in the first corpus data set, count the occurrence frequency of each character, and retain the characters whose occurrence frequency ranking is within a preset ratio; S2, filtering the first character confusion set according to the characters reserved in S1, filtering out characters that do not appear in the characters reserved in S1 in the first character confusion set, and determining whether any first easily confused character pair in the first character confusion set after filtering out has a similar character shape structure and / or a similar character pronunciation; if both the character shape structure and the character pronunciation are not similar, filtering out the first easily confused character pair; S3, generating a glyph structure and / or pinyin sequence corresponding to each character in the characters reserved by S1, calculating the edit distance between each character in the characters reserved by S1 and any other character according to the glyph structure and / or pinyin sequence, deleting easily confused character pairs whose edit distance is less than or equal to a first preset edit distance threshold, and forming a second easily confused character pair, wherein, if any character in the characters reserved by S1 is a polyphonic character, generating a plurality of pinyin sequences corresponding to the polyphonic character, calculating the edit distance between each character in the characters reserved by S1 and any other character according to the glyph structure and / or any pinyin sequence, and if any character in the characters reserved by S1 contains a preset syllable, obtaining a character in the pinyin sequence containing a similar syllable paired with the preset syllable as an easily confused character, and forming a second easily confused character pair; S4, merging the first easily confused character pair and the second easily confused character pair in the first character confusion set after being screened out as a second character confusion set; The second corpus data set includes: a second corpus data pair; The word confusion set construction module is further adapted to: perform word segmentation processing on two second corpus data in each second corpus data pair, determine whether the two words corresponding to each word segmentation position are consistent, and if not, determine that the two words are an easily confused word pair; Performing word segmentation processing on each first corpus data set in the first corpus data set, and counting the word length of each word; Two words with the same word length are combined into a word pair, and it is determined whether the two characters corresponding to each character position in the word pair have similar glyph structures and / or similar pronunciations. If the two characters corresponding to each character position have similar glyph structures and / or similar pronunciations, the word pair is determined to be an easily confused word pair, and a word confusion set is constructed according to the easily confused word pairs; The mask processing module is further adapted to: for any word to be masked, randomly select an easily confused word corresponding to the word to be masked from the word confusion set, and replace the word to be masked with the selected easily confused word; Randomly selecting characters to be masked in a first replacement ratio, for any character to be masked, randomly selecting an easily confused character corresponding to the character to be masked from the second character confusion set, and replacing the character to be masked with the selected easily confused character; Randomly selecting characters to be masked in a second replacement ratio, and replacing the characters to be masked in the second replacement ratio with preset characters; The characters to be masked in the third replacement ratio are randomly selected, and the characters to be masked in the third replacement ratio are replaced with any characters in the vocabulary.

4. A computing device comprising: A processor, a memory, a communication interface and a communication bus, wherein the processor, the memory and the communication interface communicate with each other via the communication bus; The memory is used to store at least one executable instruction, and the executable instruction enables the processor to perform operations corresponding to the spelling check model generation method as claimed in claim 1 or 2.

5. A computer storage medium, wherein at least one executable instruction is stored in the storage medium, and the executable instruction enables a processor to perform operations corresponding to the spelling check model generation method according to claim 1 or 2.

6. A computer program product, comprising at least one executable instruction, wherein the executable instruction enables a processor to execute operations corresponding to the method for generating a spelling check model according to claim 1 or 2.