Training of text correction models and text correction methods and devices

Abstract

This application provides a text correction model training method and device, relating to the field of artificial intelligence technology. The text correction model training method includes: acquiring training data, which includes random erroneous text samples, near-phonetic erroneous text samples, and near-shape erroneous text samples; determining the input vector corresponding to each character in the training data, which includes a character representation vector, a positional representation vector, a pinyin representation vector, and a character shape representation vector; and training a pre-trained language model based on the input vector to obtain a text correction model. The text correction method includes: inputting the input vector corresponding to each character in the text to be corrected into the text correction model to obtain a text correction prediction result vector; and decoding the text correction prediction result vector to obtain the corrected target text. This application can not only reduce the cost of text correction but also improve the efficiency and accuracy of text correction.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to a method and device for training a text correction model and for text correction. Background Art

[0002] Text correction falls under the category of natural language processing and is used in many intelligent scenarios. Its main purpose is to detect errors in the input of the original text and correct them using natural language processing techniques.

[0003] At present, text correction is usually achieved by defining and continuously maintaining a confusion dictionary corpus, combined with business rules. This approach is costly because it requires summarizing a large number of business rules. In addition, the efficiency and accuracy of text correction are relatively low. Summary of the Invention

[0004] This application provides a method and device for training a text correction model and for text correction, which can not only reduce the cost of text correction, but also improve the efficiency and accuracy of text correction.

[0005] Firstly, this application provides a method for training a text correction model, the method comprising:

[0006] Acquire training data, which includes random erroneous text samples, near-sound erroneous text samples, and near-shape erroneous text samples;

[0007] Determine the input vector corresponding to each character in the training data, wherein the input vector includes a character representation vector, a position representation vector, a pinyin representation vector, and a character shape representation vector;

[0008] The pre-trained language model is trained based on the input vector to obtain a text correction model.

[0009] In some implementations, prior to acquiring the training data, the process further includes:

[0010] The random error text samples, the near-sound error text samples, and the near-shape error text samples are constructed based on the original text; wherein, the random error text samples include random missing character error text samples, random extra character error text samples, random spelling error text samples, and random word order error text samples.

[0011] In some embodiments, the near-pronunciation error text sample includes near-pronunciation error text and variable-length near-pronunciation error text, and constructing the near-pronunciation error text sample includes:

[0012] Based on the pinyin corresponding to the original text and the preset confusion rules, the near-sound error text corresponding to the original text is generated, and based on the modified error segments of the random error text samples and the confusion rules, the variable-length near-sound error text corresponding to the original text is generated.

[0013] The near-shape error text samples include near-shape error text and variable-length near-shape error text samples. Constructing the near-shape error text samples includes:

[0014] Based on the original text and a preset dictionary of similar-looking characters, the similar-looking error text corresponding to the original text is generated, and based on the modified error fragments of the random error text sample and the dictionary of similar-looking characters, the variable-length similar-looking error text sample corresponding to the original text is generated.

[0015] In some implementations, determining the input vector corresponding to each character in the training data includes:

[0016] Find the character representation vector and the position representation vector corresponding to each character by looking up the embedding table;

[0017] The pinyin corresponding to each character is determined, and the pinyin corresponding to each character is input into a gated recurrent unit (GRU) network to generate the pinyin representation vector;

[0018] The four-corner code, radical information, and structural information corresponding to each character are determined. The glyph sequence formed by concatenating the four-corner code, radical information, and structural information corresponding to each character is input into the GRU network to generate the glyph representation vector.

[0019] Secondly, this application provides a text correction method, which includes:

[0020] Obtain the input vector corresponding to each character in the text to be corrected. The input vector includes a character representation vector, a position representation vector, a pinyin representation vector, and a character shape representation vector.

[0021] Based on the input vector corresponding to each character and the text correction model, a text correction prediction result vector is obtained; the text correction model is obtained according to the training method of the text correction model described in the first aspect.

[0022] The text correction prediction result vector is decoded to obtain the target text after correction of the text to be corrected.

[0023] In some implementations, decoding the text correction prediction result vector to obtain the corrected target text of the text to be corrected includes:

[0024] Determine the index corresponding to the maximum value in the last dimension of the text error correction prediction result vector, and determine the target text based on the index and a preset vocabulary.

[0025] Thirdly, this application provides a training device for a text correction model, the device comprising:

[0026] An initialization module is used to acquire training data, which includes random erroneous text samples, near-sound erroneous text samples, and near-shape erroneous text samples.

[0027] The feature extraction module is used to determine the input vector corresponding to each character in the training data. The input vector includes a character representation vector, a position representation vector, a pinyin representation vector, and a character shape representation vector.

[0028] The training module is used to train the pre-trained language model based on the input vector to obtain a text correction model.

[0029] Fourthly, this application provides a text correction device, which includes:

[0030] The feature extraction module is used to obtain the input vector corresponding to each character in the text to be corrected. The input vector includes a character representation vector, a position representation vector, a pinyin representation vector, and a character shape representation vector.

[0031] The text correction module is used to obtain a text correction prediction result vector based on the input vector corresponding to each character and the text correction model; the text correction model is obtained by the training device of the text correction model according to claim 7.

[0032] The error correction output module is used to decode the text error correction prediction result vector to obtain the target text after the text to be corrected.

[0033] Fifthly, this application provides an electronic device, including: at least one processor and a memory;

[0034] The memory stores computer-executable instructions;

[0035] The at least one processor is configured to execute computer execution instructions stored in the memory to implement the training method of the text correction model as provided in the first aspect, or to implement the text correction method as provided in the second aspect.

[0036] Sixthly, this application provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a computer, implement the training method for the text correction model provided in the first aspect, or implement the text correction method provided in the second aspect.

[0037] The text correction model training and text correction method and device provided in this application are based on a pre-trained language model and can correct various types of text errors, such as extra words, missing words, spelling errors, and grammatical errors. It can simultaneously achieve variable-length text correction and constant-length text correction, which can not only reduce the cost of text correction, but also improve the efficiency and accuracy of text correction. Attached Figure Description

[0038] Figure 1 This is a flowchart illustrating the steps of a text correction model training method provided in this application embodiment;

[0039] Figure 2 This is a flowchart illustrating the steps of a text correction method provided in an embodiment of this application.

[0040] Figure 3 This is a schematic diagram illustrating the application of a text correction model provided in the embodiments of this application;

[0041] Figure 4 This is a schematic diagram of the program modules of a training device for a text error correction model provided in the embodiments of this application;

[0042] Figure 5 This is a schematic diagram of the program modules of a text correction device provided in the embodiments of this application;

[0043] Figure 6 This is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of this application. DETAILED DESCRIPTION

[0044] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application. Furthermore, although the application content is described with reference to one or several exemplary examples, it should be understood that each aspect of these application content can also constitute a complete implementation method on its own.

[0045] It should be noted that the brief description of terms in this application is only for the convenience of understanding the following described embodiments, rather than intending to limit the embodiments of this application. Unless otherwise specified, these terms should be understood in their ordinary and common meanings.

[0046] In this application, terms such as "first", "second", "third", etc. in the specification, claims and the above-mentioned drawings are used to distinguish similar or like objects or entities, and do not necessarily mean to limit a specific order or sequence, unless otherwise noted. It should be understood that such terms can be interchanged under appropriate circumstances, for example, it is possible to implement in an order other than those given in the illustration or description of the embodiments of this application.

[0047] In addition, the terms "comprising" and "having" and any variations thereof are intended to cover but not exclude inclusion. For example, a product or device comprising a series of components does not necessarily have to be limited to those components clearly listed, but may include other components not clearly listed or inherent to these products or devices.

[0048] The term "module" used in the embodiments of this application refers to any known or later developed hardware, software, firmware, artificial intelligence, fuzzy logic or a combination of hardware or / and software code that can perform functions related to this element.

[0049] Text error correction belongs to the category of natural language processing and has been widely used in many intelligent scenarios. For example, correcting the incorrect text input by users in a search engine can improve the retrieval accuracy; in speech recognition, due to unclear pronunciation of the speaker, there may be homophone errors, such as recognizing "The Cowherd and the Weaver Girl" as "The Vagrant Weaver Girl", or due to the speaker's fast speech rate, there may be missing-word errors, such as recognizing "If Love Has天意" as "If Love天意", etc. After text error correction in speech recognition, the speech recognition accuracy will be further improved; in the text editing and proofreading industry, automated error correction prompts and error correction suggestions can greatly reduce the workload of manual proofreading.

[0050] Referring to Table 1, from the perspective of the types of error correction, there are the following error types:

[0051] Table 1

[0052] type Error Example Corrected text Phonetic error Please revise me. Please help me revise this. Similar error sorghum sorghum Multiple word errors I went swimming today. I went swimming today. Missing character error I went swimming today. I went swimming today. Word order (grammar) error It's hard to imagine Unimaginable

[0053] In this application, the error types can be mainly divided into: extra-word errors, missing-word errors, word-order errors and spelling errors. Among them, homophone errors and shape-similar errors both belong to spelling errors.

[0054] Compared to spelling errors, variable-length text correction has always been a challenging and crucial aspect of Chinese text correction. Currently, the main methods for handling text correction are pipeline-based and end-to-end methods, among which:

[0055] Pipeline-based text correction methods have the following drawbacks: Variable-length text correction typically requires pre-defining and continuously maintaining a confusion dictionary corpus. During the error detection phase, business rules are combined, or matching algorithms are used for error detection. Matching algorithms are time-consuming. In the candidate recall phase, all possible errors are recalled as an error candidate set, which may include variable-length errors. In the correction phase, candidates are ranked using language models and similarity calculations, and the best result is selected as the correction result, completing the correction process. This method requires summarizing a large number of business rules, incurring high manpower costs during construction, high subsequent maintenance costs, relatively low error correction efficiency and accuracy, and a long correction time due to numerous matching algorithms.

[0056] End-to-end text correction methods have the following drawbacks:

[0057] First, information is not fully utilized. Most models take semantic information as input, but for Chinese error correction tasks, factors such as pinyin, similarity of form, and syntax will also have an impact.

[0058] Second, most end-to-end models can only solve single-word replacement errors, but cannot solve grammatical errors, missing words, or extra words. Generative methods, such as machine translation, can solve missing words and extra words, and are very suitable for solving grammatical errors, but they may cause overcorrection or generate duplicate text.

[0059] In some implementations, pre-trained models can be applied to text correction tasks. However, at present, the length of the input text and the length of the corrected text are equal, which cannot solve the problems of extra or missing characters.

[0060] One crucial task in pre-trained language models is the MLM task, which requires masking the original text. By using masking techniques, some tokens in a sentence are removed, and the model then predicts the masked tokens, enabling it to learn the relationships between words in the sentence.

[0061] Referring to Table 2, the current masking strategies for pre-trained language models include the following:

[0062] Table 2

[0063]

[0064] To address the aforementioned technical problems, this application provides a method for training a text correction model and for text correction. This method, based on a pre-trained language model, can correct various types of text errors, such as extra or missing characters, spelling errors, and grammatical errors. It also supports both variable-length and fixed-length text correction, reducing the cost of text correction and improving its efficiency and accuracy. Detailed technical solutions are provided in the following embodiments.

[0065] Reference Figure 1 , Figure 1 This is a flowchart illustrating the steps of a text correction model training method provided in an embodiment of this application. In some embodiments of this application, the above-mentioned text correction model training method includes:

[0066] S101. Obtain training data, which includes random erroneous text samples, near-sound erroneous text samples, and near-shape erroneous text samples.

[0067] In some embodiments, a vocabulary may be constructed first.

[0068] The vocabulary in this model is larger than that of other masked pre-trained language models because it includes commonly used Chinese characters, common words, and some 3-grams. The vocabulary is obtained by performing 1-gram, 2-gram, and 3-gram statistics on the training data, retaining high-frequency words. Furthermore, a high-frequency vocabulary can be obtained by pruning the trained n-gram language model. In addition, four special characters are added: <cls> 、 <sep> 、 <unk>and ,in, <cls> 、 <sep>It is a special input marker for the BERT model. <cls>The sign is placed at the beginning of the first sentence. <sep>The flag is used to separate two input sentences. <unk>Used to replace characters in a sentence that do not appear in the vocabulary. This represents a placeholder.

[0069] n-gram is an algorithm based on statistical language models. Its basic idea is to process the text content into a sliding window of size N bytes, forming a sequence of byte segments of length N. Each byte segment is called a gram, and the nth item can be predicted based on the previous (n-1) items. Conventionally, 1-grams are called unigrams, 2-grams are called bigrams, and 3-grams are called trigrams. A 2-gram is a pairwise combination of characters, a 3-gram (trigram) is any combination of three characters, and so on.

[0070] In some embodiments, to increase the error diversity of training data samples, training data can be generated in the following manner:

[0071] 1) Construct random error text samples based on the original text, including:

[0072] Randomly delete characters from the original text to construct random text samples with missing characters;

[0073] Randomly add characters or words to the original text: Construct random multi-character error text samples;

[0074] Randomly replace words in the original text: construct random misspelled text samples;

[0075] Randomly shuffled text in the original text: randomized word order error text sample.

[0076] 2) Constructing near-phonetic error text samples:

[0077] First, based on the original text, using existing text-to-pinyin tools and rules for confusing sounds based on local accents, we constructed a text with some near-sound errors. For example, we confused "f" with "h", "q" with "x", "n" with "l" and other local accents; we confused "eng" with "en", "ing" with "in" and other nasal sounds; and we confused "ch" with "c", "zh" with "ch", "sh" with "s" and other retroflex sounds.

[0078] Secondly, on the random error text sample constructed in 1), near-sound errors with the same confusion rules are added a second time to the modified error segments to construct variable-length near-sound error text.

[0079] 3) Construct near-shape error text samples:

[0080] First, based on the original text, a similar-looking error text is constructed using a Chinese dictionary of similar-looking characters. Similar characters can be obtained from publicly available confusion sets (Wu et ai., 2013).

[0081] Secondly, on the random error text sample constructed in 1), similar near-shape errors of the same rule are added a second time to the modified error fragments to construct a variable-length near-shape error text sample.

[0082] S102. Determine the input vector corresponding to each character in the training data. The input vector includes the character representation vector, the position representation vector, the pinyin representation vector, and the character shape representation vector.

[0083] In some embodiments of this application, character representation vectors and position representation vectors can be obtained by looking up the embedding table of a corpus; pinyin representation vectors can be generated by converting text into a pinyin sequence and then inputting the pinyin sequence into a GRU network. By determining the four-corner code, radical information, and structural information corresponding to each character, the glyph sequence formed by concatenating the four-corner code, radical information, and structural information corresponding to each character can be input into a GRU network to generate the aforementioned glyph representation vectors.

[0084] For example, the four-corner code of the door is 37001, and the four-corner code of the latch is 37101; the radical information of the latch includes ['door', 'one'], and the radical information of the latch includes ['door', 'three']; the structural information of the latch is a semi-enclosed structure.

[0085] S103. The pre-trained language model is trained based on the input vector to obtain a text correction model.

[0086] In some embodiments of this application, in order to train the model, a certain percentage of the input tokens may be randomly masked and then restored, wherein a dynamically variable mask strategy is employed to suit the task of text correction.

[0087] The Mask strategy example involves masking 15% of the tokens in the input sequence. In 25% of cases, this masking is performed using similar words with multiple characters, similar words with fewer characters, or characters of equal length. In 10% of cases, word order masking is used, and in 10% of cases, no replacement is performed. More specifically, to accommodate the diversity of error correction tasks, the masking strategies for multiple characters, fewer characters, and spelling are implemented with replacements based on character shape, pronunciation, and randomness at rates of 10%, 10%, and 5%, respectively.

[0088] Given sufficient computing resources, the percentage of the masking strategy can be adjusted to obtain the optimal masking ratio.

[0089] The MASK strategy is mainly based on the following four types: Phonic Masking, Shape Masking, Random Masking, and Unchanging.

[0090] Referring to Table 3, the current masking strategies for pre-trained language models include the following:

[0091] Table 3

[0092]

[0093] Optionally, the encoding layer uses a Transformer encoder, and the structure can be similar to that of BERT. Regarding parameter settings, let's assume dc, dp, dy, and ds represent the dimensions of the character sequence, position sequence, pinyin sequence, and glyph sequence, respectively, and hy and hs represent the dimensions of the hidden states in the pinyin and glyph GRU networks. Then, dc = dp = 768, dy = ds = 32, and hy = hs = 768. To save on pre-training costs, more parameters can be set empirically. With sufficient computational resources, a grid search approach can be used to achieve better performance.

[0094] During the fine-tuning training phase, the input character sequence is expected to be the target sequence generated by the model after error correction. Examples of three types of errors are given below. Word order errors and spelling errors are similar; for error-free text, the original text is expected to be output.

[0095] Refer to Table 4, which provides examples of error correction for multiple-word errors in this application.

[0096] Table 4: Errors involving multiple characters

[0097]

[0098]

[0099] Refer to Table 5, which provides examples of corrections for missing characters in this application.

[0100] Table 5: Missing Character Errors

[0101] Location information 0 1 2 3 4 5 6 7 8 9 Missing words error you No want Heart ， want Heart Affection Shu smooth After correction you No want sad ， want Heart Affection Shu smooth

[0102] Refer to Table 6, which provides examples of spelling error corrections in this application.

[0103] Table 6: Spelling Errors

[0104] Location information 0 1 2 3 4 5 6 7 8 9 10 Spelling errors you No want Imitation Heart ， want Heart Affection Shu smooth After correction you No want hurt Heart ， want Heart Affection Shu smooth

[0105] It should be noted that in the above fine-tuning training samples, for multi-character errors, if the input character length is greater than the output character length, then the output will use... To place the errors, the model will replace the incorrect word at position 3 with the correct word for any missing word errors. Common spelling and word order errors are shown in Table 6.

[0106] Based on the content described in the above embodiments, this application also provides a text correction method, referring to... Figure 2 , Figure 2 This is a flowchart illustrating the steps of a text correction method provided in an embodiment of this application. In some embodiments of this application, the text correction method includes:

[0107] S201. Obtain the input vector corresponding to each character in the text to be corrected. The input vector includes the character representation vector, the position representation vector, the pinyin representation vector, and the character shape representation vector.

[0108] In some implementations, the vocabulary and text correction model can be loaded first. Then, the input vector corresponding to each character in the text to be corrected is obtained.

[0109] S202. Based on the input vector corresponding to each character and the text correction model, obtain the text correction prediction result vector.

[0110] The text correction model described above can be obtained according to the training method of the text correction model described in the above embodiments, which will not be repeated in this embodiment.

[0111] S203. Decode the text correction prediction result vector to obtain the target text after the text to be corrected.

[0112] In some implementations, the index corresponding to the maximum value of the last dimension of the text correction prediction result vector can be determined. The Chinese character corresponding to this index in the vocabulary is the result after correction. This means that the character with the highest output probability in the vocabulary is considered to be the correct character for each input position.

[0113] Reference Figure 3 , Figure 3 This is a schematic diagram illustrating the application of a text correction model provided in the embodiments of this application.

[0114] exist Figure 3 In this example, assuming the text to be corrected is "You shouldn't be sad", the input vector corresponding to the text to be corrected is input into the text correction model. After the output of the text correction model is processed by the fine-tuning network "MLM in bert", the corrected target text "You shouldn't be sad" can be obtained.

[0115] The text correction model training and text correction method provided in this application can correct various types of text errors, such as extra words, missing words, spelling errors, and grammatical errors. It can simultaneously achieve both variable-length and constant-length text correction, which can not only reduce the cost of text correction, but also improve the efficiency and accuracy of text correction.

[0116] Based on the content described in the above embodiments, this application also provides a training device for a text error correction model. (Refer to...) Figure 4 , Figure 4 This is a schematic diagram of the program modules of a text correction model training device provided in this application embodiment. The text correction model training device 40 includes:

[0117] The initialization module 401 is used to acquire training data, which includes random error text samples, near-sound error text samples, and near-shape error text samples.

[0118] The feature extraction module 402 is used to determine the input vector corresponding to each character in the training data. The input vector includes a character representation vector, a position representation vector, a pinyin representation vector, and a character shape representation vector.

[0119] The training module 403 is used to train the pre-trained language model based on the input vector to obtain a text correction model.

[0120] In some implementations, the initialization module 401 is further configured to:

[0121] The random error text samples, the near-sound error text samples, and the near-shape error text samples are constructed based on the original text; wherein, the random error text samples include random missing character error text samples, random extra character error text samples, random spelling error text samples, and random word order error text samples.

[0122] In some embodiments, the near-pronunciation error text sample includes near-pronunciation error text and variable-length near-pronunciation error text, and the initialization module 401 is further used for:

[0123] Based on the pinyin corresponding to the original text and the preset confusion rules, the near-sound error text corresponding to the original text is generated, and based on the modified error segments of the random error text samples and the confusion rules, the variable-length near-sound error text corresponding to the original text is generated.

[0124] In some embodiments, the near-shape error text samples include near-shape error text and variable-length near-shape error text samples, and the initialization module 401 is further configured to:

[0125] Based on the original text and a preset dictionary of similar-looking characters, the similar-looking error text corresponding to the original text is generated, and based on the modified error fragments of the random error text sample and the dictionary of similar-looking characters, the variable-length similar-looking error text sample corresponding to the original text is generated.

[0126] In some implementations, the feature extraction module 402 is specifically used for:

[0127] Find the character representation vector and the position representation vector corresponding to each character by looking up the embedding table;

[0128] Determine the pinyin corresponding to each character, and input the pinyin corresponding to each character into a gated recurrent unit (GRU) network to generate the pinyin representation vector;

[0129] The four-corner code, radical information, and structural information corresponding to each character are determined. The glyph sequence formed by concatenating the four-corner code, radical information, and structural information corresponding to each character is input into the GRU network to generate the glyph representation vector.

[0130] Based on the content described in the above embodiments, this application also provides a text correction device. (Refer to...) Figure 5 , Figure 5 This is a schematic diagram of the program modules of a text correction device 50 provided in an embodiment of this application. The text correction device 50 includes:

[0131] The feature extraction module 501 is used to obtain the input vector corresponding to each character in the text to be corrected. The input vector includes a character representation vector, a position representation vector, a pinyin representation vector, and a character shape representation vector.

[0132] The text correction module 502 is used to obtain a text correction prediction result vector based on the input vector corresponding to each character and the text correction model; the text correction model is obtained according to the training device of the text correction model described in the above embodiments.

[0133] The error correction output module 503 is used to decode the text error correction prediction result vector to obtain the target text after the text to be corrected.

[0134] In some implementations, the text correction module 502 is also used for:

[0135] Determine the index corresponding to the maximum value in the last dimension of the text error correction prediction result vector, and determine the target text based on the index and a preset vocabulary.

[0136] The text correction model training and text correction device provided in this application can correct various types of text errors, such as extra words, missing words, spelling errors, and grammatical errors. It can simultaneously achieve both variable-length and constant-length text correction, which can not only reduce the cost of text correction, but also improve the efficiency and accuracy of text correction.

[0137] Furthermore, based on the content described in the above embodiments, this application also provides an electronic device, which includes at least one processor and a memory; wherein the memory stores computer execution instructions; the at least one processor executes the computer execution instructions stored in the memory to implement the various steps in the training method of the text correction model as described in the above embodiments, or to implement the various steps in the text correction method as described in the above embodiments, which will not be repeated here.

[0138] To better understand the embodiments of this application, please refer to... Figure 6 , Figure 6 This is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of this application.

[0139] like Figure 6 As shown, the electronic device 60 of this embodiment includes: a processor 601 and a memory 602; wherein:

[0140] Memory 602 is used to store instructions executed by the computer;

[0141] The processor 601 is configured to execute computer execution instructions stored in the memory to implement the various steps in the training method of the text correction model as described in the above embodiments, or to implement the various steps in the text correction method as described in the above embodiments, which will not be repeated here.

[0142] In some implementations, the memory 602 can be either standalone or integrated with the processor 601.

[0143] When the memory 602 is set up independently, the device also includes a bus 603 for connecting the memory 602 and the processor 601.

[0144] Furthermore, based on the content described in the above embodiments, this application also provides a computer-readable storage medium storing computer-executable instructions. When the computer executes the computer-executable instructions, it implements the various steps in the training method of the text correction model as described in the above embodiments, or implements the various steps in the text correction method as described in the above embodiments. This embodiment will not be elaborated further here.

[0145] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative; for instance, the division of modules described above is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple modules may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be indirect coupling or communication connection through some interfaces, devices, or modules, and may be electrical, mechanical, or other forms.

[0146] The modules described above as separate components may or may not be physically separate. The components shown as modules may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0147] Furthermore, the functional modules in the various embodiments of this application can be integrated into one processing unit, or each module can exist physically separately, or two or more modules can be integrated into one unit. The unit integrating the above modules can be implemented in hardware or in the form of hardware plus software functional units.

[0148] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some or all of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the scope of the technical solutions of the embodiments of the present application. < / unk> < / sep> < / cls> < / sep> < / cls> < / unk> < / sep> < / cls>

Claims

1. A training method for a text error correction model, characterized in that, The method includes: Acquire training data, which includes random erroneous text samples, near-sound erroneous text samples, and near-shape erroneous text samples; The input vector corresponding to each character in the training data is determined. The input vector includes a character representation vector, a position representation vector, a pinyin representation vector, and a character shape representation vector. The character shape representation vector is generated by inputting the character shape sequence, which is formed by concatenating the four-corner code, radical information, and structural information corresponding to each character, into the GRU network. The pre-trained language model is trained based on the input vector to obtain a text correction model. The training adopts a dynamically variable length mask strategy, which includes masking 15% of the tokens in the total length of the input sequence. In 25% of cases, the tokens are replaced with similar words with more characters, similar words with fewer characters, or characters of equal length. In 10% of cases, word order masking is performed, and in 10% of cases, no replacement is performed. Furthermore, in the more-character, fewer-character, and spelling masking strategies, the replacements are performed in terms of character shape, pronunciation, and random replacement at a ratio of 10%, 10%, and 5%, respectively.

2. The method according to claim 1, characterized in that, Before acquiring the training data, the process also includes: The random error text samples, the near-sound error text samples, and the near-shape error text samples are constructed based on the original text; wherein, the random error text samples include random missing character error text samples, random extra character error text samples, random spelling error text samples, and random word order error text samples.

3. The method according to claim 2, characterized in that, The near-pronunciation error text sample includes near-pronunciation error text and variable-length near-pronunciation error text. Constructing the near-pronunciation error text sample includes: Based on the pinyin corresponding to the original text and the preset confusion rules, the near-sound error text corresponding to the original text is generated, and based on the modified error segments of the random error text samples and the confusion rules, the variable-length near-sound error text corresponding to the original text is generated. The near-shape error text samples include near-shape error text and variable-length near-shape error text samples. Constructing the near-shape error text samples includes: Based on the original text and a preset dictionary of similar-looking characters, the similar-looking error text corresponding to the original text is generated, and based on the modified error fragments of the random error text sample and the dictionary of similar-looking characters, the variable-length similar-looking error text sample corresponding to the original text is generated.

4. The method according to claim 1, characterized in that, Determining the input vector corresponding to each character in the training data includes: Find the character representation vector and the position representation vector corresponding to each character by looking up the embedding table; Determine the pinyin corresponding to each character, and input the pinyin corresponding to each character into a gated recurrent unit (GRU) network to generate the pinyin representation vector; The four-corner code, radical information, and structural information corresponding to each character are determined. The glyph sequence formed by concatenating the four-corner code, radical information, and structural information corresponding to each character is input into the GRU network to generate the glyph representation vector.

5. A text correction method, characterized in that, The method includes: Obtain the input vector corresponding to each character in the text to be corrected. The input vector includes a character representation vector, a position representation vector, a pinyin representation vector, and a character shape representation vector. Based on the input vector corresponding to each character and the text correction model, a text correction prediction result vector is obtained; the text correction model is obtained by the training method of the text correction model according to any one of claims 1 to 4. The text correction prediction result vector is decoded to obtain the target text after correction of the text to be corrected.

6. The method according to claim 5, characterized in that, Decoding the text correction prediction result vector to obtain the corrected target text of the text to be corrected includes: Determine the index corresponding to the maximum value in the last dimension of the text error correction prediction result vector, and determine the target text based on the index and a preset vocabulary.

7. A training device for a text error correction model, characterized in that, The device includes: An initialization module is used to acquire training data, which includes random erroneous text samples, near-sound erroneous text samples, and near-shape erroneous text samples. The feature extraction module is used to determine the input vector corresponding to each character in the training data. The input vector includes a character representation vector, a position representation vector, a pinyin representation vector, and a character shape representation vector. The character shape representation vector is generated by inputting the character shape sequence, which is formed by concatenating the four-corner code, radical information, and structural information corresponding to each character, into the GRU network. The training module is used to train the pre-trained language model based on the input vector to obtain a text correction model. The training employs a dynamically variable length mask strategy, which includes masking 15% of the tokens in the total length of the input sequence. Specifically, in 25% of cases, the tokens are replaced with similar words with multiple characters, similar words with fewer characters, or characters of equal length; in 10% of cases, word order masking is performed; and in 10% of cases, no replacement is performed. Furthermore, in the multiple-character, few-character, and spelling masking strategies, replacements are performed based on character shape, pronunciation, and random substitution at proportions of 10%, 10%, and 5%, respectively.

8. A text correction device, characterized in that, The device includes: The feature extraction module is used to obtain the input vector corresponding to each character in the text to be corrected. The input vector includes a character representation vector, a position representation vector, a pinyin representation vector, and a character shape representation vector. The text correction module is used to obtain a text correction prediction result vector based on the input vector corresponding to each character and the text correction model; the text correction model is obtained by the training device of the text correction model according to claim 7. The error correction output module is used to decode the text error correction prediction result vector to obtain the target text after the text to be corrected.

9. An electronic device, characterized in that, include: At least one processor and memory; The memory stores computer-executable instructions; The at least one processor is configured to execute computer execution instructions stored in the memory to implement the training method of the text correction model as described in any one of claims 1 to 4, or to implement the text correction method as described in any one of claims 5 to 6.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed by a computer, implement the training method of the text correction model as described in any one of claims 1 to 4, or implement the text correction method as described in any one of claims 5 to 6.

Images