A data enhancement method for Chinese text proofreading

Abstract

The application provides a data enhancement method for Chinese text proofreading, and relates to the technical field of artificial intelligence. The method judges the position and type of errors prone to occur in the correct source sentence through a sequence labeling model, makes up for the defects of current methods of randomly selecting error positions and error types, and makes the data closer to existing training data; in generating multi-word errors, syntax error data generated by using the model BERT is added, so that the semantic correlation of the generated error sentences is stronger; in the process of generating spelling errors, syntax error data generated by using the model BERT is added to simulate the situation of vocabulary selection errors in writing; at the same time, the spelling errors caused by pressing the wrong key when entering the text by using the keyboard in the real entry process are considered; the generated pseudo data contains common syntax error types, which can improve the robustness of the syntax correction model and the spelling correction model to a certain extent, and make the model learn more diverse and similar error sentence features to the real data.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of artificial intelligence, and in particular to a data enhancement method for Chinese text proofreading. BACKGROUND

[0002] In the big data era, the speed and frequency of information generation have greatly improved compared to before, and people's requirements for Chinese text proofreading effect are also increasing. With the development of deep learning technology, the solution to the Chinese text proofreading problem has changed from using rule matching to using deep learning models to identify and correct potential errors in the text.

[0003] The training data required by the Chinese text proofreading model is a sentence pair composed of a grammatical error sentence and a correct sentence. These sentence pairs are mostly from the grammar error correction shared task training set, and some studies will first use crawler technology to obtain low-quality text on the Internet, and then manually annotate and organize the final data. However, due to the small amount of grammatical error data in real-world scenarios and the sparsity of errors themselves, it is difficult to obtain sufficient grammatical error text data to support model learning in the actual training process. Due to the shortage of data quantity, some deep learning models cannot learn the error characteristics in the grammatical error text as much as possible, so they cannot fully play the structure and advantages of the model itself. Therefore, due to the above reasons, various Chinese text proofreading methods also consider using data enhancement to expand the training data.

[0004] Chinese grammar errors are manifested as wrong words, missing words, extra words, and disordered sequences. Among them, the wrong word type of grammar error is often handled separately and is also called spelling error. Initially, the data enhancement method for Chinese text proofreading mostly randomly selects the word segmentation results of correct sentences, and then performs operations such as replacing words, deleting words, adding words, and rearranging order, and then the processing results are used as generated pseudo-data. Subsequent work observes the error rate of existing data based on the original, and generates noise similar to grammar error forms according to prior probability to damage the text.

[0005] In 2020, a team led by Wang Chen proposed a single-language data enhancement method based on corrupt corpus. In this method, the user first sets the proportion of corrupt words, extra word errors, missing word errors, and character replacement errors according to the observation of the error rate of the corpus, and then converts the correct sentence to the generated error sentence according to the specific operation method of each error.

[0006] In 2021, Pan Lijian proposed a Chinese text proofreading corpus automatic generation technology research that describes data augmentation methods for Chinese spelling errors and Chinese grammar errors. For Chinese spelling errors, the method first randomly selects a word in the segmentation result, and then performs confusion replacement according to the error characteristics of pinyin, pronunciation, and character shape input method to generate pseudo data; for Chinese grammar errors, i.e. multi-word, few-word, and character replacement type errors, a model is trained based on the translation method to generate error sentences containing the above error types.

[0007] The data augmentation method provided in the Chinese patent "CN202110758581.5 Chinese text proofreading method and device based on data augmentation" uses the noise function in the BART (Bidirectional and Auto-Regressive Transformers) model. In use, a random number generated on a uniform distribution in [0, 1] is used as a probability to convert the input source sentence into a sentence with different types of syntax errors using one or any combination of the three methods of word masking, random word replacement, and text padding.

[0008] Although the data generated by the current data augmentation method can conform to the form of the syntax error sentences in the training data, the data quality cannot fully meet the training requirements.

[0009] Firstly, the current rule-based methods all determine the error generation position in a random manner, and then perform the corresponding noise addition process. However, random selection itself is prone to introduce uncertainty, leading to some differences between the generated errors and the original data.

[0010] Secondly, in the two schemes involving character or word modification, multi-word error and spelling error, researchers mostly do not use the capabilities of pre-trained language models. The rule-based method mostly uses random character addition to create multi-word errors, but there may be a large semantic difference from the original sentence, which does not conform to the internal logic of error data. Spelling errors often also include a small part of word selection errors, and existing schemes mostly ignore this part of the error content. Relying only on the phonetic and graphic features of spelling errors cannot create data with semantic level word selection errors.

[0011] In addition, the grammar error data generated completely relying on the model has weak generalization, high uncertainty in the generation process, and fewer controllable factors. Some complex filtering methods are often needed to determine whether the difference between the generated sentence and the original sentence is within an acceptable range. SUMMARY

[0012] The technical problem solved by the present application is to provide a data enhancement method for Chinese text correction to solve the above-mentioned problems of the prior art, so as to realize the enhancement of grammatical error data.

[0013] To solve the above technical problems, the technical solution adopted by the present application is: a data enhancement method for Chinese text correction, according to the specific form of grammatical errors and spelling errors and the error distribution situation contained therein, the positions of various grammatical errors in the text sentence are determined; combining the prior knowledge and the ability of the pre-training language model, the grammatical error content appearing in the real world is simulated, and it is ensured that the model can learn more grammatical error knowledge that does not appear in the training set; specifically including the following steps:

[0014] Step 1, according to the punctuation in the sentence, the input sequence labeling model and the BERT model are used to divide the sentence; if the length of the divided sentence is greater than 508 characters, the sentence is directly divided according to the character length of 508;

[0015] Step 2, input the divided sentence as the source sentence into the sequence labeling model;

[0016] Step 3, obtain the prediction result of the sequence labeling model on the source sentence; the sequence labeling model completes the forward propagation process according to the trained model parameters, and outputs the position of the error prone in the source sentence and the corresponding error type according to the confidence of the model;

[0017] Step 4, according to the different error types identified by the sequence labeling model, the source sentence is processed according to the corresponding error type;

[0018] I. For the characters marked as B-M in the source sentence, the marked result is regarded as a few-word error, and the characters marked as B-M are deleted;

[0019] II. For the two consecutive characters marked as B-R and I-R in the source sentence, use the Random library in Python to randomly generate an integer between 1 and 100, and according to the size of the generated integer, add characters in the middle position of the two characters in the following two ways:

[0020] (a) When the generated integer is less than or equal to 80, use the pre-training language model BERT to complete the addition of characters; first fill [MASK] characters in the middle position of the two consecutive marked characters to modify the source sentence; then input the modified source sentence into the BERT model to obtain the multi-word candidate set output vector, and combine the Softmax function to obtain the finally generated multi-word grammatical error pseudo data; during the modification of the source sentence, the following modification rules are followed:

[0021] When the generated integer is less than or equal to 20, two [MASK] characters are added between two consecutive marked characters to simulate a multi-word grammatical error; otherwise, only one [MASK] character is added.

[0022] During the modification of the source statement, the top five candidate characters from the multi-character candidate set generated by the BERT model and sorted by the Softmax function are randomly selected from a uniform distribution as pseudo data for multi-character syntax errors.

[0023] (b) When the generated integer is between 81 and 100, randomly select a single character from the Chinese character dictionary and add it;

[0024] 3. For characters marked as BW and IW out of order errors, first perform word segmentation on the source statement and extract the word segmentation results related to the BW and IW out of order error characters;

[0025] If there are at least two word segmentation results related to the BW and IW disordered characters, then two word segmentation results are randomly selected and their positions are randomly swapped according to probability to simulate disordered situations in syntax errors.

[0026] If fewer than two tokenization results are extracted that are related to the out-of-order error characters of BW and IW, then the generation of out-of-order error at that tag is abandoned.

[0027] IV. For characters marked as BS and IS spelling errors, randomly generate an integer between 1 and 100, and replace the marked characters according to the size of the generated integer using the following three methods:

[0028] (1) When the generated integer is between 1 and 10, the marked character is replaced with the [MASK] character, and then input into the BERT model to obtain the output vector. The Softmax function is used to obtain the final generated spelling error pseudo data.

[0029] (2) When the generated integer is between 11 and 15, replace the marked character with a synonym of the word marked as a BS and IS spelling error; if the synonyms library does not generate a synonym related to the marked character, then abandon the generation of error at that point.

[0030] (3) When the generated integer is between 16 and 65, a phonetic obfuscation mode is used for character replacement; the marked characters are converted into pinyin using the pypinyin library in Python, and the marked characters are replaced using the following three methods according to the size of the generated integer:

[0031] When the generated integer is between 16 and 45, the marked character is replaced by a character in the Pinyin character confusion table that is homophonous to the marked character; when the generated integer is between 46 and 55, the Pinyin converted from the marked character by using the Pinyin2Hanzi library is converted into a Chinese character, and a converted character that is different from the marked character is selected to replace the marked character;

[0032] When the generated integer is between 56 and 65, an error introduced when a human uses a computer keyboard to input is simulated; the initials and finals in Pinyin are randomly added, omitted or replaced according to the keyboard positions of the full Pinyin input method and the double Pinyin input method, the modified Pinyin is converted into a character by using the Pinyin2Hanzi library, and the marked character is replaced by the converted character;

[0033] (4) When the generated integer is between 66 and 100, a character replacement is performed in the font confusion mode;

[0034] When the generated integer is between 67 and 78, the marked character is replaced by a character in the homograph confusion table; when the generated integer is between 79 and 100, the marked character is replaced by a confusion character obtained by randomly covering the content of the character and inputting the covered character into an OCR interface;

[0035] Step 5, after all the marked characters are processed, the processing results are combined to obtain a finally generated error sentence with a syntax error;

[0036] Step 6, filter out error sentences that do not meet the requirements; when the number of syntax error words or the number of syntax errors in the generated error sentence exceeds a set value, the generated sentence is directly discarded.

[0037] The beneficial effects produced by the above technical solutions are that the data enhancement method for Chinese text proofreading provided by the application has the following advantages: (1) The sequence labeling model is used to determine the positions and types of errors that are prone to occur in the correct source sentence, so as to make up for the defects of the current method of randomly selecting error positions and error types, and make the data closer to the existing training data. (2) In the generation of multi-word errors, the syntactic error data generated by using the pre-trained language model BERT is added, so that the semantic relevance of the generated error sentences is stronger, and the defects of the current method of randomly adding characters in the generation of multi-word errors are made up. (3) In the process of generating spelling errors, the syntactic error data generated by using the pre-trained language model BERT is added to simulate the situation of lexical selection errors in writing. In addition, the spelling errors caused by pressing the wrong key when the user uses the 26-key keyboard to enter the text in the real input process are considered. (4) The generated pseudo data contains common syntactic error types, which can improve the robustness of the syntactic error correction model and the spelling error correction model to a certain extent, so that the model learns more diverse and similar error sentence features to the real data. Adding the pseudo data generated by the method to the training data can improve the related effect of the Chinese text proofreading model, especially the recall rate index, which can effectively help the model to detect more error data in different situations.

[0038] The method of the application can determine the positions where each type of error is prone to occur according to the specific form of syntactic errors and spelling errors and the error distribution situation contained therein. By combining the prior knowledge and the ability of the pre-trained language model, the syntactic error content in the real world is simulated as much as possible, and the model can learn more syntactic error knowledge that does not appear in the training set. Thus, the defects of the current various data enhancement methods deviating from the real data distribution and generating pseudo data with poor quality effect are reduced. BRIEF DESCRIPTION OF DRAWINGS

[0039] Figure 1 A flowchart of the data enhancement method for Chinese text proofreading provided by the embodiment of the application is shown in the figure.

[0040] Figure 2 A schematic diagram of the sequence labeling model provided by the embodiment of the application is shown in the figure.

[0041] Figure 3 A schematic diagram of the pre-trained language model BERT provided by the embodiment of the application is shown in the figure. DETAILED DESCRIPTION

[0042] The specific embodiments of the application will be further described in detail below with reference to the accompanying drawings and embodiments. The following embodiments are used to illustrate the application, but not to limit the scope of the application.

[0043] In this embodiment, a data enhancement method for Chinese text proofreading is provided. According to the specific form of grammatical errors and spelling errors and the error distribution situation contained therein, the positions where each type of grammatical error is prone to occur in the text sentence are determined. In combination with the prior knowledge and the ability of the pre-trained language model, the grammatical error content occurring in the real world is simulated, and it is ensured that the model can learn more grammatical error knowledge that does not appear in the training set. As shown in Figure 1 The method comprises the following steps:

[0044] Step 1. Due to the limitation of the model input length, the input sequence labeling model and the BERT model are divided into sentences according to the punctuation in the sentence. If the length of the divided sentence is greater than 508 characters, the sentence is directly divided according to the character length of 508.

[0045] Step 2. The divided sentence is input into the sequence labeling model as a source sentence.

[0046] In this embodiment, the sentence without grammatical errors after division is selected as the source sentence and input into the sequence labeling model. The sequence labeling model is composed of a pre-trained language model BERT (Bidirectional Encoder Representation from Transformers) and a CRF (Conditional Random Field), and the specific model structure is shown in Figure 2 The model structure diagram. In this embodiment, the CGED (Chinese Grammatical Error Diagnosis) competition data of previous years is used as the training data of the sequence labeling model as shown in Figure 2 The model training is completed.

[0047] Step 3. The prediction result of the sequence labeling model for the source sentence is obtained. The sequence labeling model completes the forward propagation process according to the trained model parameters, and outputs the position prone to error and the corresponding error type in the source sentence according to the confidence obtained by the model.

[0048] Step 4. The source sentence is processed according to the different error types identified by the sequence labeling model.

[0049] I. For the characters marked as B-M in the source sentence, the marked result is regarded as a few-word error, and the characters marked as B-M are deleted.

[0050] For example, the annotation result of the source sentence "Let's eat" is O O B-M O O, which means that when people write this sentence, they are likely to miss the word "eat" and write it as "Let's eat", which is a missing word error. Therefore, when the sequence labeling model marks "eat" as B-M, the delete operation is performed to process "Let's eat" as "Let's eat".

[0051] II. For the two consecutive characters marked as B-R and I-R in the source sentence, a random integer between 1 and 100 is generated using the Random library in Python. According to the size of the generated integer, the following two ways are used to add characters in the middle position of the two characters, and the frequency of multi-word and multi-word errors is configured according to the probability based on the actual grammar error situation:

[0052] (a) When the generated integer is less than or equal to 80, use the pre-trained language model BERT to complete the word addition operation; first fill the [MASK] character in the middle position of the two consecutive marked characters to modify the source sentence; then input the modified source sentence into the BERT model to get the multi-word candidate set output vector, and use the Softmax function to get the final generated multi-word grammar error pseudo data; the BERT model structure is shown in Figure 3 This operation mainly considers that the multi-word content in the multi-word grammar error data is mostly coherent in the context. BERT model benefits from a large amount of training data and mask language model task in the pre-training process, and can well perceive the context content and complete the text, so it can be as close as possible to the data distribution of the training set under the condition of determining the error position, and maintain the coherence of the text.

[0053] In this embodiment, the following detailed operations are followed during the modification of the source sentence:

[0054] When the generated integer is less than or equal to 20, two [MASK] characters are added in the middle position of the two consecutive marked characters to simulate multi-word grammar errors, and one [MASK] character is added in the remaining cases;

[0055] During the modification of the source sentence, the top five candidate characters in the multi-character candidate set generated from the BERT model and sorted by the Softmax function are randomly selected as multi-word grammar error pseudo data to avoid generating errors with low diversity and rigid performance;

[0056] (b) When the generated integer is between 81 and 100, a single character is randomly selected from the Chinese character table for addition; only single characters are added here, which aims to simulate irregular multi-word errors that occur in daily life and improve the robustness of the subsequent model training.

[0057] III. For the characters marked as B-W and I-W out-of-order errors, first use the jieba library in python to perform word segmentation on the source sentence, and extract the word segmentation results related to the B-W and I-W out-of-order error characters;

[0058] If the extracted word segmentation results related to the B-W and I-W out-of-order error characters are not less than two, randomly select two word segmentation results and randomly exchange the arrangement position of the word segmentation results according to the probability, to simulate the out-of-order situation in the syntax error;

[0059] If the extracted word segmentation results related to the B-W and I-W out-of-order error characters are less than two, abandon generating out-of-order errors at this mark;

[0060] IV. For the characters marked as B-S and I-S spelling (wrong word) error, use the Random library in Python to randomly generate an integer between 1 and 100, and replace the marked characters according to the size of the generated integer by the following three ways, which aims to construct errors by focusing on different aspects according to the probability, and increase the diversity of errors:

[0061] (1) When the generated integer is between 1 and 10, replace the marked characters with [MASK] characters, then input to the BERT model to get the output vector, and use the Softmax function to get the finally generated spelling error pseudo data, the BERT model structure is as shown in Figure 3 . This is to simulate semantic error content that is far from the pronunciation and shape, such errors are often represented as word selection errors, not spelling errors.

[0062] (2) When the generated integer is between 11 and 15, use the synonyms library in python to generate synonyms of the words marked as B-S and I-S spelling errors to replace the marked characters, generally the synonym score of synonyms provided will be lower than the set value and the length of the replaced character will be the same. If synonyms library does not generate synonyms related to the marked characters, abandon generating errors at this place; this is also to simulate word selection errors.

[0063] (3) When the generated integer is between 16 and 65, take the phonetic confusion mode to replace the characters; use the pypinyin library in python to convert the marked characters into pinyin form, and use the following three ways to replace the marked characters according to the size of the generated integer, where:

[0064] When the generated integer is between 16 and 45, the marked character is replaced by a character homophonic to the marked character in the Pinyin character confusion table; the Pinyin character confusion table is a table in which characters with the same or similar Pinyin are organized in a row, used to create homophonic or near-homophonic errors;

[0065] When the generated integer is between 46 and 55, the Pinyin converted from the marked character by using the Pinyin2Hanzi library is converted into a Chinese character, and a converted character different from the marked character is selected to replace the marked character;

[0066] When the generated integer is between 56 and 65, errors introduced when a human uses a computer keyboard to input are simulated; the initials and finals in the Pinyin are randomly added, omitted or replaced according to the keyboard layout of the full Pinyin input method and the double Pinyin input method, and then the Pinyin after modification is converted into characters by using the Pinyin2Hanzi library, and the converted characters are used to replace the marked characters;

[0067] (4) When the generated integer is between 66 and 100, the character replacement is performed in the font confusion mode;

[0068] When the generated integer is between 67 and 78, the marked character is replaced by using the font similar character confusion table; the font similar character confusion table is a table in which characters with similar font are organized in a row, used to create font similar errors;

[0069] When the generated integer is between 79 and 100, the marked character is replaced by randomly covering the character content and inputting the covered character into an OCR interface to obtain a confused character;

[0070] Step 5, after all the marked characters are processed, the processing results are merged to obtain the finally generated error sentence with grammatical errors;

[0071] Step 6, filter out error sentences that do not meet the requirements; when the number of grammatical errors in the generated error sentence exceeds 40% of the length of the sentence or the number of grammatical errors exceeds 8, the generated sentence is directly discarded.

[0072] In this embodiment, the pre-training weights used by the BERT model used in steps 2 and 4 are all pytorch versions of BERT-Base-Chinese on the HuggingFace website. BERT is composed of 12 Transformer Encoders, has 12 self-attention heads, and the number of hidden units in the feedforward network is 768. The composition of data syntax errors can be adjusted in terms of error number and type according to real scene data, so that the generated data is more close to the characteristics of the original training set.

[0073] In this embodiment, the data augmentation method of the present invention is used to augment the input statement: A reporter recently visited several hospitals in Sichuan and found that many of the patients were young people. Several frontline doctors said that diabetes, hypertension, and other diseases that were once considered "diseases of the elderly" are now being diagnosed at increasingly younger ages. "We are seeing more and more young 'diabetics' in our outpatient clinic; many young people have high blood sugar," Li Huaqi, deputy director of the Department of Endocrinology and Metabolic Diseases at the Third People's Hospital of Chengdu, told reporters. In recent years, the department has seen a trend of younger people being diagnosed with diabetes, with many young people in their twenties and thirties also developing the disease.

[0074] The sequence labeling results obtained through the sequence labeling model are as follows: BW IW IW IW OOOOOOOOO OO OOOOOOOOOOOOOOOOOOO OOOOOOOOOOOO BM OO OO OOOOOOOOOOOOOOOOO BR IR OOOOOOOOOOOOOOOO OO OOOOOOOOOOOOOOOOOOO OOOOOOOOOOOOOOOO OO OOOOOOOOOOOOOOOOOOOOO OO BS IS OOOOOOOOOOO OO OOOOOO

[0075] Based on the marking results, the corresponding error type processing results for the source statement are as follows: Recently, reporters visited several hospitals in Sichuan Province and found that many of the patients in the "geriatric" departments were young people. Several frontline doctors said that diabetes, hypertension, and other diseases previously considered "elderly diseases" are now increasingly affecting younger patients. "We're seeing more and more young 'diabetics' in our outpatient clinic; many young people have high blood sugar," Li Huaqi, deputy director of the Department of Endocrinology and Metabolic Diseases at the Third People's Hospital of Chengdu, told reporters. In recent years, the department has seen a trend of younger people dying from diabetes, with many young people in their twenties and thirties also developing the disease.

[0076] Finally, it should be noted that the above examples are only used to illustrate the technical solutions of the present application, and are not intended to limit the present application; although the present application has been described in detail with reference to the foregoing examples, those skilled in the art should understand that the technical solutions recorded in the foregoing examples can still be modified, or some or all of the technical features can be replaced by equivalents; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope defined by the claims of the present application.

Claims

1. A data enhancement method for Chinese text proofreading, characterized by: According to the punctuation in the sentence, the input sequence labeling model and the BERT model are used to divide the sentence into sentences; The divided sentence is input into the sequence labeling model as the source sentence; The sequence labeling model completes the forward propagation process according to the trained model parameters, and outputs the position of the source sentence prone to errors and the corresponding error type according to the confidence of the model; Obtain the identification result of the source sentence by the sequence labeling model; According to the different error types identified by the sequence labeling model, the source sentence is processed according to the corresponding error type; I. For the characters marked as B-M in the source sentence, the marked results are regarded as few-word errors, and the characters marked as B-M are deleted; II. For the two consecutive characters marked as B-R and I-R in the source sentence, add characters in the middle position of the two characters; III. For the characters marked as B-W and I-W, according to the segmentation results, simulate the disorder situation in the syntax error or give up generating disorder error at the marked position; IV. For the characters marked as B-S and I-S, replace the marked characters; After all the marked characters are processed, the processing results are combined to obtain the final generated error sentence with syntax errors; Filter out the error sentences that do not meet the requirements.

2. The data enhancement method for Chinese text correction according to claim 1, wherein: The specific method for adding characters in the middle position of the two consecutive characters marked as B-R and I-R in the source sentence is: Randomly generate an integer between 1 and 100, and according to the size of the generated integer, add characters in the middle position of the two characters by the following two methods: (a) When the generated integer is less than or equal to 80, use the pre-trained language model BERT to complete the addition of characters; First, fill [MASK] characters in the middle position of the two consecutive marked characters to modify the source sentence; then input the modified source sentence into the BERT model to obtain the multi-word candidate set output vector, and combine the Softmax function to obtain the finally generated multi-word syntax error pseudo data; During the modification of the source sentence, the following modification rules are followed: When the generated integer is less than or equal to 20, add two [MASK] characters in the middle position of the two consecutive marked characters to simulate multi-word syntax errors, and in other cases, only add one [MASK] character; During the modification of the source sentence, randomly select the top five candidate characters from the multi-character candidate set generated by the BERT model and sorted by the Softmax function as the multi-word syntax error pseudo data according to the uniform distribution; (b) When the generated integer is between 81 and 100, randomly select a single character from the Chinese character table for addition.

3. The data augmentation method for Chinese text correction according to claim 2, wherein: The specific method for simulating the disorder situation in the syntax error according to the segmentation results or giving up generating disorder error at the marked position for the characters marked as B-W and I-W is: First, perform segmentation on the source sentence to extract the segmentation results related to the B-W and I-W disorder error characters; If the extracted segmentation results related to the B-W and I-W out-of-order error characters are not less than two, randomly select two segmentation results to randomly exchange the arrangement positions of the segmentation results according to the probability, to simulate the out-of-order situation in the syntax error; If the extracted segmentation results related to the B-W and I-W out-of-order error characters are less than two, give up generating the out-of-order error at the mark.

4. The data enhancement method for Chinese text correction according to claim 3, wherein: The specific method for replacing the marked characters for the characters marked as B-S and I-S spelling errors is: Randomly generate an integer between 1 and 100, and replace the marked characters by the following three ways according to the size of the generated integer: (1) When the generated integer is between 1 and 10, replace the marked characters with [MASK] characters, input them into the BERT model to get the output vector, and use the Softmax function to obtain the finally generated spelling error pseudo data; (2) When the generated integer is between 11 and 15, replace the marked characters with the synonyms of the words marked as B-S and I-S spelling errors; if the synonyms library does not generate synonyms related to the marked characters, give up generating errors at this place; (3) When the generated integer is between 16 and 65, take the phonetic confusion mode to replace the characters; (4) When the generated integer is between 66 and 100, take the graph confusion mode to replace the characters; when the generated integer is between 67 and 78, use the graphically similar character confusion table to replace the marked characters; when the generated integer is between 79 and 100, replace the marked characters by randomly covering the character content and passing the covered characters into the OCR interface to obtain confused characters.

5. The data augmentation method for Chinese text correction according to claim 4, wherein: The specific method for replacing the characters when the generated integer is between 16 and 65 is: Convert the marked characters into pinyin form using the pypinyin library in python, and replace the marked characters using the following three ways according to the size of the generated integer, wherein: When the generated integer is between 16 and 45, replace the marked characters with the characters in the pinyin character confusion table that are homophonic to the marked characters; When the generated integer is between 46 and 55, convert the pinyin converted from the marked characters into Chinese characters using the Pinyin2Hanzi library, and replace the marked characters with the converted characters that are not the same as the marked characters; When the generated integer is between 56 and 65, simulate the errors introduced by humans when using a computer keyboard to input; randomly add, omit or replace the initial and final vowels in the pinyin according to the full-pinyin input method and double-pinyin input method keyboard layout, then convert the modified pinyin into characters using the Pinyin2Hanzi library, and replace the marked characters with the converted characters.

Images