Method, system and processor for optimizing chinese text correction in official document writing scenario

Abstract

The application relates to a method, system and processor for optimizing Chinese text error correction in a government document writing scenario, comprising the following steps: collecting original data sets; performing a word segmentation operation on the sentences in the original data sets to obtain a high-frequency word table; expanding the high-frequency word table; making mistakes in the multiple sentences in the original data sets, pre-training a BERT encoder according to the error-correct sentence pairs and the high-frequency word table; collecting government document writing word groups from a network to form a government document writing word table; crawling multiple correct sentences on a government document website to build a government document writing data set; making mistakes in the correct sentences in the government document writing data set, training the BERT encoder again according to the error-correct sentence pairs and the high-frequency word table; and using the optimized BERT encoder to predict the Chinese text to be processed to obtain text output after error correction. The application can learn and correct the wrong words in the government document writing scenario, and can solve the problems of multi-word and few-word errors.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of text processing, in particular to a method and system for optimizing Chinese text error correction in the context of official document writing and a processor. BACKGROUND

[0002] In the field of natural language processing (NLP), Chinese text error correction tasks usually include two types: Chinese grammar error correction (CGEC) and Chinese spelling error correction (CSC). The former aims to correct the addition, deletion and modification errors in the input sentence. The classic grammar error correction (GEC) method is mostly based on the "sequence to sequence" encoder-decoder framework, which encodes the input sentence first and then generates the error correction result by the decoder. The latter focuses on correcting the wrong characters in the input sentence, mainly through predicting each character in the sequence. Chinese spelling error correction (CSC) methods based on pre-trained language models such as BERT can be divided into two categories: 1. Directly use official supervised datasets for training; 2. Construct samples from unmarked data, pre-train the model, and then fine-tune the pre-trained model with supervised datasets. These methods draw on the success of BERT in NLP tasks, and through pre-training and fine-tuning, the performance of Chinese spelling error correction models based on sequence prediction is improved.

[0003] However, the existing technology has the following defects: on the one hand, the existing text error correction method lacks specificity and cannot effectively correct errors in the context of official document writing; on the other hand, the Chinese spelling error correction method based on sequence prediction cannot handle the case of multiple or fewer words in a sentence. SUMMARY

[0004] Therefore, the technical problem to be solved by the present application is to overcome the technical defects of the existing Chinese text error correction method in the context of official document writing, which has poor error correction ability and cannot handle multiple or fewer word errors in a sentence.

[0005] To solve the above technical problems, the present application provides a method for optimizing Chinese text error correction in the context of official document writing, comprising the following steps:

[0006] S1, collect and obtain an original dataset, the original dataset comprising a plurality of sentences;

[0007] S2, perform a word segmentation operation on the sentences in the original dataset to obtain a plurality of segmented words, and count the frequency of occurrence of each segmented word to obtain a high-frequency word table, wherein the high-frequency word table comprises segmented words with a frequency exceeding a predetermined frequency;

[0008] S3, extend the high-frequency word table to obtain a high-frequency word table capable of handling wrong characters, multiple words and fewer words, specifically comprising:

[0009] A few characters are added in front of each word in the high-frequency word table to obtain new segmentation, and the new segmentation is added to the original high-frequency word table.

[0010] A multi-character symbol is added in the high-frequency word table.

[0011] S4, make mistakes in multiple sentences in the original data set, set a preset error ratio of multiple words, few words and wrong words, obtain multiple error sentences, the error sentences and the sentences in the original data set form error-correct sentence pairs, pre-train the BERT encoder according to the error-correct sentence pairs and the high-frequency word table, and update the parameters of the BERT encoder.

[0012] S5, collect official document writing word groups from the network to form an official document writing word table; and crawl multiple correct sentences on the official document website to build an official document writing data set.

[0013] S6, make mistakes in the correct sentences in the official document writing data set in combination with the official document writing word table to obtain multiple error official document writing sentences, the error official document writing sentences and the correct sentences form error-correct sentence pairs, and train the BERT encoder again according to the error-correct sentence pairs and the high-frequency word table to optimize the BERT encoder.

[0014] S7, using the optimized BERT encoder to predict the Chinese text to be processed to obtain the error-corrected text output.

[0015] As preferred, in S4, making mistakes in multiple sentences in the original data set comprises:

[0016] The original data set is made mistakes using an error consistency strategy.

[0017] As preferred, in S6, making mistakes in the correct sentences in the official document writing data set comprises:

[0018] The correct sentences in the official document writing data set are made mistakes by random mistake making and mistake making in combination with the official document word table.

[0019] As preferred, the ratio of random mistake making to mistake making in combination with the official document word table is 1:1.

[0020] As preferred, in S6, making mistakes in the correct sentences in the official document writing data set comprises:

[0021] The correct sentences in the official document writing data set are made mistakes in multiple words, few words and wrong words.

[0022] As preferred, the original data set in S1 is derived from Wikipedia.

[0023] As preferred, the few character symbol in S3 is ADD.

[0024] As preferred, the many character symbol in S3 is underscore.

[0025] The application discloses a kind of optimization official document writing scene under Chinese text error correction system, comprising:

[0026] Original data set acquisition module, the original data set acquisition module is used to collect and obtain original data set, the original data set includes multiple sentences;

[0027] Word segmentation module, the word segmentation module is used to carry out word segmentation operation to the sentence in the original data set, obtains multiple word segmentation, the frequency of occurrence of each word segmentation is counted, and high-frequency word table is obtained, wherein the high-frequency word table includes more than the preset word frequency number Word segmentation;

[0028] Word table expansion module, the word table expansion module is used to expand the high-frequency word table, obtain the high-frequency word table capable of processing wrong characters, multiple characters and few characters, specifically including: adding a few character symbols in front of each word segmentation in the high-frequency word table, to obtain new word segmentation, and adding new word segmentation to the original high-frequency word table;Multiple character symbols are added in the high-frequency word table;

[0029] First training module, the first training module is used to make mistake to multiple sentences in original data set, set multiple words, few words and wrong characters of preset error proportion, obtain multiple error sentences, the error sentence and the sentence in original data set form error-correct sentence pair, according to error-correct sentence pair and high-frequency word table, pretrain BERT encoder, and update the parameters of BERT encoder;

[0030] Official document writing word group and data set acquisition unit, the official document writing word group and data set acquisition unit are used to collect official document writing word group from network, to constitute official document writing word table;A plurality of correct sentences on official document website are crawled, to build official document writing data set;

[0031] Second training module, the second training module is used to make mistake to correct sentence in the official document writing data set in combination with official document writing word table, to obtain multiple error official document writing sentences, the error official document writing sentence and correct sentence form error-correct sentence pair, according to error-correct sentence pair and high-frequency word table, train BERT encoder again, and optimize BERT encoder;

[0032] Output unit, the output unit is used to predict the Chinese text to be processed using the optimized BERT encoder, to obtain the text output after error correction.

[0033] The application discloses a processor used for running a program, wherein the program performs the above method when running.

[0034] The above technical solution of the application has the following advantages compared with the prior art:

[0035] 1. In view of the problem of lack of official document writing data in the Chinese spelling correction task, the application crawls official document website data and combines the official document vocabulary to make speciality errors and speciality pre-training, so that the pre-training model learns the characteristics existing in the official document writing text, thereby achieving learning and correction of the wrong characters appearing in the official document writing scene.

[0036] 2. In the Chinese spelling correction method of sequence prediction, the problem of multi-word and few-word errors cannot be solved, the application adds special characters in the vocabulary to improve the ability of the model to solve these two errors. BRIEF DESCRIPTION OF DRAWINGS

[0037] Figure 1 The flowchart of the method for optimizing Chinese text correction in the official document writing scene in the application;

[0038] Figure 2 The construction process of the vocabulary;

[0039] Figure 3 The error correction process of the BERT encoder;

[0040] Figure 4 The process of constructing the "error-correct" sentence pair;

[0041] Figure 5 The process diagram of the continuous learning of the BERT encoder;

[0042] Figure 6 The pre-training process of the BERT encoder in the official document writing scene. DETAILED DESCRIPTION

[0043] The application will be further described below in combination with the drawings and specific embodiments, so that those skilled in the art can better understand the application and implement it, but the embodiments are not limiting to the application.

[0044] Referring to Figure 1 The application discloses a method for optimizing Chinese text correction in the official document writing scene, which comprises the following steps:

[0045] S1, collect the original data set, and the original data set includes a plurality of sentences.

[0046] The original data set is derived from Wikipedia.

[0047] S2, a word segmentation operation is performed on the sentences in the original data set to obtain a plurality of word segments, and the frequency of occurrence of each word segment is counted to obtain a high-frequency word table, wherein the high-frequency word table includes word segments with a frequency exceeding a preset frequency.

[0048] S3, the high-frequency word table is extended to obtain a high-frequency word table capable of processing wrong words, multiple words and few words, specifically including:

[0049] A few character symbol is added in front of each word segment in the high-frequency word table to obtain a new word segment, and the new word segment is added to the original high-frequency word table; a multiple character symbol is added in the high-frequency word table.

[0050] In S3, the few character symbol is ADD. In S3, the multiple character symbol is an underscore.

[0051] S4, a plurality of sentences in the original data set are made wrong, a plurality of words, a few words and wrong words with a preset error ratio are set, a plurality of wrong sentences are obtained, the wrong sentences and the sentences in the original data set form a wrong-correct sentence pair, and the BERT encoder is pre-trained according to the wrong-correct sentence pair and the high-frequency word table to update the parameters of the BERT encoder.

[0052] Among them, making wrong to a plurality of sentences in the original data set includes: using an error consistency strategy to make wrong to a plurality of sentences in the original data set.

[0053] S5, collect official document writing word groups from the network to form an official document writing word table; crawl a plurality of correct sentences on the official document website to build an official document writing data set.

[0054] S6, combine the official document writing word table to make wrong to the correct sentences in the official document writing data set to obtain a plurality of wrong official document writing sentences, and the wrong official document writing sentences and the correct sentences form a wrong-correct sentence pair, and the BERT encoder is trained again according to the wrong-correct sentence pair and the high-frequency word table to optimize the BERT encoder.

[0055] Among them, making wrong to the correct sentences in the official document writing data set includes: making wrong to the correct sentences in the official document writing data set by random wrong and combining the official document word table. The proportion of random wrong and combining the official document word table is 1:1. Making wrong to the correct sentences in the official document writing data set includes: making wrong to the correct sentences in the official document writing data set by multiple words, few words and wrong words.

[0056] S7, using the optimized BERT encoder to predict the Chinese text to be processed to obtain the corrected text output.

[0057] The application also discloses a Chinese text error correction system optimized in a government document writing scenario, which comprises an original data set acquisition module, a word segmentation module, a word table expansion module, a first training module, a government document writing word group and data set acquisition unit, a second training module and an output unit.

[0058] The original data set acquisition module is used for collecting original data sets, and the original data sets comprise a plurality of sentences.

[0059] The word segmentation module is used for performing word segmentation on the sentences in the original data set to obtain a plurality of segmented words, and the frequency of occurrence of each segmented word is counted to obtain a high-frequency word table, wherein the high-frequency word table comprises segmented words with a frequency exceeding a preset frequency.

[0060] The word table expansion module is used for expanding the high-frequency word table to obtain a high-frequency word table capable of processing wrong characters, multiple characters and few characters, and specifically comprises: adding a few character symbol in front of each segmented word in the high-frequency word table to obtain new segmented words, and adding the new segmented words to the original high-frequency word table; and adding a multiple character symbol in the high-frequency word table.

[0061] The first training module is used for making mistakes in the plurality of sentences in the original data set, setting a plurality of words, a few words and wrong characters with a preset error ratio, obtaining a plurality of error sentences, and combining the error sentences and the sentences in the original data set to form error-correct sentence pairs, pre-training a BERT encoder according to the error-correct sentence pairs and the high-frequency word table, and updating the parameters of the BERT encoder.

[0062] The government document writing word group and data set acquisition unit is used for collecting government document writing word groups from the network to form a government document writing word table, and crawling a plurality of correct sentences on a government document website to build a government document writing data set.

[0063] The second training module is used for making mistakes in the correct sentences in the government document writing data set in combination with the government document writing word table to obtain a plurality of error government document writing sentences, combining the error government document writing sentences and the correct sentences to form error-correct sentence pairs, and training the BERT encoder again according to the error-correct sentence pairs and the high-frequency word table to optimize the BERT encoder.

[0064] The output unit is used for predicting a Chinese text to be processed using the optimized BERT encoder to obtain an error-corrected text output.

[0065] In the following, the technical solutions in the application are further described and explained in combination with specific embodiments.

[0066] (1) First, a large amount of unlabeled data is artificially collected, and about 38 million sentences are obtained after cleaning. A sentence example is as follows: Figure 2"①" in it. Then, these 38 million sentences are segmented and the word frequencies are counted. For example, for the sentence "The weather has turned cold recently.", the present invention segments it into "最", "近", "天", "气", "变", "冷", "了", "。". All sentences go through similar segmentation operations and the word frequencies are counted. Eventually, a word frequency table like Figure 1 "②" in it will be obtained. However, in order to avoid reducing the training speed of the model and without affecting the error correction performance of the model, the present invention does not need to retain all the segmented words. The present invention only retains the 6,000 segmented words with the highest word frequencies, Figure 2 "③" in it is the word list containing 6,000 segmented words.

[0067] The present invention only retains 6,000 Chinese segmented words, and there are two considerations for doing so: on the one hand, the number of segmented words in the word list has a great impact on the training speed of the model. The larger the word list, the slower the model training speed. Therefore, only the 6,000 segmented words with the highest word frequencies are retained; on the other hand, these 6,000 segmented words with high word frequencies are sufficient to cover commonly used Chinese characters. Therefore, there is no need to retain those Chinese characters with too low occurrence frequencies to avoid reducing the performance of the model.

[0068] The schematic diagram of the error correction of the BERT encoder using the 6,000-segmented-word list in the present invention is as Figure 3As shown. The BERT encoder needs to use a vocabulary both at the input and output. For example, when tokenizing the input sentence "The weather has turned cold recently.", the tokenization results are: "最", "进", "天", "气", "变", "冷", "了", "。", and all these tokens can be found in the vocabulary. After being corrected by BERT, the results are: "最", "近", "天", "气", "变", "冷", "了", "。", and all these tokens exist in the vocabulary. Specifically, the BERT encoder finally predicts the probability distribution of these 6000 tokens. Usually, the present invention will select the token with the highest probability as the final prediction result. For example, for the correction of the first character "最", if the final prediction result of BERT is still "最", it means that among these 6000 tokens, the probability of the token "最" is the highest, so it is selected as the prediction result of the first input token "最", and so on. When the BERT encoder corrects "进", it will predict that the current token should be modified to "近" by combining the context, so among the probability distributions of these 6000 words, "近" is the highest. This demonstrates the ability of the BERT encoder to correct typos. However, since there are no tokens representing extra or missing characters in the vocabulary, the BERT encoder cannot correct the errors of extra or missing characters in the sentence. Therefore, the present invention came up with a method of expanding the vocabulary to be able to correct the errors of extra and missing characters. The specific expansion method is as follows: Ⅰ For extra characters: Add an "_" to the vocabulary; Ⅱ For missing characters: Add an "ADD" before the 6000 Chinese tokens, such as "ADD的", "ADD一", etc., and add them to the vocabulary, such as Figure 2 "④" in, and the final vocabulary size is 12001. If a certain token in the input is predicted by the BERT encoder as "ADD的", it means that there is a missing "的" character before this token in the original sentence; if it is predicted as "_", it means that this token in the original sentence is redundant, thus achieving the correction of the errors of missing and extra characters.

[0069] (2) The error correction ability of the BERT encoder is not inherent from the beginning, but is trained through the "error - correct" sentence pairs constructed by the present invention. First, input the wrong sentence and tokenize it, and the model will output the prediction result. Then, calculate the error between the prediction result and the correct result for backpropagation, thereby updating the model parameters to make the model gradually have the error correction ability. The construction strategy is based on error consistency, artificially set the proportions of the occurrences of extra, missing, and typo errors, and use the method of automatically creating errors to process the unlabeled data. These data are used for the pre - training of BERT, thus obtaining a model that initially has the ability to correct typo, extra, and missing character errors. Figure 4 Shows the process of constructing the "error - correct" sentence pairs, Figure 5The pre-training process of the BERT encoder is illustrated to correct errors and update weight parameters.

[0070] (3) The model M1 trained in step (2) has the ability to correct errors of wrong characters, multiple characters and few characters. However, since the 380 million training corpus is in a general field, the error correction ability of the M1 model in the context of official document writing is lacking. Therefore, the present application further crawls about 390,000 correct sentences from an official document website, and artificially creates errors in combination with an official document vocabulary (the size of the vocabulary is about 1 thousand, note: the segmented words in the official document writing vocabulary are meaningful word groups, not single Chinese characters). The error ratio of ordinary error creation to error creation in combination with the official document vocabulary is 1:1. Then, the model M1 is further trained using the 390,000 "error-correct" sentence pairs until convergence, and the model M2 is obtained. Examples of creating errors in combination with the official document vocabulary for official document texts are as follows: Example 1: "Preliminary build artificial intelligence technology standards, service system and industry ecological chain.", among them, "service system", "industry", "ecological chain" are in the official document vocabulary, then the present application replaces these words with wrong characters, adds characters and deletes characters in proportion. For example: delete the character "chain" from "ecological chain" to get "ecology", and the "error-correct" sentence pair after error creation is (Preliminary build artificial intelligence technology standards, service system and industry ecology., Preliminary build artificial intelligence technology standards, service system and industry ecology ADD chain). The pre-training model of the present application learns that "ecology" is missing a "chain" character, and modifies "." to "ADD chain", indicating that "." is missing a "chain" character in front of it; Example 2: "The teacher is teaching the students carefully.", among them, "teacher", "student", "class" are in the official document vocabulary, the present application creates an error for "class" as "science", and the "error-correct" sentence pair after error creation is (The teacher is teaching the students carefully., The teacher is teaching the students ADD science.). The pre-training process of the model in the context of official document writing is as shown in Figure 6 .

[0071] Through the above-mentioned pre-training method, we take into account the wrong characters, multiple characters and few characters in the text errors, and consider the specificity of the pre-training model. Finally, the above-mentioned trained model can be used to correct the downstream official document texts containing wrong characters, multiple characters and few characters.

[0072] The present application discloses a processor for running a program, wherein the program performs the above-mentioned method when running.

[0073] Those skilled in the art will appreciate that embodiments of the application can be devised for a method, a system, or a computer program product. Accordingly, the present application can be embodied in the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present application can take the form of a computer program product on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROMs, optical storage devices, etc.) embodying computer readable program code.

[0074] The present application is described in reference to the flowchart illustrations and / or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart illustrations and / or block diagrams. Figure 1 one or more functions specified in the flowchart illustrations and / or block diagrams. Figure 1 one or more functions specified in the flowchart illustrations and / or block diagrams.

[0075] These computer program instructions can also be stored in a computer- readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instructions which implement the functions specified in the flowchart illustrations and / or block diagrams. Figure 1 one or more functions specified in the flowchart illustrations and / or block diagrams. Figure 1 one or more functions specified in the flowchart illustrations and / or block diagrams.

[0076] These computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart illustrations and / or block diagrams. Figure 1 one or more functions specified in the flowchart illustrations and / or block diagrams. Figure 1 one or more functions specified in the flowchart illustrations and / or block diagrams.

[0077] Obviously, the above-described embodiments are only examples and are not intended to limit the present application. Based on the above description, those skilled in the art can make other variations and modifications of the present application without departing from the present application. Neither requiring nor intending to limit the present application to the exact forms of implementations shown and described, it is to be understood that the application is capable of non-limiting embodiments in its broader aspects.

Claims

1. A method for optimizing Chinese text error correction in a bureaucratic writing scenario, characterized in that, The method comprises the following steps: S1, collecting an original data set, wherein the original data set comprises a plurality of sentences; S2, performing a word segmentation operation on the sentences in the original data set to obtain a plurality of segmented words, and counting the frequency of occurrence of each segmented word to obtain a high-frequency word table, wherein the high-frequency word table comprises segmented words with a frequency exceeding a preset frequency; S3, expanding the high-frequency word table to obtain a high-frequency word table capable of processing errors, multiple words, and few words, specifically comprising: adding a few character symbol in front of each segmented word in the high-frequency word table to obtain new segmented words, and adding the new segmented words to the original high-frequency word table; adding a multiple character symbol in the high-frequency word table; S4, making errors in the plurality of sentences in the original data set, setting a preset error ratio of multiple words, few words, and errors, obtaining a plurality of error sentences, and the error sentences and the sentences in the original data set forming error-correct sentence pairs, and pre-training a BERT encoder according to the error-correct sentence pairs and the high-frequency word table to update the parameters of the BERT encoder; S5, collecting official document writing word groups from the network to form an official document writing word table, and crawling a plurality of correct sentences on an official document website to build an official document writing data set; S6, making errors in the correct sentences in the official document writing data set in combination with the official document writing word table to obtain a plurality of error official document writing sentences, wherein the error official document writing sentences and the correct sentences form error-correct sentence pairs, and the BERT encoder is trained again according to the error-correct sentence pairs and the high-frequency word table to optimize the BERT encoder; when the BERT encoder is trained again according to the error-correct sentence pairs and the high-frequency word table, firstly, the error sentences are input and segmented, and the model outputs a prediction result; then, the error between the prediction result and the correct result is calculated for back propagation; S7, using the optimized BERT encoder to predict a Chinese text to be processed to obtain a corrected text output.

2. The method for correcting Chinese text errors in the scenario of optimizing official writing according to claim 1, characterized in that, In S4, making errors in the plurality of sentences in the original data set comprises: using an error consistency strategy to make errors in the plurality of sentences in the original data set.

3. The method for optimizing Chinese text error correction in a bureaucratic writing scenario according to claim 1, wherein, In S6, making errors in the correct sentences in the official document writing data set comprises: making errors in the correct sentences in the official document writing data set in a random error making manner and a combination of official document word table error making manner.

4. The method of claim 3, wherein, The ratio of the random error making manner to the combination of official document word table error making manner is 1:

1.

5. The method for optimizing Chinese text error correction in a bureaucratic writing scenario according to claim 1, wherein, In S6, making errors in the correct sentences in the official document writing data set comprises: making errors in the correct sentences in the official document writing data set in a multiple word, few word, and error making manner.

6. The method for optimizing Chinese text error correction in a bureaucratic writing scenario according to claim 1, wherein, The original data set in S1 is derived from Wikipedia.

7. The method for optimizing Chinese text error correction in a bureaucratic writing scenario according to claim 1, wherein, The few character symbol in S3 is ADD, and the multiple character symbol in S3 is an underscore.

8. A Chinese text error correction system for optimizing a document writing scenario, characterized in that, Comprise: an original data set acquisition module, the original data set acquisition module is used for collecting an original data set, and the original data set comprises a plurality of sentences; The word segmentation module is configured to perform a word segmentation operation on the sentences in the original data set to obtain a plurality of segmented words, count the frequency of occurrence of each segmented word, and obtain a high-frequency word table, wherein the high-frequency word table includes segmented words with a frequency exceeding a preset frequency number; The word table expansion module is configured to expand the high-frequency word table to obtain a high-frequency word table capable of processing errors, multiple words, and few words, specifically including: adding a few character symbol in front of each segmented word in the high-frequency word table to obtain new segmented words, and adding the new segmented words to the original high-frequency word table; and adding a multiple character symbol to the high-frequency word table; The first training module is configured to make errors in the plurality of sentences in the original data set, set a preset error ratio of multiple words, few words, and errors, obtain a plurality of error sentences, combine the error sentences and the sentences in the original data set to form error-correct sentence pairs, pre-train the BERT encoder according to the error-correct sentence pairs and the high-frequency word table, and update the parameters of the BERT encoder; The official document writing word group and data set acquisition unit is configured to collect official document writing word groups from the network to form an official document writing word table, and crawl a plurality of correct sentences on an official document website to build an official document writing data set; The second training module is configured to make errors in the correct sentences in the official document writing data set in combination with the official document writing word table to obtain a plurality of error official document writing sentences, combine the error official document writing sentences and the correct sentences to form error-correct sentence pairs, and re-train the BERT encoder according to the error-correct sentence pairs and the high-frequency word table to optimize the BERT encoder; when re-training the BERT encoder according to the error-correct sentence pairs and the high-frequency word table, first, the error sentences are input and segmented, and the model outputs a prediction result; then, the error between the prediction result and the correct result is calculated to perform back propagation; The output unit is configured to use the optimized BERT encoder to predict a Chinese text to be processed to obtain an error-corrected text output.

9. A processor, comprising: The processor is configured to run a program, wherein the program performs the method of any one of claims 1-7 when running. The processor is configured to run a program, wherein the program performs the method of any one of claims 1-7 when running.

Images