Chinese grammar error correction method based on detection and pinyin joint enhancement

Abstract

The invention relates to a Chinese grammar error correction method based on detection and pinyin joint enhancement, and belongs to the technical field of natural language processing. According to the method, firstly, an end-to-end grammar error correction basic model is constructed by utilizing an original pre-training language model BART, and the model can improve the error correction performance by utilizing the strong language representation capability of the BART. And then, after the coding stage of the BART model is finished, an error detection layer is added, and the detection layer aims to alleviate the problem of over-correction of the model by accurately identifying errors. Meanwhile, a private pinyin encoder is used for learning pinyin information representation of characters, the representation is combined with text information representation through a cross attention mechanism, and then text representation fused with pinyin information is obtained. And finally, a self-adaptive gating mechanism is adopted to filter out harmful components in the pinyin information, so that the accuracy and the reliability of the finally obtained text information representation are ensured.

Description

[0001] Technical Language

[0002] The invention relates to a Chinese grammar error correction method based on detection and pinyin joint enhancement, belonging to the technical field of natural language processing. Background Art

[0003] Chinese grammar correction is an important topic in natural language processing. Its core goal is to detect and correct grammatical irregularities in Chinese texts. Such irregularities may involve errors in word order, improper part-of-speech collocation, or problems with sentence structure. These errors will significantly affect the clarity and readability of the text. Therefore, the demand for Chinese grammar correction technology is gradually increasing.

[0004] In the pursuit of improving the accuracy and efficiency of Chinese grammar correction, it is extremely important to build efficient correction models. These models can automatically detect and fix grammatical problems in texts. At the same time, given the popularity of Chinese texts in many fields and different application scenarios, the correction models must also be able to adapt to various professional backgrounds and usage scenarios to cater to the specific needs of different users and scenarios.

[0005] The recognition of Chinese grammatical errors involves automatically determining inaccuracies in natural language sentences, such as missing or redundant components, inappropriate word order, etc. This task usually includes confirming the presence of errors, classifying them, and accurately locating them. Proper use of grammar detection technology can significantly enhance the effect of error correction.

[0006] In summary, it can be seen that Chinese grammar correction technology plays an irreplaceable role in improving the overall quality of text, optimizing user experience, and meeting the requirements of diversified applications. Grammar detection technology, as the core of achieving this goal, not only ensures the accuracy and professionalism of the text, but also reduces misunderstandings and communication barriers. In addition, it can help non-native speakers improve their writing skills and promote language education. In the field of natural language processing, it also improves the accuracy of technologies such as machine translation and speech recognition. Looking to the future, as technology continues to evolve, grammar detection technology will continue to lead the innovative development in the field of Chinese grammar correction. Summary of the invention

[0007] The present invention provides a Chinese grammar correction method based on detection and pinyin joint enhancement to solve the problem of low accuracy of Chinese grammar correction. The present invention has achieved excellent experimental results on the MuCGEC dataset and the Chinese grammar correction task.

[0008] The technical solution of the present invention is: a Chinese grammar error correction method based on detection and pinyin joint enhancement, the specific steps of the method are as follows:

[0009] Step 1. Select the pre-trained model BART as the basic model architecture, which is suitable for the needs of this task. After selection, test the effect of the basic model. In order to ensure the performance of the model, we specially use the Chinese grammar error correction dataset as the training set, validation set and test set to comprehensively evaluate the accuracy and robustness of the model;

[0010] Step 2: Collect and organize all Chinese pinyin and build a Chinese pinyin dictionary that contains all pinyin information. The purpose of building this dictionary is to provide accurate pinyin data support for subsequent models. ;

[0011] Step 3: Optimize and organize the basic model framework, and add a pinyin encoding module on this basis. The purpose of designing this module is to enable the model to effectively learn and understand pinyin information, thereby improving the overall performance of the model;

[0012] Step 4: After the pinyin encoding module, a cross attention mechanism is introduced, which is responsible for fusing pinyin information with text information. At the same time, in order to improve the purity of information processing, the present invention adds an adaptive gating mechanism to filter out harmful information that may exist in the fused text information;

[0013] Step 5: Comprehensively train and optimize the model that integrates pinyin encoding and adaptive gating mechanism. By adding a detection layer at the text encoding end, jointly optimize and adjust the model parameters to achieve the best performance indicators.

[0014] As a further solution of the present invention, the specific steps of Step 1 are as follows:

[0015] Step 1.1. First, obtain the open-source and pre-trained Chinese BART-large model, which adopts the encoding-end to decoding-end architecture design and is suitable for processing Chinese text;

[0016] Step 1.2. Next, collect and obtain open access datasets, including the Lang8 dataset, the HSK dataset, and the NLPCC2018 dataset, which will be used to train and evaluate the model;

[0017] Step 1.3: Preprocess the Lang8 and HSK datasets to remove duplicate data and data with too much noise to ensure data quality. In addition, the HSK dataset is repeated five times and integrated into the Lang8 dataset to construct the final training corpus, which enhances the diversity of training data and the generalization ability of the model.

[0018] Step 1.4. Taking the BART architecture as the base model, conduct preliminary tests on the MuCGEC dataset to obtain the benchmark performance results of the model.

[0019] As a further solution of the present invention, Step 2 includes the following:

[0020] Step 2.1. Collect all possible Chinese pinyin representations from the Internet and build a basic pinyin dictionary V p1 The dictionary contains all possible pinyin, providing comprehensive pinyin data support for subsequent processing;

[0021] Step 2.2: Use the public pinyin conversion tool Pypinyin to convert all Chinese characters in the training set into their corresponding pinyin representations P. By removing duplicate pinyin representations in the training corpus and comparing them with the basic pinyin dictionary P constructed in step 2.1, v1 Merge, and finally get a complete dictionary P containing all pinyin v .

[0022] Step 2.3, the final constructed pinyin dictionary P v According to the similarity of pinyin, similar pinyin are placed in the adjacent position in the dictionary to enhance the model's ability to learn and distinguish similar pinyin.

[0023] As a further solution of the present invention, the specific steps of Step 3 are as follows:

[0024] Step 3.1. For the input sentence X, first use the original BART dictionary to segment the words, and then use Pypinyin to segment each character x in the input sentence. i Converted into the corresponding pinyin id to represent P i ;

[0025] Step 3.2, add a pinyin embedding layer to the basic model, and use the pinyin embedding layer to represent the sparse pinyin id P i Mapping to dense phonetic vector representation

[0026] Step 3.3, the dense pinyin vector obtained in the previous step As the input of the private long short-term memory network, the pinyin context-related word representation is obtained through the private long short-term memory network encoding.

[0027] As a further solution of the present invention, the specific steps of Step 4 are as follows:

[0028] Step 4.1, when obtaining the pinyin context-related word representation of Step 3.3 Then, the output of the text encoding end in the base model is Word representation related to phonetic context As the input of the multi-head attention mechanism, the pinyin information is integrated with the text information to obtain the text representation with a large amount of pinyin information.

[0029] Step 4.2: Represent the text with a lot of pinyin information obtained in the previous step Through adaptive gating, harmful information is filtered out to obtain the final text representation h with pinyin information. i By using the final text information to represent h i As the input of the decoding end, it improves the ability of the grammatical error correction model to utilize pinyin information.

[0030] As a further solution of the present invention, the specific steps of Step 5 are as follows:

[0031] Step 5.1: Output the text encoding end in the basic model After regularization and linear changes, the model overfitting rate is reduced and discrete text representation is obtained

[0032] Step 5.2: Discrete text representation As the input of a private bidirectional long short-term memory network, the private bidirectional long short-term memory network captures possible erroneous information between text information and obtains a vector representation with detection information

[0033] Step 5.3. Vector representation with error information After the classification layer, the dense vector representation is mapped into sparse detection information

[0034] Step 5.4. Try to use conditional random fields to detect information Modeling, optimize the text encoding end by calculating the negative log-likelihood function through conditional random fields.

[0035] The beneficial effects of the present invention are:

[0036] 1. The present invention first uses the original pre-trained language model BART model to build an end-to-end grammatical error correction basic model, which can use the powerful language representation ability of the BART model to improve the error correction performance.

[0037] 2. After the encoding phase of the BART model, an error detection layer is added to mitigate the problem of over-correction of the model by accurately identifying errors. At the same time, the present invention uses a private pinyin encoder to learn the pinyin information representation of characters, and combines this representation with the text information representation through a cross-attention mechanism to obtain a text representation that incorporates the pinyin information.

[0038] 3. Adopt adaptive gating mechanism to filter out harmful components in pinyin information to ensure the accuracy and reliability of the final text information representation.

[0039] 4. The present invention integrates pinyin information into the grammar correction model to improve the model's ability to understand pinyin; the present invention constructs a Chinese grammar detection data set using a dynamic programming algorithm; the present invention uses a detection model to alleviate the problem of over-correction of the model; the present invention uses pinyin information and detection tasks to assist the model in error correction, effectively improving the accuracy of the model in correcting problems. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 It is a flow chart of the present invention. DETAILED DESCRIPTION

[0041] Example 1: Figure 1 As shown, a Chinese grammar correction method based on detection and pinyin joint enhancement, the method includes: Step 1, select the pre-trained model BART as the basic model architecture, the model is suitable for the needs of this task. After the selection, the basic model is tested for its effect. In order to ensure the performance of the model, we specially use the Chinese grammar correction dataset as the training set, validation set and test set to comprehensively evaluate the accuracy and robustness of the model;

[0042] Step 2: Collect and organize all Chinese pinyin and build a Chinese pinyin dictionary that contains all pinyin information. The purpose of building this dictionary is to provide accurate pinyin data support for subsequent models. ;

[0043] Step 3: Optimize and organize the basic model framework, and add a pinyin encoding module on this basis. The purpose of designing this module is to enable the model to effectively learn and understand pinyin information, thereby improving the overall performance of the model;

[0044] Step 4: After the pinyin encoding module, a cross attention mechanism is introduced, which is responsible for fusing pinyin information with text information. At the same time, in order to improve the purity of information processing, the present invention adds an adaptive gating mechanism to filter out harmful information that may exist in the fused text information;

[0045] Step 5: Comprehensively train and optimize the model that integrates pinyin encoding and adaptive gating mechanism. By adding a detection layer at the text encoding end, jointly optimize and adjust the model parameters to achieve the best performance indicators.

[0046] As a further solution of the present invention, the specific steps of Step 1 are as follows:

[0047] Step 1.1. First, obtain the open-source and pre-trained Chinese BART-large model, which adopts the encoding-end to decoding-end architecture design and is suitable for processing Chinese text;

[0048] Step 1.2. Next, collect and obtain open access datasets, including the Lang8 dataset, the HSK dataset, and the NLPCC2018 dataset, which will be used to train and evaluate the model;

[0049] Step 1.3: Preprocess the Lang8 and HSK datasets to remove duplicate data and data with too much noise to ensure data quality. In addition, the HSK dataset is repeated five times and integrated into the Lang8 dataset to construct the final training corpus, which enhances the diversity of training data and the generalization ability of the model.

[0050] Step 1.4. Taking the BART architecture as the base model, conduct preliminary tests on the MuCGEC dataset to obtain the benchmark performance results of the model.

[0051] As a further solution of the present invention, Step 2 includes the following:

[0052] Step 2.1. Collect all possible Chinese pinyin representations from the Internet and build a basic pinyin dictionary V p1 The dictionary contains all possible pinyin, providing comprehensive pinyin data support for subsequent processing;

[0053] Step 2.2: Use the public pinyin conversion tool Pypinyin to convert all Chinese characters in the training set into their corresponding pinyin representations P. By removing duplicate pinyin representations in the training corpus and comparing them with the basic pinyin dictionary P constructed in step 2.1, v1 Merge, and finally get a complete dictionary P containing all pinyin v .

[0054] Step 2.3, the final constructed pinyin dictionary P v According to the similarity of pinyin, similar pinyin are placed in the adjacent position in the dictionary to enhance the model's ability to learn and distinguish similar pinyin.

[0055] As a further solution of the present invention, the specific steps of Step 3 are as follows:

[0056] Step 3.1. For the input sentence X, first use the original BART dictionary to segment the words, and then use Pypinyin to segment each character x in the input sentence. i Converted into the corresponding pinyin id to represent Pi ;

[0057] Step 3.2, add a pinyin embedding layer to the basic model, and use the pinyin embedding layer to represent the sparse pinyin id P i Mapping to dense phonetic vector representation

[0058] Step 3.3, the dense pinyin vector obtained in the previous step As the input of the private long short-term memory network, the pinyin context-related word representation is obtained through the private long short-term memory network encoding.

[0059] As a further solution of the present invention, the specific steps of Step 4 are as follows:

[0060] Step 4.1, when obtaining the pinyin context-related word representation of Step 3.3 Then, the output of the text encoding end in the base model is Word representation related to phonetic context As the input of the multi-head attention mechanism, the pinyin information is integrated with the text information to obtain the text representation with a large amount of pinyin information.

[0061] Step 4.2: Represent the text with a lot of pinyin information obtained in the previous step Through adaptive gating, harmful information is filtered out to obtain the final text representation h with pinyin information. i By using the final text information to represent h i As the input of the decoding end, it improves the ability of the grammatical error correction model to utilize pinyin information.

[0062] As a further solution of the present invention, the specific steps of Step 5 are as follows:

[0063] Step 5.1: Output the text encoding end in the basic model After regularization and linear changes, the model overfitting rate is reduced and discrete text representation is obtained

[0064] Step 5.2, Discrete Text Representation As the input of a private bidirectional long short-term memory network, the private bidirectional long short-term memory network captures possible erroneous information between text information and obtains a vector representation with detection information

[0065] Step 5.3. Vector representation with error information After the classification layer, the dense vector representation is mapped into sparse detection information

[0066] Step 5.4. Try to use conditional random fields to detect information Modeling, optimize the text encoding end by calculating the negative log-likelihood function through conditional random fields.

[0067] Example 2: Figure 1 As shown, a Chinese grammar error correction method based on detection and pinyin joint enhancement, the specific steps of the method are as follows:

[0068] a1. Download the Chinese BART-large pre-trained model and the dataset of the grammatical error correction task and perform pre-processing.

[0069] 1) Download the Chinese BART-large pre-trained model and the Lang8, HSK, and MuCGEC datasets.

[0070] 2) The downloaded Lang8 and HSK data sets were preprocessed, mainly including deduplication, deletion of too long or too short sentence pairs, etc.; according to the data fusion method commonly used by predecessors, HSK was repeated five times and integrated into Lang8 as the training set of the present invention. The processing of the MuCGEC data set mainly includes deleting some data sets that cannot be labeled. Finally, 1568886 million relatively standardized grammar error correction training data were formed.

[0071] Table 1 Number of grammatical error correction datasets

[0072] Dataset name Number of datasets / sentences Lang8 1,220,906 HSK 156,870 MuCGEC-dev 1125 MuCGEC-test 6000 Lang8+HSK 1568886 NLPCC2018 2000

[0073] a2. First, collect all the pinyin representations and build a pinyin dictionary P V1 Next, we use the dynamic programming algorithm to build the grammar detection task label, then select BartForConditionalGeneration as the base model and the pre-trained model BART-large. Finally, we add the phonetic encoder, cross attention mechanism, adaptive gating and detection layer to the base model.

[0074] 1) Collect all pinyin representations and construct a pinyin dictionary P V1 .

[0075] 2) Since there is no ready-made custom corpus for error detection, a dynamic programming algorithm is designed to automatically label errors for our grammatical error correction dataset. For each source error sentence X and target correct sentence Y, we first construct a distance matrix D and a position matrix P, where D is initialized to 0 and P is initialized to (None, None). Specifically, D represents the source word x iand the target word y i The relative distance between them, and P represents the modification position between them. Then, we iterate over the sentence pairs (X, Y). If the source word x i With the target word y i Different, the cost value c is set to 1, otherwise it is set to 0. At the same time, the elements of D[i][j] are updated to the minimum of the following: D[i-1][j]+1, D[i][j-1]+1, and D[i-1][j-1]+c. Secondly, we update the elements of the position matrix P according to the distance matrix D and the cost value c. The following are the update rules: If D[i][j] is equal to D[i-1][j-1]+c, we set P[i][j] to (i-1, j-1). If D[i][j] is equal to D[i-1][j]+1, P[i][j] is updated to (i-1, j). Otherwise, P[i][j] is replaced with (i, j-1). Finally, we traverse the matrix P starting from P[n][m] to obtain the detection label. If P[i][j] is equal to (i-1, j-1) and D[i][j] is equal to D[i-1][j-1], the detection label l i is false. Otherwise, l i is marked as true.

[0076] 3) The BartForConditionalGeneration model consists of two parts: the encoder and the decoder, which work together to complete the task of grammatical error correction. The encoder uses a multi-head self-attention mechanism to contextually encode each word in the source sentence to generate a representation rich in contextual information. The decoder is similar in structure and additionally introduces a masked multi-head attention module to more effectively model the generated lexical information and ensure the grammatical and semantic correctness of the output sentence.

[0077] 4) Add a pinyin encoder to the basic model to obtain dense pinyin representation Then, a pinyin-specific bidirectional long short-term memory network is used to encode the contextual pinyin information representation. The specific formula is as follows:

[0078]

[0079] 5) When the contextual phonetic information is obtained, With contextual text information By using a multi-head cross-attention mechanism to integrate pinyin and text information, the specific formula is as follows:

[0080]

[0081]

[0082] Among them, W q ,W k ,W q ,W c is the weight matrix, d k Model embedding dimension. Finally, we filter out harmful information through adaptive gating to obtain the final text context representation:

[0083]

[0084] Among them, W1,b1 is the weight matrix. Then we get the final prediction result through the decoding end and softmax.

[0085] P(Y|X; θ)=softmax(W2decoder(y i-1 ,h i )+b2) (8)

[0086] y i-1 is the previous true value of the current predicted value. During the training process, the objective function is to minimize the cross entropy loss function, the formula is as follows:

[0087]

[0088] θ is a trainable model parameter, X = {x1, x2, …, x m} is a sentence with m word origins, Y = {y1, y2, ..., y n} is a correct sentence with n words, Y ＜t ={y1,y2,…,y t-1} is the word visible at the tth time step;

[0089] In the prediction stage, beam search decoding is used to maximize the conditional probability P(y * |x;θ) find an optimal sequence y * ;

[0090] 2) Modify the BartForConditionalGeneration basic model. The present invention adds a detection layer after the model. In order to train this newly added detection layer, the present invention uses CRF to set an independent loss function L for it. D , and add this loss function to the loss function of the original model to form a total loss function. Next, the present invention calculates the gradients of the total loss function and the original loss function to the model parameters, and uses these gradients to update the model parameters. The following is a description of the total loss function:

[0091]

[0092] in, is the detection layer loss function, is the error correction loss function.

[0093] 3) Use character-based Chinese grammar evaluation indicators. The evaluation indicators used for Chinese grammar error correction are precision P, recall R and F 0.5 , the calculation results are as follows:

[0094]

[0095] in:

[0096] TP (True Positive): The word that the model corrected from the wrong word.

[0097] FP (False Positive): The model changes the correct word to an incorrect word.

[0098] FN (False Negative): The model did not correct the wrong word.

[0099] 4) Use the Lang8+HSK text dataset as the training set to train the error correction model, adjust the hyperparameters, and obtain the detection model with the best performance.

[0100] a4. The best results were obtained by adjusting different hyperparameters on different validation sets.

[0101] 1) The final results are shown in Table 2. The publisher of the MuCGEC model released the MuCGEC dataset and integrated the results of the Seq2Seq and Seq2Edit models, significantly improving grammatical errors. SynGEC designed an error-adaptive dependency parser and integrated the parsed tree into the encoder, thereby enhancing the grammatical understanding ability of the traditional Seq2Seq model. Mixedit generates error samples by mixing and editing existing sentences to create enhanced data. It utilizes various traditional data augmentation strategies, namely direct noise, pattern noise, and back-translation, to enhance their effectiveness by combining them in a structured way. The MixEdit model is a representative work that uses the constructed pseudo corpus as external knowledge to improve the performance of the Seq2Seq model.

[0102] The present invention compares the method of the present invention with the above work. 0.5 The highest score is F. Although MuCGEC has the highest accuracy, it is the result of integrating six models and is not comparable as a single model. The final score of the model of the present invention is F. 0.5 Compared with the previous work, the score of the model of the present invention is greatly improved, which proves the effectiveness of the method of the present invention.

[0103] Table 2 Test results of the proposed model and other models on the MuCGEC test set and NLPCC2018

[0104]

[0105] At the same time, in order to verify whether the pinyin fusion module and the detection module are necessary, the present invention tests the performance of the detection module and the pinyin fusion model based on the Lang8+HSK data training of the basic model. The results are shown in Table 3. After adding the detection model, the accuracy of the correction result is significantly improved, proving that the detection module can alleviate the problem of over-correction. When both models are integrated into the basic model, the accuracy is improved by nearly 5 points. At the same time, compared with the basic model, after the detection module is introduced, F 0.5 After the introduction of the Pinyin fusion module, F 0.5 When the two modules jointly enhance the error correction model, F 0.5 The score is improved by nearly 2.57 points, which not only shows that the over-correction in the current grammatical error correction model has been alleviated, but also verifies the effectiveness of the joint constraint method adopted by the present invention.

[0106] Table 3 Test results of the basic model and the detection and pinyin enhancement models on the MuCGEC validation set

[0107]

[0108] In order to explore a more effective fusion method, the present invention has tested the following fusion methods: dot product means that the obtained fusion information is directly fused with the text information, original gating means selecting the forget gate and outputting them as the fusion method, and adaptive gating is the method selected by the present invention. The results are shown in Table 4, and the adaptive gating mechanism obtains the highest score. At the same time, it is also confirmed that the method proposed by the present invention can be used as a powerful method to improve the grammatical error correction performance.

[0109] Table 4 Test results of different methods of filtering harmful pinyin information on the MuCGEC validation set

[0110]

[0111] The specific implementation modes of the present invention are described in detail above in conjunction with the accompanying drawings, but the present invention is not limited to the above implementation modes, and various changes can be made within the knowledge scope of ordinary technicians in this field without departing from the purpose of the present invention.

Claims

1. A Chinese grammar error correction method based on detection and pinyin joint enhancement, characterized in that: The specific steps of the method include the following: Step 1. Select the pre-trained model BART as the basic model architecture, and use the Chinese grammar error correction dataset as the training set, validation set, and test set; Step 2, comprehensively collect and organize all Chinese pinyin, and build a Chinese pinyin dictionary that contains all pinyin information; Step 3: Optimize and organize the basic model framework, and add a pinyin encoding module on this basis, which is used to learn and understand pinyin information; Step 4: After the pinyin encoding module, a cross-attention mechanism is introduced, which is responsible for fusing the pinyin information with the text information. At the same time, an adaptive gating mechanism is added to filter out harmful information that may exist in the fused text information. Step 5: Comprehensively train and optimize the model that integrates pinyin encoding and adaptive gating mechanism, and jointly optimize and adjust the model parameters by adding a detection layer at the text encoding end to achieve the best performance indicators.

2. The Chinese grammar error correction method based on detection and phonetic joint enhancement according to claim 1 is characterized in that: The specific steps of Step 1 include the following: Step 1.

1. Obtain the open-source and pre-trained Chinese BART-large model, which has an encoder-decoder framework. Step 1.2, obtain the open access Lang8 dataset, HSK dataset and NlPCC2018 dataset; Step 1.3, preprocess the Lang8 dataset and the HSK dataset, remove the duplicate data and data with too much noise in the two datasets; and repeatedly integrate the HSK dataset into the Lang8 dataset several times as the final training corpus; Step 1.

4. Select the BART architecture as the basic model, test the basic model, and obtain the benchmark results.

3. The Chinese grammar error correction method based on detection and phonetic joint enhancement according to claim 1 is characterized in that: The specific steps of Step 2 include the following: Step 2.

1. Collect all possible pinyin representations P from the Internet and build a basic pinyin dictionary P v1 ; Step 2.2: Use the public tool Pypinyin to convert all Chinese characters in the training set into pinyin representations P. By removing duplicate pinyin representations in the training corpus and merging them with the basic pinyin dictionary obtained in Step 2.1, the final dictionary P containing all pinyin is obtained. v ; Step 2.3, the final dictionary P v All the pinyin representations in the image are sorted and arranged in close positions according to similar pinyins, which increases the model's ability to learn similar pinyins.

4. The Chinese grammar error correction method based on detection and phonetic joint enhancement according to claim 1 is characterized in that: The specific steps of Step 3 are as follows: Step 3.

1. Use Pypinyin to convert the input sentence into the corresponding pinyin representation P, and use the pinyin dictionary to convert each character in the input sentence into the corresponding pinyin id representation P i ; Step 3.2, add a pinyin embedding layer to the basic model, and use the pinyin embedding layer to represent the sparse pinyin id P i Mapping to dense phonetic vector representation Step 3.3, the dense pinyin vector obtained in the previous step As the input of the private long short-term memory network, the pinyin context-related word representation is obtained through the private long short-term memory network encoding.

5. The Chinese grammar error correction method based on detection and phonetic joint enhancement according to claim 1 is characterized in that: The specific steps of Step 4 are as follows: Step 4.1, when obtaining the pinyin context-related word representation of Step 3.3 Then, the output of the text encoding end in the base model is Word representation related to phonetic context As the input of the multi-head attention mechanism, the pinyin information is integrated with the text information to obtain the text representation with a large amount of pinyin information. Step 4.2: Represent the text with a lot of pinyin information obtained in the previous step Through adaptive gating, harmful information is filtered out to obtain the final text representation h with pinyin information. i ; By using the final text information to represent h i As the decoder input.

6. The Chinese grammar error correction method based on detection and phonetic joint enhancement according to claim 4 is characterized in that: The specific steps of Step 5 include the following: Step 5.1: Output the text encoding end in the basic model After regularization and linear changes, the model overfitting rate is reduced and discrete text representation is obtained Step 5.2: Discrete text representation As the input of a private bidirectional long short-term memory network, the bidirectional long short-term memory network captures possible erroneous information between text information and obtains a vector representation with detection information Step 5.

3. Vector representation with error information After the classification layer, the dense vector representation is mapped into sparse detection information Step 5.

4. Try to use conditional random fields to detect information Modeling, optimizing text encoder parameters by calculating negative log-likelihood function through conditional random fields.

Images