Chinese grammar error correction method and system based on large model fine tuning

Abstract

The invention relates to a Chinese grammar error correction method and system based on large model fine tuning. According to the method, a Qwen2-7B model is used for generating an error correction instruction data set; screening the instruction data set according to complexity, quality and diversity; performing supervised fine tuning on the Qwen2-7B model by adopting an LORA method, and emphatically optimizing a self-attention layer and a feedforward network layer so as to enhance the processing capability of the model on long-distance dependence and context information; then, a Qwen2-72B model is used for scoring the error correction data set, preference tags are generated, and the Qwen2-7B model is further optimized by using the tags, so that the response quality of the Qwen2-7B model is improved; a direct preference optimization method is used, a preference data set is combined to train the model, and parameters are adjusted to minimize a loss function, so that the tendency of the model to preference response is enhanced. According to the method, the accuracy of Chinese grammar error correction is effectively improved through a multi-level optimization strategy.

Description

[0001] Technical Language

[0002] The present invention relates to a Chinese grammar error correction method and system based on large model fine-tuning, belonging to the technical field of natural language processing. Background Art

[0003] As users become more casual in text input and voice input, grammatical errors are common in all kinds of applications. Especially in products such as text editing software, input method applications, and email applications, built-in grammar correction services have become an important part of improving user experience. An excellent grammar correction system can significantly improve user satisfaction. For example, in search query scenarios, search engines will correct queries entered by users to ensure accurate search results; the goal of Chinese grammar error correction is to automatically identify and correct potential errors in sentences while maintaining their original meaning. This not only reduces manual verification costs and saves human resources, but also provides basic support for other artificial intelligence applications, such as search queries, text recognition, voice recognition, writing assistance, and article review.

[0004] In recent years, with the breakthrough of deep learning technology in the field of text generation, the construction of Chinese grammar correction corpus has also made significant progress. The current grammar correction corpus mainly exists in the form of sentence pairs, including incorrect sentences and their corresponding correct sentences. However, the construction of large-scale Chinese grammar correction corpus faces challenges, mainly reflected in the need for rich language knowledge and background knowledge, and the high cost of manual annotation makes it difficult to obtain large-scale training data. This problem has become a key bottleneck in the research of Chinese grammar correction.

[0005] In order to solve the problem of shortage of large-scale training data, the present invention proposes a Chinese grammar correction method based on large model fine-tuning. The method first extracts sentences containing grammatical errors from a public Chinese grammar correction corpus (such as Lang8), and ensures the quality and diversity of the data through manual screening and annotation. Then, the data is processed using large models such as Qwen2-7B to generate a rich correction instruction data set. In this process, few-shot learning technology is used to improve the model's ability to recognize grammatical errors.

[0006] After the dataset is constructed, the method also introduces LORA (Low-Rank Adaptation) technology to fine-tune the model, focusing on optimizing the self-attention layer and the feedforward network layer to improve the model's ability to capture long-distance dependencies. At the same time, by constructing a preference dataset and applying the direct preference optimization (DPO) method, the quality of the error correction responses generated by the model is further improved. This multi-level optimization strategy not only enhances the model's ability to understand Chinese grammar, but also provides effective support for practical applications.

[0007] In general, the Chinese grammatical correction method based on large models aims to simulate the grammatical errors that users are prone to make in different scenarios, thereby improving the generalization performance of the correction model and providing a new idea and method for Chinese grammatical correction research. Summary of the invention

[0008] The present invention provides a Chinese grammar correction method and system for fine-tuning a large model, which can effectively improve the accuracy of the large model for Chinese text correction. The present invention has conducted sufficient experiments on four standard public Chinese spelling correction data sets to verify the effectiveness of the method.

[0009] The technical solution of the present invention is: a Chinese grammar error correction method based on large model fine-tuning, the method comprising the following steps:

[0010] Step 1: Extract enough corpus containing Chinese grammatical errors from Lang8, HSK, and FCGEC and pass it into the Qwen2-7B model to generate an error correction instruction set;

[0011] Step 2: Filter out high-quality data in the error correction instruction set based on the three indicators of complexity, quality, and diversity;

[0012] Step 3: Use the LORA method and the selected error correction instruction set to perform supervised fine-tuning on the Qwen2-7B model;

[0013] Step 4: Use the Qwen2-72B model to score the data set and generate preference labels, and then use the Qwen2-7B model to generate responses and make judgments, thus achieving self-optimization of the model, improving response quality, and obtaining a preferred data set;

[0014] Step 5: Optimize the Qwen2-7B model using the direct preference DPO method.

[0015] Furthermore, the specific steps of step 1 are as follows:

[0016] Step 1.1: Extract corpora containing typical Chinese grammatical errors and instructions, input error sentences, and output corresponding correct sentences from the public Chinese grammar error correction corpus Lang8, HSK, and FCGEC;

[0017] Step 1.2: Remove the corpus involving links and special symbols from the selected corpus, and transfer the remaining corpus into Qwen2-7B in a few-shot manner to generate richer error correction instructions and construct an error correction instruction set;

[0018] Step 1.3: Repeatedly select instructions and instructions generated by Qwen2-7B in a certain ratio, and use the output-first process to generate corresponding examples based on the selected instructions to avoid the results generated by the model being biased towards a certain label;

[0019] Step 1.4: When you identify examples where the instructions are too long or too short, or where the output of an instance is a repetition of the input, filter out these invalid generation;

[0020] Step 1.5: Compare the ROUGE-L similarity between the new instruction and all existing instructions, and add instructions with similarity less than 0.7 to the error correction instruction set.

[0021] Furthermore, the specific steps of step 2 are as follows:

[0022] Step 2.1: Iterate each instruction in the error correction instruction set multiple times to increase the complexity and quality of the instruction;

[0023] Step 2.2: Select samples from the error correction instruction set one by one and add them to the selected data set, calculate the cosine similarity of the sample to the selected data set, and filter out samples with cosine similarity greater than a threshold to increase instruction complexity.

[0024] Furthermore, the specific steps of step 3 are as follows:

[0025] Step 3.1: Fine-tune the self-attention layer, especially the part involving the encoder-decoder interaction;

[0026] Step 3.2: Set the rank of LORA, starting from a small rank value and gradually increasing it, while observing the performance changes of the model on the validation set to determine an optimal rank;

[0027] Step 3.3 sets the scaling factor of LORA to control the impact of the low-rank matrix on the original weight matrix; take the initial value of 1 as the starting point, adjust it together with the rank, constantly observe the performance of the model on the training set and the validation set, and gradually adjust the scaling factor.

[0028] Furthermore, the specific steps of step 4 are as follows:

[0029] Step 4.1: The constructed data set is passed into the Qwen2-72B model and scored according to the internal mechanism of the model. According to the model scoring, the error correction answer with the highest score is selected as the preference label to construct the preference data set;

[0030] Step 4.2: The Qwen2-7B model generates a correction answer for the same sentence based on the preference dataset and generates a response again by imitating the features of the correction options that score high in the preference dataset;

[0031] Step 4.3: The Qwen2-72B model considers and interprets the regenerated response and makes a judgment;

[0032] Step 4.4: Based on the discrimination generated by Qwen2-72B as the training signal, the Qwen2-7B model is trained again using the error correction answers that the model considers to be of high quality and the error correction answers that need to be improved, so that the training signal is aligned with the internal instructions of the Qwen2-72B model.

[0033] Furthermore, the specific steps of step 5 are as follows:

[0034] Step 5.1: Pass the preference dataset into the fine-tuned model Qwen2-7B;

[0035] Step 5.2: Use the Qwen2-72B model as a preference model to measure the consistency between the given reward function and the empirical preference data;

[0036] Step 5.3: Define the binary cross entropy loss function as the objective function of DPO to measure the difference between the model output and the preferred answer in the preference dataset, increase the log probability of the preferred response and decrease the log probability of the non-preferred response in the function, calculate the loss function gradient and use the loss function to update the model parameters;

[0037] Step 5.4: Train Qwen2-7B using the objective function and the prepared preference dataset, and adjust the model parameters to minimize the loss function so that the model is more inclined to generate the preferred response;

[0038] Step 5.5: Establish a feedback mechanism based on grammar support, collect model output feedback, and use the Qwen22-72B model to judge the output results;

[0039] Step 5.6: Based on the collected feedback and the discrimination generated by the Qwen2-72B model, repeatedly calculate the loss gradient and update the parameters to continuously improve the model performance.

[0040] The present invention also provides a Chinese grammar correction system based on large model fine-tuning, including: a module for executing the Chinese grammar correction method based on large model fine-tuning.

[0041] The beneficial effects of the present invention are:

[0042] 1. Improve the accuracy of grammatical error correction: By combining the fine-tuning technology of the large model and a rich instruction data set, the present invention can significantly improve the accuracy of Chinese grammatical error correction, reduce the missed detection rate and false detection rate of grammatical errors, and help users effectively improve the quality of their writing.

[0043] 2. Reduce training costs: The present invention uses Qwen2-72B to generate preference labels, skipping the learning steps of the traditional reward function, simplifying the model training process, reducing the cost of data annotation and processing, and making model training more efficient.

[0044] 3. Wide applicability: The technology of the present invention can be widely used in multiple fields such as education, content creation, online writing assistance, etc., providing users with more accurate and efficient grammar correction services, and has high market value and application prospects. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 is a flow chart of the present invention; DETAILED DESCRIPTION

[0046] Example 1: Figure 1 As shown in the figure, a Chinese grammar correction method based on large model fine-tuning is implemented in the following steps:

[0047] a1. Extract 100 sentences containing Chinese grammatical errors from Lang8 and pass them into the Qwen2-7B model to generate a richer instruction dataset.

[0048] The specific steps of step 1 are as follows:

[0049] Extract enough sentences containing typical Chinese grammatical errors and instructions, input (wrong sentences), and output (corresponding correct sentences) from the public Chinese grammar correction corpus Lang8, HSK, and FCGEC;

[0050] The selected corpus contains links and special symbols, and the remaining corpus is passed into Qwen2-7B in a few-shot manner to generate more abundant error correction instructions and construct an error correction instruction set;

[0051] Repeatedly select manually annotated instructions from the instruction set and instructions generated by Qwen2-7B in a ratio of 3:1, and use the output-first process to generate corresponding examples based on the selected instructions to avoid the results generated by the model being biased towards a certain label;

[0052] When identifying examples where instructions are too long or too short, or where the output of an instance is a repetition of the input, filter out these invalid generation;

[0053] Compare the ROUGE-L similarity between the new instructions and all existing instructions, and add instructions with similarity less than 0.7 to the error correction instruction set;

[0054] According to the authoritative linguistics book "Modern Chinese Dictionary", Chinese grammatical errors can be roughly divided into seven categories: chaotic structure, illogicality, incomplete components, redundant components, improper collocation, improper word order and ambiguous errors. However, ambiguous errors are often caused by lack of contextual information, and a large amount of additional knowledge is required to assist in correction. Therefore, the Chinese grammar error correction corpus constructed by the present invention mainly focuses on the first six types of Chinese grammatical errors.

[0055] From the public Chinese grammar correction corpus Lang8, 100 sentences containing the six grammatical errors were manually screened and annotated, and then passed into Qwen2-7B for instruction construction, instruction screening (ROUGE-L similarity), data enhancement and data filtering.

[0056] ROUGE-L stands for Longest Common Subsequence, which computes the longest common subsequence (LCS) between the output and the reference output. To apply LCS to summary evaluation, we regard the summary sentence as a sequence of words. The intuition is that the longer the LCS of two summary sentences, the more similar the two summaries are. We propose to use LCS-based F-measure to estimate the similarity between two summaries X of length m and Y, assuming that X is a reference summary sentence and Y is a candidate summary sentence, as follows:

[0057]

[0058]

[0059] Where LCS(X,Y) is the length of the longest common subsequence of X and Y. In DUC, β is set to a very large number ∞, and only Rlcs is considered. Equation 4 is called the LCS-based F-measure. When X = Y, ROUGE-L = 1, and when LCS(X,Y) = 0, ROUGE-L = 0

[0060] a2. Filter out high-quality data in the instruction data set based on the three indicators of complexity, high quality, and diversity.

[0061] The specific steps of a2 are as follows

[0062] Step a2.1: Each instruction in the error correction instruction set is iterated multiple times to increase the complexity of the instruction (adding conditional restrictions, specifying instructions, etc.) and the quality (improving the relevance, helpfulness, and details of the answer, etc.);

[0063] Step a2.2: Select samples from the error correction instruction set one by one and add them to the selected data set, calculate the cosine similarity of the sample to the selected data set, and filter out samples with cosine similarity greater than 0.6 to increase instruction complexity.

[0064] 1) Complexity evaluation method: The perplexity of the response is calculated by the Qwen2-7B model as a complexity indicator. The larger the perplexity value, the more difficult the data sample is. The formula is as follows:

[0065]

[0066] Among them, S stands for sentence, N is the length of the sentence, and p(w i ) is the probability of the i-th word. The first word is p(w1|w0), and w0 is START, which means the beginning of the sentence and is a placeholder.

[0067] 2) Quality evaluation method: F1-score is used to evaluate the quality of instructions. The specific formula is as follows;

[0068]

[0069] Among them, TP (True Positive): the predicted answer is correct; FP (False Positive): other classes are predicted as this class by mistake; FN (False Negative): this class label is predicted as other class labels

[0070] 3) Diversity evaluation method: Calculate the cosine similarity between sentences in the generated text. A lower average similarity value indicates a higher diversity. The formula is as follows:

[0071]

[0072] Among them, A and B are the vector representations of the sentences.

[0073] a3. Use the LORA method and error correction enhancement dataset to perform supervised fine-tuning on the Qwen2-7B model;

[0074] 1) Determine the target module for fine-tuning: self-attention layer

[0075] 1. Fine-tune the weights of the self-attention layer to enhance the model’s ability to capture long-range dependencies and contextual information.

[0076] W′=W+α·ΔW (9)

[0077] W is the original weight matrix, α is the learning rate, and ΔW is the weight update calculated based on the training data.

[0078] 2. Adjust the scale factor in the self-attention mechanism to optimize the model's learning intensity of the relationship between elements of different distances.

[0079] scalefactor=scalefactor·η (10)

[0080] Among them, scalefactor is the original scaling factor, and η is the scaling factor update ratio dynamically adjusted according to the performance of the validation set.

[0081] 3. Introduce additional supervision signals to enhance the self-attention layer’s focus on grammatical features.

[0082]

[0083] Among them, y is the true label, is the model prediction, is an additional loss term based on the grammatical structure, and λ is the weight of this term.

[0084] 2) Setting the rank of LORA: Based on gradually increasing the learning rate, the model parameters are stabilized at the beginning of training, and then the learning speed is accelerated to quickly adapt to the data characteristics;

[0085] 1. Initialize the learning rate:

[0086] A small initial learning rate α0 is set to ensure stable update of model parameters in the early stage of training.

[0087] 2. Define a learning rate increasing function λ(t), a linear function that gradually increases with time step t, then the updated learning rate can be expressed as:

[0088] α t =α0+λ(t) (12)

[0089] Among them, t is the current training step number, λ(t) is the learning rate increase function, α t is the updated learning rate.

[0090] 3. Performance monitoring and rank value determination;

[0091] During the training process, continuously monitor the performance of the model on the validation set. If the performance improves, continue to increase the learning rate; if the performance stops improving or shows a downward trend, determine an optimal learning rate as the rank value.

[0092] 3) Set the scaling factor of LORA to control the impact of the low-rank matrix on the original weight matrix. Take the initial value of 1 as the starting point, adjust it together with the rank, constantly observe the performance of the model on the training set and the validation set, and gradually adjust the scaling factor.

[0093]

[0094] Among them, β t is the scaling factor of the current step, δ is the adjustment amount of the scaling factor, and T is the total number of training steps.

[0095] a4. Use the Qwen2-72B model to score the data set and generate preference labels, then use the Qwen2-7B model to generate responses and make judgments to generate a preference data set.

[0096] 1) Use the Qwen2-7B model to score each sentence in the dataset. The model assigns a score to each sentence based on its internal mechanisms (such as language fluency and accuracy of error correction logic). Based on the model score, the sentence with the highest score is selected as the preferred label.

[0097] 2) The Qwen2-7B model generates a correction answer for the same sentence based on the preference dataset, trying to mimic the characteristics of the high-scoring correction options. The model generates responses again, trying to replicate the high-scoring characteristics observed in the preference dataset.

[0098] 3) The Qwen2-7B model evaluates each response it generates to determine its consistency with the preference label and generates an explanation for each response to help the model understand its discriminative decision.

[0099] 4) Use the discriminant generated by the model as a training signal, select high-quality error-corrected answers for rewards, and penalize answers that need improvement. Use the calculated reward value to retrain the Qwen2-7B model and optimize the model parameters to better match the internal instructions.

[0100] a5. Use DPO method to optimize Qwen2-7B model;

[0101] 1) Pass the preference dataset into the fine-tuned model Qwen2-7B.

[0102] 2) Use the Qwen2-72B model as the preference model and compare the consistency between the model output and the empirical preference data.

[0103] 3) Define the binary cross entropy loss function as the objective function of DPO, which is used to measure the difference between the model output and the preferred answer in the preference dataset. In this function, the log probability of the preferred response is increased and the log probability of the non-preferred response is decreased. The loss function gradient is calculated and the model parameters are updated using the loss function.

[0104]

[0105] Among them, y is the preferred answer, is the model output.

[0106] 4) Use the objective function and the prepared preference dataset to train Qwen2-7B and minimize the loss function so that the model is more inclined to generate preference responses.

[0107]

[0108] Here, n is the number of samples in the dataset.

[0109] 5) Establish a feedback mechanism based on grammar support, collect model output feedback, and use the Qwen2-72B model to judge the output results.

[0110] 6) Iteratively update model parameters based on the collected feedback and the judgment generated by the Qwen2-72B model

[0111]

[0112] Where η is the learning rate, is the gradient of the loss function with respect to the parameters.

[0113] The present invention also provides a Chinese grammar correction system based on large model fine-tuning, including: a module for executing the Chinese grammar correction method based on large model fine-tuning.

[0114] Verify whether this method can improve the error correction performance of the model.

[0115] 1) The present invention uses the publicly available Chinese grammar correction corpus NLPCC18-Test as the experimental data of the present invention. The commonly used accuracy (Precision), recall (Recall), and F-measure (F-measure) are used as evaluation indicators of the Chinese grammar correction model. The accuracy and recall are calculated using formulas (6) and (7), and the calculation formula of the F-measure is as follows:

[0116]

[0117] 2) In order to verify the effect of the method proposed in the present invention on improving the performance of the Chinese grammar correction model, the present invention compared the performance of Qwen2-7B, the Qwen-7B model trained using the DPO method, and GPT4 in Chinese grammar correction.

[0118] 3) The final experimental results are shown in Table 2. It can be seen that the Qwen2-7B trained by our proposed method far exceeds GPT4 in all indicators, and has corresponding improvements compared to the untrained Qwen2-7B model.

[0119] Table 1 Statistics of the dataset used

[0120]

[0121] Table 2 Performance of different models on the test set

[0122]

[0123] The specific implementation modes of the present invention are described in detail above in conjunction with the accompanying drawings, but the present invention is not limited to the above implementation modes, and various changes can be made within the knowledge scope of ordinary technicians in this field without departing from the purpose of the present invention.

Claims

1. A Chinese grammar correction method based on large model fine-tuning, characterized by: The method comprises the following steps: Step 1: Collect corpus containing Chinese grammatical errors and pass it into the Qwen2-7B model to generate an error correction instruction set; Step 2: Filter out high-quality data in the error correction instruction set based on the three indicators of complexity, quality, and diversity; Step 3: Use the LORA method and the selected error correction instruction set to perform supervised fine-tuning on the Qwen2-7B model; Step 4: Use the Qwen2-72B model to score the data set and generate preference labels, and then use the Qwen2-7B model to generate responses and make judgments, thus achieving self-optimization of the model, improving response quality, and obtaining a preferred data set; Step 5: Optimize the Qwen2-7B model using the direct preference DPO method.

2. The Chinese grammar error correction method based on large model fine-tuning according to claim 1 is characterized by: The specific steps of step 1 are as follows: Step 1.1: Extract corpora containing typical Chinese grammatical errors and instructions, input error sentences, and output corresponding correct sentences from the public Chinese grammar error correction corpus Lang8, HSK, and FCGEC; Step 1.2: Remove the corpus involving links and special symbols from the selected corpus, and transfer the remaining corpus into Qwen2-7B in a few-shot manner to generate richer error correction instructions and construct an error correction instruction set; Step 1.3: Repeatedly select instructions and instructions generated by Qwen2-7B in a certain ratio, and use the output-first process to generate corresponding examples based on the selected instructions to avoid the results generated by the model being biased towards a certain label; Step 1.4: When you identify examples where the instructions are too long or too short, or where the output of an instance is a repetition of the input, filter out these invalid generation; Step 1.5: Compare the ROUGE-L similarity between the new instruction and all existing instructions, and add instructions with similarity less than 0.7 to the error correction instruction set.

3. The Chinese grammar error correction method based on large model fine-tuning according to claim 1 is characterized in that: The specific steps of step 2 are as follows Step 2.1: Iterate each instruction in the error correction instruction set multiple times to increase the complexity and quality of the instruction; Step 2.2: Select samples from the error correction instruction set one by one and add them to the selected data set, calculate the cosine similarity of the sample to the selected data set, and filter out samples with cosine similarity greater than a threshold to increase instruction complexity.

4. The Chinese grammar error correction method based on large model fine-tuning according to claim 1 is characterized in that: The specific steps of step 3 are as follows Step 3.1: Fine-tune the self-attention layer, especially the part involving the encoder-decoder interaction; Step 3.2: Set the rank of LORA, starting from a small rank value and gradually increasing it, while observing the performance changes of the model on the validation set to determine an optimal rank; Step 3.3 sets the scaling factor of LORA to control the impact of the low-rank matrix on the original weight matrix; take the initial value of 1 as the starting point, adjust it together with the rank, constantly observe the performance of the model on the training set and the validation set, and gradually adjust the scaling factor.

5. The Chinese grammar error correction method based on large model fine-tuning according to claim 1 is characterized in that: The specific steps of step 4 are as follows Step 4.1: The constructed data set is passed into the Qwen2-72B model and scored according to the internal mechanism of the model. According to the model scoring, the error correction answer with the highest score is selected as the preference label to construct the preference data set; Step 4.2: The Qwen2-7B model generates a correction answer for the same sentence based on the preference dataset and generates a response again by imitating the features of the correction options that score high in the preference dataset; Step 4.3: The Qwen2-72B model considers and interprets the regenerated response and makes a judgment; Step 4.4: Based on the discrimination generated by Qwen2-72B as the training signal, the Qwen2-7B model is trained again using the error correction answers that the model considers to be of high quality and the error correction answers that need to be improved, so that the training signal is aligned with the internal instructions of the Qwen2-72B model.

6. The Chinese grammar error correction method based on large model fine-tuning according to claim 1 is characterized by: The specific steps of step 5 are as follows: Step 5.1: Pass the preference dataset into the fine-tuned model Qwen2-7B; Step 5.2: Use the Qwen2-72B model as a preference model to measure the consistency between the given reward function and the empirical preference data; Step 5.3: Define the binary cross entropy loss function as the objective function of DPO to measure the difference between the model output and the preferred answer in the preference dataset, increase the log probability of the preferred response and decrease the log probability of the non-preferred response in the function, calculate the loss function gradient and use the loss function to update the model parameters; Step 5.4: Train Qwen2-7B using the objective function and the prepared preference dataset, and adjust the model parameters to minimize the loss function so that the model is more inclined to generate the preferred response; Step 5.5: Establish a feedback mechanism based on grammar support, collect model output feedback, and use the Qwen22-72B model to judge the output results; Step 5.6: Based on the collected feedback and the discrimination generated by the Qwen2-72B model, repeatedly calculate the loss gradient and update the parameters to continuously improve the model performance.

7. A Chinese grammar correction system based on large model fine-tuning, characterized by: include: A module for executing the Chinese grammar correction method based on large model fine-tuning as described in any one of claims 1 to 6.

Images