A large model fine-tuning-based Chinese grammar correction method and system

Abstract

The application relates to a Chinese grammar correction method and system based on large model fine tuning. The method uses a Qwen2-7B model to generate a correction instruction data set; the instruction data set is filtered according to complexity, quality and diversity; the LORA method is used to supervise the fine tuning of the Qwen2-7B model, and the self-attention layer and the feedforward network layer are mainly optimized to enhance the processing capacity of the model for long-distance dependence and context information; then the Qwen2-72B model is used to score the correction data set, generate preference labels, and further optimize the Qwen2-7B model using the labels to improve the response quality; a direct preference optimization method is used to train the model combined with the preference data set, parameters are adjusted to minimize the loss function, and the model is enhanced to be more inclined to respond to preferences. The method effectively improves the accuracy of Chinese grammar correction through multi-level optimization strategies.

Description

[0001] Technical language

[0002] The application relates to a Chinese grammar correction method and system based on large model fine-tuning, and belongs to the technical field of natural language processing. BACKGROUND

[0003] With the increase in user randomness in text input and voice input, grammar errors are widespread in various applications. Especially in products such as word processing software, input method applications and email applications, the built-in grammar correction service has become an important part of improving user experience. A good grammar correction system can significantly improve user satisfaction. For example, in the search query scenario, the search engine will correct the user's input query to ensure accurate search results; the goal of Chinese grammar error correction is to automatically identify and correct potential errors in sentences while maintaining their original meaning. This not only reduces the cost of manual checking and saves human resources, but also provides a foundation for other artificial intelligence applications, such as search queries, character recognition, speech recognition, writing assistance and article review scenarios.

[0004] In recent years, with the breakthrough of deep learning technology in the field of text generation, the construction of Chinese grammar correction corpus has also made significant progress. The current grammar correction corpus mainly exists in the form of sentence pairs, including incorrect sentences and their corresponding correct sentences. However, constructing a large-scale Chinese grammar correction corpus faces challenges, mainly in the need for rich language knowledge and background knowledge, as well as the high cost of manual annotation, making it difficult to obtain large-scale training data. This problem has become a key bottleneck in the research of Chinese grammar correction.

[0005] In order to solve the problem of shortage of large-scale training data, the application proposes a Chinese grammar correction method based on large model fine-tuning. First, sentences containing grammar errors are extracted from public Chinese grammar correction corpus (such as Lang8), and artificial screening and annotation are used to ensure the quality and diversity of the data. Then, large models such as Qwen2-7B are used to process the data and generate a rich set of correction instruction data. In this process, few-shot learning technology is used to improve the model's ability to identify grammar errors.

[0006] After the construction of the data set, the method also introduces the LORA (Low-Rank Adaptation) technology to fine-tune the model, focusing on optimizing the self-attention layer and the feedforward network layer to improve the model's ability to capture long-distance dependencies. At the same time, by constructing a preference data set and applying the direct preference optimization (DPO) method, the quality of the model's generated correction responses is further improved. This multi-level optimization strategy not only enhances the model's understanding of Chinese grammar, but also provides effective support for practical applications.

[0007] In summary, the Chinese grammar correction method based on large models aims to simulate the grammatical errors that users are prone to make in different scenarios, thereby improving the generalization performance of the correction model and providing a new approach and method for Chinese grammar correction research. Summary of the Invention

[0008] This invention provides a method and system for fine-tuning large-scale Chinese grammar correction, which can effectively improve the accuracy of large-scale models in correcting Chinese text errors. Extensive experiments were conducted on four standard public Chinese spelling correction datasets to verify the effectiveness of the proposed method.

[0009] The technical solution of this invention is: a Chinese grammar correction method based on large model fine-tuning, the method comprising the following steps:

[0010] Step 1: Extract a sufficient amount of corpus containing Chinese grammatical errors from Lang8, HSK, and FCGEC and input it into the Qwen2-7B model to generate a set of error correction instructions;

[0011] Step 2: Select high-quality data from the error correction instruction set based on three indicators: complexity, quality, and diversity;

[0012] Step 3: Supervised fine-tuning of the Qwen2-7B model using the LORA method and the selected error correction instruction set;

[0013] Step 4: The dataset is scored and preference labels are generated using the Qwen2-72B model. Then, the Qwen2-7B model is used to generate responses and make judgments, thereby achieving self-optimization of the model and improvement of response quality, and obtaining the preference dataset.

[0014] Step 5: Optimize the Qwen2-7B model using the Direct Preferences (DPO) method.

[0015] Furthermore, the specific steps of step 1 are as follows:

[0016] Step 1.1: Extract corpora containing typical Chinese grammar errors and including instructions, input error statements, and corresponding correct output statements from publicly available Chinese grammar correction corpora such as Lang8, HSK, and FCGEC;

[0017] Step 1.2: Remove the corpus containing links and special symbols from the selected corpus, and input the remaining corpus into Qwen2-7B using a few-shot method to generate richer error correction instructions and construct an error correction instruction set;

[0018] Step 1.3: Repeatedly select instructions and instructions generated by Qwen2-7B according to a certain ratio, and generate corresponding examples according to the selected instructions using the output-first process to avoid the model generation results being biased towards a certain label;

[0019] Step 1.4: When examples such as instructions that are too long or too short, or instances where the output is a duplicate of the input, are identified, these invalid generation methods are filtered out.

[0020] Step 1.5: Compare the ROUGE-L similarity of the new instruction with all existing instructions, and add instructions with a similarity of less than 0.7 to the error correction instruction set.

[0021] Furthermore, the specific steps of step 2 are as follows:

[0022] Step 2.1: Iterate through each instruction in the error correction instruction set multiple times to increase the complexity and quality of the instructions;

[0023] Step 2.2: Select samples one by one from the error correction instruction set and add them to the selected dataset. Calculate the cosine similarity between the sample and the selected dataset, and filter out samples with a cosine similarity greater than the threshold to increase instruction complexity.

[0024] Furthermore, the specific steps of step 3 are as follows:

[0025] Step 3.1: Fine-tune the self-attention layer, especially the parts involving encoder-decoder interaction;

[0026] Step 3.2: Set the rank of LORA, starting with a small rank value and gradually increasing it, while observing the performance changes of the model on the validation set to determine an optimal rank;

[0027] Step 3.3 Set the scaling factor of LORA to control the influence of the low-rank matrix on the original weight matrix; start with an initial value of 1, adjust it together with the rank, and continuously observe the model's performance on the training and validation sets, gradually adjusting the scaling factor.

[0028] Furthermore, step 4 is specifically as follows:

[0029] Step 4.1: Input the constructed dataset into the Qwen2-72B model and score it according to the model's internal mechanism. Based on the model's scoring, select the corrected answer with the highest score as the preference label to construct the preference dataset.

[0030] Step 4.2: The Qwen2-7B model generates corrected answers for the same statement based on the preference dataset and mimics the characteristics of corrected options that score highly in the preference dataset to generate a response again;

[0031] Step 4.3: The Qwen2-72B model considers and interprets the regenerated response and makes a judgment;

[0032] Step 4.4: Using the discriminant generated by Qwen2-72B as the training signal, retrain the Qwen2-7B model with the error correction answers that the model considers to be of high quality and those that need improvement, so that the training signal is aligned with the internal indications of the Qwen2-72B model.

[0033] Furthermore, step 5 specifically involves the following steps:

[0034] Step 5.1: Input the preference dataset into the fine-tuned model Qwen2-7B;

[0035] Step 5.2: Use the Qwen2-72B model as a preference model to measure the consistency between the given reward function and empirical preference data;

[0036] Step 5.3: Define the binary cross-entropy loss function as the objective function of DPO to measure the difference between the model output and the preferred answer in the preference dataset. In this function, increase the log probability of the preferred response and decrease the log probability of the non-preferred response. Calculate the gradient of the loss function and use the loss function to update the model parameters.

[0037] Step 5.4: Train Qwen2-7B using the objective function and the prepared preference dataset, and tune the model parameters to minimize the loss function so that the model is more inclined to generate preference responses;

[0038] Step 5.5: Establish a feedback mechanism based on grammatical support, collect model output feedback, and use the Qwen22-72B model to judge the output results;

[0039] Step 5.6: Based on the collected feedback and the discrimination generated by the Qwen2-72B model, repeatedly calculate the loss gradient and update the parameters to continuously improve the model performance.

[0040] The present invention also provides a Chinese grammar correction system based on large model fine-tuning, comprising: a module for executing the Chinese grammar correction method based on large model fine-tuning.

[0041] The beneficial effects of this invention are:

[0042] 1. Improve the accuracy of grammar correction: By combining fine-tuning techniques of large models with rich instruction datasets, this invention can significantly improve the accuracy of Chinese grammar correction, reduce the missed detection rate and false detection rate of grammatical errors, and help users effectively improve the quality of their writing.

[0043] 2. Reduced training costs: This invention uses Qwen2-72B to generate preference labels, skipping the learning steps of traditional reward functions, simplifying the model training process, reducing the cost of data labeling and processing, and making model training more efficient.

[0044] 3. Wide applicability: The technology of this invention can be widely applied in multiple fields such as education, content creation, and online writing assistance, providing users with more accurate and efficient grammar correction services, and has high market value and application prospects. Attached Figure Description

[0045] Figure 1 This is a flowchart of the present invention; Detailed Implementation

[0046] Example 1: As Figure 1 As shown, a Chinese grammar correction method based on large model fine-tuning is implemented through the following steps:

[0047] a1. Extract 100 sentences containing Chinese grammatical errors from Lang8 and input them into the Qwen2-7B model to generate a richer instruction dataset.

[0048] The specific steps of step 1 are as follows:

[0049] Extract a sufficient number of sentences containing typical Chinese grammatical errors and including instructions, inputs (incorrect statements), and outputs (corresponding correct statements) from publicly available Chinese grammar correction corpora such as Lang8, HSK, and FCGEC.

[0050] Remove the corpus containing links and special symbols from the selected corpus, and input the remaining corpus into Qwen2-7B in a few-shot manner to generate richer error correction instructions and construct an error correction instruction set;

[0051] The instructions manually annotated in the instruction set and the instructions generated by Qwen2-7B are repeatedly selected in a 3:1 ratio. Based on the selected instructions, the corresponding examples are generated using an output-first process to avoid the model's generated results being biased towards a certain label.

[0052] When examples such as instructions that are too long or too short, or instances where the output is a duplicate of the input, are identified, these invalid generation methods are filtered out.

[0053] Compare the ROUGE-L similarity between the new instruction and all existing instructions, and add instructions with a similarity of less than 0.7 to the error correction instruction set;

[0054] Based on authoritative linguistics books such as the *Modern Chinese Dictionary*, Chinese grammatical errors can be broadly categorized into seven types: structural inconsistencies, illogicalities, incomplete components, redundant components, inappropriate collocations, incorrect word order, and ambiguity. However, ambiguity errors are often caused by a lack of contextual information and require substantial additional knowledge for correction. Therefore, the Chinese grammar correction corpus constructed in this invention primarily focuses on the first six types of Chinese grammatical errors.

[0055] From the publicly available Chinese grammar correction corpus Lang8, 100 sentences containing the six types of grammatical errors were manually selected and labeled, and then fed into Qwen2-7B for instruction construction, instruction filtering (ROUGE-L similarity), data augmentation, and data filtering.

[0056] ROUGE-L stands for Longest Common Subsequence, which computes the longest common subsequence (LCS) between the output and the reference output. To apply LCS to summary evaluation, we treat summary sentences as sequences of words. Intuitively, the longer the LCS of two summary sentences, the more similar the two summaries are. We propose using an LCS-based F-measure to estimate the similarity between two summaries X of length m and Y, assuming X is the reference summary sentence and Y is the candidate summary sentence, as follows:

[0057]

[0058]

[0059] Where LCS(X,Y) is the length of the longest common subsequence of X and Y. In DUC, β is set to a very large number ∞, and only Rlcs is considered. Equation 4 is called the F-metric based on LCS. ROUGE-L = 1 when X = Y, and ROUGE-L = 0 when LCS(X,Y) = 0.

[0060] a2. Select high-quality data from the instruction dataset based on three indicators: complexity, high quality, and diversity.

[0061] The specific steps for a2 are as follows:

[0062] Step a2.1: Iterate through each instruction in the error correction instruction set multiple times to increase the complexity (adding conditional constraints, specifying instructions, etc.) and quality (improving the relevance, helpfulness, and detail of the answer, etc.);

[0063] Step a2.2: Select samples one by one from the error correction instruction set and add them to the selected dataset. Calculate the cosine similarity between the sample and the selected dataset, and filter out samples with a cosine similarity greater than 0.6 to increase instruction complexity.

[0064] 1) Complexity assessment method: The perplexity of the response is calculated using the Qwen2-7B model as a complexity indicator. A higher perplexity value means that the data sample is more difficult. The formula is as follows:

[0065]

[0066] Where S represents sentence, N is sentence length, and p(w i p(w1|w0) represents the probability of the i-th word. The first word is p(w1|w0), and w0 is START, which represents the start of the sentence and is a placeholder.

[0067] 2) Quality assessment method: The quality of the instructions is assessed using the F1-score, and the specific formula is as follows;

[0068]

[0069] Among them, TP (True Positive): the predicted answer is correct; FP (False Positive): other classes are incorrectly predicted as this class; FN (False Negative): the label of this class is predicted as another class.

[0070] 3) Diversity assessment method: Calculate the cosine similarity between sentences in the generated text. A lower average similarity value indicates higher diversity. The formula is as follows:

[0071]

[0072] Here, A and B are the vector representations of the sentences.

[0073] a3. Supervised fine-tuning of the Qwen2-7B model using the LORA method and an error correction augmentation dataset;

[0074] 1) Identify the target module for fine-tuning: self-attention layer

[0075] 1. Fine-tune the weights of the self-attention layer to enhance the model's ability to capture long-distance dependencies and contextual information.

[0076] W′=W+α·ΔW (9)

[0077] W is the original weight matrix, α is the learning rate, and ΔW is the weight update amount calculated based on the training data.

[0078] 2. Adjust the scaling factor in the self-attention mechanism to optimize the model's learning strength of relationships between elements at different distances.

[0079] scalefactor=scalefactor·η (10)

[0080] Where scalefactor is the original scaling factor, and η is the scaling factor update ratio that is dynamically adjusted based on the performance of the validation set.

[0081] 3. Introduce additional supervision signals to enhance the attention of the self-attention layer to grammatical features.

[0082]

[0083] Where y is the real label. It is a model prediction. It is an additional loss term based on grammatical structure, and λ is the weight of this term.

[0084] 2) Set the rank of LORA: Based on gradually increasing the learning rate, the model parameters are stabilized in the early stage of training, and then the learning speed is accelerated to quickly adapt to the data features;

[0085] 1. Initialize the learning rate:

[0086] Set a small initial learning rate α0 to ensure stable updates of model parameters in the early stages of training.

[0087] 2. Define a learning rate function λ(t) that increases linearly with time step t. The updated learning rate can then be expressed as:

[0088] α t =α0+λ(t) (12)

[0089] Where t is the current training step number, λ(t) is the learning rate increment function, and α t This is the updated learning rate.

[0090] 3. Performance monitoring and rank determination;

[0091] During training, continuously monitor the model's performance on the validation set. If performance improves, continue increasing the learning rate; if performance no longer improves or shows a downward trend, determine an optimal learning rate as the rank value.

[0092] 3) Set the scaling factor of LORA to control the influence of the low-rank matrix on the original weight matrix. Start with an initial value of 1, adjust it in conjunction with the rank, and continuously observe the model's performance on the training and validation sets, gradually adjusting the scaling factor.

[0093]

[0094] Where, β t δ is the scaling factor for the current step, δ is the adjustment amount of the scaling factor, and T is the total number of training steps.

[0095] a4. The dataset is scored and preference labels are generated using the Qwen2-72B model. Then, the Qwen2-7B model is used to generate responses and make judgments to generate preference datasets.

[0096] 1) Use the Qwen2-7B model to score each statement in the dataset. The model assigns a score to each statement based on its internal mechanisms (such as language fluency and the accuracy of error correction logic). Select the statement with the highest score as the preference label.

[0097] 2) The Qwen2-7B model generates corrected answers for the same statement based on the preference dataset, attempting to mimic the characteristics of high-scoring corrected options. The model then generates responses again, trying to replicate the high-scoring features observed in the preference dataset.

[0098] 3) The Qwen2-7B model evaluates each of its generated responses, determines their consistency with the preference label, and generates an explanation for each response to help the model understand its discriminative decision.

[0099] 4) Use the model-generated discriminants as training signals, reward high-quality error-correcting answers, and penalize answers that need improvement. Retrain the Qwen2-7B model using the calculated reward values ​​to optimize model parameters and better align with internal instructions.

[0100] a5. Optimize the Qwen2-7B model using the DPO method;

[0101] 1) Input the preference dataset into the fine-tuned model Qwen2-7B.

[0102] 2) Using the Qwen2-72B model as the preference model, we compared the consistency between the model output and the empirical preference data.

[0103] 3) Define the binary cross-entropy loss function as the objective function of DPO, which measures the difference between the model output and the preferred answer in the preference dataset. In this function, increase the log probability of the preferred response and decrease the log probability of the non-preferred response. Calculate the gradient of the loss function and use the loss function to update the model parameters.

[0104]

[0105] Among them, y is the preferred answer. Output for the model.

[0106] 4) Train Qwen2-7B using the objective function and the prepared preference dataset, minimizing the loss function to make the model more inclined to generate preference responses.

[0107]

[0108] Where n is the number of samples in the dataset.

[0109] 5) Establish a feedback mechanism based on grammatical support, collect model output feedback, and use the Qwen2-72B model to judge the output results.

[0110] 6) Based on the collected feedback and the discrimination generated by the Qwen2-72B model, iteratively update the model parameters.

[0111]

[0112] Where η is the learning rate. It is the gradient of the loss function with respect to the parameters.

[0113] The present invention also provides a Chinese grammar correction system based on large model fine-tuning, comprising: a module for executing the Chinese grammar correction method based on large model fine-tuning.

[0114] Verify whether this method can improve the model's error correction performance.

[0115] 1) This invention uses the publicly available Chinese grammar correction corpus NLPCC18-Test as the experimental data. Precision, recall, and F-measure are commonly used as evaluation metrics for the Chinese grammar correction model. Precision and recall are calculated using equations (6) and (7), and the F-measure is calculated as follows:

[0116]

[0117] 2) To verify the effect of the proposed method on improving the performance of Chinese grammar correction models, this invention compares the performance of Qwen2-7B, the Qwen-7B model trained using the DPO method, and GPT4 in Chinese grammar correction.

[0118] 3) The final experimental results are shown in Table 2 below. It can be seen that the Qwen2-7B trained by our proposed method far outperforms GPT4 in all metrics, and also shows a corresponding improvement compared to the untrained Qwen2-7B model.

[0119] Statistical information of the dataset used in Table 1

[0120]

[0121] Table 2 shows the performance of different models on the test set.

[0122]

[0123] The specific embodiments of the present invention have been described in detail above with reference to the accompanying drawings. However, the present invention is not limited to the above embodiments. Within the scope of knowledge possessed by those skilled in the art, various changes can be made without departing from the spirit of the present invention.

Claims

1. A Chinese grammar correction method based on large model fine-tuning, characterized in that: The method includes the following steps: Step 1: Collect corpus containing Chinese grammatical errors and input it into the Qwen2-7B model to generate a set of error correction instructions; Step 2: Select high-quality data from the error correction instruction set based on three indicators: complexity, quality, and diversity; Step 3: Supervised fine-tuning of the Qwen2-7B model using the LORA method and the selected error correction instruction set; Step 4: The dataset is scored and preference labels are generated using the Qwen2-72B model. Then, the Qwen2-7B model is used to generate responses and make judgments, thereby achieving self-optimization of the model and improvement of response quality, and obtaining the preference dataset. Step 5: Optimize the Qwen2-7B model using the Direct Preferences (DPO) method; The specific steps of step 4 are as follows: Step 4.1: Input the constructed dataset into the Qwen2-72B model and score it according to the model's internal mechanism. Based on the model's scoring, select the corrected answer with the highest score as the preference label to construct the preference dataset. Step 4.2: The Qwen2-7B model generates corrected answers for the same statement based on the preference dataset and mimics the characteristics of corrected options that score highly in the preference dataset to generate a response again; Step 4.3: The Qwen2-72B model considers and interprets the regenerated response and makes a judgment; Step 4.4: Based on the discriminant generated by Qwen2-72B as the training signal, retrain the Qwen2-7B model using the error correction answers that the model considers to be of high quality and those that need improvement, so that the training signal is aligned with the internal indication of the Qwen2-72B model. The specific steps of step 5 are as follows: Step 5.1: Input the preference dataset into the fine-tuned model Qwen2-7B; Step 5.2: Use the Qwen2-72B model as a preference model to measure the consistency between the given reward function and empirical preference data; Step 5.3: Define the binary cross-entropy loss function as the objective function of DPO to measure the difference between the model output and the preferred answer in the preference dataset. In this function, increase the log probability of the preferred response and decrease the log probability of the non-preferred response. Calculate the gradient of the loss function and use the loss function to update the model parameters. Step 5.4: Train Qwen2-7B using the objective function and the prepared preference dataset, and tune the model parameters to minimize the loss function so that the model is more inclined to generate preference responses; Step 5.5: Establish a feedback mechanism based on grammatical support, collect model output feedback, and use the Qwen2-72B model to judge the output results; Step 5.6: Based on the collected feedback and the discriminant generated by the Qwen2-72B model, repeatedly calculate the loss. Gradients and updated parameters are used to continuously improve model performance.

2. The Chinese grammar correction method based on large model fine-tuning according to claim 1, characterized in that: The specific steps of step 1 are as follows: Step 1.1: Extract corpora containing typical Chinese grammar errors and including instructions, input error statements, and corresponding correct output statements from publicly available Chinese grammar correction corpora such as Lang8, HSK, and FCGEC; Step 1.2: Remove the corpus containing links and special symbols from the selected corpus, and input the remaining corpus into Qwen2-7B using a few-shot method to generate richer error correction instructions and construct an error correction instruction set; Step 1.3: Repeatedly select instructions and instructions generated by Qwen2-7B according to a certain ratio, and generate corresponding examples according to the selected instructions using the output-first process to avoid the model generation results being biased towards a certain label; Step 1.4: When an instruction is identified as too long or too short, or when the instance output is a duplicate of the input, filter out these invalid generation results. Step 1.5: Compare the ROUGE-L similarity of the new instruction with all existing instructions, and add instructions with a similarity of less than 0.7 to the error correction instruction set.

3. The Chinese grammar correction method based on large model fine-tuning according to claim 1, characterized in that: The specific steps of step 2 are as follows: Step 2.1: Iterate through each instruction in the error correction instruction set multiple times to increase the complexity and quality of the instructions; Step 2.2: Select samples one by one from the error correction instruction set and add them to the selected dataset. Calculate the cosine similarity between the sample and the selected dataset, and filter out samples with a cosine similarity greater than the threshold to increase instruction complexity.

4. The Chinese grammar correction method based on large model fine-tuning according to claim 1, characterized in that: The specific steps of step 3 are as follows: Step 3.1: Fine-tune the self-attention layer; Step 3.2: Set the rank of LORA, starting with a small rank value and gradually increasing it, while observing the performance changes of the model on the validation set to determine an optimal rank; Step 3.3: Set the scaling factor of LORA to control the influence of the low-rank matrix on the original weight matrix; start with an initial value of 1, adjust it together with the rank, and continuously observe the model's performance on the training and validation sets, gradually adjusting the scaling factor.

5. A Chinese grammar correction system based on large model fine-tuning, characterized in that: include: A module for performing the Chinese grammar correction method based on large model fine-tuning as described in any one of claims 1 to 4.

Images