Vietnamese grammar error correction corpus construction method based on error type perception of large model

Abstract

The invention relates to a Vietnamese grammar error correction corpus construction method based on error type perception of a large model, and belongs to the field of natural language processing. According to the method, firstly, a voice recognition model is used for simulating Vietnamese grammar errors in a real scene, a preliminary error correction data set is generated, then through deep analysis of distribution rules and grammar structure features of typical errors in the data set, a chain thinking cue (CoT) mechanism fusing error type features is designed in a targeted mode, and the error type features of the Vietnamese grammar errors are extracted. Guiding a large language model (LLM) to generate synthetic statements containing predetermined grammar errors in batches; thirdly, in order to enhance corpus quality, synchronously implementing a web crawler to collect a native Vietnamese text, and constructing a pure monolingual corpus through multi-layer filtering and cleaning; and finally, the generated synthetic data needs to be strictly verified and processed to ensure that the error type is consistent with a preset target, and a pre-training model normal form and a large model normal form are strengthened in a two-stage fine tuning manner, so that the generalization ability of a grammar error correction model is effectively improved, and the problem that Vietnamese grammar error correction corpus is deficient is solved.

Description

Technical Field

[0001] The present invention relates to a method for constructing Vietnamese grammatical error correction corpus based on error type perception of a large model, and belongs to the technical field of natural language processing. Background Art

[0002] As Vietnamese becomes increasingly used in global communication, more and more learners are choosing to master it as a second language. However, in this process, learners often encounter various challenges due to the interference of native language habits. In particular, the significant grammatical differences between Vietnamese and their native language easily lead to grammatical errors. Such problems are particularly prominent in fields that require extremely high linguistic accuracy, such as journalism and government affairs, and are often seen as a lack of professionalism. Therefore, the demand for addressing Vietnamese grammatical errors is growing. Research on Vietnamese grammar correction technology not only has significant academic significance, but also offers broad prospects for practical application.

[0003] In a specific language context, a grammatical correction model can automatically identify and correct grammatical errors in sentences to conform to expected grammatical standards. When an input sentence contains errors, the model can effectively correct them; conversely, if the sentence is correct, the original form is preserved. Grammatical errors typically occur at the lexical, syntactic, and semantic levels, often violating the rules of the target language or the linguistic habits of native speakers. In recent years, grammatical correction techniques based on deep contextual representations have made significant progress in resource-rich languages such as English and Chinese, but progress has been slower in low-resource languages such as Vietnamese. The main bottleneck is the lack of large-scale, publicly available Vietnamese grammatical correction datasets. Currently, there is no publicly available Vietnamese grammatical correction task data, and directly annotating Vietnamese correction corpus requires a significant investment of time and effort. Therefore, exploring a low-cost, efficient method to simulate common grammatical errors made by learners and constructing a Vietnamese grammatical correction dataset covering a variety of application scenarios has become a major challenge. Successfully addressing this challenge will significantly improve the performance of Vietnamese grammatical correction models and further promote the standardized use of Vietnamese and cross-cultural communication in the digital age. Summary of the Invention

[0004] The technical problem solved by the present invention is: the present invention provides a method for constructing Vietnamese grammar correction corpus based on error type perception of a large model, so as to solve the problem of insufficient Vietnamese grammar correction corpus. The present invention obtains relatively high-quality Vietnamese grammar correction corpus through multi-stage fine-tuning.

[0005] The technical solution of the present invention is: a method for constructing Vietnamese grammatical error correction corpus based on error type awareness of a large model, the method comprising:

[0006] Step 1. Construct pseudo data for Vietnamese grammar correction;

[0007] First, we obtain an open source Vietnamese speech dataset through online channels as the basic material. Then, we use the existing Vietnamese speech recognition model to generate pseudo data samples containing grammatical errors. Finally, we perform deduplication and denoising on the generated data, and form an initial pseudo dataset that can be used for grammatical error correction tasks through quality screening.

[0008] Step 2: Constructing synthetic grammatical error data based on error type awareness of the large language model;

[0009] First, we use a grammatical error correction and annotation tool to capture grammatical errors in pseudo-data and count the number and proportion of each error type. We then design chained thinking prompts for each error type that incorporate the characteristics of the error type, guiding the large language model to generate grammatical error data containing the specified error type. Then, we use an annotation tool to capture input sentences and output sentences from the large model, retaining only synthetic data that matches the specified error type.

[0010] Step 3: Collect massive amounts of original Vietnamese language data through web crawler technology to build a large-scale unlabeled Vietnamese benchmark database;

[0011] First, we collected a massive amount of original Vietnamese corpus using web crawler technology to construct a large-scale unlabeled original Vietnamese database. We then performed preprocessing on the data, including text cleaning, format standardization, and language feature screening, to remove invalid and abnormal samples. Finally, we conducted a quality assessment to form a structured unlabeled corpus, from which we extracted high-quality sentences that met Vietnamese standards as benchmark corpus for training grammatical error correction models.

[0012] Step 4: Utilize a pre-trained model and a large language model with Vietnamese language capabilities; strengthen the two correction paradigms through a two-stage fine-tuning approach using Vietnamese grammatical correction pseudo data and grammatical error synthetic data, thereby effectively improving the generalization ability of the grammatical correction model.

[0013] Furthermore, the Step 1 includes:

[0014] Step 1.1. Obtain an open-source Vietnamese speech dataset as the basic data.

[0015] Step 1.2: Use the existing Vietnamese speech recognition model to generate pseudo-data samples containing grammatical errors;

[0016] Step 1.3: De-duplication and denoising are performed on the generated pseudo data samples with grammatical errors to remove duplicate samples and invalid information;

[0017] Step 1.4: Filter out low-quality pseudo data samples with grammatical errors through the quality screening mechanism to form Vietnamese grammatical correction pseudo data that meets the requirements of the grammatical correction task.

[0018] Furthermore, the Step 2 includes:

[0019] Step 2.1. Use the grammatical error correction annotation tool to extract grammatical errors from the Vietnamese grammatical error correction pseudo data and calculate the distribution of error types;

[0020] Step 2.2: Design a chained thinking prompt template based on the characteristics of error type distribution, and guide the large language model to generate synthetic grammatical error data of the target error type;

[0021] Step 2.3: Use annotation tools to compare the input text with the output of the large language model to extract grammatical error samples; the input text refers to the unlabeled data used to inject grammatical errors;

[0022] Step 2.4: Perform error type matching verification and retain the grammatical error synthetic data that is completely consistent with the target error type.

[0023] Furthermore, the Step 3 includes:

[0024] Step 3.1. Use web crawler technology to collect original corpora from multiple fields to build an unlabeled Vietnamese basic database. The original corpora from multiple fields include original corpora from Vietnamese news, medical, and social media.

[0025] Step 3.2: Perform text cleaning, format standardization, and filter text that meets the statistical characteristics of Vietnamese through language models;

[0026] Step 3.3: Establish a multi-dimensional quality assessment system to form a structured unannotated corpus;

[0027] Step 3.4: Use the sliding window sampling method to extract high-quality sentences that conform to Vietnamese standards, which serve as the benchmark corpus for training the Vietnamese grammar correction model.

[0028] Furthermore, the Step 4 includes:

[0029] Step 4.1. Select a pre-trained model and a large language model with Vietnamese language capabilities as the Vietnamese grammar correction model. Pre-trained basic models with Vietnamese language capabilities include Bartpho, and large language models with Vietnamese language capabilities include the Qwen series.

[0030] Step 4.2: Perform the first stage of fine-tuning. Using the constructed Vietnamese grammar correction pseudo data, gradually strengthen the pre-trained basic model and the large language model, so that the pre-trained basic model and the large language model have preliminary grammar correction capabilities.

[0031] Step 4.3: Implement the second stage of fine-tuning, integrate the synthetic data of grammatical errors generated by the large language model, improve the recognition accuracy of the pre-trained basic model and the large language model for various types of grammatical errors, and form a multi-dimensional error correction capability.

[0032] Furthermore, the Step 4 includes:

[0033] During the pre-training model training process, the objective function is optimized by minimizing the cross entropy loss function. The cross entropy loss function formula is as follows:

[0034]

[0035] θ is the trainable pre-trained model parameter, x is the source sentence, y={y1,y2,…,y n} is a correct sentence with n words, y ＜t ={y1,y2,…,y t-1} is the word visible at time step t;

[0036] Use LoRA to fine-tune the large language model. Assume that the weight matrix of a linear layer of the large language model is LoRA decomposes a linear layer weight matrix into:

[0037] W′=W+ΔW=W+BA (2)

[0038] in and It is a low-rank matrix, and the objective function adapts the large language model to downstream tasks by optimizing the parameters of B and A.

[0039] The present invention provides a Vietnamese grammatical error correction corpus construction system based on error type awareness of a large model. The system comprises: a module for executing the Vietnamese grammatical error correction corpus construction method based on error type awareness of a large model.

[0040] The present invention provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, a method for constructing Vietnamese grammatical error correction corpus based on error type awareness of a large model is implemented.

[0041] The present invention provides a non-transitory computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the method for constructing Vietnamese grammatical error correction corpus based on error type awareness of a large model is implemented.

[0042] The present invention provides a computer program product, comprising a computer program. When the computer program is executed by a processor, the method for constructing Vietnamese grammatical error correction corpus based on error type awareness of a large model is implemented.

[0043] The beneficial effects of the present invention are:

[0044] 1. The present invention uses a Vietnamese grammatical correction corpus construction algorithm based on error type perception of a large model to automatically construct a certain scale of noisy grammatical correction corpus, and the generated corpus conforms to the grammatical errors in real scenarios;

[0045] 2. Unlike traditional manual annotation methods, this invention uses a large model of error type distribution guide words to automatically construct Vietnamese grammar correction corpus, effectively controlling the cost of corpus construction;

[0046] 3. The present invention adopts a phased fine-tuning approach. By fine-tuning two grammatical error correction paradigms, the experimental results show that the corpus finally screened out is of high quality. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] Figure 1 is a flow chart of the present invention;

[0048] Figure 2 These are the performance results of various models in the embodiments of the present invention on four error types. DETAILED DESCRIPTION

[0049] Example 1: Figure 1-Figure 2 As shown, a method for constructing Vietnamese grammatical error correction corpus based on error type awareness of a large model, the method comprising:

[0050] Step 1: Constructing Vietnamese grammatical error correction pseudo data. Obtain an open-source Vietnamese speech dataset from online channels as basic material, and then use the existing Vietnamese speech recognition model to generate Vietnamese grammatical error correction pseudo data samples containing grammatical errors. Step 1 includes:

[0051] Step 1.1. Obtain an open-source Vietnamese speech dataset as the basic data.

[0052] Step 1.2: Use the existing Vietnamese speech recognition model to generate pseudo-data samples containing grammatical errors;

[0053] Step 1.3: De-duplication and denoising are performed on the generated pseudo data samples with grammatical errors to remove duplicate samples and invalid information;

[0054] Step 1.4: Filter out low-quality pseudo data samples with grammatical errors through the quality screening mechanism to form Vietnamese grammatical correction pseudo data that meets the requirements of the grammatical correction task.

[0055] Step 2: Construct synthetic grammatical error data based on error type awareness in the large language model; obtain grammatical errors in pseudo-data and count the distribution of various error types; design chain thinking prompts for each error type that incorporate error type characteristics, guiding the large language model to generate grammatical error data containing the specified error type;

[0056] Step 2 includes:

[0057] Step 2.1. Use the grammatical error correction annotation tool to extract grammatical errors from the Vietnamese grammatical error correction pseudo data and calculate the distribution of error types;

[0058] Step 2.2: Design a chained thinking prompt template based on the characteristics of error type distribution, and guide the large language model to generate synthetic grammatical error data of the target error type;

[0059] Step 2.3: Use annotation tools to compare the input text with the output of the large language model to extract grammatical error samples; the input text refers to the unlabeled data used to inject grammatical errors;

[0060] Step 2.4: Perform error type matching verification and retain the grammatical error synthetic data that is completely consistent with the target error type.

[0061] Table 1 shows the design of the large model thinking chain for error type perception

[0062]

[0063]

[0064] Step 3: Collect massive amounts of original Vietnamese language data through web crawler technology to build a large-scale unlabeled Vietnamese benchmark database;

[0065] Step 3 includes:

[0066] Step 3.1. Use web crawler technology to collect original corpora from multiple fields to build an unlabeled Vietnamese basic database. The original corpora from multiple fields include original corpora from Vietnamese news, medical, and social media.

[0067] Step 3.2: Perform text cleaning (removing special symbols / advertising codes), format standardization (unifying encoding and paragraph separators), and filter text that meets Vietnamese statistical characteristics through language modeling.

[0068] Step 3.3: Establish a multi-dimensional quality assessment system (completeness / consistency / grammatical rationality) to form a structured unannotated corpus;

[0069] Step 3.4: Use the sliding window sampling method to extract high-quality sentences that conform to Vietnamese standards, which serve as the benchmark corpus for training the Vietnamese grammar correction model.

[0070] Step 4: Fine-tune the Vietnamese model in stages. Optimize the grammatical correction paradigms of the pre-trained model and the large oracle model: Utilize a pre-trained model and a large language model with Vietnamese language capabilities. Strengthen these two correction paradigms through a two-stage fine-tuning approach using Vietnamese grammatical correction pseudo data and synthetic grammatical error data, effectively improving the generalization capabilities of the grammatical correction model.

[0071] 1) For the error correction task, we employed the widely used Transformer architecture, transforming this task into a sequence-to-sequence problem. This architecture consists primarily of two parts: an encoder and a decoder. The encoder uses a multi-head attention mechanism to model the context of each word in the input sentence, thereby more fully capturing the associations between words. The decoder, on the other hand, is based on a similar structure but incorporates a masked multi-head self-attention module to more precisely model the generation of the target sequence. The introduction of this masking mechanism effectively prevents information leakage, ensuring that the decoder relies only on previously generated parts when generating the current word, thereby improving the accuracy and coherence of the generated results. Overall, the Transformer architecture demonstrates excellent performance in error correction tasks, thanks to its powerful context capture capabilities and reasonable masking mechanism, making the system more intelligent and reliable when correcting errors.

[0072] Furthermore, the Large Language Model demonstrates powerful capabilities in text error correction. Based on massive amounts of data and deep learning techniques, it comprehensively analyzes input text and automatically detects and corrects errors. This decoder-only model, combined with an advanced Transformer architecture, gradually generates corrected text. The Large Language Model possesses advanced semantic understanding capabilities, capable of identifying not only grammatical errors, improper syntax, and misused punctuation, but also more complex contextual logic errors. Through multiple rounds of attention, the model meticulously models the relationships between words in a sentence, ensuring that the corrected text is more semantically accurate and natural.

[0073] Step 4 includes:

[0074] Step 4.1. Select a pre-trained model and a large language model with Vietnamese language capabilities as the Vietnamese grammar correction model. Pre-trained basic models with Vietnamese language capabilities include Bartpho, and large language models with Vietnamese language capabilities include the Qwen series.

[0075] Step 4.2: Perform the first stage of fine-tuning. Using the constructed Vietnamese grammar correction pseudo data, gradually strengthen the pre-trained basic model and the large language model, so that the pre-trained basic model and the large language model have preliminary grammar correction capabilities.

[0076] Step 4.3: Implement the second stage of fine-tuning, integrate the synthetic data of grammatical errors generated by the large language model, improve the recognition accuracy of the pre-trained basic model and the large language model for various types of grammatical errors, and form a multi-dimensional error correction capability.

[0077] During the pre-training model training process, the objective function is optimized by minimizing the cross entropy loss function. The cross entropy loss function formula is as follows:

[0078]

[0079] θ is the trainable pre-trained model parameter, x is the source sentence, y={y1,y2,…,y n} is a correct sentence with n words, y ＜t ={y1,y2,…,y t-1} is the word visible at time step t;

[0080] Use LoRA to fine-tune the large language model. Assume that the weight matrix of a linear layer of the large language model is LoRA decomposes a linear layer weight matrix into:

[0081] W′=W+ΔW=W+BA (2)

[0082] in and It is a low-rank matrix, and the objective function adapts the large language model to downstream tasks by optimizing the parameters of B and A.

[0083] The present invention first uses a speech recognition model to simulate Vietnamese grammatical errors in real scenarios to generate a preliminary error correction dataset. Then, through in-depth analysis of the distribution patterns and grammatical structure characteristics of four typical error types in the dataset, a chained thinking prompt (CoT) mechanism is designed to integrate error type characteristics, guiding the large language model (LLM) to batch generate synthetic sentences containing predetermined grammatical errors. Then, to enhance the quality of the corpus, a web crawler is simultaneously implemented to collect native Vietnamese text, and a pure monolingual corpus is constructed through multi-layer filtering and cleaning. Finally, the generated synthetic data needs to be rigorously verified and processed to ensure that its error types are consistent with the preset targets. The pre-trained model paradigm and the large model paradigm are strengthened through a two-stage fine-tuning approach, thereby effectively improving the generalization ability of the grammatical correction model to address the problem of the shortage of Vietnamese grammatical correction corpus.

[0084] The present invention provides a Vietnamese grammar correction corpus construction system based on error type awareness of a large model, the system comprising:

[0085] The first building module is used to construct Vietnamese grammar correction pseudo data;

[0086] The second building block is used to construct grammatical error synthetic data based on error type awareness of the large language model;

[0087] The third construction module is used to collect massive amounts of original Vietnamese corpus through web crawler technology to build a large-scale unlabeled Vietnamese benchmark database;

[0088] An optimization module is used to leverage pre-trained models and large language models with Vietnamese language capabilities. It strengthens the two correction paradigms through a two-stage fine-tuning approach using Vietnamese grammatical correction pseudo data and synthetic grammatical error data, effectively improving the generalization ability of the grammatical correction model.

[0089] The present invention provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, a method for constructing Vietnamese grammatical error correction corpus based on error type awareness of a large model is implemented.

[0090] The present invention provides a non-transitory computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the method for constructing Vietnamese grammatical error correction corpus based on error type awareness of a large model is implemented.

[0091] The present invention provides a computer program product, comprising a computer program. When the computer program is executed by a processor, the method for constructing Vietnamese grammatical error correction corpus based on error type awareness of a large model is implemented.

[0092] The present invention adopts the precision rate P, recall rate R, and F which are commonly used in the field of grammatical error correction based on words. 0.5 As an evaluation metric for Vietnamese grammatical error correction models.

[0093]

[0094] Among them, TP (True Positives) is the number of errors correctly detected and corrected by the model, FP (False Positives) is the number of correct parts incorrectly marked or modified by the model, and FN (False Negatives) is the number of true errors that the model failed to detect and correct.

[0095] To verify the effectiveness of our proposed method in improving the performance of Vietnamese grammar correction models, we selected two mainstream grammar correction models as benchmarks. In this validation, we used the open-source BARTpho, Qwen2.5, and LLama3.1 models for pre-training and fine-tuning. Table 2 shows the source and size of the experimental corpus.

[0096] Table 2 shows the sources and scale of some of the corpus involved in the experiment.

[0097]

[0098] The data used for fine-tuning Qwen2.5 and LLama3.1 is in the form of a triple: {Prompt, Input, Output}. An example of the data format for fine-tuning a large model is shown in Table 3:

[0099] Table 3 shows an example of the data format for fine-tuning large models.

[0100]

[0101] Table 4 shows the results after the fusion of ASR and ETAGC. The present invention compares the performance of three models (Llama3.1, Qwen2.5 and BARTpho) before and after using additional data. The additional data is constructed through an error type perception generation method based on a large model, with a scale of 150,000. First of all, we can observe that after using additional data, the precision of Llama3.1 and Qwen2.5 has been significantly improved, but the recall rate has dropped slightly, but the comprehensive indicators have still been significantly improved, which shows that the additional data has effectively enhanced the accuracy of the model. In contrast, BARTpho has performed better without using additional data. The small performance improvement after adding additional data indicates that its initial performance is close to saturation. Despite this, the benefits brought by the introduction of additional data are still considerable.

[0102] Notably, all models experienced a slight decrease in recall after using the additional data. This is likely because the error-type-aware generated data focuses more on correcting specific errors, causing the model to be more conservative in generating results, thereby reducing coverage. Nevertheless, the additional data still significantly improved the overall performance of the models, especially for Llama3.1 and Qwen2.5, which initially performed poorly.

[0103] Table 4: Performance of each model on ASR-Test

[0104]

[0105] Figure 2The error correction performance of various GEC systems across four major error types is demonstrated. Overall, our proposed method (OUR) demonstrates significant performance advantages over all other models. In particular, in experiments based on the LLM, our method achieves significant improvements, exhibiting balanced performance across different error types and demonstrating good generalization. This robustness across error categories enables our method to be effectively adapted to various systems and scenarios.

[0106] The specific embodiments of the present invention are described in detail above with reference to the accompanying drawings. However, the present invention is not limited to the above embodiments. Various changes can be made within the knowledge of ordinary technicians in this field without departing from the scope of the present invention.

Claims

1. A method for constructing Vietnamese grammatical correction corpus based on error type awareness based on a large model, characterized by: The method comprises: Step 1. Construct pseudo data for Vietnamese grammar correction; Step 2: Constructing synthetic grammatical error data based on error type awareness of the large language model; Step 3: Collect massive amounts of original Vietnamese language data through web crawler technology to build a large-scale unlabeled Vietnamese benchmark database; Step 4: Utilize a pre-trained model and a large language model with Vietnamese language capabilities; strengthen the two correction paradigms through a two-stage fine-tuning approach using Vietnamese grammatical correction pseudo data and grammatical error synthetic data, thereby effectively improving the generalization ability of the grammatical correction model.

2. The method for constructing Vietnamese grammatical correction corpus based on error type awareness based on a large model according to claim 1, characterized in that: Step 1 includes: Step 1.

1. Obtain an open-source Vietnamese speech dataset as the basic data. Step 1.2: Use the existing Vietnamese speech recognition model to generate pseudo-data samples containing grammatical errors; Step 1.3: De-duplication and denoising are performed on the generated pseudo data samples with grammatical errors to remove duplicate samples and invalid information; Step 1.4: Filter out low-quality pseudo data samples with grammatical errors through the quality screening mechanism to form Vietnamese grammatical correction pseudo data that meets the requirements of the grammatical correction task.

3. The method for constructing Vietnamese grammatical correction corpus based on error type awareness based on a large model according to claim 1, characterized in that: Step 2 includes: Step 2.

1. Use the grammatical error correction annotation tool to extract grammatical errors from the Vietnamese grammatical error correction pseudo data and calculate the distribution of error types; Step 2.2: Design a chained thinking prompt template based on the characteristics of error type distribution, and guide the large language model to generate synthetic grammatical error data of the target error type; Step 2.3: Use annotation tools to compare the input text with the output of the large language model to extract grammatical error samples; the input text refers to the unlabeled data used to inject grammatical errors; Step 2.4: Perform error type matching verification and retain the grammatical error synthetic data that is completely consistent with the target error type.

4. The method for constructing Vietnamese grammatical correction corpus based on error type awareness based on a large model according to claim 1, characterized in that: Step 3 includes: Step 3.

1. Use web crawler technology to collect original corpora from multiple fields to build an unlabeled Vietnamese basic database. The original corpora from multiple fields include original corpora from Vietnamese news, medical, and social media. Step 3.2: Perform text cleaning, format standardization, and filter text that meets the statistical characteristics of Vietnamese through language models; Step 3.3: Establish a multi-dimensional quality assessment system to form a structured unannotated corpus; Step 3.4: Use the sliding window sampling method to extract high-quality sentences that conform to Vietnamese standards, which serve as the benchmark corpus for training the Vietnamese grammar correction model.

5. The method for constructing Vietnamese grammatical correction corpus based on error type awareness based on a large model according to claim 1, characterized in that: Step 4 includes: Step 4.

1. Select a pre-trained model and a large language model with Vietnamese language capabilities as the Vietnamese grammar correction model. Pre-trained basic models with Vietnamese language capabilities include Bartpho, and large language models with Vietnamese language capabilities include the Qwen series. Step 4.2: Perform the first stage of fine-tuning. Using the constructed Vietnamese grammar correction pseudo data, gradually strengthen the pre-trained basic model and the large language model, so that the pre-trained basic model and the large language model have preliminary grammar correction capabilities. Step 4.3: Implement the second stage of fine-tuning, integrate the synthetic data of grammatical errors generated by the large language model, improve the recognition accuracy of the pre-trained basic model and the large language model for various types of grammatical errors, and form a multi-dimensional error correction capability.

6. The method for constructing Vietnamese grammatical correction corpus based on error type awareness based on a large model according to claim 1, characterized in that: Step 4 includes: During the pre-training model training process, the objective function is optimized by minimizing the cross entropy loss function. The cross entropy loss function formula is as follows: θ is the trainable pre-trained model parameter, x is the source sentence, y={y1,y2,…,y n } is a correct sentence with n words, y ＜t ={y1,y2,…,y t-1 } is the word visible at time step t; Use LoRA to fine-tune the large language model. Assume that the weight matrix of a linear layer of the large language model is LoRA decomposes a linear layer weight matrix into: W′=W+ΔW=W+BA (2) in and It is a low-rank matrix, and the objective function adapts the large language model to downstream tasks by optimizing the parameters of B and A.

7. A Vietnamese grammar correction corpus construction system based on error type awareness of a large model, characterized by: The system includes: a module for executing the method for constructing Vietnamese grammatical error correction corpus based on error type awareness based on a large model as described in any one of claims 1 to 6.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the method for constructing Vietnamese grammatical correction corpus based on error type awareness based on a large model as described in any one of claims 1 to 5 is implemented.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method for constructing Vietnamese grammatical correction corpus based on error type awareness based on a large model as described in any one of claims 1 to 6 is implemented.

10. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the method for constructing Vietnamese grammatical correction corpus based on error type awareness based on a large model as described in any one of claims 1 to 6 is implemented.

Images