Large model-based text error correction method and apparatus, device, and storage medium

Abstract

A large model-based text error correction method and apparatus, a device, and a storage medium, relating to the technical field of artificial intelligence. The method comprises: on the basis of a large language model, generating first training data according to a first preset instruction, and manually annotating preset data to generate second training data, so as to construct a training data set; constructing a corresponding target data set by means of second preset instructions corresponding to a preset text error correction task and the training data set, so as to use the target data set to fine-tune the large language model to obtain a target large language model; and acquiring a text to be error-corrected, and determining error correction instructions corresponding to said text, so as to concatenate said text and the error correction instructions and then input the concatenated result into the target large language model to obtain an error-corrected target text. The data generated on the basis of the large language model is mixed with the manually annotated data, and a large model is fine-tuned with different instructions for different tasks, so that spelling error correction and grammatical error correction are chained in series, thereby effectively improving the accuracy of text error correction.

Description

A text correction method, apparatus, device, and storage medium based on a large model

[0001] This application claims priority to Chinese Patent Application No. 202410920292.4, filed on July 10, 2024, entitled "A Text Correction Method, Apparatus, Device and Storage Medium Based on a Large Model", the entire contents of which are incorporated herein by reference. Technical Field

[0002] This invention relates to the field of artificial intelligence technology, and in particular to a text correction method, apparatus, device, and storage medium based on a large model. Background Technology

[0003] In recent years, large language models (LLMs) have made significant progress in the field of artificial intelligence, especially in multilingual processing capabilities, including Chinese processing. Chinese text correction is a fundamental and challenging task in Chinese natural language processing (NLP), primarily encompassing two tasks: Chinese grammar correction and Chinese spelling correction. However, on the one hand, many current studies and methods focus mainly on solving one of these two tasks, and the training data is often modified from exams taken by non-native Chinese speakers, which may not accurately reflect the characteristics of formal Chinese texts due to limitations in vocabulary and expression. On the other hand, traditional text correction relies on small NLP models, lacking understanding of context, and typically employs a method of detecting errors first and then correcting them, often resulting in limited accuracy.

[0004] In summary, existing text correction methods have many limitations, primarily in the low quality and difficulty in obtaining training data, which does not conform to the usage habits of native Chinese speakers. Furthermore, existing solutions typically only address one aspect of text correction tasks, limiting their applicability. Additionally, the overall accuracy of text correction is currently not high. Therefore, how to further improve the accuracy of text correction is a problem that needs to be solved in this field. Summary of the Invention

[0005] In view of this, the purpose of this invention is to provide a text correction method, apparatus, device, and storage medium based on a large model. This method generates data based on a large language model, mixes it with manually annotated data, and fine-tunes the large model using different instructions for different tasks. This links spell correction and grammar correction, effectively improving the accuracy of text correction. The specific solution is as follows:

[0006] Firstly, this application provides a text correction method based on a large model, including:

[0007] Based on a preset large language model, corresponding first training data is generated according to a first preset instruction, and corresponding second training data is generated by manually annotating the preset data according to preset manual annotation rules. A training dataset is constructed based on the first training data and the second training data. The training dataset includes text spelling correction data pairs and text syntax correction data pairs.

[0008] A corresponding target dataset is constructed by using the second preset instruction corresponding to the preset text correction task and the training dataset, so as to fine-tune the preset large language model using the target dataset to obtain the target large language model; the second preset instruction includes spelling correction instruction and grammar correction instruction corresponding to the text spelling correction data pair and the text grammar correction data pair, respectively.

[0009] Obtain the text to be corrected and determine the correction instruction corresponding to the text to be corrected. Then, concatenate the text to be corrected and the correction instruction and input them into the target large language model to obtain the corrected target text.

[0010] Optionally, constructing the training dataset based on the first training data and the second training data includes:

[0011] The first training data is manually filtered based on preset data filtering rules;

[0012] The second training data and the filtered first training data are mixed according to a preset data ratio to obtain an initial dataset;

[0013] The initial dataset is deduplicated based on the textual semantic features of the training data in the initial dataset, and then data augmentation is performed on the deduplicated initial dataset to obtain the training dataset.

[0014] Optionally, the step of performing data augmentation on the deduplicated initial dataset to obtain the training dataset includes:

[0015] Identify the existing entities in the training data in the initial dataset, and identify the target entities in the data in the preset data augmentation dataset;

[0016] The target entity is used to replace the existing entity in the training data to obtain the training dataset based on the training data after replacing the existing entity.

[0017] Optionally, the step of generating corresponding first training data based on a preset large language model according to a first preset instruction includes:

[0018] Determine the fixed paradigms corresponding to preset spelling errors and preset grammar errors, and construct the corresponding first preset instruction through the fixed paradigms;

[0019] Based on the first preset instruction, error data corresponding to the preset spelling error and the preset grammar error are generated respectively, and corresponding correct data is generated based on the error data;

[0020] The text spelling correction data pair and the text grammar correction data pair are constructed based on the error data and the correct data, so as to construct the first training data based on the text spelling correction data pair and the text grammar correction data pair.

[0021] Optionally, the step of constructing the corresponding target dataset using the second preset instruction corresponding to the preset text correction task and the training dataset includes:

[0022] Determine the error correction rules for the preset text error correction task, and determine the second preset instruction corresponding to each task based on the error correction rules;

[0023] First instruction data is constructed based on the second preset instruction and the corresponding text spelling correction data pair and text grammar correction data pair, so as to construct the target dataset based on the first instruction data.

[0024] Optionally, the process of constructing the first instruction data based on the second preset instruction and the corresponding text spelling correction data pair and text grammar correction data pair further includes:

[0025] Based on the preset error-free data and the second preset instruction, corresponding second instruction data is constructed to construct the target dataset according to the first instruction data and the second instruction data.

[0026] Optionally, the step of concatenating the text to be corrected and the correction instruction and inputting them into the target large language model to obtain the corrected target text includes:

[0027] The text to be corrected and the corresponding spelling correction instruction in the correction instruction are concatenated and then input into the target large language model to obtain the spelling correction result;

[0028] The spelling correction result and the corresponding grammar correction instruction in the correction instruction are concatenated and input into the target large language model to obtain the grammar correction result, and the grammar correction result is used as the target text.

[0029] Secondly, this application provides a text correction device based on a large model, comprising:

[0030] The dataset construction module is used to generate corresponding first training data based on a preset large language model according to a first preset instruction, and to manually annotate the preset data according to preset manual annotation rules to generate corresponding second training data, and to construct a training dataset based on the first training data and the second training data; the training dataset includes text spelling correction data pairs and text syntax correction data pairs;

[0031] The model fine-tuning module is used to construct a corresponding target dataset by using a second preset instruction corresponding to a preset text error correction task and the training dataset, so as to use the target dataset to fine-tune the preset large language model to obtain a target large language model; the second preset instruction includes spelling error correction instructions and grammar error correction instructions corresponding to the text spelling error correction data pair and the text grammar error correction data pair, respectively.

[0032] The text correction module is used to acquire the text to be corrected and determine the correction instruction corresponding to the text to be corrected, so as to concatenate the text to be corrected and the correction instruction and input them into the target large language model to obtain the corrected target text.

[0033] Thirdly, this application provides an electronic device, which includes a processor and a memory; wherein the memory is used to store a computer program, which is loaded and executed by the processor to implement the aforementioned large-model-based text correction method.

[0034] Fourthly, this application provides a computer-readable storage medium for storing a computer program that, when executed by a processor, implements the aforementioned large-model-based text correction method.

[0035] This application first generates corresponding first training data based on a preset large language model according to a first preset instruction, and then manually annotates the preset data using preset manual annotation rules to generate corresponding second training data. A training dataset is constructed based on the first and second training data. Then, a corresponding target dataset is constructed using the second preset instruction corresponding to a preset text correction task and the training dataset. The preset large language model is then fine-tuned using the target dataset to obtain a target large language model. After obtaining the text to be corrected, the corresponding correction instruction is determined. The text to be corrected and the correction instruction are concatenated and input into the target large language model to obtain the corrected target text. Through the above technical solution, this application can automatically generate data using a large language model, mix in a certain amount of manually annotated data to supplement the current lack of data on major grammatical error types, and then construct different instruction data for different tasks to fine-tune the large model. By concatenating different instructions, the spelling correction function and the grammar correction function are linked together to generate the corrected text. In this way, this application can unify the solutions for two correction tasks into one model, effectively improving the effect and accuracy of text correction. Attached Figure Description

[0036] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0037] Figure 1 is a flowchart of a text correction method based on a large model provided in this application;

[0038] Figure 2 is a hybrid data flow diagram provided in this application;

[0039] Figure 3 is a flowchart of a specific text correction method based on a large model provided in this application;

[0040] Figure 4 is a flowchart of a data generation process based on a large model provided in this application;

[0041] Figure 5 is a flowchart of an error correction task processing method based on a large model provided in this application;

[0042] Figure 6 is a schematic diagram of the instructions for an error correction task provided in this application;

[0043] Figure 7 is a flowchart of a text correction process based on a large model provided in this application;

[0044] Figure 8 is a schematic diagram of a text correction device based on a large model provided in this application;

[0045] Figure 9 is a structural diagram of an electronic device provided in this application. Detailed Implementation

[0046] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0047] Chinese text correction mainly includes two tasks: Chinese grammar correction and Chinese spelling correction. However, current research methods mainly focus on solving one of these two tasks and rely on small natural language models, lacking understanding of context. As a result, the accuracy of correction is usually limited. This application, on the other hand, can automatically generate data through a large language model, mix it with a certain amount of manually annotated data, construct different instruction data for different tasks to fine-tune the large model, and then connect the spelling correction function and the grammar correction function by concatenating different instructions to generate corrected text, which can effectively improve the effect and accuracy of text correction.

[0048] Referring to Figure 1, this embodiment of the invention discloses a text correction method based on a large model, including:

[0049] Step S11: Generate corresponding first training data based on the preset large language model according to the first preset instruction, and manually annotate the preset data according to the preset manual annotation rules to generate corresponding second training data, and construct a training dataset based on the first training data and the second training data; the training dataset includes text spelling correction data pairs and text grammar correction data pairs.

[0050] Understandably, this embodiment takes into account that in text correction tasks, spelling correction tasks include errors such as phonetic and visual similarities. The corrected text is usually the same length as the original text, making data relatively easy to obtain, but data quality is difficult to control. Grammar correction tasks, on the other hand, include errors such as illogicality, incomplete components, redundant components, and incorrect word order. In these cases, the corrected text is usually different in length from the original text, making data difficult to obtain, and the data distribution is uneven due to the large number of error types. Therefore, this embodiment first generates corresponding first training data based on a preset large language model according to a first preset instruction, and then manually annotates the preset data using preset manual annotation rules to generate corresponding second training data. Based on the first and second training data, a training dataset including text spelling correction data pairs and text grammar correction data pairs is constructed. Furthermore, based on the aforementioned dataset construction process, as shown in Figure 2, when constructing the training dataset, the first training data can be manually selected based on preset data filtering rules. The second training data and the selected first training data are then mixed according to a preset data ratio to obtain an initial dataset. Next, data deduplication is performed on the initial dataset based on the textual semantic features of the training data in the initial dataset. Data augmentation is then performed on the deduplicated initial dataset to obtain the training dataset. Simultaneously, when performing data augmentation on the deduplicated initial dataset, the existing entities in the training data in the initial dataset are first identified, and the target entities in the preset data augmentation dataset are determined. The target entities are then used to replace the existing entities in the training data, thus obtaining the training dataset based on the training data with the replaced existing entities. In this way, by adopting a mixed data construction method and using clues to guide the large model in generating data of a specified error type, not only is it easier to obtain training data, but the quality and diversity of the data are also guaranteed.

[0051] Specifically, in this embodiment, the manual annotation data module can manually annotate text data errors based on clues to ensure data balance of different error types and obtain a certain amount of data. <text correct text errorThe data pairs are then combined using a data merging module. This involves mixing manually selected ChatGPT-generated data pairs with manually labeled data pairs in a specific ratio to obtain a preliminary mixed data pair. It's important to note that since both automatically generated and manually labeled data may contain duplicates, a data augmentation module is needed to extract semantic features from the text. Then, feature similarity comparison is used to remove duplicates. In this embodiment, to enhance the model's focus on specific error types and improve its robustness, the data augmentation module employs an error-invariant data augmentation method. Specifically, this involves replacing existing entities in parallel data with similar named entities. This allows the model to truly understand the fixed paradigms of different error types (e.g., in the text "The Bird's Nest Stadium can accommodate approximately 80,000 people.", "approximately" and "more" are redundant components), rather than specific nouns. In other words, named entities (such as personal names and place names) are replaced with semantically similar entities, such as replacing "Shanghai" with "Beijing". In this way, the model can still identify and correct the same grammatical errors when faced with different nouns, and through the above steps, a sufficient number of cleaned and enhanced error-correction data pairs can be obtained. <text correct text error >

[0052] Through the above technical solution, this embodiment proposes a method of manual annotation and automatic generation of large models to obtain data through a hybrid data construction module. Then, a sufficient amount of high-quality data is obtained through manual screening, and multi-task hybrid data is obtained by mixing them in a certain proportion. Finally, data augmentation is performed to improve the diversity of data and avoid the current problems that affect the training of text correction models due to data acquisition and data quality issues.

[0053] Step S12: Construct a corresponding target dataset using the second preset instruction corresponding to the preset text correction task and the training dataset, so as to fine-tune the preset large language model using the target dataset to obtain the target large language model; the second preset instruction includes spelling correction instruction and grammar correction instruction corresponding to the text spelling correction data pair and the text grammar correction data pair, respectively.

[0054] In this embodiment, after the training dataset is constructed, a corresponding target dataset needs to be constructed using the second preset instruction corresponding to the preset text correction task and the training dataset. This target dataset is then used to fine-tune the preset large language model to obtain the target large language model. Based on the above steps, the second preset instruction in this embodiment includes spelling correction instructions and grammar correction instructions corresponding to text spelling correction data pairs and text grammar correction data pairs, respectively.

[0055] Step S13: Obtain the text to be corrected and determine the correction instruction corresponding to the text to be corrected. Then, concatenate the text to be corrected and the correction instruction and input them into the target large language model to obtain the corrected target text.

[0056] In this embodiment, after the target large language model obtains the user's text to be corrected, it can first determine the correction instruction corresponding to the text to be corrected, and then concatenate the text to be corrected and the correction instruction and input them into the target large language model so that the text can be corrected by the fine-tuned large language model to obtain the corrected target text.

[0057] Through the above technical solution, in this embodiment, before performing text correction, large language models can be used to automatically generate data to supplement the lack of data for grammatical error tasks. In this process, manually labeled data is mixed with automatically generated data, and then high-quality data is obtained through manual screening. Then, semantic feature extraction and error-invariant data augmentation methods are used to improve the diversity of data and the robustness of the model. Furthermore, a large model for fine-tuning instruction data for different tasks is constructed. In this way, a text correction method based on hybrid data construction and large model is provided. By using hybrid data construction and large model instruction fine-tuning, this embodiment can better cope with most task scenarios of Chinese text correction, meet the needs of different business scenarios, and has higher text correction accuracy and efficiency.

[0058] Based on the previous embodiment, this application can construct different instruction data to fine-tune the large model for different tasks, thereby linking spelling correction and grammar correction functions through different instruction concatenation. Next, this embodiment will describe the text correction process in detail. Referring to Figure 3, this application embodiment discloses a specific text correction method based on a large model, including:

[0059] Step S21: Generate corresponding first training data based on the preset large language model according to the first preset instruction, and manually annotate the preset data according to the preset manual annotation rules to generate corresponding second training data, and construct a training dataset based on the first training data and the second training data.

[0060] In this embodiment, it is understood that generating candidate data using ChatGPT requires providing relevant clues to the model, and then using instructions to trigger the model to generate the required results. However, due to the complexity of Chinese text correction tasks, some everyday expressions, although fluent, may still contain grammatical errors from a grammatical perspective. Furthermore, for the model, directly using instructions (prompts) to trigger the generation of incorrect text from correct text or to generate correct text from input error-correcting text is difficult. Therefore, in constructing the dataset in this embodiment, a fixed paradigm corresponding to preset spelling errors and preset grammatical errors can be determined, and a corresponding first preset instruction can be constructed using the fixed paradigm. Then, error data corresponding to the preset spelling errors and preset grammatical errors can be generated according to the first preset instruction, and corresponding correct data can be generated based on the error data. Thus, text spelling correction data pairs and text grammatical correction data pairs are constructed based on the error data and correct data, and the first training data is constructed based on the text spelling correction data pairs and text grammatical correction data pairs.

[0061] As shown in Figure 4, since Chinese spelling and grammatical errors usually contain fixed patterns (clues), this embodiment constructs instructions using these clues. Figure 4 shows how to generate data using prompts for spelling and grammatical errors, respectively. The upper part of the figure shows the generation of text containing typos using spelling error clues and ChatGPT, denoted as `text`. error Understandably, since the clues include ("the wandering Weaver Girl", "the Cowherd and the Weaver Girl"), batch replacement can be performed after the data generation is complete to obtain the corresponding text. correct The lower half of the diagram shows text containing syntax errors generated using syntax error clues and ChatGPT. The method described above can be used to obtain a large number of ChatGPT-generated data pairs containing different error types. <text correct text error Furthermore, as can be seen from the previous embodiment, since the data generated by the model may inevitably have some deviations, manual screening can be performed to ensure the overall quality of the data. Moreover, since the generated data contains error clues, manual screening can be performed quickly. Therefore, compared with manual annotation, this embodiment can also reduce the amount of manual work and improve the efficiency of data generation.

[0062] In this way, using clues to guide model generation not only reduces the difficulty of text generation, but also ensures the accuracy of the generated data. In this embodiment, the clues include error types, which can also ensure that the amount of data of different types of errors is roughly balanced, thus solving the problem of insufficient data and low quality in the current field of text correction.

[0063] Step S22: Construct a corresponding target dataset using the second preset instruction corresponding to the preset text correction task and the training dataset, so as to fine-tune the preset large language model using the target dataset to obtain the target large language model.

[0064] Understandably, when fine-tuning a large language model, instruction fine-tuning is a necessary step to make the large model follow human instructions. Therefore, a sufficient amount of instruction data needs to be prepared according to the task. In this embodiment, when constructing the target dataset, the error correction rules for the preset text correction task can be determined, and the second preset instructions corresponding to each task can be determined according to the error correction rules. Then, the first instruction data is constructed based on the second preset instructions and the corresponding text spelling error correction data pairs and text syntax error correction data pairs, so as to construct the target dataset based on the first instruction data.

[0065] As shown in Figure 5, since text correction in this embodiment involves two tasks—spelling errors focus more on details (words) and grammatical errors require more contextual understanding—this embodiment avoids training different models for different tasks. Instead, it triggers the model to handle different correction tasks through different prompts. Specifically, the LLM (Large Language Model) used in this embodiment, without guidance for a specific task, may act arbitrarily when performing text correction, thus violating the basic settings and principles of Chinese text correction tasks. Most Chinese text correction datasets and evaluation metrics focus on the principle of minimal change, meaning the model needs to perform as few edits as possible on the input sentence. However, spelling error correction tasks expect to maintain the text length, while grammatical error correction tasks expect the model to make appropriate modifications (involving additions, deletions, and alterations) based on contextual understanding, allowing the model some flexibility. Therefore, in order to utilize the basic capabilities of the large model and be compatible with two different error correction tasks, different prompts as shown in Figure 6 can be used to guide the model output in this embodiment. In this way, different prompt instruction data can be constructed for different text error correction tasks, so that the fine-tuned model can have both spell correction and grammar correction capabilities, unifying the solutions for the two tasks into one model and improving the model's error correction capabilities and accuracy.

[0066] When fine-tuning the model, the results obtained from the above steps can be specifically used as follows: <text correct text error The data pairs, combined with the above prompt, form instruction data.<instruction,output> The instruction is either prompt or text. error splicing, text correctThis is directly used as the correct output result. Furthermore, in this embodiment, during the construction of the first instruction data, corresponding second instruction data can be constructed based on preset error-free data and second preset instructions, so as to construct the target dataset according to the first and second instruction data. In this way, by adding some error-free data to create instruction types, the model can more strictly return the original text when faced with error-free input, while ensuring that the model truly possesses Chinese text error correction capabilities during training.

[0067] Once the aforementioned instruction data is prepared, this embodiment can perform instruction fine-tuning of the large model. For example, using LoRa fine-tuning, the pre-trained model weights are frozen during training, and only a trainable rank decomposition matrix is ​​injected into each layer of the Transformer architecture. Finally, a large model with Chinese text correction capabilities is obtained. The pre-trained Transformer architecture model has powerful text processing capabilities, which helps to improve the accuracy of the model in the field of text correction.

[0068] Step S23: Obtain the text to be corrected and determine the correction instruction corresponding to the text to be corrected. Then, concatenate the text to be corrected and the correction instruction and input them into the target large language model to obtain the corrected target text.

[0069] In this embodiment, the text to be corrected and the corresponding spelling correction instructions are concatenated and input into the target large language model to obtain the spelling correction result. Then, the spelling correction result and the corresponding grammar correction instructions are concatenated and input into the target large language model to obtain the grammar correction result, which is then used as the target text. This embodiment mainly utilizes different prompts to trigger the model's capabilities, achieving spelling and grammar correction. Specifically, as shown in Figure 7, after inputting a text, it is first concatenated with the spelling task prompt, then input into the model to obtain the spelling correction result. This result is then concatenated with the grammar task prompt and input into the model again to obtain the final corrected result. It is understood that in this embodiment, since the spelling task only corrects possible typos without changing the text length, it preserves the original text to the greatest extent possible; therefore, spelling correction is necessary first.

[0070] Through the above technical solution, this embodiment first generates a sufficient amount of data automatically using a large language model. Then, it obtains complete training data by mixing manually labeled data. Next, it constructs text correction type instruction data and fine-tunes the large language model. This allows the previously trained model to link spelling and grammar correction functions through the concatenation of different instructions, achieving text correction for different inputs. This not only compensates for the current lack of data for grammatical error tasks but also solves the problem of uneven distribution of data for different types of errors. Furthermore, by fine-tuning the large language model and leveraging its powerful contextual understanding capabilities, it improves the model's text correction ability, overcoming the limitations of existing text correction technologies, such as low accuracy for native Chinese speakers and the fact that most methods only detect and correct spelling or grammatical errors in text correction scenarios. Meanwhile, by adding error-free data to the instruction data, the model can return the original text for correct text, improving the model's understanding of text correction tasks. Furthermore, different prompt instruction data are constructed for different text correction tasks, allowing the fine-tuned model to have both spell correction and grammar correction capabilities. The solutions for the two tasks are unified into one model, and then, through a pipeline approach, potential errors in the text are addressed step by step, improving the error correction capability and accuracy.

[0071] Referring to Figure 8, this application also discloses a text correction device based on a large model, comprising:

[0072] The dataset construction module 11 is used to generate corresponding first training data based on a preset large language model according to a first preset instruction, and to manually annotate the preset data according to preset manual annotation rules to generate corresponding second training data, and to construct a training dataset based on the first training data and the second training data; the training dataset includes text spelling correction data pairs and text syntax correction data pairs;

[0073] The model fine-tuning module 12 is used to construct a corresponding target dataset by using a second preset instruction corresponding to a preset text error correction task and the training dataset, so as to use the target dataset to fine-tune the preset large language model to obtain a target large language model; the second preset instruction includes spelling error correction instruction and grammar error correction instruction corresponding to the text spelling error correction data pair and the text grammar error correction data pair, respectively.

[0074] The text correction module 13 is used to acquire the text to be corrected and determine the correction instruction corresponding to the text to be corrected, so as to concatenate the text to be corrected and the correction instruction and input them into the target large language model to obtain the corrected target text.

[0075] This embodiment first generates corresponding first training data based on a preset large language model according to a first preset instruction. Then, it manually annotates the preset data using preset manual annotation rules to generate corresponding second training data. A training dataset is constructed based on the first and second training data. Next, a target dataset is constructed using the second preset instruction corresponding to a preset text correction task and the training dataset. The target dataset is then used to fine-tune the preset large language model to obtain a target large language model. After obtaining the text to be corrected, the corresponding correction instruction is determined. The text to be corrected and the correction instruction are concatenated and input into the target large language model to obtain the corrected target text. In this way, by automatically generating data through the large language model, mixing it with a certain amount of manually annotated data to supplement the current lack of data on major grammatical error types, and then constructing different instruction data for different tasks to fine-tune the large model, the spelling correction function and the grammar correction function are linked together through the concatenation of different instructions to generate the corrected text. Through the above technical solution, this embodiment can unify the solutions for two correction tasks into one model, effectively improving the effect and accuracy of text correction.

[0076] In some specific embodiments, the dataset construction module 11 specifically includes:

[0077] A data filtering unit is used to manually filter the first training data based on preset data filtering rules;

[0078] A data mixing unit is used to mix the second training data and the filtered first training data according to a preset data ratio to obtain an initial dataset;

[0079] The data augmentation submodule is used to deduplicate the initial dataset based on the textual semantic features of the training data in the initial dataset, and to augment the deduplicated initial dataset to obtain the training dataset.

[0080] In some specific embodiments, the data augmentation submodule specifically includes:

[0081] An entity determination unit is used to determine existing entities in the training data in the initial dataset and to determine target entities in the data in the preset data augmentation dataset.

[0082] An entity replacement unit is used to replace the existing entity in the training data with the target entity, so as to obtain the training dataset based on the training data after replacing the existing entity.

[0083] In some specific embodiments, the dataset construction module 11 specifically includes:

[0084] The instruction construction unit is used to determine the fixed paradigm corresponding to the preset spelling error and the preset grammar error, and to construct the corresponding first preset instruction through the fixed paradigm;

[0085] The data generation unit is used to generate error data corresponding to the preset spelling error and the preset grammar error respectively according to the first preset instruction, and to generate corresponding correct data based on the error data;

[0086] The data pair construction unit is used to construct the text spelling error correction data pair and the text grammar error correction data pair based on the error data and the correct data, so as to construct the first training data based on the text spelling error correction data pair and the text grammar error correction data pair.

[0087] In some specific embodiments, the model fine-tuning module 12 specifically includes:

[0088] The instruction determination unit is used to determine the error correction rules of the preset text error correction task, and to determine the second preset instruction corresponding to each task according to the error correction rules;

[0089] The first dataset construction unit is used to construct first instruction data based on the second preset instruction and the corresponding text spelling correction data pair and text syntax correction data pair, so as to construct the target dataset based on the first instruction data.

[0090] In some specific embodiments, the model fine-tuning module 12 further includes:

[0091] The second dataset construction unit is used to construct corresponding second instruction data based on preset error-free data and the second preset instruction, so as to construct the target dataset according to the first instruction data and the second instruction data.

[0092] In some specific embodiments, the text correction module 13 specifically includes:

[0093] The instruction splicing unit is used to splice the text to be corrected and the corresponding spelling correction instruction in the correction instruction and input it into the target large language model to obtain the spelling correction result;

[0094] The text correction unit is used to concatenate the spelling correction result and the corresponding grammar correction instruction in the correction instruction and input them into the target large language model to obtain the grammar correction result, and use the grammar correction result as the target text.

[0095] Furthermore, this application also discloses an electronic device. FIG9 is a structural diagram of an electronic device 20 according to an exemplary embodiment. The content in the figure should not be considered as any limitation on the scope of use of this application.

[0096] Figure 9 is a schematic diagram of the structure of an electronic device 20 provided in an embodiment of this application. The electronic device 20 may specifically include: at least one processor 21, at least one memory 22, a power supply 23, a communication interface 24, an input / output interface 25, and a communication bus 26. The memory 22 stores a computer program, which is loaded and executed by the processor 21 to implement the relevant steps in the large-model-based text correction method disclosed in any of the foregoing embodiments. Furthermore, the electronic device 20 in this embodiment may specifically be an electronic computer.

[0097] In this embodiment, the power supply 23 is used to provide operating voltage for each hardware device on the electronic device 20; the communication interface 24 can create a data transmission channel between the electronic device 20 and external devices, and the communication protocol it follows can be any communication protocol applicable to the technical solution of this application, and is not specifically limited here; the input / output interface 25 is used to acquire external input data or output data to the outside world, and its specific interface type can be selected according to specific application needs, and is not specifically limited here.

[0098] In addition, the memory 22, as a carrier for resource storage, can be a read-only memory, random access memory, disk or optical disk, etc. The resources stored thereon can include operating system 221, computer program 222, etc., and the storage method can be temporary storage or permanent storage.

[0099] The operating system 221 is used to manage and control the various hardware devices on the electronic device 20 and the computer program 222, which may be Windows Server, Netware, Unix, Linux, etc. In addition to including a computer program capable of performing the large-model-based text correction method executed by the electronic device 20 as disclosed in any of the foregoing embodiments, the computer program 222 may further include computer programs capable of performing other specific tasks.

[0100] Furthermore, this application also discloses a computer-readable storage medium for storing a computer program; wherein, when the computer program is executed by a processor, it implements the aforementioned large-model-based text correction method. Specific steps of this method can be found in the corresponding content disclosed in the foregoing embodiments, and will not be repeated here.

[0101] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to in the method section.

[0102] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0103] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein can be implemented directly by hardware, a software module executed by a processor, or a combination of both. The software module can be located in random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art.

[0104] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0105] The technical solutions provided in this application have been described in detail above. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the methods and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A text correction method based on a large model, characterized in that, include: Based on a preset large language model, corresponding first training data is generated according to the first preset instruction, and the preset data is manually annotated according to preset manual annotation rules to generate corresponding second training data, and a training dataset is constructed based on the first training data and the second training data. The training dataset includes text spelling correction data pairs and text syntax correction data pairs; A corresponding target dataset is constructed by using the second preset instruction corresponding to the preset text correction task and the training dataset, so as to fine-tune the preset large language model using the target dataset to obtain the target large language model; the second preset instruction includes spelling correction instruction and grammar correction instruction corresponding to the text spelling correction data pair and the text grammar correction data pair, respectively. Obtain the text to be corrected and determine the correction instruction corresponding to the text to be corrected. Then, concatenate the text to be corrected and the correction instruction and input them into the target large language model to obtain the corrected target text.

2. The text correction method based on a large model according to claim 1, characterized in that, The step of constructing a training dataset based on the first training data and the second training data includes: The first training data is manually filtered based on preset data filtering rules; The second training data and the filtered first training data are mixed according to a preset data ratio to obtain an initial dataset; The initial dataset is deduplicated based on the textual semantic features of the training data in the initial dataset, and then data augmentation is performed on the deduplicated initial dataset to obtain the training dataset.

3. The text correction method based on a large model according to claim 2, characterized in that, The process of augmenting the deduplicated initial dataset to obtain the training dataset includes: Identify the existing entities in the training data in the initial dataset, and identify the target entities in the data in the preset data augmentation dataset; The target entity is used to replace the existing entity in the training data to obtain the training dataset based on the training data after replacing the existing entity.

4. The text correction method based on a large model according to claim 1, characterized in that, The generation of corresponding first training data based on a preset large language model according to a first preset instruction includes: Determine the fixed paradigms corresponding to preset spelling errors and preset grammar errors, and construct the corresponding first preset instruction through the fixed paradigms; Based on the first preset instruction, error data corresponding to the preset spelling error and the preset grammar error are generated respectively, and corresponding correct data is generated based on the error data; The text spelling correction data pair and the text grammar correction data pair are constructed based on the error data and the correct data, so as to construct the first training data based on the text spelling correction data pair and the text grammar correction data pair.

5. The text correction method based on a large model according to claim 1, characterized in that, The construction of the corresponding target dataset by using the second preset instruction corresponding to the preset text correction task and the training dataset includes: Determine the error correction rules for the preset text error correction task, and determine the second preset instruction corresponding to each task based on the error correction rules; First instruction data is constructed based on the second preset instruction and the corresponding text spelling correction data pair and text grammar correction data pair, so as to construct the target dataset based on the first instruction data.

6. The text correction method based on a large model according to claim 5, characterized in that, The process of constructing the first instruction data based on the second preset instruction and the corresponding text spelling correction data pair and text grammar correction data pair further includes: Based on the preset error-free data and the second preset instruction, corresponding second instruction data is constructed to construct the target dataset according to the first instruction data and the second instruction data.

7. The text correction method based on a large model according to any one of claims 1 to 6, characterized in that, The step of concatenating the text to be corrected and the correction instruction and inputting them into the target large language model to obtain the corrected target text includes: The text to be corrected and the corresponding spelling correction instruction in the correction instruction are concatenated and then input into the target large language model to obtain the spelling correction result; The spelling correction result and the corresponding grammar correction instruction in the correction instruction are concatenated and input into the target large language model to obtain the grammar correction result, and the grammar correction result is used as the target text.

8. A text correction device based on a large model, characterized in that, include: The dataset construction module is used to generate corresponding first training data based on a preset large language model according to a first preset instruction, and to manually annotate the preset data according to preset manual annotation rules to generate corresponding second training data, and to construct a training dataset based on the first training data and the second training data. The training dataset includes text spelling correction data pairs and text syntax correction data pairs; The model fine-tuning module is used to construct a corresponding target dataset by using a second preset instruction corresponding to a preset text error correction task and the training dataset, so as to use the target dataset to fine-tune the preset large language model to obtain a target large language model; the second preset instruction includes spelling error correction instructions and grammar error correction instructions corresponding to the text spelling error correction data pair and the text grammar error correction data pair, respectively. The text correction module is used to acquire the text to be corrected and determine the correction instruction corresponding to the text to be corrected, so as to concatenate the text to be corrected and the correction instruction and input them into the target large language model to obtain the corrected target text.

9. An electronic device, characterized in that, The electronic device includes a processor and a memory; wherein the memory is used to store a computer program, which is loaded and executed by the processor to implement the large model-based text correction method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, Used to store computer programs, which, when executed by a processor, implement the large-model-based text correction method as described in any one of claims 1 to 7.

Images