Syntax error correction method, system, device and medium based on consistency learning

Abstract

The present invention relates to text error correction and artificial intelligence technology, and discloses a grammatical error correction method based on consistency learning, comprising: generating standardized predicted error labels and standardized true error labels based on pre-acquired original error text and original correct text, inputting the standardized predicted error labels and standardized true error labels into a pre-trained sequence into a sequence model to obtain a first predicted text and a second predicted text, calculating a consistency loss value between the first predicted text and the second predicted text, calculating a final loss value based on the consistency loss value, optimizing a model based on the final loss value to obtain a text error correction model, and using the text error correction model to perform text error correction on a target error text to obtain the original correct text. The present invention also proposes a grammatical error correction system, electronic device, and storage medium based on consistency learning. The present invention can improve the accuracy of text error correction.

Description

Technical Field

[0001] The present invention relates to the fields of artificial intelligence technology and text error correction technology, and in particular to a grammar error correction method, system, device and medium based on consistency learning. Background Art

[0002] Grammatical error correction is a fundamental natural language processing problem with a wide range of applications, including, but not limited to, improving the quality of machine translation, enhancing text readability, reducing grammatical errors in handwriting and speech recognition to improve text readability, and helping non-native speakers improve their language skills. Currently, error correction is primarily divided into two modules: error detection and error correction.

[0003] In the error detection module, previous work has made preliminary attempts to incorporate detection label knowledge into the error correction model to improve correction results. Sequence editing models (Seq2Edit) are used to convert a source sentence into a series of edit operations (insertion, deletion, and replacement). Despite certain limitations, such as the need for manual edit selection and the use of a dictionary, Sequence Editing Models are particularly advantageous in error correction due to their high understanding capabilities. Sequence-to-Sequence (Seq2Seq) models have been well explored in the error correction module, encoding erroneous sentences via an encoder and generating error-free tokens for each using a decoder. Sequence-to-Sequence models are better suited for generating fluent sentences, but their decoding speed is slower. Sequence-to-Sequence models, on the other hand, treat grammatical error correction as a single translation task, offering the advantage of better ability to generate corrected sentences. However, sequence-to-sequence error correction models still face the challenge of overcorrection.

[0004] However, existing technologies train the error detection module and the error correction module separately, resulting in a low coupling degree between the two modules, which affects the text correction effect. Summary of the Invention

[0005] The present invention provides a grammatical error correction method, system, device and medium based on consistency learning, the main purpose of which is to solve the problem of low efficiency and low accuracy of existing text error correction methods.

[0006] To achieve the above objectives, the present invention provides a grammatical error correction method based on consistency learning, comprising:

[0007] Obtaining the original error text and the original correct text corresponding to the original error text;

[0008] Generate a predicted label based on the original erroneous text using a sequence editing model, and generate a true label based on the original erroneous text and the target correct text using an error labeling model;

[0009] Generating a predicted error label according to the predicted label, and generating a true error label according to the true label;

[0010] Performing label normalization processing on the predicted error labels and the true error labels using a preset label processing algorithm to obtain normalized predicted error labels and normalized true error labels;

[0011] Inputting the standardized predicted error label and the standardized true error label into a pre-trained sequence into a sequence model to obtain a first predicted text and a second predicted text, determining a difference measure between the first predicted text and the second predicted text, and obtaining a consistency loss value;

[0012] Calculating loss values ​​between the first predicted text and the second predicted text and the original correct text to obtain a first loss value and a second loss value, and calculating a final loss value based on the first loss value, the second loss value, and the consistency loss value;

[0013] If the final loss value is greater than or equal to a preset loss value threshold, the final loss value is reduced as an incentive response for model training to optimize the model parameters of the sequence-to-sequence model to obtain a target text correction model, and after receiving a target error text that needs to be grammatically corrected, the target error text is corrected using the target text correction model to obtain a target correct text.

[0014] In order to solve the above problems, the present invention further provides a grammar error correction system based on consistency learning, the system comprising:

[0015] A data acquisition module, configured to acquire an original error text and an original correct text corresponding to the original error text;

[0016] a label prediction module, configured to generate a predicted label based on the original error text using a sequence editing model, generate a true label based on the original error text and the target correct text using an error annotation model, generate a predicted error label based on the predicted label, and generate a true error label based on the true label, and perform label normalization on the predicted error label and the true error label using a preset label processing algorithm to obtain a normalized predicted error label and a normalized true error label;

[0017] a loss value calculation module, configured to input the standardized predicted error label and the standardized true error label into a pre-trained sequence into a sequence model to obtain a first predicted text and a second predicted text, determine a difference measure between the first predicted text and the second predicted text to obtain a consistency loss value, calculate the loss value between the first predicted text and the second predicted text and the original correct text to obtain a first loss value and a second loss value, and calculate a final loss value based on the first loss value, the second loss value, and the consistency loss value;

[0018] The model optimization module is used to optimize the model parameters of the sequence-to-sequence model by reducing the final loss value as an incentive response for model training if the final loss value is greater than or equal to a preset loss value threshold, thereby obtaining a target text correction model, and after receiving a target error text that needs to be grammatically corrected, using the target text correction model to perform text correction on the target error text to obtain a target correct text.

[0019] In order to solve the above problem, the present invention further provides an electronic device, comprising:

[0020] at least one processor;

[0021] and, a memory communicatively coupled to the at least one processor;

[0022] The memory stores a computer program that can be executed by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can perform the above-mentioned grammatical error correction method based on consistency learning.

[0023] In order to solve the above problems, the present invention also provides a computer-readable storage medium, in which at least one computer program is stored. The at least one computer program is executed by a processor in an electronic device to implement the above-mentioned grammatical error correction method based on consistency learning.

[0024] The embodiment of the present invention obtains an original error text and an original correct text corresponding to the original error text, generates a predicted label based on the original error text using a sequence editing model, generates a true label based on the original error text and the target correct text using an error annotation model, generates a predicted error label based on the predicted label, and generates a true error label based on the true label, performs label standardization processing on the predicted error label and the true error label using a preset label processing algorithm to obtain a standardized predicted error label and a standardized true error label, inputs the standardized predicted error label and the standardized true error label into a pre-trained sequence into a sequence model, obtains a first predicted text and a second predicted text, and determines the first predicted text. The difference between the first predicted text and the second predicted text is measured to obtain a consistency loss value, the loss value between the first predicted text and the second predicted text and the original correct text is calculated to obtain a first loss value and a second loss value, the final loss value is calculated according to the first loss value, the second loss value and the consistency loss value, the model parameters of the sequence-to-sequence model are optimized with the decrease of the final loss value as the incentive response of model training, and the target text correction model is obtained. After receiving the target error text that needs to be grammatically corrected, the text correction model is used to perform text correction on the target error text to obtain the target correct text, so as to improve the coupling between the error recognition module and the error correction module, thereby improving the effect and efficiency of text correction. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 1 is a schematic diagram of an application environment of a grammatical error correction method based on consistency learning in an embodiment of the present invention;

[0026] Figure 2 A flowchart of a grammatical error correction method based on consistency learning provided by one embodiment of the present invention;

[0027] Figure 3 A functional module diagram of a grammatical error correction system based on consistency learning provided by one embodiment of the present invention;

[0028] Figure 4 A schematic structural diagram of an electronic device for implementing the grammatical error correction method based on consistency learning provided by an embodiment of the present invention.

[0029] The purpose, features and advantages of the present invention will be further described with reference to the accompanying drawings and in conjunction with the embodiments. DETAILED DESCRIPTION

[0030] It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0031] The embodiment of the present application provides a grammatical error correction method based on consistency learning. The execution subject of the grammatical error correction method based on consistency learning includes but is not limited to at least one of the electronic devices such as a server, a terminal, etc. that can be configured to execute the method provided by the embodiment of the present application. In other words, the grammatical error correction method based on consistency learning can be executed by software or hardware installed on a terminal device or a server device, and the software can be a blockchain platform. The server includes but is not limited to: a single server, a server cluster, a cloud server or a cloud server cluster, etc. The server can be an independent server, or it can be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms.

[0032] The grammar error correction method based on consistency learning provided by the embodiment of the present invention can be applied in the following situations: Figure 1In an application environment, the client communicates with the server through a network. The server can obtain the original error text and the original correct text corresponding to the original error text through the client, use the sequence editing model to generate a predicted label according to the original error text, use the error annotation model to generate a true label according to the original error text and the target correct text, generate a predicted error label according to the predicted label, and generate a true error label according to the true label, use a preset label processing algorithm to perform label standardization on the predicted error label and the true error label to obtain a standardized predicted error label and a standardized true error label, input the standardized predicted error label and the standardized true error label into the pre-trained sequence into the sequence model, and obtain a first predicted text. The present invention comprises a sequence-to-sequence model, a first prediction text, a second prediction text, and a second prediction text, and determines a difference metric between the first prediction text and the second prediction text to obtain a consistency loss value. The loss values ​​between the first prediction text and the second prediction text and the original correct text are calculated to obtain a first loss value and a second loss value. A final loss value is calculated based on the first loss value, the second loss value, and the consistency loss value. The model parameters of the sequence-to-sequence model are optimized using the decrease in the final loss value as an incentive response for model training to obtain a target text correction model. After receiving a target erroneous text that requires grammatical correction, the target erroneous text is corrected using the text correction model to obtain a target correct text. This effectively improves the efficiency and accuracy of text correction. The client can include, but is not limited to, various personal computers, laptops, smartphones, tablet computers, and portable wearable devices. The server can be implemented as an independent server or a server cluster consisting of multiple servers. The present invention is described in detail below through specific embodiments.

[0033] Reference Figure 2 FIG. 1 is a flow chart of a grammatical error correction method based on consistency learning provided by an embodiment of the present invention. In this embodiment, the grammatical error correction method based on consistency learning includes:

[0034] S1. Obtain an original error text and an original correct text corresponding to the original error text.

[0035] In the embodiment of the present invention, the original error text is a text containing grammatical errors.

[0036] In the embodiment of the present invention, the original correct text is a text obtained by correcting grammatical errors in the original erroneous text.

[0037] In an embodiment of the present invention, the original error text may be an output result of speech recognition performed based on the original correct text. For example, the original correct text may be “I don’t want it, I don’t care about those”, and the original error text may be “I don’t care about those”.

[0038] In the embodiment of the present invention, by obtaining the original error text and the original correct text corresponding to the original error text, the efficiency of subsequently generating predicted labels and true labels is improved.

[0039] S2. Generate a predicted label based on the original erroneous text using a sequence editing model, and generate a true label based on the original erroneous text and the target correct text using an error annotation model.

[0040] In this embodiment of the present invention, the sequence editing model is a model for natural language processing (NLP) tasks. It represents sequence-to-sequence transformations through a series of editing operations, each of which can replace a span of the source text with a target token or leave it unchanged. This approach has demonstrated excellent performance on a variety of NLP tasks, such as text normalization, sentence fusion, sentence splitting and rewriting, text simplification, and grammatical error correction.

[0041] In an embodiment of the present invention, generating a predicted label based on the original erroneous text using a sequence editing model includes:

[0042] Converting the original error text into a binary sequence to obtain an error text sequence;

[0043] Using the sequence editing model to perform error prediction based on the original error text to obtain a prediction error;

[0044] The erroneous text sequence is error-labeled according to the prediction error to obtain a prediction label.

[0045] In detail, converting the original error text into a binary sequence to obtain an error text sequence means that each character in the original error text is represented by "0", for example, "I watch, I don't care about the main content" can be converted into "00000000".

[0046] Specifically, labeling the error text sequence according to the predicted error to obtain a predicted label means representing the error portion of the error text sequence with a "1". If an error is identified corresponding to multiple characters, multiple "1"s are required to represent them. For example, the error text sequence can be labeled as "0110000100".

[0047] In an embodiment of the present invention, the error labeling model is used to generate a true label based on the original error text and the target correct text. The error labeling model is directly used to compare the original error text with the target correct text, and the error part is identified through the comparison, and then marked according to the error part.

[0048] In detail, the error labeling model is used to generate a true label based on the original error text and the target correct text, including:

[0049] Comparing the original erroneous text with the target correct text using the error marking model to obtain a comparison result;

[0050] The error text sequence of the original error text is marked as an error according to the comparison result to obtain a true label.

[0051] In detail, the true label can be expressed as “0110000110”, which represents the true error condition of the original error text.

[0052] In an embodiment of the present invention, a sequence editing model is used to generate a predicted label based on the original erroneous text, and an error labeling model is used to generate a true label based on the original erroneous text and the target correct text, thereby improving the efficiency of subsequently generating predicted error labels and true error labels.

[0053] S3. Generate a predicted error label according to the predicted label, and generate a true error label according to the true label.

[0054] In the embodiment of the present invention, the predicted error label and the true error label are both text labels.

[0055] In the embodiment of the present invention, generating a predicted error label according to the predicted label and generating a true error label according to the true label refers to generating a corresponding text label according to the error recognition result of the original error text.

[0056] In detail, the number of text words in the predicted error label and the true error label corresponds to "1" in the predicted label and the true label.

[0057] Specifically, generating a predicted error label based on the predicted label and generating a true error label based on the true label refers to generating the predicted error label and the true error label based on the text corresponding to the "1" in the predicted label and the true label in the original error text. For example, the predicted error label can be expressed as "table table, main", and the true predicted label can be expressed as "table table, main, inside". Since "table" corresponds to two "1", "table" needs to be duplicated and expressed as "table table".

[0058] In the embodiment of the present invention, generating a predicted error label according to the predicted label, and generating a true error label according to the true label, includes:

[0059] Obtaining an erroneous text position according to the predicted label and the true label;

[0060] Extracting the error text of the original error text according to the error text position;

[0061] The error texts are combined according to the arrangement of the original error texts to obtain predicted error labels and true error labels.

[0062] In the embodiment of the present invention, by generating a predicted error label according to the predicted label and generating a true error label according to the true label, the efficiency of subsequently generating a standardized predicted error label and a standardized true error label is improved.

[0063] S4. Perform label normalization processing on the predicted error label and the true error label using a preset label processing algorithm to obtain a normalized predicted error label and a normalized true error label.

[0064] In an embodiment of the present invention, the method of performing label normalization processing on the predicted error labels and the true error labels using a preset label processing algorithm to obtain normalized predicted error labels and normalized true error labels includes:

[0065] Add an error tag before each text in the predicted error tag and the true error tag to obtain a predicted tag tag and a true tag tag;

[0066] Adding a connection tag to the tail of the predicted label and the tail of the true label to obtain a predicted connection tag and a true connection tag;

[0067] Replacing the error content in the original error text with the label mask to obtain mask replacement text;

[0068] The mask replacement text is connected to the tail of the predicted connection label and the true connection label respectively to obtain a normalized predicted error label and a normalized true error label.

[0069] In detail, the error mark is added before each text in the predicted error label and the true error label to obtain the predicted mark label and the true mark label, which can be added before each text in the predicted error label and the true error label. <e>",For example" <e>Table, <e>host, <e>Inside".

[0070] In detail, the step of adding a connection tag to the tail of the predicted label and the tail of the real label to obtain the predicted connection tag and the real connection tag may be performed by adding a " <sep>".For example" <e>Table, <e>host, <e>Inside <sep>”.

[0071] In detail, the replacing the error content in the original error text with the label mask to obtain the mask replacement text means replacing the error content in the original error text with " <m>", for example, the normalized true error label can be expressed as " <e>Table, <e>host, <e>Inside <sep>I <m>I don't care <m> <m>some".

[0072] In an embodiment of the present invention, a preset label processing algorithm is used to perform label standardization processing on the predicted error label and the true error label to obtain a standardized predicted error label and a standardized true error label, thereby improving the efficiency of subsequent text prediction.

[0073] S5. Input the standardized predicted error label and the standardized true error label into the pre-trained sequence into the sequence model to obtain a first predicted text and a second predicted text, determine the difference measure between the first predicted text and the second predicted text, and obtain a consistency loss value.

[0074] In the embodiment of the present invention, the sequence-to-sequence model is a T5-large model.

[0075] In detail, the T5-large model is an advanced language model proposed by Google researchers. It uniformly converts various natural language processing (NLP) tasks into a text-to-text format, so that the same model and training objectives can be used to handle different tasks. The design concept of the T5 model is to convert the input text into a series of task-specific output texts.

[0076] The T5 model has multiple versions, including T5-Large, a larger model with approximately 739 million parameters, suitable for complex tasks requiring high computational resources. The T5-Large model utilizes a large amount of text data during pre-training, including data from Common Crawl and other sources such as Wikipedia. Pre-training tasks include both unsupervised and supervised tasks. Unsupervised tasks primarily learn language patterns through mask prediction (similar to the Masked Language Model task of the BERT model), while supervised tasks include various text-to-text conversion tasks.

[0077] In the embodiment of the present invention, the consistency loss value is a value used to measure the difference between the first predicted text and the second predicted text, and is an asymmetric value.

[0078] In the embodiment of the present invention, determining the difference measure between the first predicted text and the second predicted text to obtain a consistency loss value includes:

[0079] Converting the first predicted text and the second predicted text into vector representations using a bag-of-words model to obtain a first predicted vector and a second predicted vector;

[0080] The consistency loss value is calculated based on the first prediction vector and the second prediction vector using the following formula:

[0081]

[0082] Wherein, S is the consistency loss value, represents the first prediction vector, represents the second prediction vector.

[0083] In this embodiment of the present invention, using the bag-of-words model to convert the first predicted text and the second predicted text into vector representations to obtain the first predicted vector and the second predicted vector is done by counting the number of occurrences of each word in each text to form a vector. The dimension of the vector is the size of the vocabulary, and the value of each dimension represents the number of occurrences of the corresponding word in the text.

[0084] In an embodiment of the present invention, the standardized predicted error labels and the standardized true error labels are input into a pre-trained sequence model to obtain a first predicted text and a second predicted text, and a difference metric between the first predicted text and the second predicted text is determined to obtain a consistency loss value. This improves the efficiency of the subsequent calculation of the final loss value.

[0085] S6. Calculate the loss value between the first predicted text and the second predicted text and the original correct text to obtain a first loss value and a second loss value, and calculate a final loss value based on the first loss value, the second loss value and the consistency loss value.

[0086] In the embodiment of the present invention, the calculation of the loss value between the first predicted text and the second predicted text and the original correct text is to calculate the loss value between the first predicted text and the second predicted text and the original correct text using a preset rated loss function.

[0087] Specifically, the loss function is a function used in machine learning to measure the difference between a model's predicted value and the actual value. Also known as the cost function or objective function, the loss function guides parameter optimization during model training. A smaller loss function indicates that the model's predicted results are closer to the true values, generally resulting in better model performance.

[0088] In an embodiment of the present invention, the steps of calculating the loss value between the first predicted text and the second predicted text and the original correct text and determining the difference measure between the first predicted text and the second predicted text to obtain the consistency loss value are the same and will not be repeated here.

[0089] In the embodiment of the present invention, the calculating of the final loss value according to the first loss value, the second loss value, and the consistency loss value includes:

[0090] Obtaining adjustable model parameters of the sequence-to-sequence model;

[0091] Calculating the product of the adjustable model parameter and the consistency loss value to obtain a target loss value;

[0092] Calculate the sum of the target loss value, the first loss value, and the second loss value to obtain a final loss value.

[0093] In an embodiment of the present invention, the first loss value and the second loss value are obtained by calculating the loss values ​​between the first predicted text and the second predicted text and the original correct text, and the final loss value is calculated based on the first loss value, the second loss value and the consistency loss value, thereby improving the efficiency of subsequent model training.

[0094] S7. If the final loss value is greater than or equal to a preset loss value threshold, the model parameters of the sequence-to-sequence model are optimized by reducing the final loss value as an incentive response for model training to obtain a target text correction model. After receiving a target error text that needs to be grammatically corrected, the target error text is corrected using the target text correction model to obtain a target correct text.

[0095] In an embodiment of the present invention, the method of optimizing the model parameters of the sequence-to-sequence model by reducing the final loss value as an incentive response for model training to obtain a target text error correction model includes:

[0096] Using the reduction of the final loss value as an incentive response for model training, and optimizing the hyperparameters of the sequence-to-sequence model using a back-propagation algorithm to obtain an optimized model;

[0097] Replacing the sequence-to-sequence model with the optimization model, and recalculating the optimized final loss value based on the standardized predicted error labels and the standardized true error labels;

[0098] If the final loss value of the optimization is greater than or equal to the loss value threshold and the preset number of iterations has not been reached, returning to the step of reducing the final loss value as an incentive response for model training, optimizing the hyperparameters of the sequence-to-sequence model using a backpropagation algorithm, and obtaining an optimized model;

[0099] If the final loss value of the optimization is less than the loss value threshold or reaches the preset number of iterations, the currently obtained optimization model is used as the target text error correction model.

[0100] In detail, the hyperparameters of the sequence-to-sequence model include weight parameters, bias terms, and learning rates.

[0101] In an embodiment of the present invention, the above steps of obtaining the final loss value are performed with the goal of minimizing the final loss value until the model performance meets the requirements or reaches a preset number of iterations, thereby obtaining a target text error correction model, thereby improving the efficiency and training effect of model training.

[0102] In an embodiment of the present invention, after obtaining a target erroneous text that needs to be grammatically corrected, the target text correction model is used to perform text correction on the target erroneous text to obtain a target correct text.

[0103] In the embodiment of the present invention, since the consistency loss value is introduced in the training process, the text error correction model can maximize the consistency between the first predicted text and the second predicted text, greatly improving the ability of the text error correction model to perform text error correction.

[0104] In an embodiment of the present invention, the target text error correction model is used to perform text error correction on the target erroneous text to obtain the target correct text. This is done by extracting features from the target erroneous text and then inputting the features into the target text error correction model to obtain the target correct text.

[0105] In an embodiment of the present invention, a target erroneous text requiring grammatical correction is obtained, and the target erroneous text is corrected using the target text correction model to obtain a target correct text, thereby improving the accuracy and efficiency of text correction.

[0106] As can be seen, in this embodiment of the present invention, by integrating the sequence editing model error correction system with the sequence-to-sequence model error correction system, the sequence editing model and the sequence-to-sequence model are combined based on their respective understanding and generation strengths, leveraging the error detection and error correction capabilities of the sequence editing model and the sequence-to-sequence model. A detection and correction module is also proposed to introduce detection information from the sequence editing model error correction system into the sequence-to-sequence model error correction system, enabling the model to make more accurate predictions. This significantly improves the accuracy and efficiency of grammatical error correction.

[0107] like Figure 3 , which is a functional module diagram of a grammatical error correction system based on consistency learning provided by an embodiment of the present invention.

[0108] The grammatical error correction system 100 based on consistency learning described in the present invention can be installed in an electronic device. Depending on the functionality implemented, the grammatical error correction system 100 based on consistency learning can include a data acquisition module 101, a label prediction module 102, a loss value calculation module 103, and a model optimization module 104. A module described in the present invention, also referred to as a unit, refers to a series of computer program segments that can be executed by an electronic device processor and can perform a fixed function, and is stored in the memory of the electronic device.

[0109] In this embodiment, the functions of each module / unit are as follows:

[0110] The data acquisition module 101 is used to acquire the original error text and the original correct text corresponding to the original error text;

[0111] The label prediction module 102 is configured to generate a predicted label based on the original error text using a sequence editing model, generate a true label based on the original error text and the target correct text using an error annotation model, generate a predicted error label based on the predicted label, and generate a true error label based on the true label, and perform label normalization on the predicted error label and the true error label using a preset label processing algorithm to obtain a normalized predicted error label and a normalized true error label;

[0112] The loss value calculation module 103 is configured to input the standardized predicted error labels and the standardized true error labels into a pre-trained sequence into a sequence model to obtain a first predicted text and a second predicted text, determine a difference measure between the first predicted text and the second predicted text to obtain a consistency loss value, calculate the loss values ​​between the first predicted text and the second predicted text and the original correct text to obtain a first loss value and a second loss value, and calculate a final loss value based on the first loss value, the second loss value, and the consistency loss value;

[0113] The model optimization module 104 is configured to optimize the model parameters of the sequence-to-sequence model by reducing the final loss value as an incentive response for model training if the final loss value is greater than or equal to a preset loss value threshold, thereby obtaining a target text correction model, and upon receiving a target error text that requires grammatical correction, perform text correction on the target error text using the target text correction model to obtain a target correct text.

[0114] In one embodiment, the tag prediction module 102 is specifically configured to:

[0115] Converting the original error text into a binary sequence to obtain an error text sequence;

[0116] Using the sequence editing model to perform error prediction based on the original error text to obtain a prediction error;

[0117] The erroneous text sequence is error-labeled according to the prediction error to obtain a prediction label.

[0118] In one embodiment, the tag prediction module 102 is specifically configured to:

[0119] Comparing the original erroneous text with the target correct text using the error marking model to obtain a comparison result;

[0120] The error text sequence of the original error text is marked as an error according to the comparison result to obtain a true label.

[0121] In one embodiment, the tag prediction module 102 is specifically configured to:

[0122] Obtain source code data after successful deployment of the preset software;

[0123] Obtaining a changed code file in the source code data according to the code change record to obtain a changed file;

[0124] The changed code in the changed file is obtained according to the code change record.

[0125] In one embodiment, the tag prediction module 102 is specifically configured to:

[0126] Add an error tag before each text in the predicted error tag and the true error tag to obtain a predicted tag tag and a true tag tag;

[0127] Adding a connection tag to the tail of the predicted label and the tail of the true label to obtain a predicted connection tag and a true connection tag;

[0128] Replacing the error content in the original error text with the label mask to obtain mask replacement text;

[0129] The mask replacement text is connected to the tail of the predicted connection label and the true connection label respectively to obtain a normalized predicted error label and a normalized true error label.

[0130] In one embodiment, the loss value calculation module 103 is specifically configured to:

[0131] Obtaining adjustable model parameters of the sequence-to-sequence model;

[0132] Calculating the product of the adjustable model parameter and the consistency loss value to obtain a target loss value;

[0133] Calculate the sum of the target loss value, the first loss value, and the second loss value to obtain a final loss value.

[0134] Summarize the test cases that passed the execution and the test cases that failed the execution to obtain the test results.

[0135] In one embodiment, the model optimization module 104 is specifically configured to:

[0136] Using the reduction of the final loss value as an incentive response for model training, and optimizing the hyperparameters of the sequence-to-sequence model using a back-propagation algorithm to obtain an optimized model;

[0137] Replacing the sequence-to-sequence model with the optimization model, and recalculating the optimized final loss value based on the standardized predicted error labels and the standardized true error labels;

[0138] If the final loss value of the optimization is greater than or equal to the loss value threshold and the preset number of iterations has not been reached, returning to the step of reducing the final loss value as an incentive response for model training, optimizing the hyperparameters of the sequence-to-sequence model using a backpropagation algorithm, and obtaining an optimized model;

[0139] If the final loss value of the optimization is less than the loss value threshold or reaches the preset number of iterations, the currently obtained optimization model is used as the target text error correction model.

[0140] In detail, each module in the grammar error correction system 100 based on consistency learning in the embodiment of the present invention adopts the same Figure 2 The grammatical error correction method based on consistency learning described in the present invention has the same technical means and can produce the same technical effects, so I will not go into details here.

[0141] like Figure 4 , which is a structural diagram of an electronic device for a grammatical error correction method based on consistency learning provided by an embodiment of the present invention.

[0142] The electronic device 1 may include a processor 10, a memory 11, a communication bus 12, and a communication interface 13. It may also include a computer program stored in the memory 11 and executable on the processor 10, such as a grammatical error correction program based on consistency learning.

[0143] In some embodiments, the processor 10 may be composed of an integrated circuit, for example, a single packaged integrated circuit, or a plurality of packaged integrated circuits with the same or different functions, including one or more central processing units (CPUs), microprocessors, digital processing chips, graphics processors, and a combination of various control chips. The processor 10 is the control core (Control Unit) of the electronic device, connecting the various components of the entire electronic device using various interfaces and lines, and executing programs or modules stored in the memory 11 (such as a grammar error correction program based on consistency learning, etc.), as well as calling data stored in the memory 11, to perform various functions of the electronic device and process data.

[0144] The memory 11 includes at least one type of readable storage medium, and the readable storage medium includes a flash memory, a mobile hard disk, a multimedia card, a card-type memory (for example, an SD or DX memory, etc.), a magnetic memory, a disk, an optical disk, etc. In some embodiments, the memory 11 may be an internal storage unit of an electronic device, such as a mobile hard disk of the electronic device. In other embodiments, the memory 11 may also be an external storage device of an electronic device, such as a plug-in mobile hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. equipped on the electronic device. Furthermore, the memory 11 may also include both an internal storage unit of the electronic device and an external storage device. The memory 11 can not only be used to store application software and various types of data installed in the electronic device, such as the code of a grammar correction program based on consistency learning, but can also be used to temporarily store data that has been output or is to be output.

[0145] The communication bus 12 may be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus. The bus may be divided into an address bus, a data bus, a control bus, etc. The bus is configured to enable communication between the memory 11 and at least one processor 10, etc.

[0146] The communication interface 13 is used for communication between the above-mentioned electronic device and other devices, including a network interface and a user interface. Optionally, the network interface may include a wired interface and / or a wireless interface (such as a WI-FI interface, a Bluetooth interface, etc.), which is generally used to establish a communication connection between the electronic device and other electronic devices. The user interface may be a display (Display), an input unit (such as a keyboard (Keyboard)), optionally, the user interface may also be a standard wired interface, a wireless interface. Optionally, in this embodiment, the display may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, and an OLED (Organic Light-Emitting Diode, organic light-emitting diode) touch device, etc. Among them, the display may also be appropriately referred to as a display screen or a display unit, which is used to display information processed in the electronic device and to display a visual user interface.

[0147] The figure only shows an electronic device with components. Those skilled in the art will understand that the structure shown in the figure does not constitute a limitation on the electronic device, and may include fewer or more components than shown in the figure, or combine certain components, or arrange the components differently.

[0148] For example, although not shown, the electronic device may further include a power source (such as a battery) for powering various components. Preferably, the power source may be logically connected to the at least one processor 10 via a power management device, thereby implementing functions such as charge management, discharge management, and power consumption management through the power management device. The power source may further include any components such as one or more DC or AC power sources, a recharging device, a power failure detection circuit, a power converter or inverter, a power status indicator, etc. The electronic device may further include various sensors, Bluetooth modules, Wi-Fi modules, etc., which will not be described in detail here.

[0149] It should be understood that the embodiment is for illustration only and the scope of the patent application is not limited to this structure.

[0150] The grammar error correction program based on consistency learning stored in the memory 11 of the electronic device 1 is a combination of multiple instructions. When running in the processor 10, it can achieve the following:

[0151] Obtaining the original error text and the original correct text corresponding to the original error text;

[0152] Generate a predicted label based on the original erroneous text using a sequence editing model, and generate a true label based on the original erroneous text and the target correct text using an error labeling model;

[0153] Generating a predicted error label according to the predicted label, and generating a true error label according to the true label;

[0154] Performing label normalization processing on the predicted error labels and the true error labels using a preset label processing algorithm to obtain normalized predicted error labels and normalized true error labels;

[0155] Inputting the standardized predicted error label and the standardized true error label into a pre-trained sequence into a sequence model to obtain a first predicted text and a second predicted text, determining a difference measure between the first predicted text and the second predicted text, and obtaining a consistency loss value;

[0156] Calculating loss values ​​between the first predicted text and the second predicted text and the original correct text to obtain a first loss value and a second loss value, and calculating a final loss value based on the first loss value, the second loss value, and the consistency loss value;

[0157] If the final loss value is greater than or equal to a preset loss value threshold, the final loss value is reduced as an incentive response for model training to optimize the model parameters of the sequence-to-sequence model to obtain a target text correction model, and after receiving a target error text that needs to be grammatically corrected, the target error text is corrected using the target text correction model to obtain a target correct text.

[0158] Specifically, the specific implementation method of the processor 10 for the above instructions can refer to the description of the relevant steps in the corresponding embodiment of the drawings, which will not be repeated here.

[0159] Furthermore, if the modules / units integrated into the electronic device 1 are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. The computer-readable storage medium can be volatile or non-volatile. For example, the computer-readable medium can include: any entity or device capable of carrying the computer program code, a recording medium, a USB flash drive, a mobile hard drive, a magnetic disk, an optical disk, a computer memory, or a read-only memory (ROM).

[0160] The present invention further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program. When the computer program is executed by a processor of an electronic device, the computer program can implement:

[0161] Obtaining the original error text and the original correct text corresponding to the original error text;

[0162] Generate a predicted label based on the original erroneous text using a sequence editing model, and generate a true label based on the original erroneous text and the target correct text using an error labeling model;

[0163] Generating a predicted error label according to the predicted label, and generating a true error label according to the true label;

[0164] Performing label normalization processing on the predicted error labels and the true error labels using a preset label processing algorithm to obtain normalized predicted error labels and normalized true error labels;

[0165] Inputting the standardized predicted error label and the standardized true error label into a pre-trained sequence into a sequence model to obtain a first predicted text and a second predicted text, determining a difference measure between the first predicted text and the second predicted text, and obtaining a consistency loss value;

[0166] Calculating loss values ​​between the first predicted text and the second predicted text and the original correct text to obtain a first loss value and a second loss value, and calculating a final loss value based on the first loss value, the second loss value, and the consistency loss value;

[0167] If the final loss value is greater than or equal to a preset loss value threshold, the final loss value is reduced as an incentive response for model training to optimize the model parameters of the sequence-to-sequence model to obtain a target text correction model, and after receiving a target error text that needs to be grammatically corrected, the target error text is corrected using the target text correction model to obtain a target correct text.

[0168] In the embodiments provided herein, it should be understood that the disclosed devices, systems, and methods may be implemented in other ways. For example, the system embodiments described above are merely illustrative. For example, the module division is merely a logical functional division, and actual implementation may employ other division methods.

[0169] The modules described as separate components may or may not be physically separate, and the components shown as modules may or may not be physical units, that is, they may be located in one place or distributed across multiple network elements. Some or all of the modules may be selected to achieve the purpose of the solution of this embodiment according to actual needs.

[0170] In addition, the functional modules in various embodiments of the present invention may be integrated into a single processing unit, each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or hardware plus software functional modules.

[0171] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the present invention can be implemented in other specific forms without departing from the spirit or essential characteristics of the present invention.

[0172] Therefore, the embodiments should be considered in all respects as illustrative and non-restrictive, and the scope of the invention is defined by the appended claims rather than the foregoing description, and all changes that come within the meaning and range of equivalents of the claims are intended to be embraced therein. Any reference to a figure in a claim should not be construed as limiting the claim to which it relates.

[0173] The embodiments of the present application can acquire and process relevant data based on artificial intelligence technology. Artificial Intelligence (AI) is the theory, method, technology, and application system that uses digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use knowledge to achieve optimal results.

[0174] Furthermore, it is clear that the word "comprising" does not exclude other units or steps, and the singular does not exclude the plural. Multiple units or devices recited in a system claim may also be implemented by a single unit or device through software or hardware. Terms such as "first" and "second" are used to indicate names and do not imply any particular order.

[0175] Finally, it should be noted that any software tools or components not developed by our company appear in the examples for illustration purposes only and do not represent actual use. The above examples are intended to illustrate the technical solutions of the present invention and are not intended to be limiting. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art will appreciate that the technical solutions of the present invention may be modified or replaced with equivalents without departing from the spirit and scope of the technical solutions of the present invention.< / m> < / m> < / m> < / sep> < / e> < / e> < / e> < / m> < / sep> < / e> < / e> < / e> < / sep> < / e> < / e> < / e> < / e>

Claims

1. A grammatical error correction method based on consistency learning, characterized in that: The method comprises: Obtaining the original error text and the original correct text corresponding to the original error text; Generate a predicted label based on the original erroneous text using a sequence editing model, and generate a true label based on the original erroneous text and the original correct text using an error annotation model; Generating a predicted error label according to the predicted label, and generating a true error label according to the true label; Performing label normalization processing on the predicted error labels and the true error labels using a preset label processing algorithm to obtain normalized predicted error labels and normalized true error labels; Inputting the standardized predicted error label and the standardized true error label into a pre-trained sequence into a sequence model to obtain a first predicted text and a second predicted text, determining a difference measure between the first predicted text and the second predicted text, and obtaining a consistency loss value; Calculating loss values ​​between the first predicted text and the second predicted text and the original correct text to obtain a first loss value and a second loss value, and calculating a final loss value based on the first loss value, the second loss value, and the consistency loss value; If the final loss value is greater than or equal to a preset loss value threshold, the final loss value is reduced as an incentive response for model training to optimize the model parameters of the sequence-to-sequence model to obtain a target text correction model, and after receiving a target error text that needs to be grammatically corrected, the target error text is corrected using the target text correction model to obtain a target correct text.

2. The grammatical error correction method based on consistency learning according to claim 1, characterized in that: The generating a predicted label according to the original erroneous text using a sequence editing model includes: Converting the original error text into a binary sequence to obtain an error text sequence; Using the sequence editing model to perform error prediction based on the original error text to obtain a prediction error; The erroneous text sequence is error-labeled according to the prediction error to obtain a prediction label.

3. The grammatical error correction method based on consistency learning according to claim 1, characterized in that: The generating of a true label according to the original erroneous text and the original correct text by using the error labeling model includes: Comparing the original erroneous text with the original correct text using the error marking model to obtain a comparison result; The error text sequence of the original error text is marked as an error according to the comparison result to obtain a true label.

4. The grammatical error correction method based on consistency learning according to claim 1, characterized in that: Generating a predicted error label according to the predicted label, and generating a true error label according to the true label, includes: Obtaining an erroneous text position according to the predicted label and the true label; Extracting the error text of the original error text according to the error text position; The error texts are combined according to the arrangement of the original error texts to obtain predicted error labels and true error labels.

5. The grammatical error correction method based on consistency learning according to claim 1, characterized in that: The method of performing label normalization processing on the predicted error labels and the true error labels using a preset label processing algorithm to obtain normalized predicted error labels and normalized true error labels includes: Add an error tag before each text in the predicted error tag and the true error tag to obtain a predicted tag tag and a true tag tag; Adding a connection tag to the tail of the predicted label and the tail of the true label to obtain a predicted connection tag and a true connection tag; Replacing the error content in the original error text with the label mask to obtain mask replacement text; The mask replacement text is connected to the tail of the predicted connection label and the true connection label respectively to obtain a normalized predicted error label and a normalized true error label.

6. The grammatical error correction method based on consistency learning according to claim 1, characterized in that: The calculating a final loss value according to the first loss value, the second loss value, and the consistency loss value includes: Obtaining adjustable model parameters of the sequence-to-sequence model; Calculating the product of the adjustable model parameter and the consistency loss value to obtain a target loss value; Calculate the sum of the target loss value, the first loss value, and the second loss value to obtain a final loss value.

7. The grammatical error correction method based on consistency learning according to claim 1, characterized in that: The method of optimizing the model parameters of the sequence-to-sequence model by reducing the final loss value as an incentive response for model training to obtain a target text error correction model includes: Using the reduction of the final loss value as an incentive response for model training, and optimizing the hyperparameters of the sequence-to-sequence model using a back-propagation algorithm to obtain an optimized model; Replacing the sequence-to-sequence model with the optimization model, and recalculating the optimized final loss value based on the standardized predicted error labels and the standardized true error labels; If the final loss value of the optimization is greater than or equal to the loss value threshold and the preset number of iterations has not been reached, returning to the step of reducing the final loss value as an incentive response for model training, optimizing the hyperparameters of the sequence-to-sequence model using a backpropagation algorithm, and obtaining an optimized model; If the final loss value of the optimization is less than the loss value threshold or reaches the preset number of iterations, the currently obtained optimization model is used as the target text error correction model.

8. A grammar error correction system based on consistency learning, characterized in that: The system comprises: A data acquisition module, configured to acquire an original error text and an original correct text corresponding to the original error text; a label prediction module, configured to generate a predicted label based on the original error text using a sequence editing model, generate a true label based on the original error text and the original correct text using an error annotation model, generate a predicted error label based on the predicted label, and generate a true error label based on the true label, and perform label normalization on the predicted error label and the true error label using a preset label processing algorithm to obtain a normalized predicted error label and a normalized true error label; a loss value calculation module, configured to input the standardized predicted error label and the standardized true error label into a pre-trained sequence into a sequence model to obtain a first predicted text and a second predicted text, determine a difference measure between the first predicted text and the second predicted text to obtain a consistency loss value, calculate the loss value between the first predicted text and the second predicted text and the original correct text to obtain a first loss value and a second loss value, and calculate a final loss value based on the first loss value, the second loss value, and the consistency loss value; The model optimization module is used to optimize the model parameters of the sequence-to-sequence model by reducing the final loss value as an incentive response for model training if the final loss value is greater than or equal to a preset loss value threshold, thereby obtaining a target text correction model, and after receiving a target error text that needs to be grammatically corrected, using the target text correction model to perform text correction on the target error text to obtain a target correct text.

9. An electronic device, characterized in that: The electronic device comprises: at least one processor; and, a memory communicatively coupled to the at least one processor; The memory stores a computer program that can be executed by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the grammatical error correction method based on consistency learning as described in any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the grammatical error correction method based on consistency learning according to any one of claims 1 to 7 is implemented.

Images