Grammatical error correction quality assessment method and system
By using pre-trained language models and attention mechanisms in grammar error correction quality evaluation, the clues of multiple grammar error correction results were integrated, and the problem of failure to effectively utilize multiple grammar error correction results in the existing technology was solved, and the grammar was improved Effects of error correction quality evaluation.
Patent Information
- Application Number
- CN202110393905.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-04-13
- Publication Date
- 2024-11-01
- Estimated Expiration
- 2041-04-13
AI Technical Summary
The existing grammar error correction quality evaluation model fails to effectively utilize multiple grammar error correction results generated by the grammar error correction model, resulting in limited results of grammar error checking and error correction quality evaluation.
The pre-trained language model is used to construct word vector representations of input sentences and grammatical error correction results, and the node-select attention mechanism is used to obtain grammatical error correction clues from other grammatical error correction results, and integrate it to form grammatical error correction Evidence, used to evaluate the quality of grammatical error correction results.
It improves the efficiency and accuracy of grammatical error correction quality evaluation, improves the effect of grammatical error correction, and can more effectively utilize grammatical error correction evidence in multiple grammatical error correction results.
Smart Images

Figure CN113128204B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the fields of natural language processing and grammatical error correction, and in particular to a grammatical error correction quality assessment method and system integrating multiple grammatical error correction results. Background Art
[0002] Grammatical correction is one of the important tasks of text proofreading. Its purpose is to use natural language processing technology to automatically identify and correct grammatical errors in articles. Grammatical correction systems mainly provide text proofreading services for non-native English speakers. These systems are expected to be able to detect grammatical errors and provide accurate corrections. With the continuous expansion of online education demand, grammatical correction, as an important part of essay correction, has attracted widespread attention from natural language processing researchers.
[0003] With the continuous development of deep learning technology, existing grammar correction work often regards grammar correction tasks as natural language generation tasks, and applies sequence-to-sequence neural network structures and generates grammar correction results through column search decoding. When training grammar correction models, incorrect sentences are usually regarded as source languages, and correct sentences are regarded as target languages. The sequence-to-sequence model is trained by annotating the incorrect and correct sentence pairs in the corpus, so that the grammar correction model can correct a variety of grammatical errors.
[0004] For deep neural learning technology, the scale of data often determines the effect of the model. However, the scale of existing grammatical error correction data is small, which makes it difficult to fully train the generative model based on deep neural network. Therefore, existing grammatical error correction work often focuses on corpus augmentation to alleviate the data shortage problem of grammatical error correction models. In order to generate a large number of training sentence pairs, existing work often uses various data augmentation methods to rewrite a correct sentence into an incorrect sentence. Some of these works randomly replace, delete, and add operations to the correct text to form a sentence with grammatical errors, while other works apply back translation to error-free text to translate it into an ungrammatical sentence. Both methods are widely used in today's grammatical error correction systems to pre-train grammatical error correction models. Since the distribution of grammatical error data generated by such corpus augmentation technology is somewhat different from the actual distribution of grammatical errors, it will cause the model to make incorrect grammatical modifications to the model or ignore some grammatical errors, thereby limiting the effect of the grammatical error correction model.
[0005] Existing grammatical correction quality assessment models can be roughly divided into two categories. The first category is to directly use the language model to evaluate whether the grammatical correction results conform to linguistic laws and grammatical rules, and then judge the grammatical correction quality of the corrected sentence. The second category is to encode the input text and grammatical correction results, and then evaluate the accuracy of the grammatical correction by predicting the official evaluation score of the grammatical correction, and then evaluate the quality of the grammatical correction. This type of grammatical correction quality assessment method further considers the accuracy of the grammatical correction. Finally, we can re-rank multiple grammatical correction results according to the scores of the grammatical correction quality assessment model, thereby improving the effect of the grammatical correction model. With the development of pre-trained language models, models such as BERT have proven their good effects in text encoding and generating word context representations, and have been applied to a variety of natural language processing tasks. The pre-trained language model BERT is also used in grammatical correction quality assessment, which can further improve the effect of grammatical error checking or grammatical correction.
[0006] The grammatical correction model often has subtle differences in the grammatical correction results obtained from the column search decoding, and these differences usually come from the uncertainty of the generation model. The uncertainty of these grammatical correction results mainly comes from the diversity of languages, the potential ambiguity of languages, and the confidence of the generation model. The grammatical correction results obtained by the well-trained grammatical correction model through the column search decoding are usually higher than the language model score of the given input sentence. Therefore, these grammatical correction results can provide some high-quality grammatical correction clues for the grammatical correction quality assessment model, and further provide more grammatical correction evidence for the grammatical correction quality assessment to improve its effect.
[0007] However, the existing grammatical error correction quality assessment models at this stage ignore the possible grammatical error evidence from the multiple grammatical error correction results generated by the column search decoding provided by the grammatical error correction model, thereby limiting the effectiveness of grammatical error checking and grammatical error correction quality assessment. Therefore, we hope to make full use of the multiple grammatical error correction results provided by the grammatical error correction model in grammatical error correction quality assessment, so as to prompt the model of possible grammatical errors and correction results, further improve the effectiveness of grammatical error checking and grammatical error correction quality assessment, and thus improve the effectiveness of grammatical error correction.
[0008] In summary, grammatical correction quality assessment has achieved good results in grammatical correction tasks. However, how to more effectively make full use of the multiple grammatical correction results generated in the grammatical correction decoding process in the grammatical correction quality assessment model to obtain more efficient grammatical correction quality assessment results is still a very important problem to be solved. Summary of the invention
[0009] The present invention provides a grammar error correction quality assessment method and system, which are used to solve the technical defects existing in the prior art.
[0010] The present invention provides a grammatical error correction quality assessment method, comprising:
[0011] Based on a given input sentence and several grammatical correction results, a pre-trained language model is used to construct a word vector representation of the given input sentence and the grammatical correction results;
[0012] For each word vector in the grammatical correction result, the node interaction attention mechanism is used to obtain grammatical correction clues of words that can verify the current grammatical correction result from several other grammatical correction results, and the node selection attention mechanism is used to integrate the grammatical correction clues of each grammatical correction result word to obtain grammatical correction evidence;
[0013] The grammatical error correction evidence is used to perform a quality assessment on the current grammatical error correction result.
[0014] According to the grammatical error correction quality assessment method provided by the present invention, based on a given input sentence and a number of grammatical error correction results, using a pre-trained language model to construct a word vector representation of the given input sentence and the grammatical error correction results includes:
[0015] For a given input sentence s, a grammatical error correction model is used to obtain K corresponding grammatical error correction results through column search, marked as C = {c 1 ,…,c k ,…,c K};
[0016] For a given input sentence s containing m words and the kth grammatical error correction result c containing n words k , the given input sentence and the grammatical error correction result sentence pair <s,c k >Use the special symbols "[CLS]" and "[SEP]" to represent the beginning and end of a sentence for splicing, and treat the spliced content as a node, then use BERT to encode it to get the node initialization representation H k :
[0017] H k =BERT([CLS]s[SEP]c k [SEP])
[0018] The given input sentence and the grammatical error correction result sentence pair <s,c k >The node initialization represents H k Contains the hidden state representation of each word and is marked as
[0019] According to the grammatical correction quality assessment method provided by the present invention, for each word vector in the grammatical correction result, a node interaction attention mechanism is used to obtain grammatical correction clues of words that can verify the current grammatical correction result from other grammatical correction results, and a node selection attention mechanism is used to integrate the grammatical correction clues of each grammatical correction result word, and the grammatical correction evidence is obtained, which specifically includes:
[0020] Based on the <s,c k >, using the node interaction attention mechanism to calculate the attention weight α l→k Selectively read words from the lth node, select grammatical error correction clues that can support the words in the kth node, and form a fine-grained representation V of the node l→k ;
[0021] Adopting node selection attention mechanism, based on the fine-grained representation V of the node l→k Calculate the importance of the node and integrate grammatical error correction evidence from multiple nodes that can support the verification of the quality of the current word.
[0022] According to the grammatical error correction quality assessment method provided by the present invention, the grammatical error correction clues that can support the words in the kth node are selected to form a fine-grained representation V of the node. l→k include:
[0023] For the pth word in the kth node Based on words and the qth word in the lth node The correlation score between the word representations is used to calculate the weight of the node interaction attention mechanism
[0024]
[0025] Among them, W is a trainable parameter, and Two words each and Vectorized representation of ;
[0026] Using node interaction attention weights Take the weighted sum of all word representations in the lth node to get the pth word in the kth node For example, the fine-grained node representation of the lth node
[0027]
[0028] based on Get the fine-grained node representation of the lth node facing the kth node
[0029] According to the grammatical error correction quality assessment method provided by the present invention, the node selection attention mechanism is adopted, based on the fine-grained representation V of the node l→k Calculate the importance of the node and integrate grammatical error correction evidence from multiple nodes that can support the verification of the quality of the current word. Specifically, it includes:
[0030] For the given input sentence s and grammatical error correction result c in the node l The words in are weighted and the selected attention weight γ for the lth node is calculated. l ;
[0031] Using the obtained node selection attention weight γ l The fine-grained node representation V for the lth node l→k Perform weighted summation to obtain the node verification representation for the kth node and grammatical error correction clues;
[0032] For the lth node, the node selection attention weight γ l , and weightedly integrate the grammatical correction clues to obtain grammatical correction evidence that can support the verification of the current word quality.
[0033] According to the grammatical error correction quality assessment method provided by the present invention, the selected attention weight γ of the lth node l , the node verification representation of the kth node The calculation method is:
[0034] The interaction matrix M is established between the given input sentence and the grammatical error correction result in the lth node. l ;
[0035] In the interaction matrix M l Each element in The correlation between the i-th word of a given input sentence in the l-th node and the j-th word in the grammatical correction result is measured, and the end symbol "[SEP]" used to mark the end of the sentence is included in the calculation:
[0036]
[0037] Among them, W is a trainable parameter, which is calculated on the two dimensions of the given input sentence and the grammatical error correction result. The attention weight is calculated using the following mechanism and
[0038]
[0039]
[0040] Using two sets of attention weights and Get the representation h of the given input sentence and grammatical error correction result in the lth node ls and h lh :
[0041]
[0042]
[0043] Calculate the node selection attention weight γ of the lth node l :
[0044]
[0045] in, represents the multiplication operation of the corresponding positions of the elements; represents the concatenation operation;
[0046] Node selection attention weight γ l Further integration for the pth word in the kth node The fine-grained node representation of the lth node And form the p-th word facing the k-th node Node verification representation
[0047]
[0048] Get the node verification representation for the kth node
[0049] According to the grammatical error correction quality assessment method provided by the present invention, the use of the grammatical error correction evidence to perform quality assessment on the current grammatical error correction result includes:
[0050] Use the cross entropy loss function to calculate the p-th word in the k-th node The loss of word quality classification label y:
[0051]
[0052] Among them, y * Represents the actual label classification given by the labeled data;
[0053] Calculate the loss function during model training:
[0054]
[0055] For the pth word in the kth node Use the corresponding node verification representation To calculate the probability of word-level grammatical error correction quality assessment label y
[0056]
[0057] in, represents the multiplication operation of the corresponding positions of the elements; represents the concatenation operation;
[0058] The score of the quality assessment of all words in the grammatical error correction results in the kth node, that is, the probability Average and get the grammatical error correction result c for a given input sentence s k The grammatical error correction quality evaluation score f(s,c k ):
[0059]
[0060] The present invention also provides a grammar error correction quality assessment system, comprising:
[0061] A word vector construction module is used to construct a word vector representation of a given input sentence and a number of grammatical error correction results using a pre-trained language model based on a given input sentence and a number of grammatical error correction results;
[0062] The grammatical correction evidence determination module is used to obtain grammatical correction clues of words that can verify the current grammatical correction result from other grammatical correction results for the word vector in each grammatical correction result by using the node interaction attention mechanism, and integrate the grammatical correction clues of each grammatical correction result word by using the node selection attention mechanism to obtain grammatical correction evidence;
[0063] The quality assessment module is used to use the grammatical error correction evidence to perform quality assessment on the current grammatical error correction result.
[0064] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the steps of any of the above-mentioned grammatical error correction quality assessment methods are implemented.
[0065] The present invention also provides a non-transitory computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps of any of the above-mentioned grammatical error correction quality assessment methods are implemented.
[0066] The grammatical correction quality assessment method of the embodiment of the present invention can obtain grammatical correction clues of words that can verify the current grammatical correction result by utilizing multiple grammatical correction results. The designed attention mechanism can find grammatical correction evidence that is helpful for verifying the quality of the current grammatical correction result in different grammatical correction results, thereby performing quality assessment on the current grammatical correction result, having a more efficient assessment effect and improving the grammatical correction effect. BRIEF DESCRIPTION OF THE DRAWINGS
[0067] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0068] Figure 1 It is a flowchart of a grammar error correction quality assessment method provided by an embodiment of the present invention;
[0069] Figure 2 is a schematic diagram of a grammatical error correction quality assessment system provided by an embodiment of the present invention;
[0070] Figure 3 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION
[0071] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0072] The embodiment of the present invention discloses a method for evaluating the quality of grammatical error correction. Figure 1 ,include:
[0073] S1: Based on a given input sentence and several grammatical error correction results, a pre-trained language model is used to construct a word vector representation of the given input sentence and grammatical error correction results;
[0074] Pre-trained language models such as BERT can be used to significantly improve the performance of natural language processing tasks by generating efficient contextual word-level representations. BERT stands for Bidirectional Encoder Representations from Transformer, which is a bidirectional encoder representation based on Transformer. As the name suggests, BERT uses Transformer, and when processing a word, it can also consider the words before and after the word to get its meaning in the context. The attention mechanism of Transformer has a good effect in feature extraction of words in context, and intuitively, bidirectional encoding that considers context is better than unidirectional encoding that only considers the previous (or following) context. Based on the BERT pre-trained language model, a word vector representation of a given input sentence and grammatical error correction results can be constructed.
[0075] S2: For each word vector in the grammatical correction result, the node interaction attention mechanism is used to obtain grammatical correction clues of words that can verify the current grammatical correction result from other grammatical correction results, and the node selection attention mechanism is used to integrate the grammatical correction clues of each grammatical correction result word to obtain grammatical correction evidence;
[0076] The node interaction attention mechanism is to calculate the attention weight α l→k Selectively read words from the lth node, and further select grammatical error correction evidence that can support the words in the kth node.
[0077] S3: Using the grammatical error correction evidence, perform quality assessment on the current grammatical error correction result.
[0078] For each given input sentence, a grammatical error correction model can be used to obtain K corresponding grammatical error correction results through column search. This grammatical error correction model is the model finally obtained by executing the grammatical error correction quality assessment method, which is obtained by training the grammatical error correction quality assessment method of the embodiment of the present invention using multiple sample given input sentences and several sample grammatical error correction results.
[0079] The embodiment of the present invention can obtain grammatical correction clues of words that can verify the current grammatical correction result by utilizing multiple grammatical correction results. The designed attention mechanism can find grammatical correction evidence that is helpful for verifying the quality of the current grammatical correction result in different grammatical correction results, thereby performing quality assessment on the current grammatical correction result, having a more efficient assessment effect and improving the grammatical correction effect.
[0080] Furthermore, based on a given input sentence and a plurality of grammatical error correction results, using a pre-trained language model to construct a word vector representation of the given input sentence and the grammatical error correction results includes:
[0081] For a given input sentence s, a grammatical error correction model is used to obtain K corresponding grammatical error correction results through column search, marked as C = {c 1 ,…,c k ,…,c K};
[0082] For a given input sentence s containing m words and the kth grammatical error correction result c containing n words k , the given input sentence and the grammatical error correction result sentence pair <s,c k >Use the special symbols "[CLS]" and "[SEP]" to represent the beginning and end of a sentence for splicing, and treat the spliced content as a node, then use BERT to encode it to get the node initialization representation H k :
[0083] H k =BERT([CLS]s[SEP]c k [SEP])
[0084] The given input sentence and the grammatical error correction result sentence pair <s,c k >The node initialization represents H k Contains the hidden state representation of each word and is marked as
[0085] According to the grammatical correction quality assessment method provided by the present invention, for each word vector in the grammatical correction result, a node interaction attention mechanism is used to obtain grammatical correction clues of words that can verify the current grammatical correction result from other grammatical correction results, and a node selection attention mechanism is used to integrate the grammatical correction clues of each grammatical correction result word, and the grammatical correction evidence is obtained, which specifically includes:
[0086] Based on the <s,c k >, using the node interaction attention mechanism to calculate the attention weight α l→k Selectively read words from the lth node, select grammatical error correction clues that can support the words in the kth node, and form a fine-grained representation V of the node l→k ;
[0087] Adopting node selection attention mechanism, based on the fine-grained representation V of the node l→k Calculate the importance of the node and integrate grammatical error correction evidence from multiple nodes that can support the verification of the quality of the current word.
[0088] According to the grammatical error correction quality assessment method provided by the present invention, the grammatical error correction clues that can support the words in the kth node are selected to form a fine-grained representation V of the node. l→k include:
[0089] For the pth word in the kth node Based on words and the qth word in the lth node The correlation score between the word representations is used to calculate the weight of the node interaction attention mechanism
[0090]
[0091] Among them, W is a trainable parameter, and Don't be two words and Vectorized representation of ;
[0092] Using node interaction attention weights Take the weighted sum of all word representations in the lth node to get the pth word in the kth node For example, the fine-grained node representation of the lth node
[0093]
[0094] based on Get the fine-grained node representation of the lth node facing the kth node
[0095] The node selection attention mechanism is used to further integrate grammatical error correction support evidence that can support the verification of the quality of the current word from multiple nodes by calculating the importance of the node. Specifically, the node selection attention mechanism is used based on the fine-grained representation V of the node. l→k Calculate the importance of the node and integrate grammatical error correction evidence from multiple nodes that can support the verification of the quality of the current word, including:
[0096] For the lth node, given the input sentence s and the grammatical error correction result c in the node l The words in are weighted to represent them respectively, so as to calculate the selection attention weight γ for the lth node l Then, the selected attention weight γ of the node is used l The fine-grained node representation V for the lth node l→k Perform weighted summation to obtain the node verification representation for the kth node and grammatical error correction clues;
[0097] For the lth node, the node selection attention weight γ l , and weightedly integrate the grammatical correction clues to obtain grammatical correction evidence that can support the verification of the current word quality.
[0098] According to the grammatical error correction quality assessment method provided by the present invention, the selected attention weight γ of the lth node l , the node verification representation of the kth node The calculation method is:
[0099] To calculate the selected attention weight γ of a node l , establish an interaction matrix M between the given input sentence and the grammatical error correction result in the lth node l ;
[0100] In the interaction matrix M l Each element in The correlation between the i-th word of a given input sentence in the l-th node and the j-th word in the grammatical correction result is measured, and the end symbol "[SEP]" used to mark the end of the sentence is included in the calculation:
[0101]
[0102] Among them, W is a trainable parameter, which is calculated on the two dimensions of the given input sentence and the grammatical error correction result. The attention weight is calculated using the following mechanism and
[0103]
[0104]
[0105] Using two sets of attention weights and Get the representation h of the given input sentence and grammatical error correction result in the lth node ls and h lh :
[0106]
[0107]
[0108] Calculate the node selection attention weight γ of the lth node l :
[0109]
[0110] in, represents the multiplication operation of the corresponding positions of the elements; represents the concatenation operation;
[0111] Node selection attention weight γ l Further integration for the pth word in the kth node The fine-grained node representation of the lth node And form the p-th word facing the k-th node Node verification representation
[0112]
[0113] Get the node verification representation for the kth node
[0114] According to the grammatical error correction quality assessment method provided by the present invention, the use of the grammatical error correction evidence to perform quality assessment on the current grammatical error correction result includes:
[0115] Use the cross entropy loss function to calculate the p-th word in the k-th node The loss of word quality classification label y:
[0116]
[0117] Among them, y * Represents the actual label classification given by the labeled data;
[0118] Calculate the loss function during model training:
[0119]
[0120] For the pth word in the kth node Use the corresponding node verification representation To calculate the probability of word-level grammatical error correction quality assessment label y
[0121]
[0122] in, represents the multiplication operation of the corresponding positions of the elements; represents the concatenation operation;
[0123] The score of the quality assessment of all words in the grammatical error correction results in the kth node, that is, the probability Average and get the grammatical error correction result c for a given input sentence s k The grammatical error correction quality evaluation score f(s,c k ):
[0124]
[0125] The embodiment of the present invention discloses a grammatical error correction quality assessment system, see Figure 2 ,include:
[0126] A word vector construction module 10 is used to construct a word vector representation of a given input sentence and a plurality of grammatical error correction results using a pre-trained language model based on a given input sentence and a plurality of grammatical error correction results;
[0127] The grammatical correction evidence determination module 20 is used to obtain grammatical correction clues of words that can verify the current grammatical correction result from other grammatical correction results using a node interaction attention mechanism for the word vector in each grammatical correction result, and to integrate the grammatical correction clues of each grammatical correction result word using a node selection attention mechanism to obtain grammatical correction evidence;
[0128] The quality assessment module 30 is used to perform quality assessment on the current grammar correction result by using the grammar correction evidence.
[0129] Figure 3 An example of a physical structure diagram of an electronic device is shown, and the electronic device may include: a processor 310, a communication interface 320, a memory 330, and a communication bus 340, wherein the processor 310, the communication interface 320, and the memory 330 communicate with each other through the communication bus 340. The processor 310 may call the logic instructions in the memory 330 to execute a grammatical error correction quality assessment method, which includes:
[0130] S1: Based on a given input sentence and several grammatical error correction results, a pre-trained language model is used to construct a word vector representation of the given input sentence and grammatical error correction results;
[0131] S2: For each word vector in the grammatical correction result, the node interaction attention mechanism is used to obtain grammatical correction clues of words that can verify the current grammatical correction result from other grammatical correction results, and the node selection attention mechanism is used to integrate the grammatical correction clues of each grammatical correction result word to obtain grammatical correction evidence;
[0132] S3: Using the grammatical error correction evidence, perform quality assessment on the current grammatical error correction result.
[0133] In addition, the logic instructions in the above-mentioned memory can be implemented in the form of software functional units and can be stored in a computer-readable storage medium when sold or used as an independent product. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.
[0134] On the other hand, the present invention further provides a computer program product, the computer program product comprising a computer program stored on a non-transitory computer-readable storage medium, the computer program comprising program instructions, and when the program instructions are executed by a computer, the computer can perform a grammatical error correction quality assessment method, the method comprising:
[0135] S1: Based on a given input sentence and several grammatical error correction results, a pre-trained language model is used to construct a word vector representation of the given input sentence and grammatical error correction results;
[0136] S2: For each word vector in the grammatical correction result, the node interaction attention mechanism is used to obtain grammatical correction clues of words that can verify the current grammatical correction result from other grammatical correction results, and the node selection attention mechanism is used to integrate the grammatical correction clues of each grammatical correction result word to obtain grammatical correction evidence;
[0137] S3: Using the grammatical error correction evidence, perform quality assessment on the current grammatical error correction result.
[0138] In another aspect, the present invention further provides a non-transitory computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, a grammatical error correction quality assessment method is implemented, the method comprising:
[0139] S1: Based on a given input sentence and several grammatical error correction results, a pre-trained language model is used to construct a word vector representation of the given input sentence and grammatical error correction results;
[0140] S2: For each word vector in the grammatical correction result, the node interaction attention mechanism is used to obtain grammatical correction clues of words that can verify the current grammatical correction result from other grammatical correction results, and the node selection attention mechanism is used to integrate the grammatical correction clues of each grammatical correction result word to obtain grammatical correction evidence;
[0141] S3: Using the grammatical error correction evidence, perform quality assessment on the current grammatical error correction result.
[0142] The system embodiment described above is merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment. Those of ordinary skill in the art may understand and implement it without creative work.
[0143] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.
[0144] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.
Claims
1. A grammatical error correction quality assessment method, characterized in that: For text proofreading, including: Based on a given input sentence and several grammatical correction results, a pre-trained language model is used to construct a word vector representation of the given input sentence and the grammatical correction results; For each word vector in the grammatical correction result, the node interaction attention mechanism is used to obtain grammatical correction clues of words that can verify the current grammatical correction result from several other grammatical correction results, and the node selection attention mechanism is used to integrate the grammatical correction clues of each grammatical correction result word to obtain grammatical correction evidence; Using the grammatical error correction evidence, the quality of the current grammatical error correction result is evaluated, wherein: The method of constructing a word vector representation of a given input sentence and a plurality of grammatical error correction results using a pre-trained language model based on a given input sentence and a plurality of grammatical error correction results includes: For a given input sentence , using a grammar error correction model to obtain The corresponding grammatical error correction results are marked as ; For a given input sentence containing m words and the first Syntax correction results , the given input sentence and the grammatical error correction result sentence pair Use the special symbols "[CLS]" and "[SEP]" to represent the beginning and end of a sentence for splicing, and treat the spliced content as a node, then use BERT to encode it to get the node initialization representation : ; The given input sentence and the grammatical error correction result sentence pair Node initialization representation Contains the hidden state representation of each word and is marked as ; in, For the word vector in each grammatical correction result, the node interaction attention mechanism is used to obtain grammatical correction clues of words that can verify the current grammatical correction result from other grammatical correction results, and the node selection attention mechanism is used to integrate the grammatical correction clues of each grammatical correction result word, and the grammatical correction evidence obtained specifically includes: Based on the , using the node interaction attention mechanism to calculate the attention weight Selectively from Read words from the nodes and select the words that can be used for the The words in each node serve as supporting grammatical error correction clues to form a fine-grained representation of the node ; Adopt node selection attention mechanism, based on fine-grained representation of nodes Calculate the importance of the node and integrate grammatical error correction evidence from multiple nodes that can support the verification of the quality of the current word; Among them, The using the grammatical error correction evidence to perform quality assessment on the current grammatical error correction result includes: The cross entropy loss function is used to calculate The first node Words Word quality classification label Loss: ; in, Represents the actual label classification given by the labeled data; Calculate the loss function during model training: ; For The first node Words , using the corresponding node verification representation To calculate the word-level grammatical correction quality assessment label Probability : ; in," " represents the multiplication operation of corresponding positions of elements, " " represents the splicing operation; For The score of the quality assessment score of all words in the grammatical error correction results in the node, that is, the probability Average, and get , grammar correction results Grammar correction quality assessment score : 。 2. The grammatical error correction quality assessment method according to claim 1, characterized in that: The method of selecting the The words in each node serve as supporting grammatical error correction clues to form a fine-grained representation of the node include: For The first Words , according to the word and The first Words The correlation score between the word representations is used to calculate the weight of the node interaction attention mechanism : ; in, is a trainable parameter, and Two words each and Vectorized representation of ; Using node interaction attention weights For All the word representations in the nodes are weighted summed to obtain The first Words For example, Fine-grained node representation of nodes : ; based on , get the The node Fine-grained node representation of nodes .
3. The grammatical error correction quality assessment method according to claim 2, characterized in that: The node selection attention mechanism is adopted, based on the fine-grained representation of the node Calculate the importance of the node and integrate grammatical error correction evidence from multiple nodes that can support the verification of the quality of the current word. Specifically, it includes: For a given input sentence in a node and grammar correction results The words in are weighted and the value for The selected attention weight of each node ; Using the obtained node selection attention weight For Fine-grained node representation of nodes Perform weighted summation to obtain Node verification representation of nodes , and grammatical correction clues; For Node selection attention weights for nodes , and weightedly integrate the grammatical correction clues to obtain grammatical correction evidence that can support the verification of the current word quality.
4. The grammatical error correction quality assessment method according to claim 3, characterized in that: The said The selected attention weight of each node , No. Node verification representation of nodes , the calculation method is: Needle in the An interaction matrix is established between the given input sentence and the grammatical error correction result in the node ; In the interaction matrix Each element in Measured the The first node of a given input sentence words and grammatical correction results The relevance of the words, the terminator "[SEP]" used to mark the end of the sentence is included in the calculation: ; in, It is a trainable parameter, calculated on the two dimensions of the given input sentence and the grammatical correction result, and the attention weight is calculated using the following mechanism and : ; ; Using two sets of attention weights and , get the The representation of the given input sentence and grammatical error correction results in the nodes as well as : ; ; Calculate the Node selection attention weights for nodes : ; in," " represents the multiplication operation of corresponding positions of elements, " " represents the splicing operation; Node selection attention weights Further integration of The first Words No. Fine-grained node representation of nodes And form a The first Words Node verification representation : ; Get the first Node verification representation of nodes .
5. A grammatical error correction quality assessment system, characterized in that: Used for text proofreading, to implement the grammatical error correction quality assessment method according to any one of claims 1 to 4, the system comprising: A word vector construction module is used to construct a word vector representation of a given input sentence and a number of grammatical error correction results using a pre-trained language model based on a given input sentence and a number of grammatical error correction results; The grammatical correction evidence determination module is used to obtain grammatical correction clues of words that can verify the current grammatical correction result from other grammatical correction results for the word vector in each grammatical correction result by using the node interaction attention mechanism, and integrate the grammatical correction clues of each grammatical correction result word by using the node selection attention mechanism to obtain grammatical correction evidence; The quality assessment module is used to use the grammatical error correction evidence to perform quality assessment on the current grammatical error correction result.
6. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the steps of the grammatical error correction quality assessment method according to any one of claims 1 to 4 are implemented.
7. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the grammatical error correction quality assessment method according to any one of claims 1 to 4 are implemented.
