A composition correcting method and system based on collocation rhetoric grammar correction

Abstract

This invention provides a method and system for essay correction based on collocation, rhetoric, and grammar. The invention constructs a conditional random field layer. The conditional random field layer is based on Markov decision chains and can reconnect the relationships between words in a sentence. After the output of the BERT encoding unit is input into the conditional random field layer, the improved Viterbi algorithm is run to decode the final grammar modification label. The output construction unit then constructs the correct sentence based on the grammar modification label.

Description

Technical Field

[0001] This invention relates to the field of Chinese language processing methods, and more specifically, to a method and system for essay correction based on collocation, rhetoric, and grammar error correction. Background Technology

[0002] The grammar rules of Chinese are complex and changeable, which causes some Chinese grammar error problems for Chinese beginners (such as primary school students) in the process of using Chinese. The cost of manual correction is relatively high, and machine correction has gradually developed with the development of technologies such as deep learning. Incorrect collocation error is one of the common grammar errors in compositions. There are many types of it, including improper subject-predicate collocation, improper verb-object collocation, improper subject-object collocation, improper modification component-center word collocation, improper correlative word collocation, improper one-sided-two-sided collocation, etc. Traditional machine correction technologies mainly focus on word spelling errors. For example, in the sentence "Xiaoming ate an apple", "苹果" is wrongly written as "平果". For incorrect collocation errors, due to the lack of consideration of long-distance context, traditional machine correction technologies are difficult to solve long-distance collocation error problems. Traditional error correction technologies can be summarized into three methods. The first is the rule-based method. For specific error types, specific error correction rules are formulated to correct errors. These rules can only correct specific types of errors, and due to the fact that the rules are manually designed, the scalability is very poor. The second method is the statistics-based method. Text features are obtained through relevant information such as characters and words to build a language model, and errors are discovered by counting the word frequencies between words. This method lacks in-depth understanding of text semantics and cannot capture long-distance dependency relationships. For example, for the sentence with improper correlative word collocation "Regardless of the extremely adverse climate conditions and geographical environment, the athletes still overcame the difficulties and successfully climbed to the summit", the distance between the two wrongly collocated words "不管" and "仍然" is relatively far. Moreover, it is very difficult to set the word frequency threshold. Setting it too high will lead to a large number of misjudgments, and setting it too low will lead to a large number of missed error detections. The third method is the deep learning-based method. A neural network is constructed to convert an error correction problem into a sequence generation problem, and the text containing grammar errors is converted into correct text through a method similar to machine translation. However, due to the exposure bias problem (referring to the fact that the data types encountered by the model during training and testing are different. This is reflected in the different inputs used in prediction and training. During training, each word input comes from real samples, but during prediction, the current input uses the output of the previous word), the results are usually affected by the error accumulation problem (error accumulate means that during testing, when the prediction result of the previous word is incorrect, this incorrect output is used as the input for the next unit, resulting in more and more mistakes). There are also some works that propose to solve the grammar error correction problem by simply using sequence labeling instead of generation methods. In these works, all error correction operations are completed through the predicted sequence labeling, such as text deletion, insertion, and replacement. However, the pure sequence labeling strategy needs to add insertion and replacement prefixes to the original words, such as "插入-好", "替换-坏", which will expand the vocabulary to three times its original size, greatly reducing the computational efficiency.Furthermore, simple sequence labeling is inefficient and cumbersome, requiring multiple prediction steps until no new operations can be predicted.

[0003] Generally, grammar correction problems are divided into three levels: identification, localization, and correction. Identification requires the model to determine whether a sentence is grammatically incorrect; localization requires the model to pinpoint the location and type of error; and correction requires the model to output a correct sentence that retains the original semantics as much as possible. Traditional work can be summarized into three approaches. The first is rule-based methods, which formulate specific correction rules for specific error types. These rules can only correct a few specific types of errors, and because the rules are manually designed, their scalability is poor. The second approach is statistical methods, which use Markov chain-based statistical models (such as N-gram, GloVe, Word2Vec) to discover errors by statistically analyzing word frequencies. This method lacks a deep understanding of text semantics and cannot capture long-distance dependencies. Current mainstream methods are based on deep neural networks. Many of these methods utilize sequence prediction models similar to machine translation, directly translating potentially erroneous input text into corrected text. However, these methods are susceptible to exposure errors and error accumulation problems. Some works propose using sequence labeling instead of sequence generation to solve the grammar correction problem. In these works, predicted sequence labels are used to perform all error correction operations, such as text deletion, insertion, and replacement. However, relying solely on sequence labeling requires expanding the vocabulary to three times its original size, significantly reducing computational efficiency. Furthermore, the pure sequence labeling framework is inefficient and cumbersome because it requires multiple prediction steps until no new operations are predicted. These studies can handle a small number of grammatical errors, but due to a lack of understanding of text semantics, they struggle to effectively handle mismatched grammatical errors, especially long-distance mismatched errors that require understanding of contextual semantics. Summary of the Invention

[0004] This invention provides a method for correcting essays based on collocation, rhetoric, and grammar. This method can be used by language teachers to correct student essays and by parents to guide their children in writing essays. It can reduce the burden on teachers and parents and has great application value.

[0005] Another objective of this invention is to propose an essay correction system based on collocation, rhetoric, and grammar correction that applies the above-mentioned method.

[0006] To achieve the above-mentioned technical effects, the technical solution of the present invention is as follows:

[0007] A method for correcting essay errors based on collocation, rhetoric, and grammar includes the following steps:

[0008] S1: Define the task formally for Chinese sentences;

[0009] S2: Construct a collocation error example construction unit to construct collocation error examples for the Chinese sentences processed in step S1;

[0010] S3: Construct BERT encoding units to encode the Chinese sentence obtained in step S2 into a word vector;

[0011] S4: Construct a conditional random field layer unit to decode the Chinese sentence obtained in step S3;

[0012] S5: The output building unit processes the data from step S4 to obtain the final result.

[0013] Furthermore, the specific process of step S1 is as follows:

[0014] For a given Chinese sentence It will be represented in the form of word strings. ,in For Chinese word segmentation; for sentences Correct the sentence Correct the grammatical errors in the text and produce the correct sentence. , and They are not necessarily equal; It can be equal to, greater than, or less than. ;

[0015] for :when equal hour, No additional special treatment is required; when Greater than hour, Right now Some words were deleted from the text; therefore, during the training phase, they were added back. - Special identifier arrive The tail, so that equal ;this ;when Less than hour, That is, in the original sentence More information was inserted in Add special symbols to the tail This indicates that these positions may need to be converted into some new words. .

[0016] Furthermore, the specific process of step S2 is as follows:

[0017] To address the lack of training data, the collocation error construction unit uses a large number of correct sentences from an unannotated corpus to construct common examples of improper collocation of conjunctions, quantifiers, and prepositions in writing.

[0018] The collocation error sample construction unit first randomly selects a correct sentence with conjunctions, quantifiers, or prepositions from the corpus, then masks the conjunctions, quantifiers, or prepositions in the sentence, and then uses BERT to predict the probability distribution of candidate words, where the candidate words and the masked words are words of the same type, that is, if the masked word is a conjunction, the candidate word should also be a conjunction; then, the word with the highest probability that is different from the original word is selected from the probability distribution of candidate words.

[0019] Furthermore, the specific process of step S3 is as follows:

[0020] The BERT encoding unit encodes a sentence into a word vector and extracts the semantic features of the sentence; for an input sentence, this unit feeds the sentence into the BERT model to obtain the encoded text embedding vector;

[0021] The input sentence first enters the embedding layer of the BERT encoding unit. In the embedding layer, each word in the sentence is encoded with position, sentence number, and character. After encoding, the sentence enters the feature extraction encoding module, which consists of N layers of transformer encoding layers. Finally, through the L layers of transformers in the BERT encoding unit, the text representation vector group is obtained. The output of the BERT coding unit is used as the output of the unit and input to the conditional random field layer unit.

[0022] Furthermore, the specific process of step S4 is as follows:

[0023] The semantic embedding vector encoded by the BERT encoding unit is decoded to obtain the corrected answer. In the BERT encoding unit, the sentence to be corrected has already been encoded into a representation vector. The conditional random field layer fully utilizes the contextual relationships between different words to construct the probability P(Y|X) of the output sequence Y as follows:

[0024]

[0025] in, Let Z(X) represent the word at the t-th position in the output sequence Y, and Z(X) be a normalization factor. For a fractional value that is not considered in context, it is defined as , and These are learnable parameters. To take into account a transition fraction from the above, it is defined as M is a learnable matrix; since the size of matrix M is relatively large, direct learning is not effective, so a low-rank decomposition method is used, letting... ,in and All are learnable matrices smaller than M, thus reducing the learning difficulty. To compute Z(X), an improved Viterbi algorithm is used, which uses only the top-k nodes at each time step and employs maximum likelihood estimation. Given the input X, Y that maximizes P(Y|X) is selected as the output of this unit. The optimization objective is:

[0026]

[0027] In this task, one phenomenon is that most words in a sentence are correct and require no improvement. Because maximum likelihood estimation is used for parameter learning, a simple copying strategy can easily lead to a significant decrease in the loss function value. Therefore, incorrect words that need more attention actually receive less attention during training. A focus loss is used to improve the loss function, where... These are hyperparameters of the weighted penalty term:

[0028] .

[0029] Furthermore, step S5 specifically comprises:

[0030] The final output is constructed using the output Y of the conditional random field layer unit. An improved Viterbi algorithm is employed. In the Viterbi algorithm, assuming a given hidden Markov model state space S with k states, the probability of the initial state i is... The transition probability from state i to state j is The observed output is The most likely state sequence of the observed results Given by the recurrence relation:

[0031]

[0032]

[0033] Here It is the probability of the state sequence most likely corresponding to the first t observations with a final state of k. The Viterbi path can be obtained by saving the backward pointer to remember the state x used in the second equation. A function is declared. If it returns t>1, then calculate. The value of x used, or k if t=1, is thus obtained:

[0034]

[0035]

[0036] After the improvement, only the top-k nodes are used at each time step to obtain the target result. After the above operation, the output sentence is the final corrected output.

[0037] A system for correcting essay writing based on collocation, rhetoric, and grammar errors includes:

[0038] The formal definition unit of a task is used to formally define a task within a text sentence;

[0039] The mismatch error example construction unit is used to construct mismatch error examples from Chinese sentences processed by the task's formal definition unit.

[0040] The BERT encoding unit encodes the Chinese sentence obtained from the improperly matched erroneous sample construction unit into a word vector.

[0041] The Conditional Random Field layer unit decodes the Chinese sentences obtained from the BERT coding unit;

[0042] The output building unit processes the results obtained from the conditional random field layer unit to obtain the final result.

[0043] Furthermore, the collocation error example construction unit addresses the problem of insufficient training data. This unit uses a large number of correct sentences from an unlabeled corpus to construct examples of common collocation errors involving conjunctions, quantifiers, and prepositions in writing.

[0044] The collocation error sample construction unit first randomly selects a correct sentence with conjunctions, quantifiers, or prepositions from the corpus, then masks the conjunctions, quantifiers, or prepositions in the sentence, and then uses BERT to predict the probability distribution of candidate words, where the candidate words and the masked words are words of the same type, that is, if the masked word is a conjunction, the candidate word should also be a conjunction; then, the word with the highest probability that is different from the original word is selected from the probability distribution of candidate words.

[0045] Furthermore, the BERT encoding unit encodes the sentence into a word vector and extracts the semantic features of the sentence; for the input sentence, the unit inputs the sentence into the BERT model to obtain the encoded text embedding vector;

[0046] The input sentence first enters the embedding layer of the BERT encoding unit. In the embedding layer, each word in the sentence is encoded with position, sentence number, and character. After encoding, the sentence enters the feature extraction encoding module, which consists of N layers of transformer encoding layers. Finally, through the L layers of transformers in the BERT encoding unit, the text representation vector group is obtained. The output of the BERT coding unit is used as the output of the unit and input to the conditional random field layer unit.

[0047] Furthermore, the conditional random field layer unit decodes the semantic embedding vector encoded by the BERT encoding unit to obtain the corrected answer. In the BERT encoding unit, the sentence to be corrected has already been encoded into a representation vector. The conditional random field layer fully utilizes the contextual relationships between different words to construct the probability P(Y|X) of the output sequence Y as follows:

[0048]

[0049] in, Let Z(X) represent the word at the t-th position in the output sequence Y, and Z(X) be a normalization factor. For a fractional value that is not considered in context, it is defined as , and These are learnable parameters. To take into account a transition fraction from the above, it is defined as M is a learnable matrix; since the size of matrix M is relatively large, direct learning is not effective, so a low-rank decomposition method is used, letting... ,in and All are learnable matrices smaller than M, thus reducing the learning difficulty. To compute Z(X), an improved Viterbi algorithm is used, which uses only the top-k nodes at each time step and employs maximum likelihood estimation. Given the input X, Y that maximizes P(Y|X) is selected as the output of this unit. The optimization objective is:

[0050]

[0051] In this task, one phenomenon is that most words in a sentence are correct and require no improvement. Because maximum likelihood estimation is used for parameter learning, a simple copying strategy can easily lead to a significant decrease in the loss function value. Therefore, incorrect words that need more attention actually receive less attention during training. A focus loss is used to improve the loss function, where... These are hyperparameters of the weighted penalty term:

[0052] .

[0053] Furthermore, the output construction unit uses the output Y of the conditional random field layer unit to construct the final output, employing an improved Viterbi algorithm. In the Viterbi algorithm, it is assumed that given the hidden Markov model state space S, there are k states, and the probability of the initial state i is... The transition probability from state i to state j is The observed output is The most likely state sequence of the observed results Given by the recurrence relation:

[0054]

[0055]

[0056] Here It is the probability of the state sequence most likely corresponding to the first t observations with a final state of k. The Viterbi path can be obtained by saving the backward pointer to remember the state x used in the second equation. A function is declared. If it returns t>1, then calculate. The value of x used, or k if t=1, is thus obtained:

[0057]

[0058]

[0059] After the improvement, only the top-k nodes are used at each time step to obtain the target result. After the above operation, the output sentence is the final corrected output.

[0060] Compared with the prior art, the beneficial effects of the technical solution of the present invention are:

[0061] (1) One of the main problems in current Chinese grammar error correction research is the lack of training data. In the model training stage, the error sample construction unit uses a large number of correct sentences from an unlabeled corpus to generate training samples and construct Chinese collocation inappropriate examples. This method can alleviate the problem of data scarcity to a certain extent, effectively expand the scale of the training set, solve the problem of insufficient training data, and thus improve the error correction effect of the model.

[0062] (2) Based on relevant research, and considering the complexity and diversity of Chinese grammar, the BERT encoding unit of this patent uses a pre-trained BERT model to obtain the encoded semantic representation. Among them, the converter encoding layer based on the multi-head self-attention mechanism selectively focuses on the information of the input sentence, that is, assigns different weights to a certain character or word in the sentence, makes full use of the information of the sentence context, enhances the model's ability to capture contextual content, and can obtain richer semantic information, thereby improving the performance of the model;

[0063] (3) In the model proposed in this patent, each node can make full use of the bidirectional information of the context. Based on the contextual relationship between different words, the conditional random field layer unit adopts maximum likelihood estimation to obtain the most likely output, which is used as the input of the output building unit;

[0064] (4) Based on the input from the conditional random field layer unit, combined with the improved Viterbi algorithm, the output building unit can take various operations such as "keep", "delete", and "insert" to obtain the result of the initial sentence after model correction, and the length of the output sentence is not required to be the same as the initial sentence. Attached Figure Description

[0065] Figure 1 This is a flowchart of the method of the present invention;

[0066] Figure 2 This is a system structure diagram of the present invention;

[0067] Figure 3 Example of incorrect collocation;

[0068] Figure 4 This is a diagram of the BERT coding unit structure. Detailed Implementation

[0069] The accompanying drawings are for illustrative purposes only and should not be construed as limiting the scope of this patent.

[0070] To better illustrate this embodiment, some parts in the accompanying drawings may be omitted, enlarged, or reduced, and do not represent the actual product dimensions;

[0071] It will be understood by those skilled in the art that certain well-known structures and their descriptions may be omitted in the accompanying drawings.

[0072] The technical solution of the present invention will be further described below with reference to the accompanying drawings and embodiments.

[0073] Example 1

[0074] like Figure 1 As shown, a method for correcting essay errors based on collocation, rhetoric, and grammar includes the following steps:

[0075] S1: Define the task formally for Chinese sentences;

[0076] S2: Construct a collocation error example construction unit to construct collocation error examples for the Chinese sentences processed in step S1;

[0077] S3: Construct BERT encoding units to encode the Chinese sentence obtained in step S2 into a word vector;

[0078] S4: Construct a conditional random field layer unit to decode the Chinese sentence obtained in step S3;

[0079] S5: The output building unit processes the data from step S4 to obtain the final result.

[0080] Furthermore, the specific process of step S1 is as follows:

[0081] For a given Chinese sentence It will be represented in the form of word strings. ,in For Chinese word segmentation; for sentences Correct the sentence Correct the grammatical errors in the text and produce the correct sentence. ,Notice, and They are not necessarily equal; It can be equal to, greater than, or less than. ;

[0082] for :when equal hour, No additional special treatment is required; when Greater than hour, Right now Some words were deleted from the text; therefore, during the training phase, they were added back. - Special identifier arrive The tail, so that equal ;this ;when Less than hour, That is, in the original sentence More information was inserted in Add special symbols to the tail This indicates that these positions may need to be converted into some new words. .

[0083] The specific process of step S2 is as follows:

[0084] To address the lack of training data, the collocation error construction unit uses a large number of correct sentences from an unannotated corpus to construct common examples of improper collocation of conjunctions, quantifiers, and prepositions in writing.

[0085] The collocation error sample construction unit first randomly selects a correct sentence with conjunctions, quantifiers, or prepositions from the corpus, then masks the conjunctions, quantifiers, or prepositions in the sentence, and then uses BERT to predict the probability distribution of candidate words, where the candidate words and the masked words are words of the same type, that is, if the masked word is a conjunction, the candidate word should also be a conjunction; then, the word with the highest probability that is different from the original word is selected from the probability distribution of candidate words.

[0086] like Figure 3 As shown, the correctly selected sentence from the corpus, "Even if we become wealthy, we shouldn't waste," contains the conjunction "even if." After masking, it becomes "[MASK]Even if we become wealthy, we shouldn't waste." This masked sentence is input into BERT, which predicts the probability distribution. Then, words are selected and filled in using the method described above. After BERT filling, the sentence becomes "If we become wealthy, we shouldn't waste." The conjunctions "if" and "also" are mismatched, thus completing the construction of a sentence example with a mismatched conjunction.

[0087] The specific process of step S3 is as follows:

[0088] The BERT encoding unit encodes a sentence into a word vector and extracts the semantic features of the sentence; for an input sentence, this unit feeds the sentence into the BERT model to obtain the encoded text embedding vector;

[0089] The input sentence first enters the embedding layer of the BERT encoding unit. In the embedding layer, each word in the sentence is encoded with position, sentence number, and character. After encoding, the sentence enters the feature extraction encoding module, which consists of N layers of transformer encoding layers. Finally, through the L layers of transformers in the BERT encoding unit, the text representation vector group is obtained. , The output of the BERT coding unit is used as the output of the unit and input to the conditional random field layer unit.

[0090] The specific process of step S4 is as follows:

[0091] The semantic embedding vector encoded by the BERT encoding unit is decoded to obtain the corrected answer. In the BERT encoding unit, the sentence to be corrected has already been encoded into a representation vector. The conditional random field layer fully utilizes the contextual relationships between different words to construct the probability P(Y|X) of the output sequence Y as follows:

[0092]

[0093] in, Let Z(X) represent the word at the t-th position in the output sequence Y, and Z(X) be a normalization factor. For a fractional value that is not considered in context, it is defined as , and These are learnable parameters. To take into account a transition fraction from the above, it is defined as M is a learnable matrix; since the size of matrix M is relatively large, direct learning is not effective, so a low-rank decomposition method is used, letting... ,in and All are learnable matrices smaller than M, thus reducing the learning difficulty. To compute Z(X), an improved Viterbi algorithm is used, which uses only the top-k nodes at each time step and employs maximum likelihood estimation. Given the input X, Y that maximizes P(Y|X) is selected as the output of this unit. The optimization objective is:

[0094]

[0095] In this task, one phenomenon is that most words in a sentence are correct and require no improvement. Because maximum likelihood estimation is used for parameter learning, a simple copying strategy can easily lead to a significant decrease in the loss function value. Therefore, incorrect words that need more attention actually receive less attention during training. A focus loss is used to improve the loss function, where... These are hyperparameters of the weighted penalty term:

[0096] .

[0097] Step S5 consists of the following components:

[0098] The final output is constructed using the output Y of the conditional random field layer unit. An improved Viterbi algorithm is employed. In the Viterbi algorithm, assuming a given hidden Markov model state space S with k states, the probability of the initial state i is... The transition probability from state i to state j is The observed output is The most likely state sequence of the observed results Given by the recurrence relation:

[0099]

[0100]

[0101] Here It is the probability of the state sequence most likely corresponding to the first t observations with a final state of k. The Viterbi path can be obtained by saving the backward pointer to remember the state x used in the second equation. A function is declared. If it returns t>1, then calculate. The value of x used, or k if t=1, is thus obtained:

[0102]

[0103]

[0104] After the improvement, only the top-k nodes are used at each time step to obtain the target result. After the above operation, the output sentence is the final corrected output.

[0105] Example 2

[0106] like Figure 2 As shown, a composition correction method system based on collocation, rhetoric, and grammar correction includes:

[0107] The formal definition unit of a task is used to formally define a task within a text sentence;

[0108] The mismatch error example construction unit is used to construct mismatch error examples from Chinese sentences processed by the task's formal definition unit.

[0109] like Figure 4 As shown, the BERT encoding unit encodes the Chinese sentence obtained from the improperly matched erroneous sample construction unit into a word vector;

[0110] The Conditional Random Field layer unit decodes the Chinese sentences obtained from the BERT coding unit;

[0111] The output building unit processes the results obtained from the conditional random field layer unit to obtain the final result.

[0112] The collocation error construction unit addresses the problem of insufficient training data. It uses a large number of correct sentences from an unannotated corpus to construct examples of common writing errors involving conjunctions, quantifiers, and prepositions.

[0113] The collocation error sample construction unit first randomly selects a correct sentence with conjunctions, quantifiers, or prepositions from the corpus, then masks the conjunctions, quantifiers, or prepositions in the sentence, and then uses BERT to predict the probability distribution of candidate words, where the candidate words and the masked words are words of the same type, that is, if the masked word is a conjunction, the candidate word should also be a conjunction; then, the word with the highest probability that is different from the original word is selected from the probability distribution of candidate words.

[0114] The BERT encoding unit encodes a sentence into a word vector and extracts the semantic features of the sentence; for an input sentence, this unit feeds the sentence into the BERT model to obtain the encoded text embedding vector;

[0115] The input sentence first enters the embedding layer of the BERT encoding unit. In the embedding layer, each word in the sentence is encoded with position, sentence number, and character. After encoding, the sentence enters the feature extraction encoding module, which consists of N layers of transformer encoding layers. Finally, through the L layers of transformers in the BERT encoding unit, the text representation vector group is obtained. The output of the BERT coding unit is used as the output of the unit and input to the conditional random field layer unit.

[0116] The Conditional Random Field (CRF) layer decodes the semantic embedding vector encoded by the BERT coding unit to obtain the corrected answer. In the BERT coding unit, the sentence to be corrected has already been encoded into a representation vector. The conditional random field layer fully utilizes the contextual relationships between different words to construct the probability P(Y|X) of the output sequence Y as follows:

[0117]

[0118] in, Let Z(X) represent the word at the t-th position in the output sequence Y, and Z(X) be a normalization factor. For a fractional value that is not considered in context, it is defined as , and These are learnable parameters. To take into account a transition fraction from the above, it is defined as M is a learnable matrix; since the size of matrix M is relatively large, direct learning is not effective, so a low-rank decomposition method is used, letting... ,in and All are learnable matrices smaller than M, thus reducing the learning difficulty. To compute Z(X), an improved Viterbi algorithm is used, which uses only the top-k nodes at each time step and employs maximum likelihood estimation. Given the input X, Y that maximizes P(Y|X) is selected as the output of this unit. The optimization objective is:

[0119]

[0120] In this task, one phenomenon is that most words in a sentence are correct and require no improvement. Because maximum likelihood estimation is used for parameter learning, a simple copying strategy can easily lead to a significant decrease in the loss function value. Therefore, incorrect words that need more attention actually receive less attention during training. A focus loss is used to improve the loss function, where... These are hyperparameters of the weighted penalty term:

[0121] .

[0122] The output building block uses the output Y of the conditional random field layer unit to construct the final output, employing an improved Viterbi algorithm. In the Viterbi algorithm, assuming a given hidden Markov model state space S with k states, the probability of the initial state i is... The transition probability from state i to state j is The observed output is The most likely state sequence of the observed results Given by the recurrence relation:

[0123]

[0124]

[0125] Here It is the probability of the state sequence most likely corresponding to the first t observations with a final state of k. The Viterbi path can be obtained by saving the backward pointer to remember the state x used in the second equation. A function is declared. If it returns t>1, then calculate. The value of x used, or k if t=1, is thus obtained:

[0126]

[0127]

[0128] After the improvement, only the top-k nodes are used at each time step to obtain the target result. After the above operation, the output sentence is the final corrected output.

[0129] Example 3

[0130] like Figure 1 As shown, a method for correcting essay errors based on collocation, rhetoric, and grammar includes the following steps:

[0131] S1: Define the task formally for Chinese sentences;

[0132] S2: Construct a collocation error example construction unit to construct collocation error examples for the Chinese sentences processed in step S1;

[0133] S3: Construct BERT encoding units to encode the Chinese sentence obtained in step S2 into a word vector;

[0134] S4: Construct a conditional random field layer unit to decode the Chinese sentence obtained in step S3;

[0135] S5: The output building unit processes the data from step S4 to obtain the final result.

[0136] Furthermore, the specific process of step S1 is as follows:

[0137] For a given Chinese sentence It will be represented in the form of word strings. ,in For Chinese word segmentation; for sentences Correct the sentence Correct the grammatical errors in the text and produce the correct sentence. ,Notice, and They are not necessarily equal; It can be equal to, greater than, or less than. ;

[0138] for :when equal hour, No additional special treatment is required; when Greater than hour, Right now Some words were deleted from the text; therefore, during the training phase, they were added back. - Special identifier arrive The tail, so that equal ;this ;when Less than hour, That is, in the original sentence More information was inserted in Add special symbols to the tail This indicates that these positions may need to be converted into some new words. .

[0139] The specific process of step S2 is as follows:

[0140] To address the lack of training data, the collocation error construction unit uses a large number of correct sentences from an unannotated corpus to construct common examples of improper collocation of conjunctions, quantifiers, and prepositions in writing.

[0141] The collocation error sample construction unit first randomly selects a correct sentence with conjunctions, quantifiers, or prepositions from the corpus, then masks the conjunctions, quantifiers, or prepositions in the sentence, and then uses BERT to predict the probability distribution of candidate words, where the candidate words and the masked words are words of the same type, that is, if the masked word is a conjunction, the candidate word should also be a conjunction; then, the word with the highest probability that is different from the original word is selected from the probability distribution of candidate words.

[0142] like Figure 3 As shown, the correctly selected sentence from the corpus, "Even if we become wealthy, we shouldn't waste," contains the conjunction "even if." After masking, it becomes "[MASK]Even if we become wealthy, we shouldn't waste." This masked sentence is input into BERT, which predicts the probability distribution. Then, words are selected and filled in using the method described above. After BERT filling, the sentence becomes "If we become wealthy, we shouldn't waste." The conjunctions "if" and "also" are mismatched, thus completing the construction of a sentence example with a mismatched conjunction.

[0143] The specific process of step S3 is as follows:

[0144] The BERT encoding unit encodes a sentence into a word vector and extracts the semantic features of the sentence; for an input sentence, this unit feeds the sentence into the BERT model to obtain the encoded text embedding vector;

[0145] The input sentence first enters the embedding layer of the BERT encoding unit. In the embedding layer, each word in the sentence is encoded with position, sentence number, and character. After encoding, the sentence enters the feature extraction encoding module, which consists of N layers of transformer encoding layers. Finally, through the L layers of transformers in the BERT encoding unit, the text representation vector group is obtained. The output of the BERT coding unit is used as the output of the unit and input to the conditional random field layer unit.

[0146] The specific process of step S4 is as follows:

[0147] The semantic embedding vector encoded by the BERT encoding unit is decoded to obtain the corrected answer. In the BERT encoding unit, the sentence to be corrected has already been encoded into a representation vector. The conditional random field layer fully utilizes the contextual relationships between different words to construct the probability P(Y|X) of the output sequence Y as follows:

[0148]

[0149] in, Let Z(X) represent the word at the t-th position in the output sequence Y, and Z(X) be a normalization factor. For a fractional value that is not considered in context, it is defined as , and These are learnable parameters. To take into account a transition fraction from the above, it is defined as M is a learnable matrix; since the size of matrix M is relatively large, direct learning is not effective, so a low-rank decomposition method is used, letting... ,in and All are learnable matrices smaller than M, thus reducing the learning difficulty. To compute Z(X), an improved Viterbi algorithm is used, which uses only the top-k nodes at each time step and employs maximum likelihood estimation. Given the input X, Y that maximizes P(Y|X) is selected as the output of this unit. The optimization objective is:

[0150]

[0151] In this task, one phenomenon is that most words in a sentence are correct and require no improvement. Because maximum likelihood estimation is used for parameter learning, a simple copying strategy can easily lead to a significant decrease in the loss function value. Therefore, incorrect words that need more attention actually receive less attention during training. A focus loss is used to improve the loss function, where... These are hyperparameters of the weighted penalty term:

[0152] .

[0153] Step S5 consists of the following components:

[0154] The final output is constructed using the output Y of the conditional random field layer unit. An improved Viterbi algorithm is employed. In the Viterbi algorithm, assuming a given hidden Markov model state space S with k states, the probability of the initial state i is... The transition probability from state i to state j is The observed output is The most likely state sequence of the observed results Given by the recurrence relation:

[0155]

[0156]

[0157] Here It is the probability of the state sequence most likely corresponding to the first t observations with a final state of k. The Viterbi path can be obtained by saving the backward pointer to remember the state x used in the second equation. A function is declared. If it returns t>1, then calculate. The value of x used, or k if t=1, is thus obtained:

[0158]

[0159]

[0160] After the improvement, only the top-k nodes are used at each time step to obtain the target result. After the above operation, the output sentence is the final corrected output.

[0161] like Figure 2 As shown, a composition correction method system based on collocation, rhetoric, and grammar correction includes:

[0162] The formal definition unit of a task is used to formally define a task within a text sentence;

[0163] The mismatch error example construction unit is used to construct mismatch error examples from Chinese sentences processed by the task's formal definition unit.

[0164] like Figure 4 As shown, the BERT encoding unit encodes the Chinese sentence obtained from the improperly matched erroneous sample construction unit into a word vector;

[0165] The Conditional Random Field layer unit decodes the Chinese sentences obtained from the BERT coding unit;

[0166] The output building unit processes the results obtained from the conditional random field layer unit to obtain the final result.

[0167] The collocation error construction unit addresses the problem of insufficient training data. It uses a large number of correct sentences from an unannotated corpus to construct examples of common writing errors involving conjunctions, quantifiers, and prepositions.

[0168] The collocation error sample construction unit first randomly selects a correct sentence with conjunctions, quantifiers, or prepositions from the corpus, then masks the conjunctions, quantifiers, or prepositions in the sentence, and then uses BERT to predict the probability distribution of candidate words, where the candidate words and the masked words are words of the same type, that is, if the masked word is a conjunction, the candidate word should also be a conjunction; then, the word with the highest probability that is different from the original word is selected from the probability distribution of candidate words.

[0169] The BERT encoding unit encodes a sentence into a word vector and extracts the semantic features of the sentence; for an input sentence, this unit feeds the sentence into the BERT model to obtain the encoded text embedding vector;

[0170] The input sentence first enters the embedding layer of the BERT encoding unit. In the embedding layer, each word in the sentence is encoded with position, sentence number, and character. After encoding, the sentence enters the feature extraction encoding module, which consists of N layers of transformer encoding layers. Finally, through the L layers of transformers in the BERT encoding unit, the text representation vector group is obtained. The output of the BERT coding unit is used as the output of the unit and input to the conditional random field layer unit.

[0171] The Conditional Random Field (CRF) layer decodes the semantic embedding vector encoded by the BERT coding unit to obtain the corrected answer. In the BERT coding unit, the sentence to be corrected has already been encoded into a representation vector. The conditional random field layer fully utilizes the contextual relationships between different words to construct the probability P(Y|X) of the output sequence Y as follows:

[0172]

[0173] in, Let Z(X) represent the word at the t-th position in the output sequence Y, and Z(X) be a normalization factor. For a fractional value that is not considered in context, it is defined as , and These are learnable parameters. To take into account a transition fraction from the above, it is defined as M is a learnable matrix; since the size of matrix M is relatively large, direct learning is not effective, so a low-rank decomposition method is used, letting... ,in and All are learnable matrices smaller than M, thus reducing the learning difficulty. To compute Z(X), an improved Viterbi algorithm is used, which uses only the top-k nodes at each time step and employs maximum likelihood estimation. Given the input X, Y that maximizes P(Y|X) is selected as the output of this unit. The optimization objective is:

[0174]

[0175] In this task, one phenomenon is that most words in a sentence are correct and require no improvement. Because maximum likelihood estimation is used for parameter learning, a simple copying strategy can easily lead to a significant decrease in the loss function value. Therefore, incorrect words that need more attention actually receive less attention during training. A focus loss is used to improve the loss function, where... These are hyperparameters of the weighted penalty term:

[0176] .

[0177] The output building block uses the output Y of the conditional random field layer unit to construct the final output, employing an improved Viterbi algorithm. In the Viterbi algorithm, assuming a given hidden Markov model state space S with k states, the probability of the initial state i is... The transition probability from state i to state j is The observed output is The most likely state sequence of the observed results Given by the recurrence relation:

[0178]

[0179]

[0180] Here It is the probability of the state sequence most likely corresponding to the first t observations with a final state of k. The Viterbi path can be obtained by saving the backward pointer to remember the state x used in the second equation. A function is declared. If it returns t>1, then calculate. The value of x used, or k if t=1, is thus obtained:

[0181]

[0182]

[0183] After the improvement, only the top-k nodes are used at each time step to obtain the target result. After the above operation, the output sentence is the final corrected output.

[0184] The same or similar labels correspond to the same or similar parts;

[0185] The positional relationships depicted in the accompanying drawings are for illustrative purposes only and should not be construed as limiting this patent.

[0186] Obviously, the above embodiments of the present invention are merely examples for clearly illustrating the present invention, and are not intended to limit the implementation of the present invention. Those skilled in the art can make other variations or modifications based on the above description. It is neither necessary nor possible to exhaustively describe all embodiments here. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the claims of the present invention.

Claims

1. A method for essay correction based on collocation, rhetoric, and grammar error correction, characterized in that, Includes the following steps: S1: Define the task formally for Chinese sentences; the specific process of step S1 is as follows: For a given Chinese sentence It will be represented in the form of word strings. ,in For Chinese word segmentation; for sentences Correct the sentence Correct the grammatical errors in the text and produce the correct sentence. , and They are not necessarily equal; It can be equal to, greater than, or less than. ; for :when equal hour, No additional special treatment is required; when Greater than hour, Right now Some words were deleted from the text; therefore, during the training phase, they were added back. - Special identifier arrive The tail, so that equal ; this ;when Less than hour, That is, in the original sentence More information was inserted in Add special symbols to the tail This indicates that these positions may need to be converted into some new words. ; S2: Constructing a collocation error example construction unit to construct collocation error examples for the Chinese sentences processed in step S1; the specific process of step S2 is as follows: To address the lack of training data, the collocation error construction unit uses a large number of correct sentences from an unannotated corpus to construct common examples of improper collocation of conjunctions, quantifiers, and prepositions in writing. The collocation error sample construction unit first randomly selects a correct sentence with conjunctions, quantifiers, or prepositions from the corpus, then masks the conjunctions, quantifiers, or prepositions in the sentence, and then uses BERT to predict the probability distribution of candidate words, where the candidate words and the masked words are words of the same type, that is, if the masked word is a conjunction, the candidate word should also be a conjunction; then, the word with the highest probability that is different from the original word is selected from the probability distribution of candidate words. S3: Construct BERT encoding units to encode the Chinese sentence obtained in step S2 into a word vector; S4: Construct a conditional random field layer unit to decode the Chinese sentence obtained in step S3; S5: The output building unit processes the data from step S4 to obtain the final result.

2. The essay correction method based on collocation, rhetoric, and grammar correction according to claim 1, characterized in that, The specific process of step S3 is as follows: The BERT encoding unit encodes a sentence into a word vector and extracts the semantic features of the sentence; for an input sentence, this unit feeds the sentence into the BERT model to obtain the encoded text embedding vector; The input sentence first enters the embedding layer of the BERT encoding unit. In the embedding layer, each word in the sentence is encoded with position, sentence number, and character. After encoding, the sentence enters the feature extraction encoding module, which consists of N layers of transformer encoding layers. Finally, through the L layers of transformers in the BERT encoding unit, the text representation vector group is obtained. The output of the BERT coding unit is used as the output of the unit and input to the conditional random field layer unit.

3. The essay correction method based on collocation, rhetoric, and grammar correction according to claim 2, characterized in that, The specific process of step S4 is as follows: The semantic embedding vector encoded by the BERT encoding unit is decoded to obtain the corrected answer. In the BERT encoding unit, the sentence to be corrected has already been encoded into a representation vector. The conditional random field layer fully utilizes the contextual relationships between different words to construct the probability P(Y|X) of the output sequence Y as follows: in, Let Z(X) represent the word at the t-th position in the output sequence Y, and Z(X) be a normalization factor. For a fractional value that is not considered in context, it is defined as , and These are learnable parameters. To take into account a transition fraction from the above, it is defined as M is a learnable matrix; since the size of matrix M is relatively large, direct learning is not effective, so a low-rank decomposition method is used, letting... ,in and All are learnable matrices smaller than M, thus reducing the learning difficulty. To compute Z(X), an improved Viterbi algorithm is used, which uses only the top-k nodes at each time step and employs maximum likelihood estimation. Given the input X, Y that maximizes P(Y|X) is selected as the output of this unit. The optimization objective is: In this task, one phenomenon is that most words in a sentence are correct and require no improvement because maximum likelihood estimation is used for parameter learning. Therefore, a simple copying strategy can easily lead to a significant decrease in the loss function value. Consequently, incorrect words that need more attention actually receive less attention during training. A focus loss method is used to improve the loss function, where... These are hyperparameters of the weighted penalty term: 。 4. The essay correction method based on collocation, rhetoric, and grammar correction according to claim 3, characterized in that, The specific components of step S5 are as follows: The final output is constructed using the output Y of the conditional random field layer unit. An improved Viterbi algorithm is employed. In the Viterbi algorithm, assuming a given hidden Markov model state space S with k states, the probability of the initial state i is... The transition probability from state i to state j is The observed output is The most likely state sequence of the observed results Given by the recurrence relation: Here It is the probability of the state sequence most likely corresponding to the first t observations with a final state of k. The Viterbi path can be obtained by saving the backward pointer to remember the state x used in the second equation. A function is declared. If it returns t>1, then calculate. The value of x used, or k if t=1, is thus obtained: After the improvement, only the top-k nodes are used at each time step to obtain the target result, and the output sentence is the final corrected output.

5. A method system for essay correction based on collocation, rhetoric, and grammar correction, based on the essay correction method based on collocation, rhetoric, and grammar correction as described in any one of claims 1-4, characterized in that, include: The formal definition unit of a task is used to formally define a task within a text sentence; The mismatch error example construction unit is used to construct mismatch error examples from Chinese sentences processed by the task's formal definition unit. The collocation error example construction unit addresses the problem of insufficient training data. It uses a large number of correct sentences from an unannotated corpus to construct examples of common collocation errors involving conjunctions, quantifiers, and prepositions in writing. The collocation error sample construction unit first randomly selects a correct sentence with conjunctions, quantifiers, or prepositions from the corpus, then masks the conjunctions, quantifiers, or prepositions in the sentence, and then uses BERT to predict the probability distribution of candidate words, where the candidate words and the masked words are words of the same type, that is, if the masked word is a conjunction, the candidate word should also be a conjunction; then, the word with the highest probability that is different from the original word is selected from the probability distribution of candidate words. The BERT encoding unit encodes the Chinese sentence obtained from the improperly matched erroneous sample construction unit into a word vector. The Conditional Random Field layer unit decodes the Chinese sentences obtained from the BERT coding unit; The output building unit processes the results obtained from the conditional random field layer unit to obtain the final result.

6. The essay correction method system based on collocation, rhetoric, and grammar error correction according to claim 5, characterized in that, The BERT encoding unit encodes a sentence into a word vector and extracts the semantic features of the sentence; for an input sentence, the unit inputs the sentence into the BERT model to obtain the encoded text embedding vector; The input sentence first enters the embedding layer of the BERT encoding unit. In the embedding layer, each word in the sentence is encoded with position, sentence number, and character. After encoding, the sentence enters the feature extraction encoding module, which consists of N layers of transformer encoding layers. Finally, through the L layers of transformers in the BERT encoding unit, the text representation vector group is obtained. The output of the BERT coding unit is used as the output of the unit and input to the conditional random field layer unit.

7. The essay correction method system based on collocation, rhetoric, and grammar error correction according to claim 6, characterized in that, The conditional random field layer unit decodes the semantic embedding vector encoded by the BERT encoding unit to obtain the corrected answer. In the BERT encoding unit, the sentence to be corrected has already been encoded into a representation vector. The conditional random field layer fully utilizes the contextual relationships between different words to construct the probability P(Y|X) of the output sequence Y as follows: in, Let Z(X) represent the word at the t-th position in the output sequence Y, and Z(X) be a normalization factor. For a fractional value that is not considered in context, it is defined as , and These are learnable parameters. To take into account a transition fraction from the above, it is defined as M is a learnable matrix; since the size of matrix M is relatively large, direct learning is not effective, so a low-rank decomposition method is used, letting... ,in and All are learnable matrices smaller than M, thus reducing the learning difficulty. To compute Z(X), an improved Viterbi algorithm is used, which uses only the top-k nodes at each time step and employs maximum likelihood estimation. Given the input X, Y that maximizes P(Y|X) is selected as the output of this unit. The optimization objective is: In this task, one phenomenon is that most words in a sentence are correct and require no improvement because maximum likelihood estimation is used for parameter learning. Therefore, a simple copying strategy can easily lead to a significant decrease in the loss function value. Consequently, incorrect words that need more attention actually receive less attention during training. A focus loss method is used to improve the loss function, where... These are hyperparameters of the weighted penalty term: 。 8. The essay correction method system based on collocation, rhetoric, and grammar error correction according to claim 6, characterized in that, The output construction unit uses the output Y of the conditional random field layer unit to construct the final output, employing an improved Viterbi algorithm. In the Viterbi algorithm, assuming a given hidden Markov model state space S with k states, the probability of the initial state i is... The transition probability from state i to state j is The observed output is The most likely state sequence of the observed results Given by the recurrence relation: Here It is the probability of the state sequence most likely corresponding to the first t observations with a final state of k. The Viterbi path can be obtained by saving the backward pointer to remember the state x used in the second equation. A function is declared. If it returns t>1, then calculate. The value of x used, or k if t=1, is thus obtained: After the improvement, only the top-k nodes are used at each time step to obtain the target result, and the output sentence is the final corrected output.

Images