Sentence intelligent error correction method, device, computer equipment and storage medium

Abstract

The present invention relates to the technical field of detection models, and discloses a method, device, computer equipment and storage medium for intelligent error correction of sentences. The method predicts the dependency relationship between each word in the incorrect sentence by inputting the incorrect sentence into a preset dependency detection model, and obtains the dependency probability associated with each word; compares the dependency probability associated with each word with a preset dependency threshold, and records the words associated with a dependency probability less than the preset dependency threshold as words to be corrected; after masking the words to be corrected, a masked sentence to be corrected is obtained, and the masked sentence to be corrected is input into a preset language model, and correction prediction is performed on the words to be corrected to obtain predicted replacement words; the predicted replacement words corresponding to the words to be corrected are replaced with the words to be corrected, and the masked sentence to be corrected after replacement is recorded as the correct sentence corresponding to the incorrect sentence. The present invention improves the accuracy and efficiency of intelligent error correction of sentences.

Description

Technical Field

[0001] The present invention relates to the field of detection model technology, and in particular to a sentence intelligent error correction method, device, computer equipment and storage medium. Background Art

[0002] In the process of writing meeting minutes or official documents, single typos or multiple consecutive typos often occur. In order to improve the writing quality of the text, it is very necessary to intelligently identify and correct typos in the text.

[0003] In the prior art, typos in text are usually intelligently identified in the following manner: a sentence with errors in the text is input into an encoder and a decoder is used to decode each word in the text one by one and output the correct sentence. However, this method has the following shortcomings: the output of each step of the decoder depends on the output of the decoder in the previous step. When there is an error in the output of the decoder in the previous step, the error will continue to propagate, resulting in a low accuracy rate in correcting the erroneous sentences. Summary of the invention

[0004] The embodiments of the present invention provide a sentence intelligent error correction method, apparatus, computer equipment and storage medium to solve the problem of low accuracy in error correction of sentences with errors.

[0005] A sentence intelligent error correction method, comprising:

[0006] receiving a sentence correction instruction containing an erroneous sentence;

[0007] Input the erroneous sentence into a preset dependency detection model, predict the dependency relationship between the words in the erroneous sentence, obtain the dependency probability between each word and its corresponding dependent word, and associate each word with its corresponding dependency probability; the dependent word refers to at least one word located before or after a word and adjacent to it;

[0008] Comparing the dependency probability associated with each word with a preset dependency threshold, and recording the words associated with a dependency probability less than the preset dependency threshold as words to be corrected;

[0009] After masking the words to be corrected, a masked sentence to be corrected is obtained, and the masked sentence to be corrected is input into a preset language model, and correction prediction is performed on the words to be corrected to obtain a predicted replacement word corresponding to the words to be corrected;

[0010] The predicted replacement word corresponding to the word to be corrected is used to replace the word to be corrected, and the masked sentence to be corrected after the replacement is recorded as the correct sentence corresponding to the incorrect sentence.

[0011] A sentence intelligent error correction device, comprising:

[0012] An error correction instruction receiving module, used for receiving a sentence error correction instruction containing an erroneous sentence;

[0013] A dependency probability determination module, used for inputting the erroneous sentence into a preset dependency detection model, predicting the dependency relationship between the words in the erroneous sentence, obtaining the dependency probability between each word and its corresponding dependent word, and associating each word with its corresponding dependency probability; the dependent word refers to at least one word located before or after a word and adjacent to it;

[0014] A module for determining words to be corrected, used for comparing the dependency probability associated with each word with a preset dependency threshold, and recording the words associated with a dependency probability less than the preset dependency threshold as words to be corrected;

[0015] An error correction prediction module is used to perform masking processing on the words to be corrected to obtain masked sentences to be corrected, and input the masked sentences to be corrected into a preset language model to perform error correction prediction on the words to be corrected to obtain predicted replacement words corresponding to the words to be corrected;

[0016] The word replacement module is used to replace the word to be corrected with the predicted replacement word corresponding to the word to be corrected, and record the masked sentence to be corrected after the replacement as the correct sentence corresponding to the wrong sentence.

[0017] A computer device includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the above-mentioned intelligent sentence error correction method is implemented.

[0018] A computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the above-mentioned intelligent sentence error correction method is implemented.

[0019] The above-mentioned intelligent sentence correction method, device, computer equipment and storage medium receive a sentence correction instruction containing an erroneous sentence; input the erroneous sentence into a preset attachment detection model, predict the dependency relationship between the words in the erroneous sentence, obtain the dependency probability between each word and its corresponding attached word, and associate each word with the dependency probability corresponding to it; the attached word refers to at least one word located before or after a word and adjacent to it; compare the dependency probability associated with each word with a preset dependency threshold, and record the words associated with a dependency probability less than the preset dependency threshold as words to be corrected; after masking the words to be corrected, obtain a masked sentence to be corrected, and input the masked sentence to be corrected into a preset language model, perform correction prediction on the words to be corrected, and obtain a predicted replacement word corresponding to the words to be corrected; replace the words to be corrected with the predicted replacement word corresponding to the words to be corrected, and record the masked sentence to be corrected after the replacement as the correct sentence corresponding to the erroneous sentence.

[0020] The present invention uses a detection-before-error correction method to determine the words that may be wrong in the wrong sentence by means of dependency probability, thereby improving the accuracy of sentence wrong word detection; and after masking the words that may be wrong, error correction is replaced by a preset language model, thereby reducing the error correction workload of the preset language model and improving the efficiency of intelligent sentence error correction. At the same time, the preset affiliation detection model is separated from the preset language model independently, so that the two models can run in parallel, thereby improving the efficiency of intelligent sentence error correction. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments of the present invention will be briefly introduced below. Obviously, the accompanying drawings in the following description are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative labor.

[0022] Figure 1 Schematic diagram of an application environment of a sentence intelligent error correction method in one embodiment of the present invention;

[0023] Figure 2 is a flow chart of a sentence intelligent error correction method in one embodiment of the present invention;

[0024] Figure 3 is a flow chart of step S20 in the sentence intelligent error correction method in one embodiment of the present invention;

[0025] Figure 4 is a flow chart of step S201 in the sentence intelligent error correction method in one embodiment of the present invention;

[0026] Figure 5 is a flow chart of step S40 in the sentence intelligent error correction method in one embodiment of the present invention;

[0027] Figure 6 It is a principle block diagram of a sentence intelligent error correction device in one embodiment of the present invention;

[0028] Figure 7 It is a principle block diagram of a dependency probability determination module in a sentence intelligent error correction device in one embodiment of the present invention;

[0029] Figure 8 It is a principle block diagram of a word information processing unit in a sentence intelligent error correction device in one embodiment of the present invention;

[0030] Fig. 9 It is a principle block diagram of an error correction prediction module in a sentence intelligent error correction device in one embodiment of the present invention;

[0031] Fig.10 is a schematic diagram of a computer device in one embodiment of the present invention. DETAILED DESCRIPTION

[0032] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0033] The sentence intelligent error correction method provided by the embodiment of the present invention can be applied as follows: Figure 1 Specifically, the sentence intelligent error correction method is applied in a sentence intelligent error correction system, which includes the following: Figure 1 The client and server shown in the figure communicate with each other through the network to solve the problem of low accuracy of error correction for sentences with errors. The client, also known as the user end, refers to a program corresponding to the server that provides local services to customers. The client can be installed on, but not limited to, various personal computers, laptops, smart phones, tablets, and portable wearable devices. The server can be implemented as an independent server or a server cluster consisting of multiple servers.

[0034] In one embodiment, if Figure 2 As shown, a sentence intelligent error correction method is provided, and the method is applied in Figure 1 The server in the example is used as an example to illustrate the process, including the following steps:

[0035] S10: receiving a sentence correction instruction containing an erroneous sentence;

[0036] The sentence error correction instruction may be actively sent by the user, or may be actively triggered when the user types an incorrect sentence in the system. An incorrect sentence refers to a sentence with typos or syntactic errors.

[0037] S20: inputting the erroneous sentence into a preset dependency detection model, predicting the dependency relationship between the words in the erroneous sentence, obtaining the dependency probability between each word and its corresponding dependent word, and associating each word with its corresponding dependency probability; the dependent word refers to at least one word located before or after a word and adjacent to it;

[0038] Among them, the preset subsidiary relationship detection model is used to determine the dependency probability between each word and its corresponding subsidiary word, and the dependency probability refers to the probability that each word needs to rely on the existence of the subsidiary word in the wrong sentence. It can be understood that the subsidiary word can be an adjacent word before or after a word, or two adjacent words before or after a word, or an adjacent word before a word, and two adjacent words after a word; for example, assuming a sentence is "I ate a lot at noon today", then for the word noon, the corresponding subsidiary words can be "today" and "eat", or "I", "today" and "eat".

[0039] In one embodiment, if Figure 3 As shown, step S20 includes:

[0040] S201: Processing the word information of the erroneous sentence to obtain a forward hidden layer vector and a reverse hidden layer vector corresponding to the erroneous sentence; the forward hidden layer vector is determined by sorting the words in the erroneous sentence in a forward order; the reverse hidden layer vector is determined by sorting the words in the erroneous sentence in a reverse order;

[0041] It can be understood that information processing refers to operations including word segmentation, word embedding processing, and context information extraction of erroneous sentences.

[0042] In a specific embodiment, if Figure 4 As shown, step S201 includes:

[0043] S2011: After performing word segmentation processing on the erroneous sentence, determining a word sequence corresponding to the erroneous sentence;

[0044] Exemplarily, the erroneous sentence can be segmented using the Jieba word segmentation method to obtain the words in the erroneous sentence, and after replacing the codes corresponding to the words in the preset word dictionary, the word sequence corresponding to the erroneous sentence is determined; wherein the preset word dictionary refers to the words in other sample documents that are pre-encoded, and then in step S2011, the codes corresponding to the words in the erroneous sentence can be searched in the preset word dictionary for replacement, thereby obtaining a word sequence.

[0045] S2012: performing word embedding processing on the word sequence by a word embedding method to obtain a word vector sequence corresponding to the word sequence;

[0046] Among them, word embedding processing refers to the process of replacing each word in a word sequence with a word vector that can be recognized and trained by a preset affiliation detection model.

[0047] Specifically, after word segmentation is performed on the erroneous sentence and the word sequence corresponding to the erroneous sentence is determined, each word in the word sequence is embedded through a word embedding method (such as: through the skip-gram model of Word2Vec, the CBOW model of Word2Vec or the GloVe word vector, etc.), that is, a word vector that can be recognized and trained by a preset affiliation detection model is used to replace each word in the word sequence, thereby obtaining a word vector sequence corresponding to the word sequence.

[0048] S2013: Perform forward context information extraction on the word vector sequence through the information extraction module in the preset affiliation detection model to obtain the forward hidden layer vector; and perform reverse context information extraction on the word vector sequence to obtain the reverse hidden layer.

[0049] Among them, context information extraction includes extraction of previous information and extraction of following information, and the information extraction module is a Bilstm (Bi-directional Long Short-Term Memory) module. It can be understood that after the word sequence is embedded by the word embedding method to obtain the word vector sequence corresponding to the word sequence, the word vector sequence is arranged in the forward order of each word, and then the information extraction module in the preset affiliation detection model can be used to extract the previous information based on the forward order of each word in the word vector sequence. It can be understood that the second word vector in the word vector sequence needs to be determined based on the first word vector, and then the forward hidden layer vector is obtained; similarly, the information extraction module in the preset affiliation detection model is used to extract the following information based on the forward order of each word in the word vector sequence. It can be understood that from the last word vector of the word vector sequence, the second to last word vector needs to be determined based on the first to last word vector, and then the reverse hidden layer vector is obtained.

[0050] S202: inputting the forward hidden layer vector into the first linear transformation layer module in the preset affiliation detection model to determine a first word feature corresponding to the forward hidden layer vector; and simultaneously inputting the reverse hidden layer vector into the second linear transformation layer module in the preset affiliation detection model to determine a second word feature corresponding to the reverse hidden layer vector;

[0051] Specifically, after word information processing is performed on the erroneous sentence to obtain a forward hidden layer vector and a reverse hidden layer vector corresponding to the erroneous sentence, the forward hidden layer vector is input into the first linear transformation layer module in the preset affiliation detection model, and the forward hidden layer vector is linearly mapped to determine the first word feature corresponding to the forward hidden layer vector; at the same time, the reverse hidden layer vector is input into the second linear transformation layer module in the preset affiliation detection model, and the reverse hidden layer vector is linearly mapped to determine the second word feature corresponding to the reverse hidden layer vector. Among them, the first word feature refers to the features representing the attached word and the attached word, the attached word refers to each word in the erroneous sentence, and the attached word refers to at least one word located before or after an attached word and adjacent to it. It can be understood that since the forward hidden layer vector is determined according to the forward order of each word, for the first word feature, the attached word is the word that comes first in the forward order as an attached word, and the remaining adjacent words are attached words.

[0052] Further, assuming that the word features corresponding to each word are determined to be (in, is the word feature corresponding to the nth vector in the forward hidden layer vector, that is, the word feature of the nth word in the wrong sentence in the forward order), and (in, is the word feature corresponding to the nth vector in the reverse hidden layer vector, that is, the word feature of the nth word in the wrong sentence in the reverse order). It can be understood that for the first word in the wrong sentence in the forward order, its corresponding word feature in the forward hidden layer vector is The corresponding feature in the reverse hidden layer vector is, That is the last word feature.

[0053] Furthermore, according to the above as well as Determine the first word feature as: The second word features are: Among them, Warc-head and barc-head are trainable parameters of the first linear transformation layer module, and Warc-dep and barc-dep are trainable parameters of the second linear transformation layer module.

[0054] S203: Determine the dependency probability between each word and its corresponding subsidiary word through a double affine attention mechanism according to the first word feature and the second word feature.

[0055] Specifically, the forward hidden layer vector is input into the first linear transformation layer module in the preset affiliation detection model to determine the first word feature corresponding to the forward hidden layer vector; at the same time, the reverse hidden layer vector is input into the second linear transformation layer module in the preset affiliation detection model to determine the second word feature corresponding to the reverse hidden layer vector, and then all the first word features are stacked, and then the dependency probability between each word and its corresponding affiliated word is determined through a dual affine attention mechanism based on the second word features and the stacked first word features.

[0056] Furthermore, the dependency probability between each word and its corresponding subsidiary word can be determined by the following formula:

[0057]

[0058] in, is the dependency probability between the nth word and its corresponding attached word in the wrong sentence; Harc-head is obtained by stacking the features of the first word; is the word feature corresponding to the nth vector in the reverse hidden layer vector; Hhead is the forward hidden layer vector; W and U are the training parameters of the dual affine attention mechanism.

[0059] In this embodiment, the ability to detect incorrect words in incorrect sentences is improved through the dual affine attention mechanism. The dual affine attention mechanism can well associate the relationship between words, and can better combine contextual information to improve the accuracy and efficiency of incorrect word detection.

[0060] S30: comparing the dependency probability associated with each of the words with a preset dependency threshold, and recording the words associated with a dependency probability less than the preset dependency threshold as words to be corrected;

[0061] Among them, the preset dependency threshold can be manually determined based on historical sample data. For example, the preset dependency threshold can be 0.8, 0.9, etc. The preset dependency threshold can be adjusted according to the accuracy requirements of the specific application scenario. For example, if a higher accuracy rate of corrected words is required, the preset dependency threshold can be set to a higher value, such as 0.95.

[0062] It can be understood that after the erroneous sentence is input into the preset dependency detection model, the dependency relationship between the words in the erroneous sentence is predicted to obtain the dependency probability between each word and its corresponding subsidiary words, and after each word is associated with the dependency probability corresponding to it, the dependency probability associated with each word is compared with the preset dependency threshold. When the dependency probability associated with the word is less than the preset dependency threshold, it indicates that the dependency relationship between the word and other words in the erroneous sentence is weak, that is, the word may be an erroneous word, and therefore the word associated with the dependency probability less than the preset dependency threshold is recorded as a word to be corrected.

[0063] Furthermore, when the dependency probability associated with the word is greater than or equal to a preset dependency threshold, it indicates that the word has a strong dependency relationship with other words in the wrong sentence, that is, the word is not an incorrect word.

[0064] S40: After performing mask processing on the word to be corrected, a masked sentence to be corrected is obtained, and the masked sentence to be corrected is input into a preset language model, and correction prediction is performed on the word to be corrected to obtain a predicted replacement word corresponding to the word to be corrected;

[0065] It can be understood that masking refers to using special characters to cover up the word to be corrected so as to make correction prediction for the word at that position.

[0066] In one embodiment, if Figure 5 As shown, step S40 includes the following steps:

[0067] S401: converting and encoding the masked sentence through the encoding module in the preset language model to obtain an encoding vector corresponding to the masked sentence; the encoding vector includes an error correction encoding vector corresponding to the word to be corrected;

[0068] Among them, the encoding module is essentially a vector conversion layer in a preset language model, and the encoding module is used to convert the masked sentence to be corrected into a recognizable encoding vector.

[0069] Specifically, after comparing the dependency probability associated with each of the words with a preset dependency threshold and recording the words associated with a dependency probability less than the preset dependency threshold as words to be corrected, the masked sentence is converted and encoded through the encoding module in the preset language model to obtain an encoding vector corresponding to the masked sentence.

[0070] S402: Linearly mapping the code vector to determine the matching scores of the error correction code vector to each word to be replaced in a preset character dictionary;

[0071] It can be understood that the preset character dictionary can be obtained by performing word segmentation, encoding and other processing based on historical sample data, and the preset character dictionary stores encoding vectors corresponding to multiple words.

[0072] Specifically, the masked sentence is converted and encoded through the encoding module in the preset language model to obtain an encoding vector corresponding to the masked sentence, the encoding vector is linearly mapped, and the error correction encoding vector is matched with each encoding vector in the preset character dictionary. For example, the error correction encoding vector can be matched with each encoding vector in the preset character dictionary by determining the Euclidean distance between the error correction encoding vector and the encoding vector in the preset character dictionary, and then the matching score of the error correction encoding vector for each word to be replaced in the preset character dictionary is determined based on the Euclidean distance.

[0073] S403: performing normalization processing on each of the matching scores to obtain a matching probability corresponding to each of the matching scores;

[0074] S404: Record the to-be-replaced word with the largest matching probability among the matching probabilities and which is greater than or equal to a preset matching threshold as the predicted replacement word.

[0075] Among them, the preset matching threshold can be 0.9, 0.95, etc.

[0076] Specifically, after linear mapping the coding vector and determining that the error correction coding vector belongs to the matching scores of each word to be replaced in the preset character dictionary, each matching score is normalized to obtain a matching probability corresponding to each matching score, and then the word to be replaced with the largest matching probability among the matching probabilities and greater than a preset matching threshold is recorded as the predicted replacement word.

[0077] Furthermore, if the maximum value of the matching probability in the matching probability is less than the preset matching threshold, the word corresponding to the maximum value of the matching probability is sent to the preset recipient, which can be the sender of the sentence correction instruction, to prompt that a predicted replacement word matching the word at this position has not been found.

[0078] S50: replacing the word to be corrected with the predicted replacement word corresponding to the word to be corrected, and recording the masked sentence to be corrected after the replacement as the correct sentence corresponding to the erroneous sentence.

[0079] Specifically, after masking the word to be corrected, a masked sentence to be corrected is obtained, and the masked sentence to be corrected is input into a preset language model, and correction prediction is performed on the word to be corrected. After the predicted replacement word corresponding to the word to be corrected is obtained, the word to be corrected is replaced with the predicted replacement word corresponding to the word to be corrected, and the masked sentence to be corrected after replacement is recorded as the correct sentence corresponding to the wrong sentence, thereby completing the entire sentence intelligent correction process.

[0080] In this embodiment, by adopting the method of first detecting and then correcting, the possible erroneous words in the erroneous sentence are judged by dependency probability, thereby improving the judgment accuracy; and after the possible erroneous words are masked, they are corrected and replaced by the preset language model, thereby reducing the error correction workload of the preset language model and improving the efficiency of intelligent error correction of sentences.

[0081] In one embodiment, before step S40, the method further includes:

[0082] (1) obtaining a set of correct sample sentences; the set of correct sample sentences comprising at least one correct sample sentence;

[0083] It can be understood that the correct sample sentence refers to a sentence without typos and syntactic errors, and the correct sample sentence can be extracted from documents in various application scenarios.

[0084] (2) performing masking processing on the correct sample sentence to obtain a sample masked sentence; the sample masked sentence contains at least one sample masked word;

[0085] Specifically, this step includes the following steps:

[0086] Selecting a first mask word from the correct sample sentence, replacing the first mask word with a preset replacement character, and recording the replaced first mask word as the sample predicted word;

[0087] A second mask word is selected from the sample predicted words, and a preset homophone word is used to replace the selected second mask word.

[0088] Specifically, after obtaining the set of correct sample sentences, a first mask word is selected from the correct sample sentences, and the first mask word is replaced by a preset replacement character, and the replaced first mask word is recorded as the sample predicted word. Preferably, the selection probability of selecting the first mask word can be 12% (the selection probability is an experimental value), that is, the probability of each word in the correct sample sentence being selected as the first mask word is 12%; further, in order to improve the model's error correction ability for erroneous sentences, a second mask word is selected from the sample predicted words, and a preset homophone is used to replace the selected second mask word, thereby creating a scenario of homophone replacement errors (such as in intelligent multi-round conversations, robots often make homophone errors), and the first mask words other than the second mask word are not replaced, and only their corresponding positions are recorded. In this way, the error correction training process of homophone replacement can be achieved, and the training process of mask error correction can also be achieved.

[0089] (3) inputting the sample masked sentence into a preset training model including initial parameters, performing mask prediction on the sample masked words, and obtaining sample predicted words corresponding to the sample masked words;

[0090] Specifically, after masking the correct sample sentence to obtain a sample masked sentence, the sample masked sentence is converted and encoded to obtain a sample encoding vector corresponding to the sample masked sentence; the sample encoding vector is linearly mapped to determine the matching scores of each to-be-replaced word in the preset character dictionary that belongs to each first masked word and the second masked word; each matching score is normalized to obtain a matching probability corresponding to each matching score, and the to-be-replaced word with the largest matching probability among the matching probabilities and greater than or equal to the preset matching threshold is recorded as a sample predicted word corresponding to the first masked word or the second masked word.

[0091] (4) after replacing the sample mask word with the sample prediction word, recording the sample mask sentence after the replacement as the sample prediction sentence;

[0092] (5) determining a prediction loss value of the preset training model according to the sample prediction sentence and the correct sample sentence corresponding thereto;

[0093] Specifically, after the sample masked sentence is input into a preset training model including initial parameters, mask prediction is performed on the sample masked words to obtain sample predicted words corresponding to the sample masked words, the sample predicted words are replaced with the sample masked words, and the replaced sample masked sentences are recorded as sample predicted sentences; and then the prediction loss value of the preset training model is determined based on the sample predicted sentences and the corresponding correct sample sentences, and the prediction loss value can be determined by a cross entropy loss function.

[0094] (6) When the predicted loss value does not reach the preset convergence condition, the initial parameters of the preset training model are updated and iterated until the predicted loss value reaches the preset convergence condition, and the preset training model after convergence is recorded as the preset language model.

[0095] It can be understood that the convergence condition can be the condition that the predicted loss value is less than the set threshold, that is, when the predicted loss value is less than the set threshold, the training is stopped; the convergence condition can also be the condition that the predicted loss value is very small and will not decrease after 10,000 calculations, that is, when the predicted loss value is very small and will not decrease after 10,000 calculations, the training is stopped, and the preset training model after convergence is recorded as the preset language model.

[0096] Furthermore, after determining the prediction loss value of the preset training model based on the sample prediction sentence and the correct sample sentence corresponding thereto, when the prediction loss value does not reach the preset convergence condition, the initial parameters of the preset training model are adjusted according to the prediction loss value, and the sample mask sentence corresponding to the correct sample sentence is re-input into the preset training model after adjusting the initial parameters, so that when the prediction loss value corresponding to the correct sample sentence reaches the preset convergence condition, another correct sample sentence in the correct sample sentence set is selected, and the above steps (1) to (5) are performed to obtain the prediction loss value corresponding to the correct sample sentence, and when the prediction loss value does not reach the preset convergence condition, the initial parameters of the preset training model are adjusted again according to the prediction loss value, so that the prediction loss value corresponding to the correct sample sentence reaches the preset convergence condition.

[0097] In this way, after the preset training model is trained with all the correct sample sentences in the correct sample sentence set, the results output by the preset training model can continue to approach the accurate results, making the recognition accuracy higher and higher, until the prediction loss values ​​corresponding to all correct sample sentences reach the preset convergence conditions, and the preset training model after convergence is recorded as the preset language model.

[0098] It should be understood that the order of execution of the steps in the above embodiment does not necessarily mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the present invention.

[0099] In one embodiment, a sentence intelligent error correction device is provided, which corresponds one-to-one to the sentence intelligent error correction method in the above embodiment. Figure 6 As shown, the sentence intelligent error correction device includes an error correction instruction receiving module 10, a dependency probability determination module 20, a to-be-corrected word determination module 30, an error correction prediction module 40 and a word replacement module 50. The functional modules are described in detail as follows:

[0100] The error correction instruction receiving module 10 is used to receive a sentence error correction instruction containing an erroneous sentence;

[0101] The dependency probability determination module 20 is used to input the erroneous sentence into a preset dependency detection model, predict the dependency relationship between the words in the erroneous sentence, obtain the dependency probability between each word and its corresponding dependent word, and associate each word with its corresponding dependency probability; the dependent word refers to at least one word located before or after a word and adjacent to it;

[0102] The to-be-corrected word determination module 30 is used to compare the dependency probability associated with each word with a preset dependency threshold, and record the words associated with a dependency probability less than the preset dependency threshold as the to-be-corrected words;

[0103] The error correction prediction module 40 is used to perform masking processing on the word to be corrected to obtain a masked sentence to be corrected, and input the masked sentence to be corrected into a preset language model to perform error correction prediction on the word to be corrected to obtain a predicted replacement word corresponding to the word to be corrected;

[0104] The word replacement module 50 is used to replace the word to be corrected with the predicted replacement word corresponding to the word to be corrected, and record the masked sentence to be corrected after the replacement as the correct sentence corresponding to the erroneous sentence.

[0105] Preferably, if Figure 7 As shown, the dependency probability determination module 20 includes:

[0106] The word information processing unit 201 is used to perform word information processing on the erroneous sentence to obtain a forward hidden layer vector and a reverse hidden layer vector corresponding to the erroneous sentence; the forward hidden layer vector is determined by sorting the words in the erroneous sentence in a forward order; the reverse hidden layer vector is determined by sorting the words in the erroneous sentence in a reverse order;

[0107] The linear mapping unit 202 is used to input the forward hidden layer vector into the first linear transformation layer module in the preset affiliation detection model to determine the first word feature corresponding to the forward hidden layer vector; and simultaneously input the reverse hidden layer vector into the second linear transformation layer module in the preset affiliation detection model to determine the second word feature corresponding to the reverse hidden layer vector;

[0108] The dependency probability determination unit 203 determines the dependency probability between each of the words and its corresponding subsidiary words through a double affine attention mechanism according to the first word feature and the second word feature.

[0109] Preferably, if Figure 8 As shown, the word information processing unit 201 includes:

[0110] A word sequence determination subunit 2011 is used to determine a word sequence corresponding to the erroneous sentence after performing word segmentation processing on the erroneous sentence;

[0111] A word embedding processing subunit 2012 is used to perform word embedding processing on the word sequence by a word embedding method to obtain a word vector sequence corresponding to the word sequence;

[0112] The information extraction subunit 2013 is used to perform forward context information extraction on the word vector sequence through the information extraction module in the preset affiliation detection model to obtain the forward hidden layer vector; and simultaneously perform reverse context information extraction on the word vector sequence to obtain the reverse hidden layer.

[0113] Preferably, if Fig. 9 As shown, the error correction prediction module 40 includes:

[0114] The conversion encoding unit 401 is used to convert the masked sentence through the encoding module in the preset language model to obtain an encoding vector corresponding to the masked sentence; the encoding vector includes an error correction encoding vector corresponding to the word to be corrected;

[0115] A matching score determination unit 402, configured to perform linear mapping on the code vector to determine a matching score of each word to be replaced in a preset character dictionary for the error correction code vector;

[0116] A matching probability determination unit 403, configured to normalize each of the matching scores to obtain a matching probability corresponding to each of the matching scores;

[0117] The predicted replacement word determination unit 404 is configured to record the to-be-replaced word with the largest matching probability among the matching probabilities and greater than or equal to a preset matching threshold as the predicted replacement word.

[0118] Preferably, the sentence intelligent error correction device also includes:

[0119] A sample sentence set acquisition module, used to acquire a correct sample sentence set; the correct sample sentence set includes at least one correct sample sentence;

[0120] A mask processing module, used for performing mask processing on the correct sample sentence to obtain a sample masked sentence; the sample masked sentence contains at least one sample masked word;

[0121] A mask prediction module, used for inputting the sample masked sentence into a preset training model including initial parameters, performing mask prediction on the sample masked words, and obtaining sample predicted words corresponding to the sample masked words;

[0122] A sample prediction sentence recording module, configured to replace the sample mask word with the sample prediction word, and then record the replaced sample mask sentence as a sample prediction sentence;

[0123] A prediction loss value determination module, used to determine the prediction loss value of the preset training model according to the sample prediction sentence and the correct sample sentence corresponding thereto;

[0124] A parameter updating module is used to update and iterate the initial parameters of the preset training model when the predicted loss value does not reach the preset convergence condition, until the predicted loss value reaches the preset convergence condition, and record the preset training model after convergence as the preset language model.

[0125] The mask processing module comprises:

[0126] A first mask word selection unit, configured to select a first mask word from the correct sample sentence, replace the first mask word with a preset replacement character, and record the replaced first mask word as the sample prediction word;

[0127] The second mask word selection unit is used to select a second mask word from the sample prediction words and replace the selected second mask word with a preset homophone word.

[0128] For the specific definition of the intelligent error correction device for sentences, please refer to the definition of the intelligent error correction method for sentences above, which will not be repeated here. Each module in the above-mentioned intelligent error correction device for sentences can be implemented in whole or in part by software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory in the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0129] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Fig.10 As shown. The computer device includes a processor, a memory, a network interface and a database connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store the data used for the intelligent error correction of sentences in the above-mentioned embodiment. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a method for intelligent error correction of sentences is implemented.

[0130] In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the intelligent sentence error correction method in the above embodiment is implemented.

[0131] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the sentence intelligent error correction method in the above embodiment is implemented.

[0132] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0133] Those skilled in the art can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example. In actual applications, the above-mentioned functions can be distributed and completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.

[0134] The embodiments described above are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included in the protection scope of the present invention.

Claims

1. A sentence intelligent error correction method, characterized in that: include: receiving a sentence correction instruction containing an erroneous sentence; Input the erroneous sentence into a preset dependency detection model, predict the dependency relationship between the words in the erroneous sentence, obtain the dependency probability between each word and its corresponding dependent word, and associate each word with its corresponding dependency probability; the dependent word refers to at least one word located before or after a word and adjacent to it; Comparing the dependency probability associated with each of the words with a preset dependency threshold, and recording the words associated with a dependency probability less than the preset dependency threshold as words to be corrected; After masking the words to be corrected, a masked sentence to be corrected is obtained, and the masked sentence to be corrected is input into a preset language model, and correction prediction is performed on the words to be corrected to obtain a predicted replacement word corresponding to the words to be corrected; Replacing the word to be corrected with the predicted replacement word corresponding to the word to be corrected, and recording the masked sentence to be corrected after the replacement as the correct sentence corresponding to the wrong sentence; The step of inputting the erroneous sentence into a preset dependency detection model, predicting the dependency relationship between the words in the erroneous sentence, and obtaining the dependency probability between the words and the corresponding dependent words includes: Processing the word information of the erroneous sentence to obtain a forward hidden layer vector and a reverse hidden layer vector corresponding to the erroneous sentence; the forward hidden layer vector is determined by sorting the words in the erroneous sentence in a forward order; the reverse hidden layer vector is determined by sorting the words in the erroneous sentence in a reverse order; Input the forward hidden layer vector into the first linear transformation layer module in the preset affiliation detection model to determine a first word feature corresponding to the forward hidden layer vector; and simultaneously input the reverse hidden layer vector into the second linear transformation layer module in the preset affiliation detection model to determine a second word feature corresponding to the reverse hidden layer vector; According to the first word feature and the second word feature, the dependency probability between each word and its corresponding subsidiary word is determined through a double affine attention mechanism.

2. The sentence intelligent error correction method according to claim 1, characterized in that: Performing word information processing on the erroneous sentence to obtain a forward hidden layer vector and a reverse hidden layer vector corresponding to the erroneous sentence includes: After performing word segmentation processing on the erroneous sentence, determining a word sequence corresponding to the erroneous sentence; Perform word embedding processing on the word sequence by a word embedding method to obtain a word vector sequence corresponding to the word sequence; Through the information extraction module in the preset affiliation detection model, forward context information extraction is performed on the word vector sequence to obtain the forward hidden layer vector; at the same time, reverse context information extraction is performed on the word vector sequence to obtain the reverse hidden layer.

3. The sentence intelligent error correction method according to claim 1, characterized in that: The step of inputting the masked sentence into a preset language model, performing error correction prediction on the word to be corrected, and obtaining a predicted replacement word corresponding to the word to be corrected includes: The masked sentence is converted and encoded by the encoding module in the preset language model to obtain an encoding vector corresponding to the masked sentence; the encoding vector includes an error correction encoding vector corresponding to the word to be corrected; Performing linear mapping on the code vector to determine the matching score of each word to be replaced in a preset character dictionary for the error correction code vector; Normalizing each of the matching scores to obtain a matching probability corresponding to each of the matching scores; The word to be replaced with the largest matching probability among the matching probabilities and greater than or equal to a preset matching threshold is recorded as the predicted replacement word.

4. The sentence intelligent error correction method according to claim 1, characterized in that: Before inputting the masked sentence into the preset language model, the method further includes: Acquire a correct sample sentence set; the correct sample sentence set includes at least one correct sample sentence; Performing masking processing on the correct sample sentence to obtain a sample masked sentence; the sample masked sentence contains at least one sample masked word; Inputting the sample masked sentence into a preset training model including initial parameters, performing mask prediction on the sample masked words, and obtaining sample predicted words corresponding to the sample masked words; After replacing the sample mask word with the sample prediction word, recording the sample mask sentence after the replacement as the sample prediction sentence; Determine the prediction loss value of the preset training model according to the sample prediction sentence and the correct sample sentence corresponding thereto; When the predicted loss value does not reach the preset convergence condition, the initial parameters of the preset training model are updated and iterated until the predicted loss value reaches the preset convergence condition, and the preset training model after convergence is recorded as the preset language model.

5. The sentence intelligent error correction method according to claim 4, characterized in that: The masking process is performed on the correct sample sentence to obtain a sample masked sentence, including: Selecting a first mask word from the correct sample sentence, replacing the first mask word with a preset replacement character, and recording the replaced first mask word as the sample predicted word; A second mask word is selected from the sample predicted words, and a preset homophone word is used to replace the selected second mask word.

6. A sentence intelligent error correction device, characterized in that: include: An error correction instruction receiving module, used for receiving a sentence error correction instruction containing an erroneous sentence; A dependency probability determination module, used for inputting the erroneous sentence into a preset dependency detection model, predicting the dependency relationship between the words in the erroneous sentence, obtaining the dependency probability between each word and its corresponding dependent word, and associating each word with its corresponding dependency probability; the dependent word refers to at least one word located before or after a word and adjacent to it; A module for determining words to be corrected, used for comparing the dependency probability associated with each word with a preset dependency threshold, and recording the words associated with a dependency probability less than the preset dependency threshold as words to be corrected; An error correction prediction module is used to perform masking processing on the words to be corrected to obtain masked sentences to be corrected, and input the masked sentences to be corrected into a preset language model to perform error correction prediction on the words to be corrected to obtain predicted replacement words corresponding to the words to be corrected; A word replacement module, used for replacing the word to be corrected with the predicted replacement word corresponding to the word to be corrected, and recording the masked sentence to be corrected after the replacement as the correct sentence corresponding to the wrong sentence; The dependency probability determination module comprises: A word information processing unit, used for performing word information processing on the erroneous sentence to obtain a forward hidden layer vector and a reverse hidden layer vector corresponding to the erroneous sentence; the forward hidden layer vector is determined by sorting the words in the erroneous sentence in a forward order; the reverse hidden layer vector is determined by sorting the words in the erroneous sentence in a reverse order; A linear mapping unit, used for inputting the forward hidden layer vector into a first linear transformation layer module in the preset affiliation detection model, performing feature mapping on the forward hidden layer vector and the reverse hidden layer vector to obtain a first word feature corresponding to the forward hidden layer vector; and simultaneously inputting the reverse hidden layer vector into a second linear transformation layer module in the preset affiliation detection model to obtain a second word feature corresponding to the reverse hidden layer vector; The dependency probability determination unit determines the dependency probability between each of the words and its corresponding subordinate words through a double affine attention mechanism according to the first word feature and the second word feature.

7. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, it implements the sentence intelligent error correction method as described in any one of claims 1 to 5.

8. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the intelligent sentence error correction method as described in any one of claims 1 to 5 is implemented.

Images