Text correction, text correction model training method and device

Abstract

This invention provides a text correction method and apparatus, including a text correction model training method. The method includes: determining the answer text to be corrected, the corresponding question text, and the knowledge text associated with the question text; inputting the answer text, question text, and knowledge text into a text correction model to obtain the text correction result of the answer text output by the text correction model; and training the text correction model with the objective of minimizing the difference between predicted erroneous entities and their labels, and minimizing the difference between the text correction prediction result and the text correction label. The text correction method and apparatus provided by this invention train the text correction model by jointly performing an error entity recognition task and a text correction task, thereby enabling the error entity recognition task and the text correction task to provide mutual auxiliary and reinforcing information, further improving the text correction accuracy of the text correction model.

Description

Technical Field

[0001] This invention relates to the field of natural language processing technology, and in particular to a text correction method and apparatus for training a text correction model. Background Technology

[0002] Large language models refer to natural language processing models with a huge number of parameters and powerful language expression capabilities, capable of handling various natural language tasks such as machine translation, automatic question answering, and language generation. However, large language models suffer from the illusion problem, which means that the text generated by the model does not follow the original input text or does not conform to the facts, generating completely fabricated information that makes its output appear correct, but is essentially wrong.

[0003] Currently, the common approach to address this is to input the question text and the associated knowledge text into a large language model, which then uses the knowledge text to enhance the contextual understanding of the question text. However, large language models can introduce new illusion problems depending on the length and quality of the augmented content each time. Summary of the Invention

[0004] This invention provides a text correction method and apparatus for training a text correction model, which addresses the shortcomings of existing large language models in terms of illusion problems.

[0005] This invention provides a text correction method, comprising:

[0006] Identify the answer text to be corrected, the corresponding question text, and the knowledge text associated with the question text;

[0007] The answer text, the question text, and the knowledge text are input into the text correction model to obtain the text correction result of the answer text output by the text correction model.

[0008] The text correction model is trained with the goal of minimizing the difference between the predicted incorrect entity and the incorrect entity label, and minimizing the difference between the text correction prediction result and the text correction label. The predicted incorrect entity is determined based on the word segmentation encoding features and sentence segmentation encoding features of the sample text, and the text correction prediction result is determined based on the word segmentation decoding features and sentence segmentation encoding features of the sample text. The sample text includes sample answer text, sample question text, and sample knowledge text.

[0009] According to a text correction method provided by the present invention, the step of inputting the answer text, the question text, and the knowledge text into a text correction model to obtain the text correction result of the answer text output by the text correction model includes:

[0010] Based on the word segmentation encoding layer of the text correction model, the answer text, the question text, and the knowledge text are segmented and encoded to obtain text word segmentation encoding features;

[0011] Based on the sentence segmentation encoding layer of the text correction model, the text word segmentation encoding features are segmented into sentences to obtain text sentence segmentation encoding features;

[0012] Based on the word segmentation and decoding layer of the text correction model, the text word segmentation encoding features are segmented and decoded to obtain text word segmentation decoding features;

[0013] Based on the text correction layer of the text correction model, the text sentence encoding features and the text word decoding features are applied to correct the answer text, and the text correction result is obtained.

[0014] According to a text correction method provided by the present invention, the step of performing word segmentation encoding on the answer text, the question text, and the knowledge text to obtain text segmentation encoding features includes:

[0015] By concatenating the answer text, the question text, and the knowledge text, a concatenated text is obtained.

[0016] Feature extraction is performed on the spliced ​​text to obtain spliced ​​text features;

[0017] The concatenated text features are segmented and encoded to obtain the text segmentation encoding features.

[0018] According to a text correction method provided by the present invention, the step of performing word segmentation encoding on the concatenated text features to obtain the text word segmentation encoding features includes:

[0019] Attention features are obtained by performing attention calculations on the spliced ​​text features;

[0020] The attention features are segmented and encoded to obtain the text segmentation encoding features.

[0021] According to a text correction method provided by the present invention, the step of determining the knowledge text includes:

[0022] Based on the semantic information of the question text, question text features are generated;

[0023] Based on the semantic information of each candidate knowledge text, generate features for each candidate knowledge text;

[0024] Based on the features of the question text and the features of each candidate knowledge text, the semantic relevance between the question text and each candidate knowledge text is determined, and the candidate knowledge text with a semantic relevance greater than a threshold is taken as the knowledge text corresponding to the question text.

[0025] This invention also provides a method for training a text correction model, comprising:

[0026] Obtain sample text, which includes sample answer text, sample question text, and sample knowledge text;

[0027] Based on the word segmentation encoding features and sentence segmentation encoding features of the sample text, the predicted incorrect entities of the sample answer text are determined;

[0028] Based on the word segmentation decoding features and sentence segmentation encoding features of the sample text, the text correction prediction result of the sample answer text is determined;

[0029] The initial model is trained with the goal of minimizing the difference between the predicted incorrect entity and the incorrect entity label, and minimizing the difference between the text correction prediction result and the text correction label, to obtain the text correction model.

[0030] The present invention also provides a text correction device, comprising:

[0031] The text determination unit is used to determine the answer text to be corrected, the question text corresponding to the answer text, and the knowledge text associated with the question text;

[0032] The text correction unit is used to input the answer text, the question text, and the knowledge text into the text correction model to obtain the text correction result of the answer text output by the text correction model;

[0033] The text correction model is trained with the goal of minimizing the difference between the predicted incorrect entity and the incorrect entity label, and minimizing the difference between the text correction prediction result and the text correction label. The predicted incorrect entity is determined based on the word segmentation encoding features and sentence segmentation encoding features of the sample text, and the text correction prediction result is determined based on the word segmentation decoding features and sentence segmentation encoding features of the sample text. The sample text includes sample answer text, sample question text, and sample knowledge text.

[0034] The present invention also provides a text correction model training device, comprising:

[0035] The text acquisition unit is used to acquire sample text, which includes sample answer text, sample question text, and sample knowledge text.

[0036] An entity determination unit is used to determine the predicted incorrect entities in the sample answer text based on the word segmentation encoding features and sentence segmentation encoding features of the sample text.

[0037] The correction prediction unit is used to determine the text correction prediction result of the sample answer text based on the word segmentation decoding features and sentence segmentation encoding features of the sample text;

[0038] The model training unit is used to train the initial model with the goal of minimizing the difference between the predicted incorrect entity and the incorrect entity label, and minimizing the difference between the text correction prediction result and the text correction label, to obtain the text correction model.

[0039] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the text correction method as described above or to implement the text correction model training method as described above.

[0040] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the text correction method as described above or the text correction model training method as described above.

[0041] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the text correction method as described above or the text correction model training method as described above.

[0042] The text correction, text correction model training method and apparatus provided by the present invention aim to minimize the difference between the predicted erroneous entity and the erroneous entity label, and to minimize the difference between the text correction prediction result and the text correction label. That is, the text correction model is trained by jointly performing the erroneous entity recognition task and the text correction task, so that the erroneous entity recognition task and the text correction task can provide each other with auxiliary and reinforcing information, thereby further improving the text correction accuracy of the text correction model. Attached Figure Description

[0043] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0044] Figure 1 This is a flowchart illustrating the text correction method provided by the present invention;

[0045] Figure 2 This is a flowchart illustrating an implementation of step 120 in the text correction method provided by the present invention.

[0046] Figure 3 This is a flowchart illustrating an implementation of step 121 in the text correction method provided by the present invention.

[0047] Figure 4 This is a flowchart illustrating the knowledge text determination method provided by the present invention;

[0048] Figure 5 This is a flowchart illustrating another text correction method provided by the present invention;

[0049] Figure 6 This is a schematic diagram of the structure of the text correction model provided by the present invention;

[0050] Figure 7 This is a flowchart illustrating the text correction model training method provided by the present invention;

[0051] Figure 8 This is a schematic diagram of the text correction device provided by the present invention;

[0052] Figure 9 This is a schematic diagram of the structure of the text correction model training device provided by the present invention;

[0053] Figure 10 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0054] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0055] Currently, large language models suffer from two types of illusion problems. The first is the illusion of not following the question text, meaning that the answer text of the large language model does not strictly follow the content of the question text and fabricates content outside the answer text. The second is the illusion of not following objective facts, meaning that the answer text of the large language model contradicts objective facts.

[0056] To address the aforementioned illusion problem in large language models, current approaches often involve inputting the question text and associated knowledge text into the large language model. The knowledge text then augments the question text with contextual data to enhance understanding, thus mitigating the illusion problem. However, large language models introduce new illusion problems depending on the length and quality of each augmentation.

[0057] Furthermore, some approaches rely on the belief that large language models possess sufficient internal knowledge, hoping to guide the correct answer by allowing the model to generate its own thought processes, rather than directly providing a conclusion. This involves the large language model breaking down complex tasks, reasoning step-by-step, and ultimately arriving at a more accurate answer based on the knowledge and analysis mentioned in the reasoning. However, in generating reasoning steps, the large language model generates more information points. While it uses these points to guide more accurate reasoning in the next step, more knowledge points also increase the possibility of illusions and the risk of a snowballing effect of misinformation.

[0058] In response, this invention provides a text correction method. Figure 1 This is a flowchart illustrating the text correction method provided by the present invention, as shown below. Figure 1 As shown, this method can be applied to text correction in the medical field, such as correcting answer text in the medical field, and it can also be applied to text correction in other fields, such as correcting answer text in the education field. The following embodiment uses answer text in the medical field as an example to illustrate the method, which includes the following steps:

[0059] Step 110: Determine the answer text to be corrected, the corresponding question text, and the knowledge text associated with the question text;

[0060] Step 120: Input the answer text, question text, and knowledge text into the text correction model to obtain the text correction result of the answer text output by the text correction model;

[0061] The text correction model is trained with the goal of minimizing the difference between the predicted incorrect entity and the incorrect entity label, and minimizing the difference between the text correction prediction result and the text correction label. The predicted incorrect entity is determined based on the word segmentation encoding features and sentence segmentation encoding features of the sample text, and the text correction prediction result is determined based on the word segmentation decoding features and sentence segmentation encoding features of the sample text. The sample text includes sample answer text, sample question text, and sample knowledge text.

[0062] Specifically, the question text refers to the text of the question or query posed by the user or system to the large language model. For example, the question text could be "Please summarize based on the given medical record content. Medical record content: XXX". The answer text is the text of the response or query result generated by the large language model based on the question text. Due to the illusion problem in large language models, the answer text generated by the large language model based on the question text may contain illusions; that is, the answer text can be understood as text that needs illusion correction.

[0063] Knowledge text refers to text that stores and organizes knowledge and is in the same domain as the question text. Knowledge text provides enhanced information corresponding to the question text. It can explain terms in the question text or provide background information to deepen understanding of the question text's context. Optionally, knowledge text associated with the question text can be selected from among the candidate texts based on the semantic relevance between the question text and candidate knowledge texts in the medical knowledge base.

[0064] Furthermore, the answer text, question text, and knowledge text can be text directly input by the user, text obtained by performing OCR (Optical Character Recognition) on the image input by the user, or text obtained by performing speech recognition on the voice input by the user. This embodiment of the invention does not specifically limit these.

[0065] After obtaining the answer text, question text, and knowledge text, these are input into a text correction model. The text correction model then corrects the answer text based on the enhanced information provided by the knowledge text corresponding to the question text, as well as the semantic information of the question text itself, yielding a text correction result. The text correction result can be the corrected text after correcting the answer text, or it can be the erroneous entities in the answer text and their corresponding corrected entities; this embodiment of the invention does not specifically limit this.

[0066] The text correction model is trained with the goal of minimizing the difference between the predicted incorrect entity and the incorrect entity label, and minimizing the difference between the text correction prediction result and the text correction label.

[0067] Predicting erroneous entities can be understood as entities in the predicted sample answer text that appear illusory, i.e., entities that need to be corrected. The prediction of erroneous entities is based on the word segmentation encoding features and sentence segmentation encoding features of the sample text. Word segmentation encoding features can be understood as the vector representations of each word in the sample text, used to characterize the semantic information of each word. Sentence segmentation encoding features can be understood as the vector representations of each sentence in the sample text, used to characterize the semantic information of each sentence. The smaller the difference between the predicted erroneous entity and the erroneous entity label, the higher the accuracy of the text correction model in identifying erroneous entities in the sample answer text.

[0068] The text correction prediction result can be understood as the predicted correction result of erroneous entities in the sample answer text. The text correction prediction result is determined based on the word segmentation decoding features and sentence segmentation encoding features of the sample text. The word segmentation decoding features can be understood as the word segmentation vector representation after decoding the word segmentation encoding features. The smaller the difference between the text correction prediction result and the text correction label, the higher the accuracy of the text correction model in correcting the sample answer text.

[0069] Furthermore, training the text correction model to minimize the difference between the predicted erroneous entities and their labels can be understood as training the text correction model specifically for the erroneous entity recognition task. Similarly, training the text correction model to minimize the difference between the text correction prediction result and the text correction label can be understood as training the text correction model specifically for the text correction task. Since the predicted erroneous entities in the erroneous entity recognition task represent erroneous entities in the sample answer text, and the text correction prediction result in the text correction task represents the predicted correction result for erroneous entities in the sample answer text, the erroneous entity recognition task can use the erroneous entities identified from the sample answer text as auxiliary enhancement information for the text correction task. Conversely, the text correction task can use the text correction result of the sample answer text as auxiliary enhancement information for the erroneous entity recognition task. This results in the final trained text correction model not only accurately identifying erroneous entities in the answer text but also accurately correcting the answer text, further improving the correction accuracy of the text correction model.

[0070] Furthermore, the text correction model in this embodiment of the invention corrects the answer text based on the answer text output by the large language model, the question text corresponding to the answer text, and the knowledge text associated with the question text. That is, the text correction model is decoupled from the large language model, thereby avoiding the problem of new illusions caused by the large language model when performing reasoning in traditional methods.

[0071] The text correction method provided in this invention aims to minimize the difference between the predicted erroneous entity and the erroneous entity label, as well as the difference between the text correction prediction result and the text correction label. That is, the text correction model is trained by jointly performing the erroneous entity recognition task and the text correction task, so that the erroneous entity recognition task and the text correction task can provide each other with auxiliary and enhancing information, thereby further improving the text correction accuracy of the text correction model.

[0072] Based on the above embodiments, Figure 2 This is a flowchart illustrating an implementation of step 120 in the text correction method provided by the present invention, as shown below. Figure 2 As shown, step 120 includes:

[0073] Step 121: Based on the text correction model, the word segmentation encoding layer performs word segmentation encoding on the answer text, question text, and knowledge text to obtain text word segmentation encoding features.

[0074] Specifically, the word segmentation encoding layer is used to segment and encode the input answer text, question text, and knowledge text, and output the corresponding text word segmentation encoding features. Further, the word segmentation encoding layer performs word segmentation encoding on the answer text, question text, and knowledge text respectively, thereby extracting the word segmentation semantics of each text and obtaining the text word segmentation encoding features.

[0075] There are many ways to perform word segmentation encoding on text. For example, you can use a Long Short-Term Memory network to extract contextual features from the word vectors of each word in the input text, or you can use a Transformer encoder to extract contextual features from the word vectors of each word in the input text.

[0076] Step 122: Based on the sentence encoding layer of the text correction model, the text word segmentation encoding features are segmented into sentences to obtain the text sentence encoding features.

[0077] Specifically, the word segmentation encoding layer is used to segment and encode the input text word segmentation encoding features into sentences and output the corresponding text sentence encoding features. Further, the sentence encoding layer segments and encodes the text word segmentation encoding features into sentences, thereby extracting the sentence semantics from the word segmentation encoding features to obtain the text sentence encoding features.

[0078] Step 123: Based on the text correction model, the word segmentation decoding layer performs word segmentation decoding on the text word segmentation encoding features to obtain the text word segmentation decoding features.

[0079] Specifically, the word segmentation decoding layer is used to decode the text word segmentation encoding features, thereby obtaining the text word segmentation decoding features. During the decoding process, the text word segmentation encoding features can be decoded using the Transformer's decoder to obtain the text word segmentation decoding features.

[0080] Step 124: Based on the text correction model, the text correction layer applies text sentence encoding features and text word decoding features to perform text correction on the answer text, and obtain the text correction result.

[0081] Specifically, the text correction layer combines text sentence encoding features and text word segmentation decoding features to correct the answer text and obtain the corrected text result. Since text sentence encoding features represent the semantic information of the text's sentences, and text word segmentation decoding features are obtained by decoding the text word segmentation encoding features, the text word segmentation decoding features can represent the semantic information of the text's word segments. Therefore, the contextual information carried in the semantic information of the sentences can be used as guiding information to instruct the text correction layer to correct erroneous entities in the answer text based on the semantic information of word segments, thus accurately obtaining the corrected text result.

[0082] Based on any of the above embodiments Figure 3 This is a flowchart illustrating an implementation of step 121 in the text correction method provided by the present invention, as shown below. Figure 3 As shown, in step 121, the answer text, question text, and knowledge text are segmented and encoded to obtain text segmentation encoding features, including:

[0083] Step 1211: Combine the answer text, question text, and knowledge text to obtain the combined text;

[0084] Step 1212: Extract features from the concatenated text to obtain the concatenated text features;

[0085] Step 1213: Perform word segmentation encoding on the spliced ​​text features to obtain text word segmentation encoding features.

[0086] Specifically, when combining answer text, question text, and knowledge text, preset symbols can be used. For example, if the answer text is A, the question text is E, and the knowledge text is K, the symbol "&" can be used to combine these texts, resulting in the combined text "A&E&K".

[0087] Furthermore, concatenated text features are used to represent the semantic information of the concatenated text, which can be represented using word embeddings. Further, the concatenated text features are segmented and encoded to extract the semantic information of the concatenated text, resulting in text segmentation encoding features.

[0088] Based on any of the above embodiments, step 1213 includes:

[0089] Attention features are obtained by performing attention calculations on the spliced ​​text features.

[0090] The attention features are segmented and encoded to obtain the text segmentation encoding features.

[0091] Specifically, when performing attention calculation on the concatenated text features, the relationships between the words in the concatenated text features can be determined. These relationships can include similarity between words, contextual relevance between words, etc. In other words, the attention features obtained after performing attention calculation on the concatenated text features are used to characterize the relationships between the words in the concatenated text features. Optionally, attention features can be obtained by performing attention calculation on the concatenated text features based on the self-attention mechanism of the Transformer model.

[0092] Since attention features are used to characterize the relationship between words in the spliced ​​text features, when encoding words using attention features, the semantics of the spliced ​​text can be accurately extracted based on the relationship between words, thus accurately obtaining the text word segmentation encoding features.

[0093] Based on any of the above embodiments Figure 4 This is a flowchart illustrating the knowledge text determination method provided by the present invention, as shown below. Figure 4 As shown, the steps for determining knowledge text include:

[0094] Step 410: Generate question text features based on the semantic information of the question text;

[0095] Step 420: Generate features for each candidate knowledge text based on the semantic information of each candidate knowledge text;

[0096] Step 430: Based on the features of the question text and the features of each candidate knowledge text, determine the semantic relevance between the question text and each candidate knowledge text, and take the candidate knowledge text with a semantic relevance greater than the threshold as the knowledge text corresponding to the question text.

[0097] Specifically, candidate knowledge texts can be texts from a medical knowledge base. Question text features are used to characterize the semantic information of the question text, and candidate knowledge text features are used to characterize the semantic information of each candidate knowledge text. The greater the semantic relevance between the question text and each candidate knowledge text, the stronger the association between the corresponding candidate knowledge text and the question text, and consequently, the higher the credibility of the enhanced information provided by the corresponding candidate knowledge text in relation to the question text.

[0098] If the semantic relevance of any candidate knowledge text is greater than the threshold, it indicates that the credibility of the enhanced information provided by the corresponding candidate knowledge text for the question text is high. Therefore, the corresponding candidate knowledge text is used as the knowledge text corresponding to the question text. Thus, the text correction model can better understand the contextual information of the question text based on the enhanced information provided by the knowledge text, and thus accurately correct the answer text corresponding to the question text.

[0099] Optionally, the medical knowledge base may also include a vector library for storing the sentence vectors of each candidate knowledge text in the medical knowledge base. When determining the knowledge text, the semantic relevance between the question text and each candidate knowledge text can be determined based on the distance between the sentence vector of the question text and the sentence vectors of each candidate knowledge text; the greater the distance, the lower the semantic relevance.

[0100] Based on any of the above embodiments Figure 5 This is a flowchart illustrating another text correction method provided by the present invention, as shown below. Figure 5 As shown, the method includes:

[0101] Determine the user's question text and input it into the Large Language Model (LLM) to obtain the answer text corresponding to the question text output by the LLM.

[0102] Simultaneously, knowledge texts with semantic relevance greater than a threshold to the question text are extracted from the knowledge base. The question text, answer text, and knowledge text are then input into a text correction model to obtain the text correction result corresponding to the answer text output by the model. The text correction model can be constructed based on the HeroNet (Hallucination Recognition and Correction Network) model.

[0103] Figure 6 This is a schematic diagram of the text correction model provided by the present invention, as shown below. Figure 6 As shown, the text correction model includes a word segmentation encoding layer, a sentence segmentation encoding layer, a word segmentation decoding layer, an error entity recognition layer, and a text correction layer. The word segmentation encoding layer performs word segmentation encoding on the answer text, question text, and knowledge text to obtain text word segmentation encoding features; the sentence segmentation encoding layer performs sentence segmentation encoding on the text word segmentation encoding features to obtain text sentence segmentation encoding features; the word segmentation decoding layer performs word segmentation decoding on the text word segmentation encoding features to obtain text word segmentation decoding features; the error entity recognition layer identifies error entities in the answer text based on the text word segmentation encoding features and text sentence segmentation encoding features; and the text correction layer corrects the answer text based on the text sentence segmentation encoding features and text word segmentation decoding features to obtain the text correction result.

[0104] Optionally, the word segmentation encoding layer can be constructed based on the encoder of the Transformer model, and the word segmentation decoding layer can be constructed based on the decoder of the Transformer model. The Transformer model is a neural network model based on a self-attention mechanism, which is as follows:

[0105]

[0106] The self-attention mechanism allows the model to perform "self-attention" when processing input features, that is, to calculate the vector representation of each token based on the importance of tokens in the input features. Here, the query vector Q, key vector K, and value vector V are vectors obtained by multiplying the input embedding by the trainable matrix and then performing a linear transformation.

[0107] The self-attention mechanism of the Transformer model can effectively capture long-distance dependencies in the input features. Furthermore, the encoder of the Transformer model is mainly used to convert the input features into a set of vector representations containing semantic information, while the decoder is used to generate the target sequence, i.e., generate new text, based on these vector representations.

[0108] Since the error entity recognition task is highly helpful for the text correction task, this embodiment of the invention uses the same model framework (text correction model) to learn both tasks simultaneously to enhance their mutual capabilities. The inputs to both tasks are identical, including the question text E, the answer text A to be corrected, and the knowledge text K associated with the question text E, where K = [k1, k2, ..., kk]. n For the erroneous entity identification task f Hner Its prediction result is That is, based on the given E, A, and K, find the erroneous entities that exist in A: in, Indicate i in A th The token is the beginning of an error entity. The intermediate token indicating the erroneous entity. This indicates that the token at that location is not an erroneous entity. On the other hand, for the text correction task f... Hcor Its output is the newly generated sentence after correction. in This is used to predict a specific token from the vocabulary list for the model.

[0109] When acquiring word segmentation encoding features, the question text E, the answer text A, and the knowledge text K are concatenated to obtain the concatenated text X = [[CLS],Q,[SEP],A,[SEP],K,[SEP]], where each sentence in K is also separated by [SEP]. The concatenated text is then processed through the embedding layer f. Embed Obtain the concatenated text features H from the Embedding layer 0 H 0 It is then input into a word segmentation encoding layer f, which consists of n stacked Transformer encoding blocks. nBlockGet each token in f nBlock The vector representation output of the final layer, i.e., the word segmentation encoding feature H. n :

[0110]

[0111]

[0112] Where n∈{1,2,…,n} corresponds to the number of coding blocks in the word segmentation coding layer, N represents the sequence length of the concatenated text, and d represents the dimension of the coding vector.

[0113] In obtaining word segmentation encoding features H n Then, each sentence segmented by [SEP] is input into the Max-pooling layer in the sentence segmentation encoding layer to convert the token vector corresponding to the sentence into a sentence vector.

[0114]

[0115] in, The i represents the i-th node in the Max-pooling process. th Sentences i Indicates the i-th th The set of all tokens contained in each sentence, where m represents the number of sentences.

[0116] Next, Input sentence interaction layer f sInter The role of the sentence interaction layer is to make the vectors of each sentence... Learn the most relevant information about each other. sInter This can be implemented based on a Bert layer with hyperparameter L, utilizing its multi-head attention mechanism:

[0117]

[0118] I s It contains the sentence vectors after all sentence interactions and the vector representation of the special token [CLS], where the sentence segmentation encoding features are... Represent f sInter The final output I S The vector representation corresponding to [CLS].

[0119] Error entity recognition layer is used to reference sentence encoding features Predict whether each token in the answer text A is an erroneous entity. Specifically, first obtain the word segmentation encoding features H. n A set H is formed from the token vectors in all sentences A. A Then HA in i th token vectors and sentence encoding features After concatenation, the fully connected layer in the erroneous entity recognition layer is reduced to 1 dimension, and then the i-th element is obtained through the Sigmoid function. th The probability that the token is an erroneous entity.

[0120]

[0121] in, This represents the parameters of the fully connected layer in the error entity recognition layer.

[0122] For the word segmentation decoding layer, when the word segmentation decoding layer f dec Received the first time-stamp [BOS] special token vector v0 and the word segmentation encoding layer f nBlock Output word segmentation encoding features H n Decoding then begins, yielding the word segmentation decoding features of the decoder at the first time step. And decode word segmentation features Sentence segmentation encoding features The data is concatenated and fed into a fully connected layer of the text correction layer for dimensionality transformation, which is then applied to the vocabulary size. Finally, the features are segmented and decoded. After passing through the Softmax layer of the text correction layer, the probability distribution of words in the vocabulary at the first time step is obtained.

[0123]

[0124]

[0125] in The parameters represent learnable parameters, and V is the size of the vocabulary. The above describes the process of generating the first token, the word segmentation and decoding layer f. dec In generating the t-th th When calculating the token vector at each time step, the vector v of the token output at the previous time step will be used. t-1 As input at the current moment, an autoregressive generation method is implemented until the special character [EOS] is generated and generation stops:

[0126] v t-1 =f′ Embed (token t-1 )

[0127]

[0128]

[0129]

[0130] Where t∈{1,2,…,l}, l represents the length of the output target sequence, and f′ Embed H is the embedding layer of the word segmentation and decoding layer. n It is the reference vector that is needed for cross-attention.

[0131] Furthermore, the loss function of the text correction model consists of two parts. The first loss function comes from the error entity identification task. The text correction model identifies error entities for each word in the answer text A and predicts the probability p that each word is an error entity. A The probability p of using an incorrect entity A The loss is denoted as Loss when compared with the incorrect entity label. Hner The second loss function comes from the text correction task. The text correction model corrects the answer text A, generates the corrected target sequence, and generates the target word probabilities. The loss between the predicted text correction result and the true target word probability (text correction label) is denoted as Loss. Hcor The joint loss function of the two is the loss function (Loss) of the text correction model. Total :

[0132] Loss Total =αLoss Hner +βLoss Hcor

[0133] Where α and β are hyperparameters, and α + β = 1.

[0134] Based on any of the above embodiments Figure 7 This is a flowchart illustrating the text correction model training method provided by the present invention, as shown below. Figure 7 As shown, the method includes:

[0135] Step 710: Obtain sample text, which includes sample answer text, sample question text, and sample knowledge text;

[0136] Step 720: Based on the word segmentation encoding features and sentence segmentation encoding features of the sample text, determine the predicted incorrect entities in the sample answer text;

[0137] Step 730: Based on the word segmentation decoding features and sentence segmentation encoding features of the sample text, determine the text correction prediction result of the sample answer text;

[0138] Step 740: Train the initial model with the goal of minimizing the difference between the predicted incorrect entity and the incorrect entity label, and minimizing the difference between the text correction prediction result and the text correction label, to obtain the text correction model.

[0139] Specifically, predicted erroneous entities can be understood as entities in the predicted sample answer text that exhibit hallucinations, i.e., erroneous entities that need to be corrected. The prediction of erroneous entities is based on the word segmentation encoding features and sentence segmentation encoding features of the sample text. Word segmentation encoding features can be understood as the vector representations of each word in the sample text, used to characterize the semantic information of each word. Sentence segmentation encoding features can be understood as the vector representations of each sentence in the sample text, used to characterize the semantic information of each sentence. The smaller the difference between the predicted erroneous entity and the erroneous entity label, the higher the accuracy of the text correction model in identifying erroneous entities in the sample answer text.

[0140] The text correction prediction result can be understood as the predicted correction result of erroneous entities in the sample answer text. The text correction prediction result is determined based on the word segmentation decoding features and sentence segmentation encoding features of the sample text. The word segmentation decoding features can be understood as the word segmentation vector representation after decoding the word segmentation encoding features. The smaller the difference between the text correction prediction result and the text correction label, the higher the accuracy of the text correction model in correcting the sample answer text.

[0141] Furthermore, training the text correction model to minimize the difference between the predicted erroneous entities and their labels can be understood as training the text correction model specifically for the erroneous entity recognition task. Similarly, training the text correction model to minimize the difference between the text correction prediction result and the text correction label can be understood as training the text correction model specifically for the text correction task. Since the predicted erroneous entities in the erroneous entity recognition task represent erroneous entities in the sample answer text, and the text correction prediction result in the text correction task represents the predicted correction result for erroneous entities in the sample answer text, the erroneous entity recognition task can use the erroneous entities identified from the sample answer text as auxiliary enhancement information for the text correction task. Conversely, the text correction task can use the text correction result of the sample answer text as auxiliary enhancement information for the erroneous entity recognition task. This results in the final trained text correction model not only accurately identifying erroneous entities in the answer text but also accurately correcting the answer text, further improving the correction accuracy of the text correction model.

[0142] The text correction model training method provided by this invention aims to minimize the difference between the predicted erroneous entity and the erroneous entity label, as well as the difference between the text correction prediction result and the text correction label. That is, the text correction model is trained by jointly performing the erroneous entity recognition task and the text correction task, so that the erroneous entity recognition task and the text correction task can provide each other with auxiliary and reinforcing information, thereby further improving the text correction accuracy of the text correction model.

[0143] The text correction apparatus provided by the present invention is described below. The text correction apparatus described below can be referred to in correspondence with the text correction method described above.

[0144] Based on any of the above embodiments Figure 8 This is a schematic diagram of the text correction device provided by the present invention, as shown below. Figure 8 As shown, the device includes:

[0145] The text determination unit 810 is used to determine the answer text to be corrected, the question text corresponding to the answer text, and the knowledge text associated with the question text;

[0146] The text correction unit 820 is used to input the answer text, question text, and knowledge text into the text correction model to obtain the text correction result of the answer text output by the text correction model;

[0147] The text correction model is trained with the goal of minimizing the difference between the predicted incorrect entity and the incorrect entity label, and minimizing the difference between the text correction prediction result and the text correction label. The predicted incorrect entity is determined based on the word segmentation encoding features and sentence segmentation encoding features of the sample text, and the text correction prediction result is determined based on the word segmentation decoding features and sentence segmentation encoding features of the sample text. The sample text includes sample answer text, sample question text, and sample knowledge text.

[0148] Based on any of the above embodiments, the text correction unit 820 includes:

[0149] The word segmentation encoding unit is used in the word segmentation encoding layer based on the text correction model to perform word segmentation encoding on the answer text, question text, and knowledge text to obtain text word segmentation encoding features;

[0150] The sentence segmentation encoding unit is used in the sentence segmentation encoding layer based on the text correction model to perform sentence segmentation encoding on the text word segmentation encoding features to obtain the text sentence segmentation encoding features;

[0151] The word segmentation and decoding unit is used in the word segmentation and decoding layer based on the text correction model to perform word segmentation and decoding on the text word segmentation encoding features to obtain the text word segmentation and decoding features;

[0152] The text correction unit is used in the text correction layer based on the text correction model. It applies text sentence encoding features and text word decoding features to correct the answer text and obtain the text correction result.

[0153] Based on any of the above embodiments, the word segmentation encoding unit includes:

[0154] The splicing unit is used to splice the answer text, question text, and knowledge text to obtain the spliced ​​text.

[0155] The extraction unit is used to extract features from the concatenated text to obtain the concatenated text features;

[0156] The first encoding unit is used to encode the features of the concatenated text into word segments, thereby obtaining the text word segmentation encoding features.

[0157] Based on any of the above embodiments, the encoding unit includes:

[0158] The computational unit is used to perform attention calculations on the features of the concatenated text to obtain attention features;

[0159] The second encoding unit is used to encode the attention features into words, thus obtaining the text word segmentation encoding features.

[0160] Based on any of the above embodiments, the device further includes:

[0161] The first generation unit is used to generate question text features based on the semantic information of the question text;

[0162] The second generation unit is used to generate features of each candidate knowledge text based on the semantic information of each candidate knowledge text.

[0163] The text selection unit is used to determine the semantic relevance between the question text and each candidate knowledge text based on the features of the question text and the features of each candidate knowledge text, and to select the candidate knowledge text with a semantic relevance greater than a threshold as the knowledge text corresponding to the question text.

[0164] The text correction model training device provided by the present invention is described below. The text correction model training device described below can be referred to in correspondence with the text correction model training method described above.

[0165] Based on any of the above embodiments Figure 9 This is a schematic diagram of the text correction model training device provided by the present invention, as shown below. Figure 9 As shown, the device includes:

[0166] The text acquisition unit 910 is used to acquire sample text, which includes sample answer text, sample question text, and sample knowledge text.

[0167] The entity determination unit 920 is used to determine the predicted incorrect entities in the sample answer text based on the word segmentation encoding features and sentence segmentation encoding features of the sample text.

[0168] The correction prediction unit 930 is used to determine the text correction prediction result of the sample answer text based on the word segmentation decoding features and sentence segmentation encoding features of the sample text.

[0169] Model training unit 940 is used to train the initial model with the goal of minimizing the difference between the predicted incorrect entity and the incorrect entity label, and minimizing the difference between the text correction prediction result and the text correction label, to obtain the text correction model.

[0170] Figure 10 This is a schematic diagram of the structure of the electronic device provided by the present invention, such as... Figure 10 As shown, the electronic device may include: a processor 1010, a memory 1020, a communications interface 1030, and a communications bus 1040, wherein the processor 1010, the memory 1020, and the communications interface 1030 communicate with each other through the communications bus 1040. The processor 1010 can call logical instructions in the memory 1020 to execute a text correction method, which includes: determining the answer text to be corrected, the question text corresponding to the answer text, and the knowledge text associated with the question text; inputting the answer text, the question text, and the knowledge text into a text correction model to obtain the text correction result of the answer text output by the text correction model; the text correction model is trained with the goal of minimizing the difference between predicted erroneous entities and erroneous entity labels, and minimizing the difference between the text correction prediction result and the text correction label; the predicted erroneous entities are determined based on the word segmentation encoding features and sentence segmentation encoding features of the sample text, and the text correction prediction result is determined based on the word segmentation decoding features and sentence segmentation encoding features of the sample text, wherein the sample text includes sample answer text, sample question text, and sample knowledge text.

[0171] Alternatively, a text correction model training method can be implemented, comprising: acquiring sample text, including sample answer text, sample question text, and sample knowledge text; determining the predicted incorrect entities of the sample answer text based on the word segmentation encoding features and sentence segmentation encoding features of the sample text; determining the text correction prediction result of the sample answer text based on the word segmentation decoding features and sentence segmentation encoding features of the sample text; and training an initial model with the objectives of minimizing the difference between the predicted incorrect entities and the incorrect entity labels, and minimizing the difference between the text correction prediction result and the text correction labels, to obtain a text correction model.

[0172] Furthermore, the logical instructions in the aforementioned memory 1020 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, essentially, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0173] On the other hand, the present invention also provides a computer program product, the computer program product comprising a computer program stored on a non-transitory computer-readable storage medium, the computer program comprising program instructions, wherein when the program instructions are executed by a computer, the computer is able to execute the text correction method provided by the above methods, the method comprising: determining the answer text to be corrected, the question text corresponding to the answer text, and the knowledge text associated with the question text; inputting the answer text, the question text, and the knowledge text into a text correction model to obtain the text correction result of the answer text output by the text correction model; the text correction model being trained with the objective of minimizing the difference between predicted erroneous entities and erroneous entity labels, and minimizing the difference between the text correction prediction result and the text correction label; the predicted erroneous entities being determined based on the word segmentation encoding features and sentence segmentation encoding features of the sample text, the text correction prediction result being determined based on the word segmentation decoding features and sentence segmentation encoding features of the sample text, the sample text including sample answer text, sample question text, and sample knowledge text.

[0174] Alternatively, a text correction model training method can be implemented, comprising: acquiring sample text, including sample answer text, sample question text, and sample knowledge text; determining the predicted incorrect entities of the sample answer text based on the word segmentation encoding features and sentence segmentation encoding features of the sample text; determining the text correction prediction result of the sample answer text based on the word segmentation decoding features and sentence segmentation encoding features of the sample text; and training an initial model with the objectives of minimizing the difference between the predicted incorrect entities and the incorrect entity labels, and minimizing the difference between the text correction prediction result and the text correction labels, to obtain a text correction model.

[0175] In another aspect, the present invention also provides a non-transitory computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the aforementioned text correction methods. The method includes: determining an answer text to be corrected, a corresponding question text, and knowledge text associated with the question text; inputting the answer text, the question text, and the knowledge text into a text correction model to obtain a text correction result of the answer text output by the text correction model; the text correction model is trained with the objective of minimizing the difference between predicted erroneous entities and erroneous entity labels, and minimizing the difference between the text correction prediction result and the text correction label; the predicted erroneous entities are determined based on the word segmentation encoding features and sentence segmentation encoding features of the sample text, and the text correction prediction result is determined based on the word segmentation decoding features and sentence segmentation encoding features of the sample text; the sample text includes sample answer text, sample question text, and sample knowledge text.

[0176] Alternatively, a text correction model training method can be implemented, comprising: acquiring sample text, including sample answer text, sample question text, and sample knowledge text; determining the predicted incorrect entities of the sample answer text based on the word segmentation encoding features and sentence segmentation encoding features of the sample text; determining the text correction prediction result of the sample answer text based on the word segmentation decoding features and sentence segmentation encoding features of the sample text; and training an initial model with the objectives of minimizing the difference between the predicted incorrect entities and the incorrect entity labels, and minimizing the difference between the text correction prediction result and the text correction labels, to obtain a text correction model.

[0177] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0178] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0179] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A text correction method, characterized in that, include: Identify the answer text to be corrected, the corresponding question text, and the knowledge text associated with the question text; The answer text, the question text, and the knowledge text are input into the text correction model to obtain the text correction result of the answer text output by the text correction model. The text correction model is trained with the goal of minimizing the difference between the predicted incorrect entity and the incorrect entity label, and minimizing the difference between the text correction prediction result and the text correction label. The predicted incorrect entity is determined based on the word segmentation encoding features and sentence segmentation encoding features of the sample text, and the text correction prediction result is determined based on the word segmentation decoding features and sentence segmentation encoding features of the sample text. The sample text includes sample answer text, sample question text, and sample knowledge text.

2. The text correction method according to claim 1, characterized in that, The step of inputting the answer text, the question text, and the knowledge text into the text correction model to obtain the text correction result of the answer text output by the text correction model includes: Based on the word segmentation encoding layer of the text correction model, the answer text, the question text, and the knowledge text are segmented and encoded to obtain text word segmentation encoding features; Based on the sentence segmentation encoding layer of the text correction model, the text word segmentation encoding features are segmented into sentences to obtain text sentence segmentation encoding features; Based on the word segmentation and decoding layer of the text correction model, the text word segmentation encoding features are segmented and decoded to obtain text word segmentation decoding features; Based on the text correction layer of the text correction model, the text sentence encoding features and the text word decoding features are applied to correct the answer text, and the text correction result is obtained.

3. The text correction method according to claim 2, characterized in that, The step of segmenting and encoding the answer text, the question text, and the knowledge text to obtain text segmentation encoding features includes: By concatenating the answer text, the question text, and the knowledge text, a concatenated text is obtained. Feature extraction is performed on the spliced ​​text to obtain spliced ​​text features; The concatenated text features are segmented and encoded to obtain the text segmentation encoding features.

4. The text correction method according to claim 3, characterized in that, The step of segmenting and encoding the concatenated text features to obtain the text segmentation encoding features includes: Attention features are obtained by performing attention calculations on the spliced ​​text features; The attention features are segmented and encoded to obtain the text segmentation encoding features.

5. The text correction method according to any one of claims 1 to 4, characterized in that, The steps for determining the knowledge text include: Based on the semantic information of the question text, question text features are generated; Based on the semantic information of each candidate knowledge text, generate features for each candidate knowledge text; Based on the features of the question text and the features of each candidate knowledge text, the semantic relevance between the question text and each candidate knowledge text is determined, and the candidate knowledge text with a semantic relevance greater than a threshold is taken as the knowledge text corresponding to the question text.

6. A method for training a text correction model, characterized in that, include: Obtain sample text, which includes sample answer text, sample question text, and sample knowledge text; Based on the word segmentation encoding features and sentence segmentation encoding features of the sample text, the predicted incorrect entities of the sample answer text are determined; Based on the word segmentation decoding features and sentence segmentation encoding features of the sample text, the text correction prediction result of the sample answer text is determined; The initial model is trained with the goal of minimizing the difference between the predicted incorrect entity and the incorrect entity label, and minimizing the difference between the text correction prediction result and the text correction label, to obtain the text correction model.

7. A text correction device, characterized in that, include: The text determination unit is used to determine the answer text to be corrected, the question text corresponding to the answer text, and the knowledge text associated with the question text; The text correction unit is used to input the answer text, the question text, and the knowledge text into the text correction model to obtain the text correction result of the answer text output by the text correction model; The text correction model is trained with the goal of minimizing the difference between the predicted incorrect entity and the incorrect entity label, and minimizing the difference between the text correction prediction result and the text correction label. The predicted incorrect entity is determined based on the word segmentation encoding features and sentence segmentation encoding features of the sample text, and the text correction prediction result is determined based on the word segmentation decoding features and sentence segmentation encoding features of the sample text. The sample text includes sample answer text, sample question text, and sample knowledge text.

8. A text correction model training device, characterized in that, include: The text acquisition unit is used to acquire sample text, which includes sample answer text, sample question text, and sample knowledge text. An entity determination unit is used to determine the predicted incorrect entities in the sample answer text based on the word segmentation encoding features and sentence segmentation encoding features of the sample text. The correction prediction unit is used to determine the text correction prediction result of the sample answer text based on the word segmentation decoding features and sentence segmentation encoding features of the sample text; The model training unit is used to train the initial model with the goal of minimizing the difference between the predicted incorrect entity and the incorrect entity label, and minimizing the difference between the text correction prediction result and the text correction label, to obtain the text correction model.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the text correction method as described in any one of claims 1 to 5, or the text correction model training method as described in claim 6.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the text correction method as described in any one of claims 1 to 5, or the text correction model training method as described in claim 6.

Images