Text correction model training method, text correction method and device

Abstract

The present disclosure provides a text correction model training method, a text correction method and a device, and relates to the technical field of computers, in particular to the field of natural language processing. The text correction model training method comprises: obtaining a sample sentence and an actual reference object of a pronoun in the sample sentence, and obtaining a historical sentence of the sample sentence; performing a first specified length mask on the pronoun in the sample sentence; the first specified length is determined based on the character length of the candidate noun in the historical sentence; and training the text correction model based on the sample sentence, the historical sentence, the masked sample sentence and the actual reference object of the pronoun until the text correction model converges. The technical solution of the present disclosure can obtain a text correction model with high precision, thereby improving the accuracy of text correction and improving the user experience.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of computers, in particular to the field of natural language processing, and specifically to a text correction model training method, a text correction method and device. BACKGROUND

[0002] In the field of natural language processing, a simultaneous interpretation product can perform speech recognition on the content spoken by a speaker in real time and output the result. When the content spoken by the speaker includes names of multiple persons or things in succession and pronouns appear in the subsequent content, the output of the pronouns is often inaccurate, for example, "he" is output as "it", which affects the accuracy of the speech recognition output and the user experience. SUMMARY

[0003] The present disclosure provides a text correction model training method, a text correction method and device.

[0004] According to a first aspect of the present disclosure, a text correction model training method is provided, comprising:

[0005] obtaining a sample sentence and an actual reference object of a pronoun in the sample sentence, and obtaining a history sentence of the sample sentence;

[0006] masking the pronoun in the sample sentence by a first specified length; the first specified length is determined based on the character length of a candidate noun in the history sentence;

[0007] training the text correction model based on the sample sentence, the history sentence, the masked sample sentence and the actual reference object of the pronoun until the text correction model converges.

[0008] According to a second aspect of the present disclosure, a text correction method is provided, comprising:

[0009] obtaining a target sentence and a history sentence of the target sentence;

[0010] masking a pronoun in the target sentence by a second specified length; the second specified length is determined based on the character length of a candidate noun in the history sentence of the target sentence;

[0011] inputting the target sentence, the history sentence of the target sentence and the masked target sentence into the text correction model, and obtaining a reference probability distribution of the masking position based on the output of the text correction model; the text correction model is pre-trained by the text correction model training method provided by any of the embodiments of the present disclosure;

[0012] determining the reference object of the pronoun in the target sentence according to the reference probability distribution;

[0013] updating the text content of the pronoun in the target sentence according to the reference object of the pronoun in the target sentence.

[0014] According to a third aspect of the present disclosure, a training apparatus of a text correction model is provided, comprising:

[0015] a first obtaining module configured to obtain a sample sentence and an actual reference object of a pronoun in the sample sentence, and obtain a history sentence of the sample sentence;

[0016] a first masking module configured to mask the pronoun in the sample sentence by a first specified length; the first specified length is determined based on a character length of a candidate noun in the history sentence;

[0017] a training module configured to train the text correction model based on the sample sentence, the history sentence, the masked sample sentence, and the actual reference object of the pronoun, until the initial text correction model converges.

[0018] According to a fourth aspect of the present disclosure, a text correction apparatus is provided, comprising:

[0019] a second obtaining module configured to obtain a target sentence, and obtain a history sentence of the target sentence;

[0020] a second masking module configured to mask a pronoun in the target sentence by a second specified length; the second specified length is determined based on a character length of a candidate noun in the history sentence of the target sentence;

[0021] a probability determining module configured to input the target sentence, the history sentence of the target sentence, and the masked target sentence into the text correction model, and obtain a reference probability distribution of a masking position based on an output of the text correction model; the text correction model is pre-trained by the training apparatus of the text correction model provided by any one of the embodiments of the present disclosure;

[0022] an object determining module configured to determine a reference object of the pronoun in the target sentence according to the reference probability distribution;

[0023] an updating module configured to update a text content of the pronoun in the target sentence according to the reference object of the pronoun in the target sentence.

[0024] According to a fifth aspect of the present disclosure, an electronic device is provided, comprising:

[0025] at least one processor; and

[0026] a memory connected in communication with the at least one processor; wherein

[0027] the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the training method of the text correction model or the text correction method provided by any one of the embodiments of the present disclosure.

[0028] According to a sixth aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, where the computer instructions are used to cause a computer to execute the training method of the text correction model or the text correction method provided by any of the embodiments of the present disclosure.

[0029] According to a seventh aspect of the present disclosure, a computer program product is provided, comprising a computer program which, when executed by a processor, implements the training method of the text correction model or the text correction method provided by any of the embodiments of the present disclosure.

[0030] The technical solutions of the present disclosure can achieve at least the following beneficial effects:

[0031] The sample data used for training the text correction model includes a sample sentence and a historical sentence of the sample sentence, and the sample sentence and the historical sentence contain relatively complete context information; the pronoun in the sample sentence is masked based on the character length of the candidate noun in the historical sentence, which can improve the adaptability of the mask length to the character length of the candidate noun; the text correction model is trained based on the context information and the masking of the pronoun in the sample sentence, which can improve the precision of the trained text correction model, and the pronoun in the target sentence is corrected based on the trained text correction model and the masking of the pronoun in the target sentence, which can improve the accuracy of the correction, obtain more accurate text content, and further improve the user experience.

[0032] It should be understood that the content described in this part is not intended to identify key or important features of the embodiments of the present disclosure, nor to limit the scope of the present disclosure. Other features of the present disclosure will become apparent from the following description. BRIEF DESCRIPTION OF DRAWINGS

[0033] The accompanying drawings are used to better understand the present solution and do not limit the present disclosure. Among them:

[0034] Figure 1 is a schematic diagram of speech recognition errors in the related art;

[0035] Figure 2 is a schematic diagram of the principle of correcting speech recognition errors in the related art;

[0036] Figure 3 is a flowchart of a training method of a text correction model provided by an embodiment of the present disclosure;

[0037] Figure 4 is a partial flowchart of another training method of a text correction model provided by an embodiment of the present disclosure;

[0038] Figure 5 is a flowchart of a text correction method provided by an embodiment of the present disclosure;

[0039] Figure 6 A schematic diagram of one principle of the text correction method provided by the embodiments of the present disclosure;

[0040] Figure 7 A schematic diagram of the structural framework of a training device of a text correction model provided by the embodiments of the present disclosure;

[0041] Figure 8 A schematic diagram of the structural framework of a text correction device provided by the embodiments of the present disclosure;

[0042] Figure 9 A schematic diagram of the structural framework of an electronic device provided by the embodiments of the present disclosure. DETAILED DESCRIPTION

[0043] Exemplary embodiments of the present disclosure are described herein with reference to the accompanying drawings, which include various details of the embodiments of the present disclosure to assist in the understanding thereof, and should be considered as merely exemplary. Thus, those skilled in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the present disclosure. Also, in the following description, descriptions of well-known functions and constructions are omitted for clarity and conciseness.

[0044] Figure 1 Three examples of the current simultaneous interpretation product in the process of speech recognition output are shown, and the pronoun error recognition appears in the three examples. In the first example, according to the relationship between the context, the actual referent of "he" in the third sentence (here, the different sentences are divided by a period and a question mark) is the noun "my wife" representing a person, and "he" in the third sentence should be "she", but it is incorrectly identified in Figure 1 ; in the second example, according to the relationship between the context, the actual referent of "he" in the third sentence is the noun "ipad" representing a thing, and "he" in the third sentence should be "it", but it is incorrectly identified in Figure 1 ; in the third example, according to the relationship between the context, the actual referent of "he" in the third sentence is the noun "dozens of dramas" representing a thing, and "he" in the third sentence should be "it", but it is incorrectly identified in Figure 1 .

[0045] In view of the above problems, the current usually uses a pre-training language model SpanBERT based on word segmentation as shown in Figure 2 to predict the probability that the referent of each pronoun y i is a certain noun x i in the context, and selects the noun with the highest probability as the predicted referent.

[0046] InFigure 2 In particular, Fixed-length spans indicates that the candidate noun and the candidate pronoun in the given context are of the same length, in Figure 2 which the length is 3; Span Embeddings indicates the span representation vector, x i represents the Transformer encoding of the candidate noun, i.e., the representation vector, y i represents the Transformer encoding of the candidate pronoun, i.e., the representation vector, s(x i , y i ) represents the correlation between x i and y i integrated with the candidate item encoding x i and y i , s c (x i , y i ) represents the correlation between x i and y i without integrating the candidate item encoding x i and y i , p(x i | y i ) represents the probability that the referent of y i is x i .

[0047] Figure 2 As shown in the manner, only the candidate noun and the candidate pronoun are considered without considering the whole information of the context, the model cannot form a complete understanding of the sentence or the chapter according to the input of the candidate noun and the candidate pronoun, and further cannot accurately identify the referent of the candidate pronoun.

[0048] In addition, Figure 2 the manner requires that the input sequence is fixed-length, which leads to the fact that some candidates need to be expanded or cropped when input. As shown in the example, Figure 2 all the input candidates must be 3 tokens, “ipad” is one token, needs to be expanded, Figure 2 to “ipad above” in the example; “he” is one token, needs to be expanded, Figure 2 to “he brain” in the example; “dozens of episodes” is four tokens, needs to be cropped, Figure 3 to “dozens of” in the example. Expansion introduces redundant information, and cropping loses key information, both of which will lead to inaccurate identification of the referent of the candidate pronoun.

[0049] The technical solutions of the present disclosure and how the technical solutions of the present disclosure solve the above technical problems will be described in detail below with specific embodiments.

[0050] According to an embodiment of the present disclosure, the present disclosure provides a training method of a text correction model, as shown in the following. Figure 4 The training method comprises the following steps.

[0051] In S301, a sample sentence and an actual referent of a pronoun in the sample sentence are obtained, and a history sentence of the sample sentence is obtained.

[0052] In an embodiment of the present disclosure, the sample sentence can include any number of sentences that include a pronoun and have incorrect text content of the pronoun. The history sentence of the sample sentence can be a specified range of sentences before the sample sentence, for example, the first M sentences before the sample sentence, where M is a positive integer, and the specific value of M can be set according to actual requirements.

[0053] In an embodiment of the present disclosure, the division of each sentence can be divided according to specified punctuation marks, and the content between two specified punctuation marks is a sentence. In one example, the specified punctuation marks can include a period, a question mark, and an exclamation mark. In other examples, other punctuation marks can also be used as specified punctuation marks according to actual requirements.

[0054] In an optional implementation, the history sentence of the sample sentence is obtained, comprising: obtaining the history sentence of the sample sentence pre-annotated with candidate nouns. In one example, the candidate nouns in the history sentence can be annotated by a part-of-speech tagging model. The part-of-speech tagging (POS tagging) model can be used to annotate all nouns in the input history sentence as candidate nouns, and can also be used to annotate other parts of speech in the input history sentence, such as verbs and pronouns. In other examples, according to different actual requirements, part of the nouns annotated by the part-of-speech tagging model can also be used as candidate nouns.

[0055] The part-of-speech tagging model in an embodiment of the present disclosure can be any one of a Hidden Markov Model (HMM), a Conditional Random Fields (CRFs) model, etc. The present disclosure does not limit this, and any function of annotating candidate nouns in the history sentence in an embodiment of the present disclosure can be implemented.

[0056] The history sentence pre-annotated with candidate nouns not only provides more complete previous information for subsequent training of the text correction model, but also provides specific candidate nouns, so that the training process of the text correction model can be targeted based on the candidate nouns, which can improve the training accuracy and training efficiency.

[0057] In another optional implementation, the historical sentence of the sample sentence can be a sentence without pre-labeled candidate nouns, and in this case, all the words in the historical sentence can be regarded as candidate nouns.

[0058] S302, the pronoun in the sample sentence is masked with a first specified length.

[0059] The first specified length is determined based on the character length of the candidate noun in the historical sentence of the sample sentence.

[0060] In an optional implementation, in the case where the historical sentence of the sample sentence includes one candidate noun, the pronoun in the sample sentence is masked based on the character length of the one candidate noun as the first specified length.

[0061] In another optional implementation, in the case where the historical sentence includes multiple candidate nouns, the maximum character length is determined among the character lengths of the multiple candidate nouns, and the pronoun in the sample sentence is masked based on the maximum character length.

[0062] Optionally, when the pronoun in the sample sentence is masked with the first specified length, the pronoun in the sample sentence can be masked ([MASK]), and then the length of the mask is expanded to the first specified length, and the masking process can be expressed by the following expression:

[0063] s = masked(S N ) Expression (1)

[0064] In expression (1), S N is the sample sentence, and s is the masked sample sentence.

[0065] In an example, if the sample sentence is “he went on a business trip yesterday”, and the maximum character length of the multiple candidate nouns in the foregoing text is 2, then the pronoun “he” in the sample sentence is masked with the maximum character length to obtain the sentence “[MASK][MASK] went on a business trip yesterday”.

[0066] Based on the maximum character length among the character lengths of the multiple candidate nouns, the pronoun in the sample sentence is masked, which can make the masking length large enough to meet the requirement of matching each candidate noun and the masking position, without the need to crop the candidate nouns.

[0067] S303, based on the sample sentence, the historical sentence of the sample sentence, the masked sample sentence, and the actual referent of the pronoun in the sample sentence, the text correction model is trained until the text correction model converges.

[0068] In an optional implementation, the training of the text correction model includes at least one iteration training, and each iteration training includes:

[0069] inputting the sample sentence, the historical sentence of the sample sentence and the masked sample sentence into the current text correction model, obtaining a reference probability distribution of the mask position based on an output of the current text correction model; determining a loss function value of the current text correction model according to an actual reference object of the pronoun in the sample sentence and the reference probability distribution output by the current text correction model, and adjusting parameters of the current text correction model according to the loss function value; wherein the current text correction model is an initial text correction model or a text correction model trained in the last iteration.

[0070] In the inputting of the sample sentence, the historical sentence of the sample sentence and the masked sample sentence into the current text correction model, all text contents of the sample sentence, the historical sentence of the sample sentence and the masked sample sentence are input into the current text correction model, and the current text correction model is trained based on all the text contents of the sample sentence, the historical sentence of the sample sentence and the masked sample sentence to improve the training accuracy.

[0071] In the embodiments of the present disclosure, iterative training of the text correction model can improve the training effect, and the obtained sample sentence and its historical sentence can be multiple groups to meet the demand of iterative training. When the number of sample sentences and their historical sentences is large enough, a text correction model with high precision can be trained.

[0072] In an optional implementation, as shown in Figure 5 the sample sentence, the historical sentence of the sample sentence and the masked sample sentence are input into the current text correction model, and the current reference object of the pronoun is obtained based on an output of the current text correction model, including the following steps S401-S403:

[0073] S401, inputting the sample sentence and the historical sentence of the sample sentence into an encoder (Encoder) in the current text correction model, and obtaining an encoding result of the sample sentence and an encoding result of the historical sentence of the sample sentence based on an output of the encoder.

[0074] In the embodiments of the present disclosure, the encoder can include a Transformer network or a Hierarchy Transformer network, or can be other networks, which are not limited in the embodiments of the present disclosure, and can only realize the function of the encoder in the embodiments of the present disclosure. The encoding result of the encoder includes hidden states of the sample sentence and hidden states of the historical sentence of the sample sentence, and specifically includes hidden states of each word in the sample sentence and hidden states of each word in the historical sentence of the sample sentence.

[0075] In an example, the encoding manner of the encoder can be expressed by the following expression:

[0076] ei =Encoder(S i ) Expression (2)

[0077] In Expression (2), S i represents the historical statement of the sample statement and the i-th statement in the sample statement, for example, in the sample statement "If he said he had 500 T in his head, I think that's conservative.", and the historical statement of the sample statement "Ah, my wife's iPad is full of dozens of dramas on the air. How much?", "Ah, my wife's iPad is full of dozens of dramas on the air." is the 1st statement, i.e., S1, "How much?" is the 2nd statement, i.e., S2, and "If he said he had 500 T in his head, I think that's conservative." is the 3rd statement, i.e., S3.e i is the hidden state of S i , including the hidden state of each word in S i .

[0078] When the sample statement and its historical statement, and the masked sample statement are input into the encoder, all the text content of the sample statement and its historical statement, and the masked sample statement is input into the encoder, which can realize the chapter-level encoding based on the context content.

[0079] S402, input the encoding result of the sample statement, the encoding result of the historical statement of the sample statement and the masked sample statement into the decoder (Decoder) in the current text correction model, and obtain the decoding result of the masked sample statement based on the output of the decoder.

[0080] In the embodiments of the present disclosure, the decoder can include a Transformer network and a cross-attention network, wherein the Transformer network can also be replaced by other networks, such as a HierarchyTransformer network, and the cross-attention network can also be replaced by other networks, and the embodiments of the present disclosure are not limited thereto, as long as the functions of the decoder of the embodiments of the present disclosure can be realized.

[0081] In one example, the decoding process of the decoder can be expressed by the following expression:

[0082] x=Transformer(s) Expression (3)

[0083] h=Decoder(x,e1,e2,…) Expression (4)

[0084] In Expression (3), s represents the masked sample statement, and x represents the semantic representation of the masked sample statement s output by the Transformer network.

[0085] In expression (4), e1 represents the hidden state of the first sentence output by the encoder; e2 represents the hidden state of the second sentence output by the encoder; h represents the vector representation of the masked sample sentence s output by the decoder, and h has a size of N*d, where N is the character length of the original sentence, and d can be determined according to actual conditions, for example, d is set to 512 or 1024.

[0086] In one example, the specific implementation of expression (4) described above can be expressed by the following expression:

[0087] α ij =softmax(x i ,e j ) expression (5)

[0088] h i =α ij ·e j expression (6)

[0089] In expression (5), x i represents the semantic representation of the i th character in the masked sample sentence s output by the Transformer network; e j represents the hidden state of the j th sentence output by the encoder; and α ij represents the attention of x i to e j .

[0090] In expression (6), h i represents the vector representation of the i th character in the masked sample sentence s output by the decoder.

[0091] The input of the decoder not only includes the encoding results of the sample sentence and its historical sentences, but also includes the content of the masked sample sentence, which can strengthen the understanding of the logical relationship between the sample sentence and the historical sentence by the decoder, and is helpful to obtain a more reliable decoding result for the masked sample sentence.

[0092] S403, sequence labeling is performed on the decoding result at the masked position in the decoding result to obtain a reference probability distribution of the masked position.

[0093] The specific way of implementing sequence labeling is not limited in the embodiments of the present disclosure, and the function of sequence labeling in the embodiments of the present disclosure can be implemented. In one example, the decoding result at the masked position can be sequence labeled by a softmax activation function, and the result of sequence labeling is the probability distribution of all candidate words at the masked position, where all candidate words can be a first specified length of character combinations formed by all characters in a preset dictionary. The expression of this way is as follows:

[0094] t i= softmax(h i ) expression (7)

[0095] In expression (7), h i has the same meaning as expression (6), t i represents a probability distribution obtained by performing sequence labeling on h i , and the probability distribution is specifically a probability distribution of all characters in a preset dictionary at the i-th character position in the sentence s.

[0096] In the specific scenario of performing sequence labeling on the mask position, the value of i in expression (7) only needs to consider the mask position in the sentence s, and does not need to consider other positions in the sentence s, and the preset dictionary usually includes each character of the candidate noun formed in the historical sentence, and thus the probability distribution (probability distribution as a pronoun) of each character of the candidate noun at each mask position in the sentence s can be obtained, which is formed by the probability value of the mask part formed by each candidate noun at each mask position.

[0097] In one example, if the sample sentence after masking is “[MASK][MASK] went on a business trip yesterday”, and the candidate nouns are “wife” and “friend”, then the probability values of the three characters “too”, “friend”, and “friend” at each mask position [MASK] can be obtained by expression (7), which can be represented as P(t1-too), P(t2-too), P(t1-friend), P(t2-friend), P(t1-friend), and P(t2-friend). If the character lengths of several candidate nouns are different, the difference in length can be filled with a special character, which can be ε, and then the probability values of the special character ε at each mask position [MASK] can be determined according to the above manner P(t1-ε), P(t2-ε).

[0098] Multiplying the probability values P(t1-too) and P(t2-too) can obtain the probability value of the candidate noun “wife” at the two mask positions [MASK][MASK], and multiplying the probability values P(t1-friend) and P(t2-friend) can obtain the probability value of the candidate noun “friend” at the two mask positions, and thus the probability distribution of the candidate nouns “wife” and “friend” at the two mask positions [MASK][MASK] can be obtained.

[0099] The sequence labeling operation in the embodiments of the present disclosure can determine the probability distribution of each candidate noun at the mask position of the sample sentence based on the decoding result, and can provide a limited range (a selection range formed by each candidate noun) for the determination of the subsequent referent object in the application stage of the model, so as to facilitate the rapid selection of the correct referent object in the range. The decoder can support the input of candidate nouns of different lengths, and in the process of determining the probability distribution, the length of the candidate noun can be padded with special characters to make the lengths of all candidate nouns consistent.

[0100] In an optional implementation, in the process of training the text error correction model, the loss function can be a cross-entropy function, and the expression thereof is as follows:

[0101] L=cross_entropy(t i ,golden i ) Expression (8)

[0102] In expression (8), the meaning of t i is the same as that in expression (7), and golden i is the referent object at the i th character position in the sample sentence. In the specific scenario of sequence labeling for the mask position, the value of i in expression (8) only needs to consider the mask position in the sentence s, and does not need to consider other positions in the sentence s. The loss function value obtained according to expression (8) can adjust the parameters of the encoder and the decoder.

[0103] According to the embodiments of the present disclosure, the present disclosure further provides a text error correction method, as shown in Figure 6 , the text error correction method comprises the following steps.

[0104] S501, obtaining a target sentence and a history sentence of the target sentence.

[0105] In the embodiments of the present disclosure, when the simultaneous interpretation product performs real-time speech recognition on the speech content of the speaker, the target sentence can be the current sentence obtained by real-time speech recognition, and in other non-real-time speech recognition scenarios, the target sentence can also be any one to-be-corrected sentence in a specified text range. The history sentence of the target sentence can be a specified range of sentences before the target sentence, for example, the first M sentences of the target sentence, M is a positive integer, and the specific value of M can be set according to actual needs. In the subsequent content of the embodiments of the present disclosure, the case where M is 2 will be taken as an example for specific introduction.

[0106] In an optional implementation, the historical sentence of the target sentence is obtained, including: obtaining the historical sentence of the target sentence pre-annotated with the candidate noun. Referring to the foregoing embodiment content, the candidate noun in the historical sentence can be annotated by the part-of-speech tagging model, which can be used to annotate all or part of the nouns in the input historical sentence as the candidate noun, and can also be used to annotate other parts of speech, such as verbs, pronouns, etc. The specific type of the part-of-speech tagging model can refer to the related content of the foregoing embodiment, which will not be described here.

[0107] The historical sentence pre-annotated with the candidate noun not only provides more complete preceding information for the text correction model, but also provides specific candidate nouns, narrows down the selection range of the referent, so that the correct referent can be quickly determined based on the candidate noun, and the accuracy and efficiency of correcting the pronoun in the target sentence can be improved.

[0108] In another optional implementation, the historical sentence of the target sentence can be a sentence without pre-annotated candidate nouns, at this time, all the words of the historical sentence can be regarded as candidate nouns.

[0109] S502, the pronoun in the target sentence is masked with a second specified length.

[0110] The second specified length is determined based on the character length of the candidate noun in the historical sentence of the target sentence.

[0111] In an optional implementation, in the case where the historical sentence of the target sentence includes one candidate noun, the pronoun in the target sentence is masked based on the character length of the one candidate noun (as the second specified length).

[0112] In an optional implementation, in the case where the historical sentence of the target sentence includes multiple candidate nouns, the maximum character length (as the second specified length) is determined among the character lengths of the multiple candidate nouns; the pronoun in the target sentence is masked based on the maximum character length.

[0113] Optionally, when the pronoun in the target sentence is masked with the second specified length, the pronoun in the target sentence can be masked ([MASK]), and then the length of the mask is expanded to the second specified length. The expression of the masking process can refer to the foregoing expression (1).

[0114] In an example, as Figure 6As shown, if the target sentence is the sentence S3 "If say he has 500T in his mind, I think that is conservative", the maximum character length of the multiple candidate nouns in the preceding text is 4, then the sentence obtained after masking the pronoun "he" in the target sentence with the maximum character length is "If say [MASK] [MASK] [MASK] [MASK] has 500T in his mind, I think that is conservative".

[0115] Based on the maximum character length in the character lengths of the multiple candidate nouns, the pronoun in the target sentence is masked, so that the masking length is large enough to meet the requirement of matching each candidate noun with the masking position, without the need to crop the candidate nouns.

[0116] S503, input the target sentence, the historical sentence of the target sentence, and the masked target sentence into a text correction model, and obtain the reference probability distribution of the masking position based on the output of the text correction model.

[0117] The text correction model is pre-trained by the training method of the text correction model provided by any of the embodiments of the present disclosure.

[0118] In an optional implementation, the target sentence and the historical sentence of the target sentence are input into an encoder (Encoder) in the text correction model, and the encoding results of the target sentence and the historical sentence thereof are obtained based on the output of the encoder; the encoding results of the target sentence and the historical sentence thereof, and the masked sample sentence are input into a decoder (Decoder) in the text correction model, and the decoding results of the masked target sentence are obtained based on the output of the decoder; the decoding results of the masking position in the decoding results are sequentially labeled to obtain the reference probability distribution of the masking position.

[0119] The specific implementation of the encoder, the decoder, and the sequence labeling can refer to the previous embodiment content, such as the related content of expressions (2) to (7), and the content of the sample sentence is replaced with the content of the target sentence.

[0120] S504, determine the reference object of the pronoun in the target sentence according to the reference probability distribution of the masking position in the target sentence.

[0121] In an optional implementation, the maximum probability value can be selected from the probability values of each candidate noun at the masking position in the reference probability distribution, and the candidate noun corresponding to the maximum probability value is taken as the reference object of the pronoun in the target sentence. The specific determination method of the reference probability distribution can refer to the related content of expression (7).

[0122] In the above embodiment, the reference probability distribution of the masking position in the target sentence is obtained based on the output of the text correction model. Figure 6In the example shown, the target sentence after the mask is "If say [MASK] [MASK] [MASK] [MASK] brain filled with 500T, I think that is conservative", the candidate nouns in the historical sentences S1 and S2 are "my wife", "ipad" and "dozens of dramas", the probability values of "I", "too", "ipad", "dozens", "ten", "dozens" and the special character ε at each mask position [MASK] can be obtained by the expression (7), and the probability distribution of the completed candidate nouns "my wife ε", "ipad ε ε ε" and "dozens of dramas" at the four mask positions [MASK] [MASK] [MASK] [MASK] can be obtained, that is, the reference probability distribution of the mask position.

[0123] In one example, if the probability values of "my wife ε", "ipad ε ε ε" and "dozens of dramas" at the mask positions [MASK] [MASK] [MASK] [MASK] are 0.38, 0.22 and 0.05 respectively, the candidate noun with the largest probability value "my wife" can be selected.

[0124] S505, updating the text content of the pronoun in the target sentence according to the reference object of the pronoun in the target sentence.

[0125] Specifically, according to the reference object of the pronoun in the target sentence and the association relationship between the noun and the pronoun, the correct text content of the pronoun in the target sentence is determined, and the correct text content is used to replace the previous incorrect text content. Wherein, the association relationship between the reference object and the pronoun can be set in advance, for example, the noun representing a male character has an association relationship with the pronoun "he", the noun representing a female character has an association relationship with the pronoun "she", and the noun representing a thing or an animal has an association relationship with the pronoun "it".

[0126] In the example shown, Figure 7 The correct reference object of the determined pronoun "he" is "my wife", and the pronoun "he" is updated to "she".

[0127] According to the embodiments of the present disclosure, the present disclosure also provides a training device of a text correction model, as shown in Figure 8 The training device comprises a first acquisition module 701, a first mask module 702 and a training module 703.

[0128] The first acquisition module 701 is configured to acquire a sample sentence and an actual reference object of a pronoun in the sample sentence, and acquire a historical sentence of the sample sentence.

[0129] The first mask module 702 is configured to mask the pronoun in the sample sentence by a first specified length; the first specified length is determined based on the character length of the candidate noun in the historical sentence.

[0130] The training module 703 is configured to train the text correction model based on the sample sentence, the historical sentence, the masked sample sentence, and the actual referent of the pronoun until the text correction model converges.

[0131] In an optional implementation, the first obtaining module 701 is specifically configured to: obtain the historical sentence of the sample sentence pre-labeled with the candidate noun.

[0132] In an optional implementation, the first masking module 702 is specifically configured to: in a case where the historical sentence includes a plurality of candidate nouns, determine a maximum character length from character lengths of the plurality of candidate nouns; and mask the pronoun in the sample sentence based on the maximum character length.

[0133] In an optional implementation, the training of the text correction model includes at least one iteration training; and the training module 703 is specifically configured to: in each iteration training, input the sample sentence, the historical sentence, and the masked sample sentence into a current text correction model, obtain a referent probability distribution of the masked position based on an output of the current text correction model, the current text correction model being the initial text correction model or a text correction model trained in a previous iteration training; determine a loss function value of the current text correction model according to the actual referent and the referent probability distribution, and adjust parameters of the current text correction model according to the loss function value.

[0134] In an optional implementation, the training module 703 is specifically configured to: input the sample sentence and the historical sentence into an encoder in the current text correction model, obtain an encoding result of the sample sentence and an encoding result of the historical sentence based on an output of the encoder; input the encoding result of the sample sentence, the encoding result of the historical sentence, and the masked sample sentence into a decoder in the current text correction model, obtain a decoding result of the masked sample sentence based on an output of the decoder; and perform sequence labeling on the decoding result of the masked position in the decoding result to obtain the referent probability distribution of the masked position.

[0135] The functions of each module in the training apparatus of the text correction model provided in the embodiments of the present disclosure can be referred to the corresponding description in the embodiments of the training method of the text correction model provided in the embodiments of the present disclosure, which will not be repeated here.

[0136] According to the embodiments of the present disclosure, the present disclosure further provides a text correction apparatus, as shown in Figure 9 The apparatus includes a second obtaining module 801, a second masking module 802, a probability determining module 803, an object determining module 804, and an updating module 805.

[0137] The second obtaining module 801 is configured to obtain a target sentence and obtain a historical sentence of the target sentence.

[0138] The second mask module 802 is configured to mask a pronoun in the target sentence according to a second specified length; the second specified length is determined based on the character length of the candidate noun in the historical sentence of the target sentence.

[0139] The probability determination module 803 is configured to input the target sentence, the historical sentence of the target sentence, and the masked target sentence into a text correction model, and obtain a reference probability distribution of the masked position based on the output of the text correction model; the text correction model is pre-trained by the training device of the text correction model provided in any embodiment of the present disclosure.

[0140] The object determination module 804 is configured to determine the reference object of the pronoun in the target sentence according to the reference probability distribution.

[0141] The update module 805 is configured to update the text content of the pronoun in the target sentence according to the reference object of the pronoun in the target sentence.

[0142] In an optional implementation, the second acquisition module 801 is specifically configured to: acquire the historical sentence of the target sentence which is pre-labeled with the candidate noun.

[0143] In an optional implementation, the second mask module 802 is specifically configured to: in a case where the historical sentence of the target sentence includes a plurality of candidate nouns, determine a maximum character length from the character lengths of the plurality of candidate nouns; and mask the pronoun in the target sentence based on the maximum character length.

[0144] The functions of each module in the text correction device provided in the embodiments of the present disclosure can be referred to the corresponding description in the embodiments of the text correction method provided in the present disclosure, which will not be repeated here.

[0145] In the technical solutions of the present disclosure, the acquisition, storage and application of user personal information comply with relevant laws and regulations and do not violate public order and good customs.

[0146] According to the embodiments of the present disclosure, the present disclosure further provides an electronic device, a non-transitory computer-readable storage medium and a computer program product.

[0147] The electronic device provided by the present disclosure includes at least one processor and a memory connected in communication with the at least one processor; the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the training method of the text correction model or the text correction method provided in any embodiment of the present disclosure.

[0148] The non-transitory computer-readable storage medium provided by the present disclosure stores computer instructions for causing a computer to execute the training method of the text correction model or the text correction method provided by any of the embodiments of the present disclosure.

[0149] The computer program product provided by the present disclosure comprises a computer program, and the computer program, when executed by a processor, implements the training method of the text correction model or the text correction method provided by any of the embodiments of the present disclosure.

[0150] Figure 9 A schematic block diagram of an example electronic device 900 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptops, desktops, tablets, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular telephones, smart phones, wearable devices, and other similar computing devices. The components shown here, their connections and relationships, and their functions, are meant to be examples only, and are not meant to limit implementations of the present disclosure described and / or claimed in this document.

[0151] As shown in ​ The device 900 includes a computing unit 901 that can perform various appropriate actions and processes in accordance with a computer program stored in a read-only memory (ROM) 902 or a computer program loaded from a storage unit 908 into a random access memory (RAM) 903. Various programs and data required for the operation of the device 900 can also be stored in the RAM 903. The computing unit 901, the ROM 902, and the RAM 903 are connected to each other through a bus 904. An input / output (I / O) interface 905 is also connected to the bus 904.

[0152] Various components in the device 900 are connected to the I / O interface 905, including an input unit 906, such as a keyboard, a mouse, etc.; an output unit 907, such as various types of displays, speakers, etc.; a storage unit 908, such as a magnetic disk, an optical disk, etc.; and a communication unit 909, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 909 allows the device 900 to exchange information / data with other devices through a computer network, such as the Internet, and / or various telecommunication networks.

[0153] The computing unit 901 can be various general and / or special purpose processing components with processing and computing capabilities. Some examples of the computing unit 901 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various specialized artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 901 performs various methods and processes described above. For example, in some embodiments, the above-described methods can be implemented as a computer software program tangibly embodied in a machine-readable medium, such as the storage unit 908. In some embodiments, portions or all of the computer program can be loaded and / or installed onto the device 900 via the ROM 902 and / or the communication unit 909. When the computer program is loaded onto the RAM 903 and executed by the computing unit 901, one or more steps of the above-described methods can be performed. Alternatively, in other embodiments, the computing unit 901 can be configured to perform the above-described methods by any other suitable means, such as by means of firmware.

[0154] Various implementations of the systems and techniques described above can be realized in digital electronic circuitry, integrated circuitry, a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), a system on a chip (SOC), a programmable logic device (PLD), a computer hardware, firmware, software, and / or combinations thereof. These various implementations can include implementation in one or more computer programs that are executable and / or interpretable on a programmable system including at least one programmable processor, which can be special or general purpose, coupled to receive data and instructions from, and to transmit data and instructions to, a storage system, at least one input device, and at least one output device.

[0155] Program code for carrying out methods of the present disclosure can be written in any combination of one or more programming languages. The program code can be provided to a processor or controller of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the program code, when executed by the processor or controller, produces a means for implementing the functions / operations specified in the flowchart and / or block diagram. The program code can be executed entirely on a machine, partially on a machine, partially on a machine and partially on a remote machine or entirely on a remote machine or server.

[0156] In the context of this disclosure, a machine-readable medium can be a tangible medium that contains or stores a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include but is not limited to an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of the machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0157] To provide for interaction with a user, the systems and techniques described here can be implemented on a computer having a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user and a keyboard and a pointing device (e.g., a mouse or a trackball) by which the user can provide input to the computer. Other kinds of devices can be used to provide for interaction with a user as well; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form, including acoustic, speech, or tactile input.

[0158] The systems and techniques described here can be implemented in a computing system that includes a back end component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front end component (e.g., a user computer having a graphical user interface or a Web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such back end, middleware, or front end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network (LAN), a wide area network (WAN), and the Internet.

[0159] The computer system can include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. The server can be a cloud server, a server of a distributed system, or a server combined with a blockchain.

[0160] It should be understood that the various forms of flow shown above can be used to reorder, add, or remove steps. For example, the steps described in the present disclosure can be performed in parallel, in series, or in a different order, as long as the desired results of the technology disclosed in the present disclosure are achieved, which is not limited herein.

[0161] The specific implementation described above does not constitute a limitation on the protection scope of the present disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent replacements, and improvements made within the spirit and principles of the present disclosure shall be included in the protection scope of the present disclosure.

Claims

1. A method for training a text correction model, comprising: obtaining a sample sentence and an actual referent of a pronoun in the sample sentence, and obtaining a history sentence of the sample sentence; masking the pronoun in the sample sentence by a first specified length; the first specified length is determined based on the character length of a candidate noun in the history sentence; training a text correction model based on the sample sentence, the history sentence, the masked sample sentence, and the actual referent of the pronoun until the text correction model converges; the masking the pronoun in the sample sentence by the first specified length comprises: in the case where the history sentence includes multiple candidate nouns, determining a maximum character length from the character lengths of the multiple candidate nouns; masking the pronoun in the sample sentence based on the maximum character length. 2.The method of Claim 1, wherein, the obtaining the history sentence of the sample sentence comprises: obtaining the history sentence of the sample sentence pre-labeled with candidate nouns. 3.The method of Claim 1 or 2, wherein, the training comprises at least one iteration, and each iteration comprises: inputting the sample sentence, the history sentence, and the masked sample sentence into a current text correction model, obtaining a referent probability distribution of a mask position based on the output of the current text correction model; the current text correction model is an initial text correction model or a text correction model trained in the last iteration; determining a loss function value of the current text correction model according to the actual referent and the referent probability distribution, and adjusting the parameters of the current text correction model according to the loss function value. 4.The method of Claim 3, wherein, the inputting the sample sentence, the history sentence, and the masked sample sentence into the current text correction model, and obtaining the referent probability distribution of the mask position based on the output of the current text correction model comprises: inputting the sample sentence and the history sentence into an encoder in the current text correction model, obtaining an encoding result of the sample sentence and an encoding result of the history sentence based on the output of the encoder; inputting the encoding result of the sample sentence, the encoding result of the history sentence, and the masked sample sentence into a decoder in the current text correction model, obtaining a decoding result of the masked sample sentence based on the output of the decoder; performing sequence labeling on the decoding result of the mask position to obtain the referent probability distribution of the mask position.

5. A text correction method, comprising: obtaining a target sentence and a history sentence of the target sentence; masking a pronoun in the target sentence by a second specified length; the second specified length is determined based on the character length of a candidate noun in the history sentence of the target sentence; inputting the target sentence, the history sentence of the target sentence, and the masked target sentence into a text correction model, and obtaining a referent probability distribution of a mask position based on the output of the text correction model; the text correction model is pre-trained by the method for training a text correction model according to any one of claims 1-4; determining a referent of the pronoun in the target sentence according to the referent probability distribution. updating text content of a pronoun in the target sentence according to an object referred to by the pronoun in the target sentence; the second specified length of the pronoun in the target sentence includes: in the case where the historical sentence of the target sentence includes a plurality of candidate nouns, determining a maximum character length from character lengths of the plurality of candidate nouns; masking the pronoun in the target sentence based on the maximum character length.

6. The text correction method of claim 5, wherein, the historical sentence of the target sentence includes: obtaining the historical sentence of the target sentence pre-labeled with candidate nouns.

7. A training device of a text correction model, comprising: a first obtaining module configured to obtain a sample sentence and an actual object referred to by a pronoun in the sample sentence, and obtain a historical sentence of the sample sentence; a first masking module configured to mask the pronoun in the sample sentence by a first specified length; the first specified length is determined based on character lengths of candidate nouns in the historical sentence; a training module configured to train a text correction model based on the sample sentence, the historical sentence, the masked sample sentence, and the actual object referred to by the pronoun, until the text correction model converges; the first masking module is specifically configured to: in the case where the historical sentence includes a plurality of candidate nouns, determine a maximum character length from character lengths of the plurality of candidate nouns; and mask the pronoun in the sample sentence based on the maximum character length.

8. The apparatus of claim 7, wherein, the first obtaining module is specifically configured to: obtain the historical sentence of the sample sentence pre-labeled with candidate nouns.

9. The text correction model training apparatus according to claim 7 or 8, wherein, the training includes at least one iteration training; the training module is specifically configured to: in each iteration training, input the sample sentence, the historical sentence, and the masked sample sentence into a current text correction model, obtain a probability distribution of a reference of a masking position based on an output of the current text correction model; the current text correction model is an initial text correction model or a text correction model trained in a previous iteration; determine a loss function value of the current text correction model according to the actual object referred to and the probability distribution of the reference, and adjust parameters of the current text correction model according to the loss function value.

10. The apparatus of claim 9, wherein, the training module is specifically configured to: input the sample sentence and the historical sentence into an encoder in the current text correction model, and obtain an encoding result of the sample sentence and an encoding result of the historical sentence based on an output of the encoder; input the encoding result of the sample sentence, the encoding result of the historical sentence, and the masked sample sentence into a decoder in the current text correction model, and obtain a decoding result of the masked sample sentence based on an output of the decoder; perform sequence labeling on the decoding result of the masking position in the decoding result, and obtain the probability distribution of the reference of the masking position.

11. A text correction device, comprising: a second obtaining module configured to obtain a target sentence, and obtain a historical sentence of the target sentence; a second masking module configured to mask a pronoun in the target sentence by a second specified length. The second specified length is determined based on the character length of the candidate noun in the historical sentence of the target sentence; The probability determination module is configured to input the target sentence, the historical sentence of the target sentence, and the masked target sentence into a text correction model, and obtain a reference probability distribution of the mask position based on an output of the text correction model; the text correction model is pre-trained by the training device of the text correction model in any one of claims 7-10; The object determination module is configured to determine the reference object of the pronoun in the target sentence according to the reference probability distribution; The update module is configured to update the text content of the pronoun in the target sentence according to the reference object of the pronoun in the target sentence. The second mask module is specifically configured to: In the case that the historical sentence of the target sentence includes a plurality of candidate nouns, determine the maximum character length in the character lengths of the plurality of candidate nouns; and mask the pronoun in the target sentence based on the maximum character length.

12. The text correction apparatus according to claim 11, wherein The second obtaining module is specifically configured to: Obtain the historical sentence of the target sentence pre-labeled with the candidate noun.

13. An electronic device comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the training method of the text correction model of any one of claims 1-4, or the text correction method of any one of claims 5-6.

14. A non-transitory computer readable storage medium having stored thereon computer instructions, wherein, The computer instructions are used to enable the computer to perform the training method of the text correction model of any one of claims 1-4, or the text correction method of any one of claims 5-6.

15. A computer program product comprising a computer program which, when executed by a processor, implements the training method of the text correction model of any one of claims 1-4, or the text correction method of any one of claims 5-6.

Images