Co-reference resolution method, training method and device for co-reference resolution model

Abstract

The present disclosure provides a method for reference resolution, a method for training a reference resolution model, and an apparatus. The reference resolution method includes: obtaining a current utterance to be processed; inputting the current utterance into a reference resolution detection sub-model of a reference resolution model to obtain a predicted insertion position of a semantically missing word in the current utterance and / or a predicted deletion position of a word that needs to be replaced; inputting the predicted insertion position and / or predicted deletion position of the current utterance, as well as the historical conversation corresponding to the current utterance, into a resolution completion sub-model of the reference resolution model to obtain a predicted position of the word with semantic missing at the predicted insertion position in the historical utterance and / or a predicted position of the replacement word corresponding to the word that needs to be replaced at the predicted deletion position in the historical utterance. The present disclosure can solve the problem that reference resolution schemes perform poorly in multi-round conversations.

Description

Technical Field

[0001] The disclosed embodiments relate to the field of artificial intelligence technology, and more particularly to a method for coreference resolution, a method for training a coreference resolution model, and a device. Background Art

[0002] Human-computer dialogue is an important research direction in the field of artificial intelligence. In recent years, it has attracted much attention in academia and industry. Remarkable progress has been made in algorithm technology, and it is widely used in products such as smart speakers and customer service robots.

[0003] In current discourse systems (such as intelligent customer service robots and smart speakers), ambiguous sentences with incomplete information often exist due to the human's colloquial way of expression. One solution is to use the conversation context to complete and resolve references in these incomplete and ambiguous sentences, thereby clarifying the ambiguity. This is also known as dialogue rewriting. However, existing reference resolution solutions work well for single-turn conversations but perform poorly in multi-turn conversations. Summary of the Invention

[0004] The embodiments of the present disclosure provide a method for coreference resolution, a method for training a coreference resolution model, and an apparatus for resolving the problem that existing coreference resolution schemes perform poorly in multi-round conversations.

[0005] In order to solve the above technical problems, the present disclosure is implemented as follows:

[0006] In a first aspect, an embodiment of the present disclosure provides a method for reference resolution, comprising:

[0007] Get the current utterance to be processed;

[0008] Inputting the current utterance into a coreference resolution detection sub-model of a coreference resolution model to obtain a predicted insertion position of a semantically missing word in the current utterance and / or a predicted deletion position of a word to be replaced;

[0009] The predicted insertion position and / or predicted deletion position of the current utterance, as well as the historical conversation corresponding to the current utterance, are input into the resolution and completion sub-model of the reference resolution model to obtain the predicted position of the semantically missing word at the predicted insertion position in the historical utterance and / or the predicted position of the replacement word corresponding to the word to be replaced at the predicted deletion position in the historical utterance.

[0010] Optionally, the reference resolution detection sub-model includes two groups of binary classification networks, wherein the input of one group of binary classification networks is the current speech, and the output is a 01 sequence used to indicate whether the position of each word in the current speech is the starting position or the ending position of the predicted insertion position; the input of the other group of binary classification networks is the current speech, and the output is a 01 sequence used to indicate whether the position of each word in the current speech is the starting position or the ending position of the predicted deletion position.

[0011] Optionally, the binary classification network uses the following formula to determine whether the position of each word in the current speech is the starting position or the ending position of the predicted insertion position or the predicted deletion position:

[0012]

[0013]

[0014] Among them, l is used to indicate whether it is a predicted insertion position or a predicted deletion position. Indicates the starting position, Indicates the end position, h l [i] is the encoding vector representation of the i-th word in the current utterance after passing through the transformer encoder, and represents the weight vector, and represents the bias term, and σ is the sigmoid activation function.

[0015] Optionally, the predicted insertion position and / or predicted deletion position of the current utterance, as well as the historical conversation corresponding to the current utterance, are input into the resolution and completion sub-model of the coreference resolution model to obtain the predicted position of the semantically missing word at the predicted insertion position in the historical utterance and / or the predicted position of the replacement word corresponding to the word to be replaced at the predicted deletion position in the historical utterance, including:

[0016] Calculating an average vector of a starting position and an ending position of a predicted insertion position of the current utterance and / or an average vector of a starting position and an ending position of a predicted deletion position;

[0017] Using the average vector as a condition, semantically processing the encoding vector obtained by encoding the current utterance using a transformer encoder using conditional layer normalization is performed to obtain a semantic output variable after conditional layer normalization;

[0018] The semantic output variable and the historical discourse of the current discourse are input into the resolution and completion sub-model to obtain the predicted position of the semantically missing word at the predicted insertion position in the historical discourse and / or the predicted position of the replacement word corresponding to the word to be replaced at the predicted deletion position in the historical discourse.

[0019] Optionally, the semantic output variable can be expressed as:

[0020] H cln =λ×(h l-1 -μ h ) / (σ h +ε)+β

[0021] Among them, H cln Output variables for the semantics;

[0022] j l-1 is the encoding vector obtained by encoding the current utterance using a transformer encoder;

[0023] H c is the average vector;

[0024] are two transformation matrices, which transform the input mean vector dimension into the same dimension as λ and β;

[0025] H c =avg(h l [s],h l [e]), avg() represents the average function, s represents the starting position, e represents the end position, h l is the encoding vector obtained by encoding the input using the transformer encoder;

[0026] μ h , σ h are respectively the mean and variance of the feature vectors of different dimensions of the encoding vector corresponding to the current speech at the current layer, H is the hidden layer dimension, k is equal to 1~H;

[0027] ε is a minimum value.

[0028] Optionally, the resolution and completion sub-model uses the following formula to determine the predicted position in the historical discourse:

[0029] p s =softmax(W s H cln +b s )

[0030] p e =softmax(W e H cln +b e )

[0031] Among them, p s is the starting position of the predicted position in the historical discourse, p e is the end position of the predicted position in the historical discourse, softmax is the normalized exponential function, W s and W e represents the weight vector, b s and b e represents the bias term, H cln Output variables for the semantics.

[0032] In a second aspect, the present disclosure provides a method for training a coreference resolution model, including:

[0033] Acquire multiple dialogue samples, each of which includes a historical utterance, a current utterance, and a reference utterance of the current utterance, wherein the reference utterance is a utterance obtained by completing semantic gaps and / or performing coreference resolution on the current utterance;

[0034] Obtaining annotated data for the conversation sample, the annotated data including: a real insertion position of a semantically missing word and / or a real deletion position of a word to be replaced in the current utterance, and a real position of the word with the semantically missing word at the real insertion position in the historical utterance and / or a real position of a replacement word corresponding to the word to be replaced at the real deletion position in the historical utterance;

[0035] Training a coreference resolution detection sub-model of a coreference resolution model to be trained based on the current utterance and the annotated data of the conversation sample to obtain a trained coreference resolution detection sub-model, wherein the trained coreference resolution detection sub-model receives an utterance as input and outputs a predicted insertion position of a semantically missing word and / or a predicted deletion position of a word to be replaced in the input utterance;

[0036] According to the annotated data of the conversation sample and the predicted insertion position and / or predicted deletion position of the current utterance of the conversation sample, the resolution and completion sub-model of the reference resolution model to be trained is trained to obtain a trained resolution and completion sub-model, wherein the input of the trained resolution and completion sub-model is the predicted insertion position and / or predicted deletion position of the utterance, and the output is the predicted position of the semantically missing word at the predicted insertion position in the historical utterance of the input utterance and / or the predicted position of the replacement word of the word to be replaced at the predicted deletion position in the historical utterance of the input utterance.

[0037] Optionally, obtaining the annotated data of the conversation sample includes:

[0038] Matching a substring in the current utterance of the conversation sample with a substring in the reference utterance to obtain a matching result;

[0039] If the matching result is a substring that only exists in the current utterance, determining the substring as a word that needs to be replaced, and the actual deletion position of the word that needs to be replaced in the current utterance, finding a replacement font corresponding to the word that needs to be replaced in the reference utterance, and marking the actual position of the replacement word corresponding to the word that needs to be replaced at the actual deletion position in the historical utterance in the historical utterance;

[0040] If the matching result is that a substring exists only in the reference utterance, it is determined that there is a semantic omission in the current utterance, and a real insertion position of the semantic omission in the current utterance is determined.

[0041] Optionally, training the coreference resolution detection sub-model of the coreference resolution model to be trained based on the current speech and the annotated data of the conversation sample includes:

[0042] Input step: inputting the current utterance of the dialogue sample into the coreference resolution detection sub-model to obtain the predicted insertion position of the semantic missing and / or the predicted deletion position of the word to be replaced in the current utterance;

[0043] Determining a value of a first objective function based on the predicted insertion position and / or predicted deletion position output by the coreference resolution detection sub-model and the actual insertion position and / or actual deletion position of the current utterance;

[0044] If the value of the first objective function does not meet the first condition, adjust the parameters of the reference resolution detection sub-model and return to the input step until the value of the first objective function meets the first condition, thereby obtaining the trained reference resolution detection sub-model.

[0045] Optionally, the reference resolution detection sub-model includes two groups of binary classification networks, wherein the input of one group of binary classification networks is the current speech, and the output is a 01 sequence used to indicate whether the position of each word in the current speech is the starting position or the ending position of the predicted insertion position; the input of the other group of binary classification networks is the current speech, and the output is a 01 sequence used to indicate whether the position of each word in the current speech is the starting position or the ending position of the predicted deletion position.

[0046] Optionally, the binary classification network uses the following formula to determine whether the position of each word in the current speech is the starting position or the ending position of the predicted insertion position or the predicted deletion position:

[0047]

[0048]

[0049] Among them, l is used to indicate whether it is a predicted insertion position or a predicted deletion position. Indicates the starting position, Indicates the end position, h l [i] is the encoding vector representation of the i-th word in the current utterance after passing through the transformer encoder, and represents the weight vector to be trained, and Represents the bias term to be trained, and σ is the sigmoid activation function.

[0050] Optionally, the first objective function is expressed using the following formula:

[0051]

[0052] Among them, loss tag is the first objective function, n is the length of the current utterance, Indicates the starting position of the predicted insertion position or predicted deletion position, Indicates the end position of the predicted insertion position or predicted deletion position of the current utterance, Indicates the starting position of the actual insertion position or the actual deletion position of the current speech, represents the end position of the actual insertion position or the actual deletion position of the current utterance, i represents the i-th word in the current utterance, and j represents the j-th word in the current utterance.

[0053] Optionally, training a resolution and completion sub-model of a coreference resolution model to be trained based on the annotated data of the conversation sample and the predicted insertion position and / or predicted deletion position of the current utterance of the conversation sample includes:

[0054] Preprocessing step: calculating the average vector of the starting position and the ending position of the predicted insertion position of the current utterance and / or the average vector of the starting position and the ending position of the predicted deletion position; using the average vector as a condition, performing semantic processing on the encoding vector obtained after encoding the current utterance using the transformer encoder using conditional layer normalization to obtain a semantic output variable after conditional layer normalization;

[0055] Input step: inputting the semantic output variable and the historical discourse of the conversation sample into the resolution and completion sub-model to obtain the predicted position of the semantically missing word at the predicted insertion position in the historical discourse and / or the predicted position of the replacement word corresponding to the word to be replaced at the predicted deletion position in the historical discourse; determining the value of the second objective function based on the predicted position and the actual position in the historical discourse;

[0056] If the value of the second objective function does not meet the second condition, adjust the parameters of the digestion and completion sub-model and return to the input step until the value of the second objective function meets the second condition, thereby obtaining the trained digestion and completion sub-model.

[0057] Optionally, the semantic output variable can be expressed as:

[0058] H cln =λ×(h l-1 -μ h ) / (σ h +ε)+β

[0059] Among them, H cln Output variables for the semantics;

[0060] h l-1 is the encoding vector obtained by encoding the current utterance using a transformer encoder;

[0061] H c is the average vector;

[0062] are two transformation matrices, which transform the input mean vector dimension into the same dimension as λ and β;

[0063] H c =avg(h l [s],h l [e]), avg() represents the average value function, s represents the starting position, e represents the end position, j l is the encoding vector obtained by encoding the input using the transformer encoder;

[0064] μ h , σ h are respectively the mean and variance of the feature vectors of different dimensions of the encoding vector corresponding to the current speech at the current layer, H is the hidden layer dimension, k is equal to 1~H;

[0065] ε is a minimum value.

[0066] Optionally, the resolution and completion sub-model uses the following formula to determine the predicted position in the historical discourse:

[0067] p s =softmax(W s H cln +b s )

[0068] p e =softmax(W e H cln +b e )

[0069] Among them, p s is the starting position of the predicted position in the historical discourse, p e is the end position of the predicted position in the historical discourse, softmax is the normalized exponential function, W s and W e represents the weight vector to be trained, b s and b e Represents the bias term to be trained, H cln Output variables for the semantics.

[0070] Optionally, the second objective function is expressed using the following formula:

[0071]

[0072] Among them, loss mrc is the second objective function, n is the length of the historical discourse, p s represents the starting position of the predicted position of the historical discourse, p e represents the end position of the predicted position of the historical discourse, y s Indicates the starting position of the real position of the historical discourse, y e represents the end position of the real position of the historical discourse, i represents the i-th word in the historical discourse, and j represents the j-th word in the historical discourse.

[0073] In a third aspect, an embodiment of the present disclosure provides a reference digestion device, comprising:

[0074] A first acquisition module is used to acquire the current utterance to be processed;

[0075] A first processing module is configured to input the current utterance into a coreference resolution detection sub-model of a coreference resolution model to obtain a predicted insertion position of a semantically missing word in the current utterance and / or a predicted deletion position of a word to be replaced;

[0076] The second processing module is used to input the predicted insertion position and / or predicted deletion position of the current discourse, as well as the historical conversation corresponding to the current discourse, into the resolution and completion sub-model of the reference resolution model, to obtain the predicted position of the semantically missing word at the predicted insertion position in the historical discourse and / or the predicted position of the replacement word corresponding to the word to be replaced at the predicted deletion position in the historical discourse.

[0077] In a fourth aspect, an embodiment of the present disclosure provides a training device for a reference resolution model, comprising:

[0078] A first acquisition module is configured to acquire a plurality of dialogue samples, each of which includes a historical utterance, a current utterance, and a reference utterance of the current utterance, wherein the reference utterance is an utterance obtained by completing semantic gaps and / or performing coreference resolution on the current utterance;

[0079] a second acquisition module, configured to acquire annotated data of the conversation sample, the annotated data including: a real insertion position of a semantically missing word and / or a real deletion position of a word to be replaced in the current utterance, and a real position of the word with the semantically missing word at the real insertion position in the historical utterance and / or a real position of a replacement word corresponding to the word to be replaced at the real deletion position in the historical utterance;

[0080] A first training module is configured to train a coreference resolution detection sub-model of a coreference resolution model to be trained based on the current utterance and annotated data of the conversation sample to obtain a trained coreference resolution detection sub-model, wherein the trained coreference resolution detection sub-model receives an utterance as input and outputs a predicted insertion position of a semantically missing word and / or a predicted deletion position of a word to be replaced in the input utterance;

[0081] The second training module is used to train the resolution and completion sub-model of the reference resolution model to be trained based on the annotated data of the dialogue sample and the predicted insertion position and / or predicted deletion position of the current discourse of the dialogue sample to obtain a trained resolution and completion sub-model, wherein the input of the trained resolution and completion sub-model is the predicted insertion position and / or predicted deletion position of the discourse, and the output is the predicted position of the semantically missing word at the predicted insertion position in the historical discourse of the input discourse and / or the predicted position of the replacement word of the word to be replaced at the predicted deletion position in the historical discourse of the input discourse.

[0082] In a fifth aspect, an embodiment of the present disclosure provides an electronic device, comprising a processor, a memory, and a program or instruction stored in the memory and executable on the processor, wherein the program or instruction, when executed by the processor, implements the steps of the method for dereferencing described in the first aspect above, or, when executed by the processor, implements the steps of the method for training the dereferencing model described in the second aspect above.

[0083] In a sixth aspect, an embodiment of the present disclosure provides a readable storage medium storing a program or instruction. When the program or instruction is executed by a processor, the method for reference resolution described in the first aspect is implemented, or the steps of the training method for the reference resolution model described in the second aspect are implemented.

[0084] In the embodiment of the present disclosure, in the reference resolution scheme, the historical discourse of the current discourse to be processed is taken into consideration, which can solve the problem that the reference resolution scheme performs poorly in multi-round dialogues. BRIEF DESCRIPTION OF THE DRAWINGS

[0085] Various other advantages and benefits will become apparent to those skilled in the art upon reading the detailed description of the preferred embodiment below. The accompanying drawings are for illustration purposes only and are not to be considered as limiting the present disclosure. The same reference symbols are used throughout the drawings to represent the same components. In the drawings:

[0086] Figure 1 Schematic diagram of the process of the reference digestion method according to an embodiment of the present disclosure;

[0087] Figure 2 Schematic diagram of the output result of the reference resolution detection sub-model according to an embodiment of the present disclosure;

[0088] Figure 3 Schematic diagram of the structure of the transformer encoder according to an embodiment of the present disclosure;

[0089] Figure 4 Schematic diagram of a flow chart of a training method for a reference resolution model according to an embodiment of the present disclosure;

[0090] Figure 5 A schematic diagram of a specific usage scenario of the reference resolution model according to an embodiment of the present disclosure for reference resolution;

[0091] Figure 6 is a schematic structural diagram of a digestion device according to an embodiment of the present disclosure;

[0092] Figure 7 Schematic diagram of the structure of a training device for a reference resolution model according to an embodiment of the present disclosure;

[0093] Figure 8Schematic diagram of the structure of an electronic device according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0094] The following will clearly and completely describe the technical solutions in the embodiments of the present disclosure in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present disclosure, not all of them. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present disclosure without making any creative efforts shall fall within the scope of protection of the present disclosure.

[0095] First, let's briefly explain the meaning of "reference." In linguistics and everyday language, a shortened or pronoun is often used to replace a term that appeared previously. This phenomenon is known in linguistics as "reference," or "reference." Formally, the process of grouping different references representing the same entity into an equivalent set is called reference resolution. Entity: An objective object in the real world. Reference: A statement that refers to an entity.

[0096] Please refer to Figure 1 The present disclosure also provides a method for deconstruction, comprising:

[0097] Step 11: Get the current utterance to be processed;

[0098] Step 12: Inputting the current utterance into the coreference resolution detection sub-model of the coreference resolution model to obtain the predicted insertion position of the semantic omission in the current utterance and / or the predicted deletion position of the word to be replaced;

[0099] Step 13: Input the predicted insertion position and / or predicted deletion position of the current utterance, as well as the historical conversation corresponding to the current utterance, into the resolution and completion sub-model of the reference resolution model to obtain the predicted position of the semantically missing word at the predicted insertion position in the historical utterance and / or the predicted position of the replacement word corresponding to the word to be replaced at the predicted deletion position in the historical utterance.

[0100] In the embodiment of the present disclosure, in the reference resolution scheme, the historical discourse of the current discourse to be processed is taken into consideration, which can solve the problem that the reference resolution scheme performs poorly in multi-round dialogues.

[0101] The following describes the implementation methods of the coreference resolution detection sub-model and the resolution completion sub-model respectively.

[0102] (1) Reference resolution detection sub-model

[0103] The reference resolution detection sub-model is used to detect the predicted insertion positions of semantic omissions and the predicted deletion positions of words that need to be replaced in the current input utterance.

[0104] Optionally, the reference resolution detection sub-model includes two groups of binary classification networks, wherein the input of one group of binary classification networks is the current speech, and the output is a 01 sequence used to indicate whether the position of each word in the current speech is the starting position or the ending position of the predicted insertion position; the input of the other group of binary classification networks is the current speech, and the output is a 01 sequence used to indicate whether the position of each word in the current speech is the starting position or the ending position of the predicted deletion position.

[0105] For example, a multi-turn conversation includes the following:

[0106] Utterance A: What movie has a high rating?

[0107] Utterance B: It’s called Slumdog Millionaire and it’s directed by Lovrin Tandan

[0108] Current Utterance: Forget it, I don’t like this director.

[0109] Among them, "Let's forget it, I don't like this director" is the current utterance, and the first two sentences are historical utterances. After inputting the current utterance into the reference resolution detection sub-model, we can get the following Figure 2 The two sequences shown in FIG2 are as follows, one of which is a detetion sequence, including two lines of sequences. One line of sequences is used to indicate whether the position of each word in the current utterance is the starting position (start) of the predicted deletion position, and the other line of sequences is used to indicate whether the position of each word in the current utterance is the ending position (end) of the predicted deletion position. Figure 2 It can be seen that in the first row of the detetion sequence, the position corresponding to "this" is 1, indicating that the position corresponding to "this" is the starting position of the predicted deletion position. In the second row of the sequence, the position corresponding to "performance" is 1, indicating that the position corresponding to "performance" is the end position of the predicted deletion position. The other sequence is the Inertaion sequence, which includes two rows of sequences. One row of sequences is used to indicate whether the position of each word in the current speech is the starting position (start) of the predicted insertion position, and the other row of sequences is used to indicate whether the position of each word in the current speech is the end position (end) of the predicted insertion position. Figure 2It can be seen that in the first row sequence of the Inertaion sequence, the position corresponding to "also" is 1, indicating that the position corresponding to "also" is the starting position of the predicted insertion position. In the second row sequence, the position corresponding to "also" is 1, indicating that the position corresponding to "also" is the ending position of the predicted insertion position. "Also" being both the starting position and the ending position means that words missing in semantics need to be inserted before "also".

[0110] Of course, in some other embodiments of the present disclosure, in the 01 sequence, the meanings indicated by 0 and 1 can also be interchanged.

[0111] In the embodiments of the present disclosure, optionally, the binary classification network uses the following formula to determine whether the position of each character in the current utterance is the starting position or the ending position of a predicted insertion position or a predicted deletion position:

[0112]

[0113]

[0114] Among them, l is used to represent whether it is a predicted insertion position or a predicted deletion position, represents the starting position, represents the ending position, h l [i] is the encoded vector representation of the i-th character in the current utterance after passing through the transformer encoder, and represent weight vectors, and represent bias terms, and σ is the sigmoid activation function.

[0115] [[ID=三十二]]It should be noted that for whether the position of each character in the current utterance is the starting position of a predicted insertion position, and for whether the position of each character in the current utterance is the starting position of a predicted insertion position, the formula is used. However, when using the formula for both, the weight vector and / or the bias term in the formula can be different.

[0116] Similarly, for whether the position of each character in the current utterance is the ending position of a predicted insertion position, and for whether the position of each character in the current utterance is the ending position of a predicted insertion position, the formula is used. However, when using the formula for both, the weight vector and / or the bias term in the formula can be different. In the embodiments of the present disclosure, the transformer encoder can be implemented in the following manner:

[0117] h0=XW t +W p +W s

[0118] h l =Transformer(h n-1 ),n∈[1,2,3,..,N]

[0119] Among them, W t 、W p 、W s They are word embedding matrix, position embedding matrix and paragraph embedding matrix respectively, n represents the number of layers of the transformer encoder, and taking the dialogue in the above content as an example, X represents the current input speech, which includes three parts: Utterance A, Utterance B, and Current Utterance. In the embodiment of the present disclosure, when inputting X into the transformer encoder, Utterance A and Utterance B can be connected with "," and then input into the transformer encoder in the form of [CLS]Utterance A, Utterance B[SEP]Current Utterance[SEP].

[0120] Please refer to Figure 3 , Figure 3 is a schematic diagram of the structure of a transformer encoder according to an embodiment of the present disclosure. The transformer encoder may include at least one Figure 3 Each transformer block shown in the figure consists of a Multi-Head Attention layer and a Feed Forward network. The Multi-Head Attention layer distributes attention computation across different subspaces, enabling multi-faceted attention learning. Both the Multi-Head Attention and Feed Forward layers are followed by an Add&Norm layer, which incorporates a residual structure and layer normalization.

[0121] (2) Eliminating and completing sub-models

[0122] The resolution and completion sub-model is used to determine from the historical discourse the predicted position of the semantically missing word at the predicted insertion position detected by the reference resolution detection sub-model in the historical discourse and / or the predicted position of the replacement word corresponding to the word to be replaced at the predicted deletion position in the historical discourse.

[0123] In the embodiment of the present disclosure, in the digestion and completion, the average vector of the starting position or the ending position of the predicted insertion position and / or the predicted deletion position of the detected current utterance can be used as a condition, and the current utterance encoded by the transformer encoder can be semantically processed through Conditional Layer Normalization (CLN), and then the digestion and completion text prediction can be performed using Machine Reading Comprehension (MRC).

[0124] In an embodiment of the present disclosure, optionally, the predicted insertion position and / or predicted deletion position of the current utterance, as well as the historical conversation corresponding to the current utterance, are input into the resolution and completion sub-model of the coreference resolution model to obtain the predicted position of the semantically missing word at the predicted insertion position in the historical utterance and / or the predicted position of the replacement word corresponding to the word to be replaced at the predicted deletion position in the historical utterance, including:

[0125] Step 131: Calculate the average vector of the starting position and the ending position of the predicted insertion position of the current utterance and / or the average vector of the starting position and the ending position of the predicted deletion position;

[0126] Step 132: Using the average vector as a condition, semantically processing the encoding vector obtained by encoding the current utterance using a transformer encoder using conditional layer normalization is performed to obtain a semantic output variable after conditional layer normalization.

[0127] Step 133: Input the semantic output variable and the historical discourse of the current discourse into the resolution and completion sub-model to obtain the predicted position of the semantically missing word at the predicted insertion position in the historical discourse and / or the predicted position of the replacement word corresponding to the word to be replaced at the predicted deletion position in the historical discourse.

[0128] That is, if a preset insertion position with semantic omissions is detected in the current discourse, the average vector of the starting position and the ending position of the predicted insertion position is calculated, and then the average vector is used as a condition to perform semantic processing on the encoding vector obtained after encoding the current discourse using a transformer encoder using conditional layer normalization to obtain a semantic output variable after conditional layer normalization, and the semantic output variable and the historical discourse of the current discourse are input into the resolution and completion sub-model to obtain the predicted position of the word with semantic omissions at the predicted insertion position in the historical discourse.

[0129] If a preset deletion position of a word that needs to be replaced is detected in the current discourse, the average vector of the starting position and the ending position of the predicted deletion position is calculated, and then the average vector is used as a condition to perform semantic processing on the encoding vector obtained after encoding the current discourse using the transformer encoder using conditional layer normalization to obtain the semantic output variable after conditional layer normalization, and the semantic output variable and the historical discourse of the current discourse are input into the resolution and completion sub-model to obtain the predicted position of the replacement word of the word that needs to be replaced at the predicted deletion position in the historical discourse.

[0130] Optionally, the semantic output variable can be expressed as:

[0131] H cln =λ×(h l-1 -μ h ) / (σ h +ε)+β

[0132] Among them, H cln Output variables for the semantics;

[0133] h l-1 is the encoding vector obtained by encoding the current utterance using a transformer encoder;

[0134] H c is the average vector; wherein the first λ in the formula is equivalent to adding an offset to the second λ In the trained digestion and completion sub-model, the latter λ is a fixed value;

[0135] are two transformation matrices, which transform the input mean vector dimension into the same dimension as λ and β. Similarly, the first β in the formula is equivalent to adding an offset to the second β. In the trained digestion and completion sub-model, the latter β is a fixed value;

[0136] H c =avg(h l [s],h l [e]), avg() represents the average function, s represents the starting position, e represents the end position, h l is the encoding vector obtained by encoding the input using the transformer encoder;

[0137] μ h , σ hare respectively the mean and variance of the feature vectors of different dimensions of the encoding vector corresponding to the current speech at the current layer, H is the hidden layer dimension, k is equal to 1~H;

[0138] ε is a minimum value (to prevent the denominator from being 0).

[0139] Optionally, the resolution and completion sub-model uses the following formula to determine the predicted position in the historical discourse:

[0140] p s =softmax(W s H cln +b s )

[0141] p e =softmax(W e H cln +b e )

[0142] Among them, p s is the starting position of the predicted position in the historical discourse, p e is the end position of the predicted position in the historical discourse, softmax is the normalized exponential function, W s and W e represents the weight vector, b s and b e represents the bias term, H cln Output variables for the semantics.

[0143] It should be noted that the starting position of the predicted position of the word with missing semantics at the predicted insertion position in the historical discourse, and the starting position of the predicted position of the corresponding replacement word of the word to be replaced at the predicted deletion position in the historical discourse, are both calculated using the formula

[0144] p s =softmax(W s H cln +b s ), however, when both use the formula, the weight vector W in the formula s and / or bias term b s , can be different.

[0145] Similarly, the predicted ending position of the semantically missing word at the predicted insertion position in the historical discourse, and the predicted ending position of the corresponding replacement word at the predicted deletion position in the historical discourse, are both calculated using formula p e =softmax(W e H cln +be ), however, when both use the formula, the weight vector W in the formula e and / or bias term b e , can be different.

[0146] Please refer to Figure 4 The present disclosure also provides a method for training a coreference resolution model, including:

[0147] Step 41: Acquire multiple dialogue samples, each of which includes a historical utterance, a current utterance, and a reference utterance of the current utterance, wherein the reference utterance is an utterance obtained by completing semantic gaps and / or performing coreference resolution on the current utterance.

[0148] Step 42: Acquire annotated data for the conversation sample, the annotated data including: the actual insertion position of the semantically missing word in the current utterance and / or the actual deletion position of the word to be replaced, as well as the actual position of the semantically missing word at the actual insertion position in the historical utterance and / or the actual position of the replacement word corresponding to the word to be replaced at the actual deletion position in the historical utterance;

[0149] Step 43: Training a coreference resolution detection sub-model of the coreference resolution model to be trained based on the current utterance and the annotated data of the conversation sample to obtain a trained coreference resolution detection sub-model, wherein the trained coreference resolution detection sub-model takes an utterance as input and outputs predicted insertion positions of semantically missing words and / or predicted deletion positions of words to be replaced in the input utterance;

[0150] Step 44: Based on the annotated data of the conversation sample and the predicted insertion position and / or predicted deletion position of the current utterance of the conversation sample, the resolution and completion sub-model of the reference resolution model to be trained is trained to obtain a trained resolution and completion sub-model, wherein the input of the trained resolution and completion sub-model is the predicted insertion position and / or predicted deletion position of the utterance, and the output is the predicted position of the semantically missing word at the predicted insertion position in the historical utterance of the input utterance and / or the predicted position of the replacement word of the word to be replaced at the predicted deletion position in the historical utterance of the input utterance.

[0151] In the disclosed embodiment, during the training of the coreference resolution model, historical utterances of the current utterance to be processed are taken into consideration, so that the trained coreference resolution model can perform better in multiple rounds of dialogue.

[0152] In the embodiment of the present disclosure, optionally, obtaining the annotated data of the conversation sample includes:

[0153] Step 421: matching a substring in the current utterance of the conversation sample with a substring in the reference utterance to obtain a matching result;

[0154] Step 422: If the matching result is a substring that only exists in the current utterance, determine the substring as a word to be replaced, and the actual deletion position of the word to be replaced in the current utterance, find the replacement font corresponding to the word to be replaced in the reference utterance, and mark the actual position of the replacement word corresponding to the actual deletion position in the historical utterance in the historical utterance.

[0155] Step 423: If the matching result is that a substring exists only in the reference utterance, it is determined that there is a semantic gap in the current utterance, and the actual insertion position of the semantic gap in the current utterance is determined.

[0156] For example, a sample conversation might be as follows:

[0157] A: What movie has a high rating?

[0158] B: It's called Slumdog Millionaire, directed by Lovrin Tandan

[0159] A: Forget it. I don’t like this director.

[0160] A+: Forget about Slumdog Millionaire. I don’t like Lovelin Tandan.

[0161] Among them, the first two sentences are historical discourse, the third sentence is current discourse, and the fourth sentence is reference discourse.

[0162] Taking the above dialogue sample as an example, the substring in the current discourse of the dialogue sample (Forget it, I don't like this director) is matched with the substring in the reference discourse (Forget it, Slumdog Millionaire, I don't like Loveline Tandon).

[0163] (1) If the matching result is a substring that exists in both the current utterance and the reference utterance, the substring remains unchanged, for example, "Forget it, I don't like it";

[0164] (2) If the matching result is a substring that only exists in the current utterance, determine that the substring is a word that needs to be replaced, and the actual deletion position of the word that needs to be replaced in the current utterance, find the replacement font corresponding to the word that needs to be replaced in the reference utterance, and mark the actual position of the replacement word corresponding to the actual deletion position in the historical utterance, for example, "this director" should be replaced with "Lovlin Tandan";

[0165] (3) If the matching result is a substring that only exists in the reference utterance, it is determined that there is a semantic omission in the current utterance, and the actual insertion position of the semantic omission in the current utterance is determined, such as the need to insert "slumdog millionaire" before "still".

[0166] Through the above operations, you can obtain the labeled data for training the model.

[0167] Optionally, training the coreference resolution detection sub-model of the coreference resolution model to be trained based on the current speech and the annotated data of the conversation sample includes:

[0168] Input step: inputting the current utterance of the dialogue sample into the coreference resolution detection sub-model to obtain the predicted insertion position of the semantic missing and / or the predicted deletion position of the word to be replaced in the current utterance;

[0169] Determining a value of a first objective function based on the predicted insertion position and / or predicted deletion position output by the coreference resolution detection sub-model and the actual insertion position and / or actual deletion position of the current utterance;

[0170] If the value of the first objective function does not meet the first condition, adjust the parameters of the reference resolution detection sub-model and return to the input step until the value of the first objective function meets the first condition, thereby obtaining the trained reference resolution detection sub-model.

[0171] The following describes the training methods for the coreference resolution detection sub-model and the resolution completion sub-model respectively.

[0172] (1) Training method of reference resolution detection sub-model

[0173] In an embodiment of the present disclosure, optionally, the reference resolution detection submodel includes two groups of binary classification networks, wherein the input of one group of binary classification networks is the current speech, and the output is a 01 sequence for indicating whether the position of each word in the current speech is the starting position or the ending position of the predicted insertion position, and the input of the other group of binary classification networks is the current speech, and the output is a 01 sequence for indicating whether the position of each word in the current speech is the starting position or the ending position of the predicted deletion position.

[0174] In the embodiment of the present disclosure, optionally, the binary classification network uses the following formula to determine whether the position of each word in the current speech is the starting position or the ending position of the predicted insertion position or the predicted deletion position:

[0175]

[0176]

[0177] Among them, l is used to indicate whether it is a predicted insertion position or a predicted deletion position. Indicates the starting position, Indicates the end position, h l [i] is the encoding vector representation of the i-th word in the current utterance after passing through the transformer encoder, and represents the weight vector to be trained, and Represents the bias term to be trained, and σ is the sigmoid activation function.

[0178] In the embodiment of the present disclosure, optionally, the first objective function is expressed by the following formula:

[0179]

[0180] Among them, loss tag is the first objective function, n is the length of the current utterance, Indicates the starting position of the predicted insertion position or predicted deletion position, Indicates the end position of the predicted insertion position or predicted deletion position of the current utterance, Indicates the starting position of the actual insertion position or the actual deletion position of the current speech, represents the end position of the actual insertion position or the actual deletion position of the current utterance, i represents the i-th word in the current utterance, and j represents the j-th word in the current utterance.

[0181] In the formula, n+2 refers to the length of the additional input CLS and SEP when the input method is input into the transformer in the form of "[CLS]Utterance A,Utterance B[SEP]CurrentUtterance[SEP]".

[0182] Of course, in the embodiment of the present application, if other input methods are used, n+2 in the formula can also be updated accordingly.

[0183] (2) Training method of the elimination and completion sub-model

[0184] In an embodiment of the present disclosure, optionally, training a resolution and completion sub-model of a coreference resolution model to be trained based on the annotated data of the conversation sample and the predicted insertion position and / or predicted deletion position of the current utterance of the conversation sample includes:

[0185] Preprocessing step: calculating the average vector of the starting position and the ending position of the predicted insertion position of the current utterance and / or the average vector of the starting position and the ending position of the predicted deletion position; using the average vector as a condition, performing semantic processing on the encoding vector obtained after encoding the current utterance using the transformer encoder using conditional layer normalization to obtain a semantic output variable after conditional layer normalization;

[0186] Input step: inputting the semantic output variable and the historical discourse of the conversation sample into the resolution and completion sub-model to obtain the predicted position of the semantically missing word at the predicted insertion position in the historical discourse and / or the predicted position of the replacement word corresponding to the word to be replaced at the predicted deletion position in the historical discourse; determining the value of the second objective function based on the predicted position and the actual position in the historical discourse;

[0187] If the value of the second objective function does not meet the second condition, adjust the parameters of the digestion and completion sub-model and return to the input step until the value of the second objective function meets the second condition, thereby obtaining the trained digestion and completion sub-model.

[0188] In the embodiment of the present disclosure, optionally, the semantic output variable can be expressed as:

[0189] H cln =λ×(h l-1 -μ h ) / (σ h +ε)+β

[0190] Among them, H cln Output variables for the semantics;

[0191] h l-1 is the encoding vector obtained by encoding the current speech using the transformer encoder; the starting value of the subscript index value of h is 0, h l-1 Indicates that the last layer index value of the l-layer network is l-1.

[0192] H c is the average vector; wherein the first λ in the formula is equivalent to adding an offset to the second λ The latter λ has an initial value and is continuously updated during the training process of the elimination and completion sub-model;

[0193] are two transformation matrices, which transform the input average vector dimension into the same dimension as λ and β; the first β in the formula is equivalent to adding an offset to the second β. The latter β has an initial value and is continuously updated during the training process of the elimination and completion sub-model.

[0194] H c =avg(h l [s],h l [e]), avg() represents the average function, s represents the starting position, e represents the end position, h l is the encoding vector obtained by encoding the input using the transformer encoder;

[0195] μ h , σ h are respectively the mean and variance of the feature vectors of different dimensions of the encoding vector corresponding to the current speech at the current layer, H is the hidden layer dimension, k is equal to 1~H; μ h , σ h Here h refers to the dimension of the network hidden layer;

[0196] ε is a minimum value.

[0197] In the embodiment of the present disclosure, optionally, the resolution and completion sub-model uses the following formula to determine the predicted position in the historical discourse:

[0198] p s =softmax(W s H cln +b s )

[0199] p e =softmax(W e H cln +b e )

[0200] Among them, p s is the starting position of the predicted position in the historical discourse, p e is the end position of the predicted position in the historical discourse, softmax is the normalized exponential function, W s and W e represents the weight vector to be trained, b s and b e Represents the bias term to be trained, H cln Output variables for the semantics.

[0201] In the embodiment of the present disclosure, optionally, the second objective function is expressed by the following formula:

[0202]

[0203] Among them, lossmrc is the second objective function, n is the length of the historical discourse, p s represents the starting position of the predicted position of the historical discourse, p e represents the end position of the predicted position of the historical discourse, y s Indicates the starting position of the real position of the historical discourse, y e represents the end position of the real position of the historical discourse, i represents the i-th word in the historical discourse, and j represents the j-th word in the historical discourse.

[0204] In the embodiment of the present disclosure, two tasks (referential resolution detection and resolution completion) are multi-task learned in a cascade structure. That is, in the first level task, the reference resolution detection task is performed, and in the second level task, the resolution completion task is performed. The two levels of tasks can share the same encoding layer (Transformer Encoder) (please refer to Figure 5 ), training and optimization are performed simultaneously, and the training optimization target is the sum of the optimization targets of the two tasks, namely:

[0205] loss=loss tag +loss mrc

[0206] Through the coreference resolution detection task, the positions of words that need to be coreference resolved or semantically completed in the current discourse are obtained. Then, through conditional layer normalization, the semantic information of the coreference resolution words is integrated into the text prediction task of resolution and completion, and the resolution and completion text prediction is performed using machine reading comprehension (MRC).

[0207] Please refer to Figure 6 The embodiment of the present disclosure further provides a reference digestion device 60, comprising:

[0208] A first acquisition module 61 is used to acquire the current speech to be processed;

[0209] A first processing module 62 is configured to input the current utterance into a coreference resolution detection sub-model of a coreference resolution model to obtain a predicted insertion position of a semantically missing word and / or a predicted deletion position of a word to be replaced in the current utterance;

[0210] The second processing module 63 is used to input the predicted insertion position and / or predicted deletion position of the current utterance, as well as the historical conversation corresponding to the current utterance, into the resolution and completion sub-model of the reference resolution model, to obtain the predicted position of the semantically missing word at the predicted insertion position in the historical utterance and / or the predicted position of the replacement word corresponding to the word to be replaced at the predicted deletion position in the historical utterance.

[0211] Optionally, the reference resolution detection sub-model includes two groups of binary classification networks, wherein the input of one group of binary classification networks is the current speech, and the output is a 01 sequence used to indicate whether the position of each word in the current speech is the starting position or the ending position of the predicted insertion position; the input of the other group of binary classification networks is the current speech, and the output is a 01 sequence used to indicate whether the position of each word in the current speech is the starting position or the ending position of the predicted deletion position.

[0212] Optionally, the binary classification network uses the following formula to determine whether the position of each word in the current speech is the starting position or the ending position of the predicted insertion position or the predicted deletion position:

[0213]

[0214]

[0215] Among them, l is used to indicate whether it is a predicted insertion position or a predicted deletion position. Indicates the starting position, Indicates the end position, h l [i] is the encoding vector representation of the i-th word in the current utterance after passing through the transformer encoder, and represents the weight vector, and represents the bias term, and σ is the sigmoid activation function.

[0216] Optionally, the second processing module 63 includes:

[0217] A calculation submodule 631 is configured to calculate an average vector of a start position and an end position of a predicted insertion position of the current utterance and / or an average vector of a start position and an end position of a predicted deletion position;

[0218] A first processing submodule is configured to perform semantic processing on an encoding vector obtained by encoding the current utterance using a transformer encoder using the average vector as a condition and using conditional layer normalization to obtain a semantic output variable after conditional layer normalization;

[0219] The second processing sub-module is used to input the semantic output variable and the historical discourse of the current discourse into the resolution and completion sub-model to obtain the predicted position of the semantically missing word at the predicted insertion position in the historical discourse and / or the predicted position of the replacement word corresponding to the word to be replaced at the predicted deletion position in the historical discourse.

[0220] Optionally, the semantic output variable can be expressed as:

[0221] H cln =λ×(h l-1 -μ h ) / (σ h +ε)+β

[0222] Among them, H cln Output variables for the semantics;

[0223] h l-1 is the encoding vector obtained by encoding the current utterance using a transformer encoder;

[0224] H c is the average vector;

[0225] are two transformation matrices, which transform the input mean vector dimension into the same dimension as λ and β;

[0226] H c =avg(h l [s],h l [e]), avg() represents the average function, s represents the starting position, e represents the end position, h l is the encoding vector obtained by encoding the input using the transformer encoder;

[0227] μ h , σ h are respectively the mean and variance of the feature vectors of different dimensions of the encoding vector corresponding to the current speech at the current layer, H is the hidden layer dimension, k is equal to 1~H;

[0228] ε is a minimum value.

[0229] Optionally, the resolution and completion sub-model uses the following formula to determine the predicted position in the historical discourse:

[0230] p s =softmax(W s H cln +b s )

[0231] p e =softmax(W e H cln +b e )

[0232] Among them, p s is the starting position of the predicted position in the historical discourse, p e is the end position of the predicted position in the historical discourse, softmax is the normalized exponential function, W s and W e represents the weight vector, b s and b e represents the bias term, H cln Output variables for the semantics.

[0233] Please refer to Figure 7 The present disclosure also provides a training device 70 for a reference resolution model, including:

[0234] A first acquisition module 71 is configured to acquire a plurality of dialogue samples, each of which includes a historical utterance, a current utterance, and a reference utterance of the current utterance, wherein the reference utterance is an utterance obtained by completing semantic gaps and / or performing coreference resolution on the current utterance;

[0235] A second acquisition module 72 is configured to acquire annotated data of the conversation sample, the annotated data including: a real insertion position of a semantically missing word and / or a real deletion position of a word to be replaced in the current utterance, and a real position of the word with the semantically missing word at the real insertion position in the historical utterance and / or a real position of a replacement word corresponding to the word to be replaced at the real deletion position in the historical utterance;

[0236] A first training module 73 is configured to train a coreference resolution detection sub-model of a coreference resolution model to be trained based on the current utterance and the annotated data of the conversation sample to obtain a trained coreference resolution detection sub-model, wherein the trained coreference resolution detection sub-model receives an utterance as input and outputs predicted insertion positions of semantically missing words and / or predicted deletion positions of words to be replaced in the input utterance;

[0237] The second training module 74 is used to train the resolution and completion sub-model of the reference resolution model to be trained based on the annotated data of the dialogue sample and the predicted insertion position and / or predicted deletion position of the current discourse of the dialogue sample to obtain a trained resolution and completion sub-model, wherein the input of the trained resolution and completion sub-model is the predicted insertion position and / or predicted deletion position of the discourse, and the output is the predicted position of the semantically missing word at the predicted insertion position in the historical discourse of the input discourse and / or the predicted position of the replacement word of the word to be replaced at the predicted deletion position in the historical discourse of the input discourse.

[0238] Optionally, the second acquisition module 72 is used to match a substring in the current discourse of the dialogue sample with a substring in the reference discourse to obtain a matching result; if the matching result is that a substring exists only in the current discourse, determine that the substring is a word that needs to be replaced, and the actual deletion position of the word that needs to be replaced in the current discourse, find the replacement font corresponding to the word that needs to be replaced in the reference discourse, and mark the actual position of the replacement word corresponding to the actual deletion position in the historical discourse; if the matching result is that a substring exists only in the reference discourse, determine that there is a semantic deficiency in the current discourse, and determine the actual insertion position of the semantic deficiency in the current discourse.

[0239] Optionally, the first training module 73 is configured to perform the following operations:

[0240] Input step: inputting the current utterance of the dialogue sample into the coreference resolution detection sub-model to obtain the predicted insertion position of the semantic missing and / or the predicted deletion position of the word to be replaced in the current utterance;

[0241] Determining a value of a first objective function based on the predicted insertion position and / or predicted deletion position output by the coreference resolution detection sub-model and the actual insertion position and / or actual deletion position of the current utterance;

[0242] If the value of the first objective function does not meet the first condition, adjust the parameters of the reference resolution detection sub-model and return to the input step until the value of the first objective function meets the first condition, thereby obtaining the trained reference resolution detection sub-model.

[0243] Optionally, the reference resolution detection sub-model includes two groups of binary classification networks, wherein the input of one group of binary classification networks is the current speech, and the output is a 01 sequence used to indicate whether the position of each word in the current speech is the starting position or the ending position of the predicted insertion position; the input of the other group of binary classification networks is the current speech, and the output is a 01 sequence used to indicate whether the position of each word in the current speech is the starting position or the ending position of the predicted deletion position.

[0244] Optionally, the binary classification network uses the following formula to determine whether the position of each word in the current speech is the starting position or the ending position of the predicted insertion position or the predicted deletion position:

[0245]

[0246]

[0247] Among them, l is used to indicate whether it is a predicted insertion position or a predicted deletion position. Indicates the starting position, Indicates the end position, h l [i] is the encoding vector representation of the i-th word in the current utterance after passing through the transformer encoder, and represents the weight vector to be trained, and Represents the bias term to be trained, and σ is the sigmoid activation function.

[0248] Optionally, the first objective function is expressed using the following formula:

[0249]

[0250] Among them, loss tag is the first objective function, n is the length of the current utterance, Indicates the starting position of the predicted insertion position or predicted deletion position, Indicates the end position of the predicted insertion position or predicted deletion position of the current utterance, Indicates the starting position of the actual insertion position or the actual deletion position of the current speech, represents the end position of the actual insertion position or the actual deletion position of the current utterance, i represents the i-th word in the current utterance, and j represents the j-th word in the current utterance.

[0251] Optionally, the second training module 74 is configured to perform the following operations:

[0252] Preprocessing step: calculating the average vector of the starting position and the ending position of the predicted insertion position of the current utterance and / or the average vector of the starting position and the ending position of the predicted deletion position; using the average vector as a condition, performing semantic processing on the encoding vector obtained after encoding the current utterance using the transformer encoder using conditional layer normalization to obtain a semantic output variable after conditional layer normalization;

[0253] Input step: inputting the semantic output variable and the historical discourse of the conversation sample into the resolution and completion sub-model to obtain the predicted position of the semantically missing word at the predicted insertion position in the historical discourse and / or the predicted position of the replacement word corresponding to the word to be replaced at the predicted deletion position in the historical discourse; determining the value of the second objective function based on the predicted position and the actual position in the historical discourse;

[0254] If the value of the second objective function does not meet the second condition, adjust the parameters of the digestion and completion sub-model and return to the input step until the value of the second objective function meets the second condition, thereby obtaining the trained digestion and completion sub-model.

[0255] Optionally, the semantic output variable can be expressed as:

[0256] H cln =λ×(h l-1 -μ h ) / (σ h +ε)+β

[0257] Among them, H cln Output variables for the semantics;

[0258] h l-1 is the encoding vector obtained by encoding the current utterance using a transformer encoder;

[0259] H c is the average vector;

[0260] are two transformation matrices, which transform the input mean vector dimension into the same dimension as λ and β;

[0261] H c =avg(h l [s],h l [e]), avg() represents the average function, s represents the starting position, e represents the end position, h l is the encoding vector obtained by encoding the input using the transformer encoder;

[0262] μ h , σ h are respectively the mean and variance of the feature vectors of different dimensions of the encoding vector corresponding to the current speech at the current layer, H is the hidden layer dimension, k is equal to 1~H;

[0263] ε is a minimum value.

[0264] Optionally, the resolution and completion sub-model uses the following formula to determine the predicted position in the historical discourse:

[0265] p s =softmax(W s H cln +b s )

[0266] p e =softmax(W e H cln +b e )

[0267] Among them, p s is the starting position of the predicted position in the historical discourse, p e is the end position of the predicted position in the historical discourse, softmax is the normalized exponential function, W s and W e represents the weight vector to be trained, b s and b e Represents the bias term to be trained, H cln Output variables for the semantics.

[0268] Optionally, the second objective function is expressed using the following formula:

[0269]

[0270] Among them, loss mrc is the second objective function, n is the length of the historical discourse, p s represents the starting position of the predicted position of the historical discourse, p e represents the end position of the predicted position of the historical discourse, y s Indicates the starting position of the real position of the historical discourse, y e represents the end position of the real position of the historical discourse, i represents the i-th word in the historical discourse, and j represents the j-th word in the historical discourse.

[0271] Please refer to Figure 8The embodiment of the present disclosure also provides an electronic device 80, including a processor 81, a memory 82, and a program or instruction stored in the memory 82 and executable on the processor 81, wherein the program or instruction, when executed by the processor 81, implements the steps of the above-mentioned reference resolution method, or, when executed by the processor 81, implements the steps of the above-mentioned training method of the reference resolution model.

[0272] An embodiment of the present disclosure provides a readable storage medium storing a program or instruction. When the program or instruction is executed by a processor, the program or instruction implements the above-mentioned reference resolution method, or implements the steps of the training method of the above-mentioned reference resolution model.

[0273] The readable storage medium includes a computer-readable storage medium, such as a computer read-only memory ROM, a random access memory RAM, a magnetic disk or an optical disk.

[0274] It should be noted that, in this article, the terms "comprise", "include" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the statement "comprises a ..." does not exclude the presence of other identical elements in the process, method, article or device comprising the element. In addition, it should be noted that the scope of the methods and devices in the embodiments of the present application is not limited to performing functions in the order shown or discussed, and may also include performing functions in a substantially simultaneous manner or in the opposite order according to the functions involved. For example, the described method may be performed in an order different from that described, and various steps may also be added, omitted, or combined. In addition, the features described with reference to certain examples may be combined in other examples.

[0275] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus the necessary general hardware platform, and of course can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a computer software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for enabling a terminal (which can be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in each embodiment of the present application.

[0276] The embodiments of the present disclosure are described above in conjunction with the accompanying drawings, but the present disclosure is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present disclosure, ordinary technicians in this field can also make many forms without departing from the scope of protection of the purpose of the present disclosure and the claims, all of which are protected by the present disclosure.

Claims

1. A method for reference resolution, characterized in that: include: Get the current utterance to be processed; Inputting the current utterance into a coreference resolution detection sub-model of a coreference resolution model to obtain a predicted insertion position of a semantically missing word in the current utterance and / or a predicted deletion position of a word to be replaced; Inputting the predicted insertion position and / or predicted deletion position of the current utterance, as well as the historical utterance corresponding to the current utterance, into the resolution and completion sub-model of the coreference resolution model, to obtain the predicted position of the semantically missing word at the predicted insertion position in the historical utterance and / or the predicted position of the replacement word corresponding to the word to be replaced at the predicted deletion position in the historical utterance; The predicted insertion position and / or predicted deletion position of the current utterance, as well as the historical utterance corresponding to the current utterance, are input into the resolution and completion sub-model of the coreference resolution model to obtain the predicted position of the semantically missing word at the predicted insertion position in the historical utterance and / or the predicted position of the replacement word corresponding to the word to be replaced at the predicted deletion position in the historical utterance, including: Calculating an average vector of a starting position and an ending position of a predicted insertion position of the current utterance and / or an average vector of a starting position and an ending position of a predicted deletion position; Using the average vector as a condition, semantically processing the encoding vector obtained by encoding the current utterance using a transformer encoder using conditional layer normalization is performed to obtain a semantic output variable after conditional layer normalization; The semantic output variable and the historical discourse of the current discourse are input into the resolution and completion sub-model to obtain the predicted position of the semantically missing word at the predicted insertion position in the historical discourse and / or the predicted position of the replacement word corresponding to the word to be replaced at the predicted deletion position in the historical discourse.

2. The method according to claim 1, characterized in that The reference resolution detection sub-model includes two groups of binary classification networks, wherein the input of one group of binary classification networks is the current speech, and the output is a 01 sequence used to indicate whether the position of each word in the current speech is the starting position or the ending position of the predicted insertion position; the input of the other group of binary classification networks is the current speech, and the output is a 01 sequence used to indicate whether the position of each word in the current speech is the starting position or the ending position of the predicted deletion position.

3. The method according to claim 2, characterized in that The binary classification network uses the following formula to determine whether the position of each word in the current utterance is the starting position or the ending position of the predicted insertion position or the predicted deletion position: Among them, l is used to indicate whether it is a predicted insertion position or a predicted deletion position. Indicates the starting position, Indicates the end position, h l [i] is the encoding vector representation of the i-th word in the current utterance after passing through the transformer encoder, and represents the weight vector, and represents the bias term, and σ is the sigmoid activation function.

4. The method according to claim 1, wherein The semantic output variable can be expressed as: H cln =λ×(h l-1 -m h ) / (s h +e)+b Among them, H cln Output variables for the semantics; h l-1 is the encoding vector obtained by encoding the current utterance using a transformer encoder; H c is the average vector; are two transformation matrices, which transform the input mean vector dimension into the same dimension as λ and β; H c =avg(h l [s],h l [e]), avg() represents the average function, s represents the starting position, e represents the end position, h l is the encoding vector obtained by encoding the input using the transformer encoder; μ h , σ h are respectively the mean and variance of the feature vectors of different dimensions of the encoding vector corresponding to the current speech at the current layer, H is the hidden layer dimension, k is equal to 1~H; ε is a minimum value.

5. The method according to claim 1, wherein The resolution and completion sub-model uses the following formula to determine the predicted position in the historical discourse: p s =softmax(W s H cln +b s ) p e =softmax(W e H cln +b e ) Among them, p s is the starting position of the predicted position in the historical discourse, p e is the end position of the predicted position in the historical discourse, softmax is the normalized exponential function, W s and W e represents the weight vector, b s and b e represents the bias term, H cln Output variables for the semantics.

6. A training method for a coreference resolution model, characterized in that: include: Acquire multiple dialogue samples, each of which includes a historical utterance, a current utterance, and a reference utterance of the current utterance, wherein the reference utterance is a utterance obtained by completing semantic omissions and / or performing coreference resolution on the current utterance; Obtaining annotated data for the conversation sample, the annotated data including: a real insertion position of a semantically missing word and / or a real deletion position of a word to be replaced in the current utterance, and a real position of the word with the semantically missing word at the real insertion position in the historical utterance and / or a real position of a replacement word corresponding to the word to be replaced at the real deletion position in the historical utterance; Training a coreference resolution detection sub-model of a coreference resolution model to be trained based on the current utterance and the annotated data of the conversation sample to obtain a trained coreference resolution detection sub-model, wherein the trained coreference resolution detection sub-model receives an utterance as input and outputs a predicted insertion position of a semantically missing word and / or a predicted deletion position of a word to be replaced in the input utterance; Based on the annotated data of the conversation sample and the predicted insertion position and / or predicted deletion position of the current utterance of the conversation sample, a resolution and completion sub-model of the coreference resolution model to be trained is trained to obtain a trained resolution and completion sub-model, wherein the input of the trained resolution and completion sub-model is the predicted insertion position and / or predicted deletion position of the utterance, and the output is the predicted position of the semantically missing word at the predicted insertion position in the historical utterance of the input utterance and / or the predicted position of the replacement word of the word to be replaced at the predicted deletion position in the historical utterance of the input utterance; The step of training the resolution and completion sub-model of the coreference resolution model to be trained based on the annotated data of the conversation sample and the predicted insertion position and / or predicted deletion position of the current utterance of the conversation sample includes: Preprocessing step: calculating the average vector of the starting position and the ending position of the predicted insertion position of the current utterance and / or the average vector of the starting position and the ending position of the predicted deletion position; using the average vector as a condition, performing semantic processing on the encoding vector obtained after encoding the current utterance using the transformer encoder using conditional layer normalization to obtain a semantic output variable after conditional layer normalization; Input step: inputting the semantic output variable and the historical discourse of the conversation sample into the resolution and completion sub-model to obtain the predicted position of the semantically missing word at the predicted insertion position in the historical discourse and / or the predicted position of the replacement word corresponding to the word to be replaced at the predicted deletion position in the historical discourse; determining the value of the second objective function based on the predicted position and the actual position in the historical discourse; If the value of the second objective function does not meet the second condition, adjust the parameters of the digestion and completion sub-model and return to the input step until the value of the second objective function meets the second condition, thereby obtaining the trained digestion and completion sub-model.

7. The method according to claim 6, characterized in that Obtaining the labeled data of the conversation sample includes: Matching a substring in the current utterance of the conversation sample with a substring in the reference utterance to obtain a matching result; If the matching result is a substring that only exists in the current utterance, determining the substring as a word that needs to be replaced, and the actual deletion position of the word that needs to be replaced in the current utterance, finding a replacement font corresponding to the word that needs to be replaced in the reference utterance, and marking the actual position of the replacement word corresponding to the word that needs to be replaced at the actual deletion position in the historical utterance in the historical utterance; If the matching result is that a substring exists only in the reference utterance, it is determined that there is a semantic omission in the current utterance, and a real insertion position of the semantic omission in the current utterance is determined.

8. The method according to claim 6, characterized in that Training the coreference resolution detection sub-model of the coreference resolution model to be trained according to the current speech and the annotated data of the conversation sample includes: Input step: inputting the current utterance of the dialogue sample into the coreference resolution detection sub-model to obtain the predicted insertion position of the semantic missing and / or the predicted deletion position of the word to be replaced in the current utterance; Determining a value of a first objective function based on the predicted insertion position and / or predicted deletion position output by the coreference resolution detection sub-model and the actual insertion position and / or actual deletion position of the current utterance; If the value of the first objective function does not meet the first condition, adjust the parameters of the reference resolution detection sub-model and return to the input step until the value of the first objective function meets the first condition, thereby obtaining the trained reference resolution detection sub-model.

9. The method according to claim 8, characterized in that The reference resolution detection sub-model includes two groups of binary classification networks, wherein the input of one group of binary classification networks is the current speech, and the output is a 01 sequence used to indicate whether the position of each word in the current speech is the starting position or the ending position of the predicted insertion position; the input of the other group of binary classification networks is the current speech, and the output is a 01 sequence used to indicate whether the position of each word in the current speech is the starting position or the ending position of the predicted deletion position.

10. The method according to claim 9, characterized in that The binary classification network uses the following formula to determine whether the position of each word in the current utterance is the starting position or the ending position of the predicted insertion position or the predicted deletion position: Among them, l is used to indicate whether it is a predicted insertion position or a predicted deletion position. Indicates the starting position, Indicates the end position, h l [i] is the encoding vector representation of the i-th word in the current utterance after passing through the transformer encoder, and represents the weight vector to be trained, and Represents the bias term to be trained, and σ is the sigmoid activation function.

11. The method according to claim 8, characterized in that The first objective function is expressed by the following formula: Among them, loss tag is the first objective function, n is the length of the current utterance, Indicates the starting position of the predicted insertion position or predicted deletion position, Indicates the end position of the predicted insertion position or predicted deletion position of the current utterance, Indicates the starting position of the actual insertion position or the actual deletion position of the current speech, represents the end position of the actual insertion position or the actual deletion position of the current utterance, i represents the i-th word in the current utterance, and j represents the j-th word in the current utterance.

12. The method according to claim 6, characterized in that The semantic output variable can be expressed as: H cln =λ×(h l-1 -m h ) / (s h +e)+b Among them, H cln Output variables for the semantics; h l-1 is the encoding vector obtained by encoding the current utterance using a transformer encoder; H c is the average vector; are two transformation matrices, which transform the input mean vector dimension into the same dimension as λ and β; H c =avg(h l [s],h l [e]), avg() represents the average function, s represents the starting position, e represents the end position, h l is the encoding vector obtained by encoding the input using the transformer encoder; μ h , σ h are respectively the mean and variance of the feature vectors of different dimensions of the encoding vector corresponding to the current speech at the current layer, H is the hidden layer dimension, k is equal to 1~H; ε is a minimum value.

13. The method according to claim 6, characterized in that The resolution and completion sub-model uses the following formula to determine the predicted position in the historical discourse: p s =softmax(W s H cln +b s ) p e =softmax(W e H cln +b e ) Among them, p s is the starting position of the predicted position in the historical discourse, p e is the end position of the predicted position in the historical discourse, softmax is the normalized exponential function, W s and W e represents the weight vector to be trained, b s and b e Represents the bias term to be trained, H cln Output variables for the semantics.

14. The method according to claim 6, characterized in that The second objective function is expressed by the following formula: Among them, loss mrc is the second objective function, n is the length of the historical discourse, p s represents the starting position of the predicted position of the historical discourse, p e represents the end position of the predicted position of the historical discourse, y s Indicates the starting position of the real position of the historical discourse, y e represents the end position of the real position of the historical discourse, i represents the i-th word in the historical discourse, and j represents the j-th word in the historical discourse.

15. A reference digestion device, characterized in that: include: A first acquisition module is used to acquire the current utterance to be processed; A first processing module is configured to input the current utterance into a coreference resolution detection sub-model of a coreference resolution model to obtain a predicted insertion position of a semantically missing word in the current utterance and / or a predicted deletion position of a word to be replaced; a second processing module, configured to input the predicted insertion position and / or predicted deletion position of the current utterance, and the historical utterance corresponding to the current utterance, into a resolution and completion sub-model of the coreference resolution model, to obtain a predicted position of the semantically missing word at the predicted insertion position in the historical utterance and / or a predicted position of a replacement word corresponding to the word to be replaced at the predicted deletion position in the historical utterance; The predicted insertion position and / or predicted deletion position of the current utterance, as well as the historical utterance corresponding to the current utterance, are input into the resolution and completion sub-model of the coreference resolution model to obtain the predicted position of the semantically missing word at the predicted insertion position in the historical utterance and / or the predicted position of the replacement word corresponding to the word to be replaced at the predicted deletion position in the historical utterance, including: Calculating an average vector of a starting position and an ending position of a predicted insertion position of the current utterance and / or an average vector of a starting position and an ending position of a predicted deletion position; Using the average vector as a condition, semantically processing the encoding vector obtained by encoding the current utterance using a transformer encoder using conditional layer normalization is performed to obtain a semantic output variable after conditional layer normalization; The semantic output variable and the historical discourse of the current discourse are input into the resolution and completion sub-model to obtain the predicted position of the semantically missing word at the predicted insertion position in the historical discourse and / or the predicted position of the replacement word corresponding to the word to be replaced at the predicted deletion position in the historical discourse.

16. A training device for a reference resolution model, characterized in that: include: A first acquisition module is configured to acquire a plurality of dialogue samples, each of which includes a historical utterance, a current utterance, and a reference utterance of the current utterance, wherein the reference utterance is an utterance obtained by completing semantic gaps and / or performing coreference resolution on the current utterance; a second acquisition module, configured to acquire annotated data of the conversation sample, the annotated data including: a real insertion position of a semantically missing word and / or a real deletion position of a word to be replaced in the current utterance, and a real position of the word with the semantically missing word at the real insertion position in the historical utterance and / or a real position of a replacement word corresponding to the word to be replaced at the real deletion position in the historical utterance; A first training module is configured to train a coreference resolution detection sub-model of a coreference resolution model to be trained based on the current utterance and annotated data of the conversation sample to obtain a trained coreference resolution detection sub-model, wherein the trained coreference resolution detection sub-model receives an utterance as input and outputs a predicted insertion position of a semantically missing word and / or a predicted deletion position of a word to be replaced in the input utterance; a second training module, configured to train a resolution and completion sub-model of a coreference resolution model to be trained based on the annotated data of the conversation sample and the predicted insertion position and / or predicted deletion position of the current utterance of the conversation sample, to obtain a trained resolution and completion sub-model, wherein the input of the trained resolution and completion sub-model is the predicted insertion position and / or predicted deletion position of the utterance, and the output is the predicted position of the semantically missing word at the predicted insertion position in the historical utterance of the input utterance and / or the predicted position of the replacement word of the word to be replaced at the predicted deletion position in the historical utterance of the input utterance; The step of training the resolution and completion sub-model of the coreference resolution model to be trained based on the annotated data of the conversation sample and the predicted insertion position and / or predicted deletion position of the current utterance of the conversation sample includes: Preprocessing step: calculating the average vector of the starting position and the ending position of the predicted insertion position of the current utterance and / or the average vector of the starting position and the ending position of the predicted deletion position; using the average vector as a condition, performing semantic processing on the encoding vector obtained after encoding the current utterance using the transformer encoder using conditional layer normalization to obtain a semantic output variable after conditional layer normalization; Input step: inputting the semantic output variable and the historical discourse of the conversation sample into the resolution and completion sub-model to obtain the predicted position of the semantically missing word at the predicted insertion position in the historical discourse and / or the predicted position of the replacement word corresponding to the word to be replaced at the predicted deletion position in the historical discourse; determining the value of the second objective function based on the predicted position and the actual position in the historical discourse; If the value of the second objective function does not meet the second condition, adjust the parameters of the digestion and completion sub-model and return to the input step until the value of the second objective function meets the second condition, thereby obtaining the trained digestion and completion sub-model.

17. An electronic device, characterized in that: The method comprises a processor, a memory, and a program or instruction stored in the memory and executable on the processor, wherein the program or instruction, when executed by the processor, implements the steps of the method for resolving references as described in any one of claims 1 to 5, or implements the steps of the method for training a resolving reference model as described in any one of claims 6 to 14.

18. A readable storage medium, characterized in that The readable storage medium stores a program or instruction, and when the program or instruction is executed by the processor, it implements the reference resolution method as described in any one of claims 1 to 5, or implements the steps of the training method of the reference resolution model as described in any one of claims 6 to 14.

Images