Machine learning model training method, abstract generation method and device

Abstract

The present disclosure provides a machine learning model training method, an abstract generation method and device, and relates to the field of artificial intelligence. The machine learning model training method comprises: supplementing each content-incomplete sentence in a dialogue sample with a preceding sentence of the content-incomplete sentence to generate a corresponding first rewritten sentence; updating the dialogue sample by using the first rewritten sentence to obtain a first dialogue text; processing the first dialogue text by using a machine learning model to generate a first abstract; determining a first loss function according to the first abstract and an abstract sample of the dialogue sample; and training the machine learning model according to the first loss function. The present disclosure can effectively improve the accuracy of dialogue abstracts.

Description

Technical Field

[0001] This disclosure relates to the field of artificial intelligence, and in particular to a machine learning model training method, a summary generation method, and an apparatus. Background Technology

[0002] In existing technologies, summary generation methods mainly include extraction methods and summarization methods. Extraction methods involve extracting multiple important sentences from the text and combining them into a summary. Summarization methods typically employ an end-to-end generative architecture, first using an encoder to map text information into a high-dimensional vector, and then using a decoder to decode from the high-dimensional vector to generate a summary. Summary of the Invention

[0003] The inventors noted that in the extraction method described above, since sentences are directly extracted from the text and pieced together to form a summary without modifying any words, the accuracy of the generated summary is low because the extracted sentences may not necessarily reflect the overall content of the text. In contrast, the summarization method utilizes a machine learning model to understand the text content and summarize it, resulting in a more accurate summary. However, incomplete sentences in the text can affect the accuracy of the generated summary. For example, in everyday conversations, incomplete sentences are common, causing the machine learning model to misunderstand the dialogue and leading to a biased conversation summary.

[0004] Accordingly, this disclosure provides a machine learning model training scheme that can effectively improve the accuracy of dialogue summarization.

[0005] According to a first aspect of the present disclosure, a method for training a machine learning model is provided, comprising: supplementing each incomplete statement in a dialogue sample with the preceding statement of each incomplete statement to generate a corresponding first rewritten statement; updating the dialogue sample with the first rewritten statement to obtain a first dialogue text; processing the first dialogue text with a machine learning model to generate a first summary; determining a first loss function based on the first summary and a summary sample of the dialogue sample; and training the machine learning model based on the first loss function.

[0006] In some embodiments, supplementing each incomplete statement in the dialogue sample with the preceding statement of each incomplete statement in the dialogue sample includes: extracting all the statements to be supplemented in the dialogue sample, and the set of preceding statements corresponding to each statement to be supplemented; using the set of preceding statements corresponding to the t-th statement to be supplemented to supplement the content of the t-th statement to be supplemented, so as to obtain a first rewritten statement corresponding to the t-th statement to be supplemented, where 1≤t≤T, and T is the total number of all the statements to be supplemented.

[0007] In some embodiments, extracting all the statements to be supplemented in the dialogue sample, and the set of preceding statements corresponding to each statement to be supplemented includes: in the dialogue sample, taking the m-th statement as the statement to be supplemented, where 1 < m≤M, and M is the total number of statements in the dialogue sample; taking the first statement to the (m - 1)-th statement as the set of preceding statements of the m-th statement.

[0008] In some embodiments, supplementing the content of the t-th statement to be supplemented includes: generating a first word sequence using all the preceding statements in the set of preceding statements corresponding to the t-th statement to be supplemented; generating a second word sequence using the t-th statement to be supplemented; extracting the vectors of each word in the first word sequence to generate a first vector set, and extracting the vectors of each word in the second word sequence to generate a second vector set; respectively using a plurality of different similarity functions to process each vector in the first vector set and each vector in the second vector set to generate a feature mapping matrix; using a skip connection layer to process the feature mapping matrix to generate an editing matrix M; according to the numerical value of the element M kl in the editing matrix M, using the k-th word in the first word sequence to edit the l-th word in the second word sequence, where 1≤k≤K, K is the total number of words in the first word sequence, and 1≤l≤L, L is the total number of words in the second word sequence.

[0009] In some embodiments, using the k-th word in the first word sequence to edit the l-th word in the second word sequence includes: if the element M kl is within the neighborhood of a first numerical value, inserting the k-th word in the first word sequence behind the l-th word in the second word sequence.

[0010] In some embodiments, using the k-th word in the first word sequence to edit the l-th word in the second word sequence includes: if the element M klThe value of the first word sequence is in the neighborhood of the second value, and the k-th word in the first word sequence is inserted before the l-th word in the second word sequence, wherein the second value is less than the first value.

[0011] In some embodiments, editing the l-th word of the second word sequence using the k-th word of the first word sequence includes: if the element M kl The value is within the neighborhood of the third value, and the k-th word in the second word sequence is replaced by the k-th word in the first word sequence, wherein the third value is less than the second value.

[0012] In some embodiments, editing the l-th word of the second word sequence using the k-th word of the first word sequence includes: if the element M kl If the value is in the neighborhood of the fourth value, no operation is performed, where the fourth value is less than the third value.

[0013] In some embodiments, generating the feature mapping matrix includes: processing the k-th vector in the first vector set and the l-th vector in the second vector set using n distinct similarity functions to obtain n relevance scores for the k-th word in the first word sequence and the l-th word in the second word sequence, where n is a natural number greater than 1; obtaining the relevance score between the k-th word in the first word sequence and the l-th word in the second word sequence based on the n relevance scores; and generating a feature mapping matrix based on the relevance score between each word in the first word sequence and each word in the second word sequence.

[0014] In some embodiments, n is 3; the similarity function includes a dot product function, a cosine similarity calculation function, and a bilinear similarity function.

[0015] In some embodiments, extracting the vectors of each word in the first word sequence to generate a first vector set, and extracting the vectors of each word in the second word sequence to generate a second vector set, includes: fusing the first word sequence and the second word sequence to obtain a third word sequence; and processing the third word sequence using a bidirectional long short-term memory model to generate the first vector set including the vectors of each word in the first word sequence, and the second vector set including the vectors of each word in the second word sequence.

[0016] In some embodiments, the method further includes: using the context of the dialogue sample, replacing each pronoun in the dialogue sample with corresponding name information to generate a corresponding second rewritten statement; updating the dialogue sample using the second rewritten statement to obtain a second dialogue text; combining the first dialogue text and the second dialogue text to generate a third dialogue text; processing the third dialogue text using the machine learning model to generate a second summary; determining a second loss function based on the second summary and the summary sample of the dialogue sample; and training the machine learning model based on the second loss function.

[0017] In some embodiments, replacing each pronoun in the dialogue sample with the corresponding name information includes: counting all name information in the dialogue sample; extracting all coreference chains in the dialogue sample using a preset coreference parsing system; if the number of pronouns J in the i-th coreference chain is not less than a preset number threshold, then randomly replacing the j-th pronoun in the i-th coreference chain with one of the all name information, 1≤i≤I, where I is the total number of coreference chains, 1≤j≤J; if the replacement confidence of the j-th pronoun output by the coreference parsing system is not less than a first preset confidence threshold, then replacing the j-th pronoun with the currently used name information.

[0018] In some embodiments, replacing each pronoun in the dialogue sample with the corresponding name information further includes: if the number J of pronouns in the i-th coreference chain is greater than 0 and less than the quantity threshold, then the j-th pronoun is randomly replaced with one of the name information; if the replacement confidence of the j-th pronoun output by the coreference parsing system is not less than a second preset confidence threshold, then the j-th pronoun is replaced with the currently used name information, wherein the second preset confidence threshold is greater than the first preset confidence threshold.

[0019] In some embodiments, replacing each pronoun in the dialogue sample with the corresponding name information further includes: if the number of pronouns in the i-th coreference chain is 0, then deleting the i-th coreference chain.

[0020] According to a second aspect of the present disclosure, a machine learning model training apparatus is provided, comprising: a first training module configured to supplement each incomplete statement in a dialogue sample with the preceding statement of each incomplete statement to generate a corresponding first rewritten statement, and to update the dialogue sample with the first rewritten statement to obtain a first dialogue text; a second training module configured to process the first dialogue text using a machine learning model to generate a first summary; and a third training module configured to determine a first loss function based on the first summary and a summary sample of the dialogue sample, and to train the machine learning model based on the first loss function.

[0021] According to a third aspect of the present disclosure, a machine learning model training apparatus is provided, comprising: a memory configured to store instructions; and a processor coupled to the memory, the processor being configured to execute instructions stored in the memory to implement the method as described in any of the above embodiments.

[0022] According to a fourth aspect of the present disclosure, a summary generation method is provided, comprising: supplementing each incomplete statement in a dialogue to be processed with the preceding statement of each incomplete statement to generate a corresponding first rewritten statement; updating the dialogue to be processed with the first rewritten statement to obtain a first dialogue text; and processing the first dialogue text with a machine learning model to generate a first summary, wherein the machine learning model is trained using the machine learning model training method described in any of the above embodiments.

[0023] In some embodiments, the method further includes: using the context of the dialogue to be processed, replacing each pronoun in the dialogue to be processed with corresponding name information to generate a corresponding second rewritten statement; using the second rewritten statement to update the dialogue to be processed to obtain a second dialogue text; combining the first dialogue text and the second dialogue text to generate a third dialogue text; and processing the third dialogue text using the machine learning model to generate a second summary.

[0024] According to a fifth aspect of the present disclosure, a summary generation apparatus is provided, comprising: a first processing module configured to supplement each incomplete statement in a dialogue to be processed with the preceding statement of each incomplete statement to generate a corresponding first rewritten statement, and to update the dialogue to be processed with the first rewritten statement to obtain a first dialogue text; and a second processing module configured to process the first dialogue text using a machine learning model to generate a first summary, wherein the machine learning model is trained using the machine learning model training method of any one of claims 1-17.

[0025] According to a sixth aspect of the present disclosure, a summary generation apparatus is provided, comprising: a memory configured to store instructions; and a processor coupled to the memory, the processor being configured to execute instructions stored in the memory to implement the method as described in any of the above embodiments.

[0026] According to a seventh aspect of the present disclosure, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer instructions that, when executed by a processor, implement the method as described in any of the above embodiments.

[0027] Other features and advantages of this disclosure will become clear from the following detailed description of exemplary embodiments with reference to the accompanying drawings. Attached Figure Description

[0028] To more clearly illustrate the technical solutions in the embodiments of this disclosure or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this disclosure. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0029] Figure 1 This is a flowchart illustrating a machine learning model training method according to an embodiment of the present disclosure;

[0030] Figure 2 This is a flowchart illustrating the process of supplementing a statement to be supplemented according to an embodiment of this disclosure;

[0031] Figure 3 This is a schematic diagram of the architecture for supplementing supplementary statements according to an embodiment of this disclosure;

[0032] Figure 4 This is a flowchart illustrating a machine learning model training method according to another embodiment of the present disclosure;

[0033] Figure 5 This is a schematic diagram of the structure of a machine learning model training apparatus according to an embodiment of the present disclosure;

[0034] Figure 6 This is a schematic diagram of the structure of a machine learning model training apparatus according to another embodiment of the present disclosure;

[0035] Figure 7 This is a schematic flowchart of a summary generation method according to an embodiment of the present disclosure;

[0036] Figure 8 This is a flowchart illustrating a summary generation method according to another embodiment of this disclosure;

[0037] Figure 9 This is a schematic diagram of the structure of an abstract generation apparatus according to an embodiment of the present disclosure;

[0038] Figure 10 This is a schematic diagram of the structure of a summary generation apparatus according to another embodiment of the present disclosure. Detailed Implementation

[0039] The technical solutions of the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this disclosure, and not all embodiments. The following description of at least one exemplary embodiment is merely illustrative and is in no way intended to limit this disclosure or its application or use. All other embodiments obtained by those skilled in the art based on the embodiments of this disclosure without creative effort are within the scope of protection of this disclosure.

[0040] Unless otherwise specifically stated, the relative arrangement, numerical expressions, and values ​​of the components and steps set forth in these embodiments do not limit the scope of this disclosure.

[0041] At the same time, it should be understood that, for ease of description, the dimensions of the various parts shown in the accompanying drawings are not drawn according to actual scale.

[0042] Techniques, methods, and equipment known to those skilled in the art may not be discussed in detail, but where appropriate, such techniques, methods, and equipment should be considered part of the specification.

[0043] In all examples shown and discussed herein, any specific values ​​should be interpreted as merely exemplary and not as limitations. Therefore, other examples of exemplary embodiments may have different values.

[0044] It should be noted that similar labels and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be discussed further in subsequent figures.

[0045] The inventors noticed that in everyday conversations, dialogue sentences are often incomplete, which leads to a discrepancy between the dialogue summary generated by the machine learning model and the actual dialogue content.

[0046] For example, Ann and Josh's conversation goes like this:

[0047] Ann: Do you still have laptops?

[0048] Josh: Yes.

[0049] Ann: I'll pay $200.

[0050] Josh: The price is $250 and there are no discounts for laptops.

[0051] Ann: Does it come with a bag? Are there any other accessories?

[0052] Josh: I have a bag and a small USB mouse.

[0053] Ann: Sounds good. I'll take it. Where can I pick it up?

[0054] The original meaning of the above conversation is: Ann wants to buy Josh's laptop for $200. Josh doesn't want to negotiate the price. Ann buys the laptop and accessories for $250.

[0055] Due to the incomplete utterances in the above conversation, when using the existing pre-trained denoising autoencoder BART to process the above conversation, the obtained summary is:

[0056] Ann wants to buy the laptop for $200, but the price is non-negotiable. Ann took Josh's bag.

[0057] From the above comparison, it can be seen that due to the incomplete content of the dialogue statements, such as the last sentence in the above conversation, the summary generated by the machine learning model is inconsistent with the dialogue content.

[0058] Therefore, the present disclosure can obtain a more accurate dialogue summary by supplementing the incomplete statements in the dialogue.

[0059] Figure 1 It is a schematic flowchart of a method for training a machine learning model according to an embodiment of the present disclosure. In some embodiments, the following method for training a machine learning model is executed by a machine learning model training device.

[0060] In step 101, use the preceding sentences of each incomplete statement in the dialogue sample to supplement each incomplete statement to generate a corresponding first rewritten statement. <​​​​​​​​​

[0064] In some embodiments, the above steps for supplementing the content of the t-th statement to be supplemented are as follows: Figure 2 As shown.

[0065] In step 201, the first word sequence is generated using all the preceding statements in the set of preceding statements corresponding to the t-th statement to be supplemented.

[0066] For example, the t-th statement to be supplemented is u t The corresponding set of statements above is C = {u1, ..., u} t-1}. Concatenate all preceding statements in set C into a first word sequence c = {c1, ..., c2}. K The first word sequence contains K words.

[0067] In step 202, the second word sequence is generated using the t-th statement to be supplemented.

[0068] For example, u t ={x1, ..., x L The elements within the parentheses form the second word sequence. The second word sequence contains L words.

[0069] In step 203, the vectors of each word in the first word sequence are extracted to generate a first vector set, and the vectors of each word in the second word sequence are extracted to generate a second vector set.

[0070] In some embodiments, the first word sequence and the second word sequence are fused to obtain a third word sequence. The third word sequence is then processed using BiLSTM (Bidirectional Long Short Term Memory) to generate a first vector set including the word vectors of the first word sequence and a second vector set including the word vectors of the second word sequence, as shown in equation (1) below:

[0071] BiLSTM(c;u) t ) = (g1, ..., g K h1, ..., h L (1)

[0072] Among them, g k For word c in the first word sequence k The vector, h l For word x in the second word sequence l The vector.

[0073] In step 204, each vector in the first vector set and each vector in the second vector set are processed using multiple different similarity functions to generate a feature mapping matrix.

[0074] In some embodiments, n distinct similarity functions are used to analyze the k-th vector g in the first vector set. k The l-th vector h in the second vector set l Process to obtain the k-th word c in the first word sequence. k The l-th word x in the second word sequence l We have n relevance scores, where n is a natural number greater than 1. Next, based on these n relevance scores, we obtain the k-th word c in the first word sequence. k The l-th word x in the second word sequence l The correlation is then determined. A feature mapping matrix is ​​generated based on the correlation between each word in the first word sequence and each word in the second word sequence.

[0075] For example, let n be 3. Similarity functions include the dot product function, the cosine similarity function, and the bilinear similarity function. Then, the k-th word c in the first word sequence... k The l-th word x in the second word sequence l Correlation F(x) l c k As shown in formula (2).

[0076] F(x l c k )=[h l ⊙g k cos(h) l g k );h l W Bi g k (2)

[0077] Where ⊙ is the vector g k sum vector h l The dot product operation between them, where cos is the vector g. k sum vector h l The cosine similarity calculation function between them, W Bi For vector g k sum vector h l The bilinear similarity function between them.

[0078] Based on this, a feature mapping matrix is ​​generated according to the correlation between each word in the first word sequence and each word in the second word sequence.

[0079] In step 205, the feature mapping matrix is ​​processed using a skip connection layer to generate the edit matrix M.

[0080] In some embodiments, the feature mapping matrix F is processed using five CNNs (Convolutional Neural Networks) with skip connections, as shown in equations (3) to (7).

[0081] F′=CNN(F) (3)

[0082] F″=CNN(Pool(F′)) (4)

[0083] F″′=DeConv(CNN(Pool(F″))) (5)

[0084] F″″=DeConv(CNN(F″′,F″)) (6)

[0085] M=FeedForward(CNN(F″″,F′)) (7)

[0086] Among them, CNN() is the convolution operation, Pool() is the MaxPooling operation, DeConv() is the deconvolution operation, and FeedForward() is the feedforward operation.

[0087] It should be noted that while similarity functions can be used to describe the relevance between words from multiple perspectives, they only capture local information. Furthermore, this disclosure utilizes skip connection layers to process the feature mapping matrix, thereby effectively incorporating global information, which is highly beneficial for rewriting incomplete sentences.

[0088] In step 206, based on the element M in the edit matrix M kl The numerical value is used to edit the l-th word in the second word sequence using the k-th word in the first word sequence, where 1≤k≤K, K is the total number of words in the first word sequence, and 1≤l≤L is the total number of words in the second word sequence.

[0089] In some embodiments, editing includes replacement, insertion, and no operation.

[0090] In some embodiments, if element M kl The value of the first value is in the neighborhood of the k-th word c in the first word sequence. k Insert the l-th word x in the second word sequence l Behind.

[0091] For example, the first value is 3.

[0092] In some embodiments, if element M kl The value of the second value is in the neighborhood of the k-th word c in the first word sequence. kInsert the l-th word x in the second word sequence l The first value is less than the second value.

[0093] For example, the second value is 2.

[0094] In some embodiments, if element M kl The value is within the neighborhood of the third value, using the k-th word c in the first word sequence. k Replace the l-th word x in the second word sequence l The third value is less than the second value.

[0095] For example, the third value is 1.

[0096] In some embodiments, if element M kl If the value is in the neighborhood of the fourth value, no operation is performed, where the fourth value is less than the third value.

[0097] For example, the fourth value is 0.

[0098] Through the above processing, the t-th statement to be supplemented, u, can be... t Rewrite to obtain the corresponding rewrite statements.

[0099] In some embodiments, the architecture for generating the edit matrix M is as follows: Figure 3 As shown.

[0100] like Figure 3 In the context representation layer, the vectors of each word in the first word sequence are extracted using the above formula (1) to generate a first vector set, and the vectors of each word in the second word sequence are extracted to generate a second vector set.

[0101] Next, in the local information encoding layer, the k-th word c in the first word sequence is calculated using the above formula (2). k The l-th word x in the second word sequence l Correlation F(x) l c k Based on this, a feature mapping matrix F is generated according to the correlation between each word in the first word sequence and each word in the second word sequence.

[0102] Next, in the global information encoding layer, the feature mapping matrix is ​​processed through the skip connection layer using the above formulas (3) to (7) to generate the edit matrix M.

[0103] After generating the encoding matrix M, if element M kl If the value of M is close to 0, the corresponding operation is no operation. klThe value is close to 1, so the k-th word c in the first word sequence is... k Replace the l-th word x in the second word sequence l If element M kl The value is close to 2, so the k-th word c in the first word sequence is... k Insert the l-th word x in the second word sequence l In front of element M. kl The value is close to 3, so the k-th word c in the first word sequence is... k Insert the l-th word x in the second word sequence l Behind.

[0104] return Figure 1 In step 102, the dialogue sample is updated using the first rewrite statement to obtain the first dialogue text.

[0105] Through the above processing, the t-th statement to be supplemented u t Rewrite to obtain the corresponding rewrite statements. Based on this, rewriting statements For the corresponding statement u in the dialogue sample t Update to obtain the first dialogue text D rew .

[0106] In step 103, the first dialogue text is processed using a machine learning model to generate a first summary.

[0107] For example, the machine learning model is the BART model to be trained.

[0108] In step 104, a first loss function is determined based on the first summary and the summary samples of the dialogue samples.

[0109] In step 105, the machine learning model is trained according to the first loss function.

[0110] In the machine learning model training method provided in the above embodiments of this disclosure, incomplete sentences in the dialogue sample are first supplemented, and then the machine learning model is trained using the supplemented dialogue sample, thereby helping the trained machine learning model to obtain more accurate dialogue summaries.

[0111] For example, Hannah and Amanda's dialogue D-1 is as follows:

[0112] Hannah: Hey, do you have Betty's phone number?

[0113] Amanda: Let me check.

[0114] Amanda: Sorry, I couldn't find it.

[0115] Amanda: Did you ask Larry?

[0116] Amanda: He called her last time.

[0117] Hannah: I don't know him well.

[0118] In the dialogue above, the third and fourth statements are incomplete, which affects the summary generation. By utilizing... Figures 1 to 3 In any embodiment, the above dialogue is processed, and the resulting dialogue content D-2 is as follows.

[0119] Hannah: Hey, do you have Betty's phone number?

[0120] Amanda: Let me check.

[0121] Amanda: Sorry, I couldn't find Betty's phone number.

[0122] Amanda: Did you ask Larry if he had Betty's phone number?

[0123] Amanda: He called her last time.

[0124] Hannah: I don't know him well.

[0125] The processing results show that the third and fourth statements in the dialogue have been completed. This is beneficial for generating a correct summary.

[0126] Figure 4 This is a schematic flowchart illustrating a machine learning model training method according to another embodiment of the present disclosure. In some embodiments, the following machine learning model training method is performed by a machine learning model training apparatus. Figure 4 and Figure 1 The difference is that, in Figure 4 In the illustrated embodiment, a step of replacing pronouns is also added.

[0127] In step 401, each incomplete statement is supplemented using the preceding statement in the dialogue sample to generate the corresponding first rewritten statement.

[0128] In step 402, the dialogue sample is updated using the first rewrite statement to obtain the first dialogue text D. rew .

[0129] In step 403, using the context of the dialogue sample, each pronoun in the dialogue sample is replaced with the corresponding name information to generate the corresponding second rewritten statement.

[0130] In some embodiments, all name information in the dialogue sample is first counted.

[0131] Next, a pre-defined coreference resolution system is used to extract all coreference chains from the dialogue samples. It should be noted that since the pre-defined coreference resolution system is not the inventive point of this invention, it will not be described in detail here.

[0132] If the number of pronouns J in the i-th coreference chain is not less than a preset threshold, then the j-th pronoun in the i-th coreference chain is randomly replaced with one of the name information from all the name information, 1≤i≤I, where I is the total number of coreference chains, and 1≤j≤J. If the replacement confidence of the j-th pronoun output by the coreference parsing system is not less than a first preset confidence threshold, then the j-th pronoun is replaced with the currently used name information.

[0133] For example, the quantity threshold is 5. The first confidence threshold is set to 0.5.

[0134] For example, if the number of pronouns J in the i-th coreference chain is greater than or equal to 5, then each pronoun in the i-th coreference chain is randomly replaced with one of the names in the entire set of names. If the replacement confidence P(x) of the j-th pronoun in the i-th coreference chain output by the coreference parsing system is... ij If the value is greater than or equal to 0.5, then the j-th pronoun will be replaced with the currently used name information.

[0135] In some embodiments, if the number J of pronouns in the i-th coreference chain is greater than 0 and less than a quantity threshold, then the j-th pronoun is randomly replaced with one of the name information. If the replacement confidence of the j-th pronoun output by the coreference parsing system is not less than a second preset confidence threshold, then the j-th pronoun is replaced with the currently used name information, wherein the second preset confidence threshold is greater than the first preset confidence threshold.

[0136] For example, the quantity threshold is 5. The second reliability threshold is set to 0.8.

[0137] For example, if the number of pronouns in the i-th coreference chain is 0 < J < 5, then each pronoun in the i-th coreference chain is randomly replaced with one of the names from all the name information. If the replacement confidence P(x) of the j-th pronoun in the i-th coreference chain output by the coreference parsing system is... ij If the value is greater than or equal to 0.8, then the j-th pronoun will be replaced with the currently used name information.

[0138] In some embodiments, if the number of pronouns in the i-th coreference chain is 0, then the i-th coreference chain is deleted. That is, if there are no pronouns in the i-th coreference chain, then no pronoun replacement processing is required.

[0139] return Figure 4 In step 404, the dialogue sample is updated using the second rewrite statement to obtain the second dialogue text D.aug .

[0140] In step 405, the first dialogue text and the second dialogue text are combined to generate the third dialogue text D. rws .

[0141] In step 406, the third dialogue text is processed using a machine learning model to generate a second summary.

[0142] In step 407, a second loss function is determined based on the second summary and the summary samples of the dialogue samples.

[0143] In step 408, the machine learning model is trained according to the second loss function.

[0144] For example, by replacing the pronouns in the above dialogue D-1 with , we get the following dialogue D-3:

[0145] Hannah: Hey, Amanda, do you have Betty's phone number?

[0146] Amanda: Let me check.

[0147] Amanda: Sorry, I couldn't find it.

[0148] Amanda: Did you ask Larry?

[0149] Amanda: Larry called Betty last time.

[0150] Hannah: I'm not familiar with Larry.

[0151] Therefore, the above processing can replace pronouns in the dialogue with their corresponding names. Next, by merging dialogues D-2 and D-3, a more complete dialogue can be obtained, as shown in dialogue D-4.

[0152] Hannah: Hey, Amanda, do you have Betty's phone number?

[0153] Amanda: Let me check.

[0154] Amanda: Sorry, I couldn't find Betty's phone number.

[0155] Amanda: Did you ask Larry if he had Betty's phone number?

[0156] Amanda: Larry called Betty last time.

[0157] Hannah: I'm not familiar with Larry.

[0158] Through the above processing, incomplete sentences in the dialogue are supplemented, and pronouns used in the dialogue are replaced with their corresponding names, which is more conducive to generating a correct summary.

[0159] Figure 5 This is a schematic diagram of the structure of a machine learning model training apparatus according to an embodiment of this disclosure. Figure 5 As shown, the machine learning model training device includes a first training module 51, a second training module 52, and a third training module 53.

[0160] The first training module 51 is configured to supplement each incomplete sentence in the dialogue sample with the preceding sentence of each incomplete sentence to generate a corresponding first rewritten sentence, and to update the dialogue sample with the first rewritten sentence to obtain the first dialogue text.

[0161] In some embodiments, the first training module 51 extracts all the sentences to be supplemented in the dialogue samples, as well as the set of preceding sentences corresponding to each sentence to be supplemented. Next, using the set of preceding sentences corresponding to the t-th sentence to be supplemented, the t-th sentence to be supplemented is supplemented to obtain the first rewritten sentence corresponding to the t-th sentence to be supplemented, where 1≤t≤T, and T is the total number of all sentences to be supplemented.

[0162] In some embodiments, in a dialogue sample, the m-th statement is taken as the statement to be supplemented, 1 < m ≤ M, where M is the total number of statements in the dialogue sample, and the 1st to the (m-1)th statements are taken as the set of preceding statements of the m-th statement.

[0163] In some embodiments, the first training module 51 generates a first word sequence using all preceding statements in the set of preceding statements corresponding to the t-th statement to be supplemented.

[0164] For example, the t-th statement to be supplemented is u t The corresponding set of statements above is C = {u1, ..., u} t-1}. Concatenate all preceding statements in set C into a first word sequence c = {c1, ..., c2}. K The first word sequence contains K words.

[0165] Next, the first training module 51 uses the t-th sentence to be supplemented to generate the second word sequence.

[0166] For example, u t ={x1, ..., x L The elements within the parentheses form the second word sequence. The second word sequence contains L words.

[0167] Next, the first training module 51 extracts the vectors of each word in the first word sequence to generate a first vector set, and extracts the vectors of each word in the second word sequence to generate a second vector set.

[0168] In some embodiments, the first training module 51 fuses the first word sequence and the second word sequence to obtain a third word sequence. The third word sequence is then processed using BiLSTM to generate a first vector set including the word vectors of the first word sequence and a second vector set including the word vectors of the second word sequence, as shown in equation (1) above.

[0169] Next, the first training module 51 processes each vector in the first vector set and each vector in the second vector set using multiple different similarity functions to generate a feature mapping matrix.

[0170] In some embodiments, the first training module 51 uses n distinct similarity functions to train the k-th vector g in the first vector set. k The l-th vector h in the second vector set l Process to obtain the k-th word c in the first word sequence. k The l-th word x in the second word sequence l We have n relevance scores, where n is a natural number greater than 1. Next, based on these n relevance scores, we obtain the k-th word c in the first word sequence. k The l-th word x in the second word sequence l The correlation is then determined. A feature mapping matrix is ​​generated based on the correlation between each word in the first word sequence and each word in the second word sequence.

[0171] For example, let n be 3. Similarity functions include the dot product function, the cosine similarity function, and the bilinear similarity function.

[0172] Based on this, a feature mapping matrix is ​​generated according to the correlation between each word in the first word sequence and each word in the second word sequence.

[0173] Next, the first training module 51 processes the feature mapping matrix using a skip connection layer to generate an edit matrix M.

[0174] In some embodiments, the feature mapping matrix F is processed using five CNNs (Convolutional Neural Networks) with skip connections, as shown in equations (3) to (7).

[0175] Next, the first training module 51 adjusts the elements M in the edit matrix M.kl The numerical value is used to edit the l-th word in the second word sequence using the k-th word in the first word sequence, where 1≤k≤K, K is the total number of words in the first word sequence, and 1≤l≤L is the total number of words in the second word sequence.

[0176] In some embodiments, editing includes replacement, insertion, and no operation.

[0177] In some embodiments, if element M kl The value of the first value is in the neighborhood of the k-th word c in the first word sequence. k Insert the l-th word x in the second word sequence l Behind.

[0178] For example, the first value is 3.

[0179] In some embodiments, if element M kl The value of the second value is in the neighborhood of the k-th word c in the first word sequence. k Insert the l-th word x in the second word sequence l The first value is less than the second value.

[0180] For example, the second value is 2.

[0181] In some embodiments, if element M kl The value is within the neighborhood of the third value, using the k-th word c in the first word sequence. k Replace the l-th word x in the second word sequence l The third value is less than the second value.

[0182] For example, the third value is 1.

[0183] In some embodiments, if element M kl If the value is in the neighborhood of the fourth value, no operation is performed, where the fourth value is less than the third value.

[0184] For example, the fourth value is 0.

[0185] Through the above processing, the t-th statement to be supplemented, u, can be... t Rewrite to obtain the corresponding rewrite statements.

[0186] Through the above processing, the t-th statement to be supplemented u t Rewrite to obtain the corresponding rewrite statements. Based on this, rewriting statements For the corresponding statement u in the dialogue sample t Update to obtain the first dialogue text Drew .

[0187] The second training module 52 is configured to process the first dialogue text using a machine learning model to generate a first summary.

[0188] For example, the machine learning model is the BART model to be trained.

[0189] The third training module 53 is configured to determine a first loss function based on the first summary and the summary samples of the dialogue samples, and to train the machine learning model based on the first loss function.

[0190] In some embodiments, the first training module 51 uses the context of the dialogue sample to replace each pronoun in the dialogue sample with the corresponding name information to generate the corresponding second rewritten statement.

[0191] For example, first, collect all the name information in the dialogue sample.

[0192] Next, a pre-defined coreference resolution system is used to extract all coreference chains from the dialogue samples. It should be noted that since the pre-defined coreference resolution system is not the inventive point of this invention, it will not be described in detail here.

[0193] If the number of pronouns J in the i-th coreference chain is not less than a preset threshold, then the j-th pronoun in the i-th coreference chain is randomly replaced with one of the name information from all the name information, 1≤i≤I, where I is the total number of coreference chains, and 1≤j≤J. If the replacement confidence of the j-th pronoun output by the coreference parsing system is not less than a first preset confidence threshold, then the j-th pronoun is replaced with the currently used name information.

[0194] For example, the quantity threshold is 5. The first confidence threshold is set to 0.5.

[0195] For example, if the number of pronouns J in the i-th coreference chain is greater than or equal to 5, then each pronoun in the i-th coreference chain is randomly replaced with one of the names in the entire set of names. If the replacement confidence P(x) of the j-th pronoun in the i-th coreference chain output by the coreference parsing system is... ij If the value is greater than or equal to 0.5, then the j-th pronoun will be replaced with the currently used name information.

[0196] In some embodiments, if the number J of pronouns in the i-th coreference chain is greater than 0 and less than a quantity threshold, then the j-th pronoun is randomly replaced with one of the name information. If the replacement confidence of the j-th pronoun output by the coreference parsing system is not less than a second preset confidence threshold, then the j-th pronoun is replaced with the currently used name information, wherein the second preset confidence threshold is greater than the first preset confidence threshold.

[0197] For example, the quantity threshold is 5. The second reliability threshold is set to 0.8.

[0198] For example, if the number of pronouns in the i-th coreference chain is 0 < J < 5, then each pronoun in the i-th coreference chain is randomly replaced with one of the names from all the name information. If the replacement confidence P(x) of the j-th pronoun in the i-th coreference chain output by the coreference parsing system is... ij If the value is greater than or equal to 0.8, then the j-th pronoun will be replaced with the currently used name information.

[0199] In some embodiments, if the number of pronouns in the i-th coreference chain is 0, then the i-th coreference chain is deleted. That is, if there are no pronouns in the i-th coreference chain, then no pronoun replacement processing is required.

[0200] Next, the first training module 51 updates the dialogue samples using the second rewritten statements to obtain the second dialogue text D. aug .

[0201] The second training module 52 combines the first and second dialogue texts to generate the third dialogue text D. rws The third dialogue text is processed using a machine learning model to generate a second summary.

[0202] The third training module 53 determines the second loss function based on the second summary and the summary samples of the dialogue samples, and trains the machine learning model based on the second loss function.

[0203] Figure 6 This is a schematic diagram of the structure of a machine learning model training apparatus according to another embodiment of this disclosure. Figure 6 As shown, the machine learning model training device includes a memory 61 and a processor 62.

[0204] Memory 61 is used to store instructions, and processor 62 is coupled to memory 61. Processor 62 is configured to execute instructions based on the memory storage, as shown in the example. Figure 1 , 2 The method involved in any of the embodiments in 4.

[0205] like Figure 6 As shown, the machine learning model training device also includes a communication interface 63 for exchanging information with other devices. Additionally, the machine learning model training device includes a bus 64, through which the processor 62, communication interface 63, and memory 61 communicate with each other.

[0206] Memory 61 may include high-speed RAM, and may also include non-volatile memory, such as at least one disk drive. Memory 61 may also be a memory array. Memory 61 may also be divided into blocks, and the blocks may be combined into virtual volumes according to certain rules.

[0207] Furthermore, processor 62 may be a central processing unit (CPU), an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement embodiments of the present disclosure.

[0208] This disclosure also relates to a computer-readable storage medium storing computer instructions that, when executed by a processor, implement... Figure 1 , 2 The method involved in any of the embodiments in 4.

[0209] Figure 7 This is a schematic flowchart illustrating a summary generation method according to an embodiment of the present disclosure. In some embodiments, the following summary generation method is performed by a summary generation apparatus.

[0210] In step 701, each incomplete statement in the dialogue to be processed is supplemented using the preceding statement of each incomplete statement to generate the corresponding first rewritten statement.

[0211] For example, the solution provided in the above embodiments of this disclosure can be used to supplement each incomplete sentence.

[0212] In step 702, the dialogue to be processed is updated using the first rewrite statement to obtain the first dialogue text.

[0213] In step 703, the first dialogue text is processed using a machine learning model to generate a first summary, wherein the machine learning model utilizes... Figure 5 or Figure 6 The machine learning model is trained using the training method described in any embodiment.

[0214] In the summary generation method provided in the above embodiments of this disclosure, by supplementing incomplete sentences in the dialogue to be processed, the machine learning model can generate a more accurate dialogue summary.

[0215] Figure 8 This is a schematic flowchart illustrating a summary generation method according to another embodiment of the present disclosure. In some embodiments, the following summary generation method is performed by a summary generation apparatus.

[0216] In step 801, each incomplete statement in the dialogue to be processed is supplemented using the preceding statement of each incomplete statement to generate the corresponding first rewritten statement.

[0217] For example, the solution provided in the above embodiments of this disclosure can be used to supplement each incomplete sentence.

[0218] In step 802, the dialogue to be processed is updated using the first rewrite statement to obtain the first dialogue text.

[0219] In step 803, using the context of the dialogue to be processed, each pronoun in the dialogue to be processed is replaced with the corresponding name information to generate the corresponding second rewritten statement.

[0220] For example, the solution provided in the above embodiments of this disclosure can be used to replace pronouns in a statement.

[0221] In step 804, the dialogue to be processed is updated using the second rewrite statement to obtain the second dialogue text.

[0222] In step 805, the first dialogue text and the second dialogue text are combined to generate the third dialogue text.

[0223] In step 806, the third dialogue text is processed using a machine learning model to generate a second summary. The machine learning model utilizes... Figure 5 or Figure 6 The machine learning model is trained using the training method described in any embodiment.

[0224] In the summary generation method provided in the above embodiments of this disclosure, by supplementing incomplete sentences in the dialogue to be processed and replacing pronouns in the sentences, the machine learning model can generate more accurate dialogue summaries.

[0225] Figure 9 This is a schematic diagram of the structure of a summary generation apparatus according to an embodiment of this disclosure. Figure 9 As shown, the abstract generation device includes a first processing module 91 and a second processing module 92.

[0226] The first processing module 91 is configured to supplement each incomplete statement in the dialogue to be processed using the preceding statement of each incomplete statement to generate a corresponding first rewritten statement, and to update the dialogue to be processed using the first rewritten statement to obtain the first dialogue text.

[0227] The second processing module 92 is configured to process the first dialogue text using a machine learning model to generate a first summary, wherein the machine learning model utilizes... Figure 5 or Figure 6 The machine learning model is trained using the training method described in any embodiment.

[0228] In some embodiments, the first processing module 91 further utilizes the context of the dialogue to be processed to replace each pronoun in the dialogue with the corresponding name information to generate a corresponding second rewritten statement, and uses the second rewritten statement to update the dialogue to be processed to obtain the second dialogue text.

[0229] The second processing module 92 combines the first and second dialogue texts to generate a third dialogue text, and then processes the third dialogue text using a machine learning model to generate a second summary. The machine learning model utilizes... Figure 5 or Figure 6 The machine learning model is trained using the training method described in any embodiment.

[0230] Figure 10 This is a schematic diagram of the structure of a summary generation apparatus according to another embodiment of this disclosure. Figure 10 As shown, the abstract generation device includes a memory 1001, a processor 1002, a communication interface 1003, and a bus 1004. Figure 10 and Figure 6 The difference is that, in Figure 10 In the illustrated embodiment, the processor 1002 is configured to implement memory-based instruction execution as follows: Figure 7 or Figure 8 The method involved in any of the embodiments.

[0231] This disclosure also relates to a computer-readable storage medium storing computer instructions that, when executed by a processor, implement... Figure 7 or Figure 8 The method involved in any of the embodiments.

[0232] The following specific examples illustrate this disclosure.

[0233] The dialogue between Mia and Tom is as follows:

[0234] Mia: Can anyone help me buy a plane ticket? I don't have a credit card right now.

[0235] Tom: You can use mine!

[0236] Mia: Should I send you the link to buy the plane tickets?

[0237] Tom: Just give me the flight numbers, company names, and your personal information that I might need.

[0238] Mia: That's great, thank you Tom.

[0239] By supplementing the incomplete statements using the solution provided in the above embodiments of this disclosure, the resulting dialogue H1 is as follows:

[0240] Mia: Can anyone help me buy a plane ticket? I don't have a credit card right now.

[0241] Tom: You can use my credit card to help Mia buy a plane ticket!

[0242] Mia: Should I send you the link to buy the plane tickets?

[0243] Tom: Just give me the data I may need to buy a ticket, including the flight, the company, and your personal information.

[0244] Mia: Great, thank you, Tom.

[0245] Next, through the solution provided by the above embodiments of the present disclosure, further replace the pronouns in the sentence, and the obtained dialogue H2 is as follows:

[0246] Mia: Can anyone help Mia buy a ticket? Mia doesn't have a credit card now.

[0247] Tom: Mia can use Tom's credit card to help Mia buy a ticket!

[0248] Mia: Should Mia send Tom the link to buy a ticket?

[0249] Tom: Tom may need the data to buy a ticket. Just give Tom the flight, the company, and Mia's personal information.

[0250] Mia: Great, thank you, Tom.

[0251] Next, input the above dialogue H2 into the trained abstract generation model BART for beam search encoding operation. For example, the three words with the highest probability of the first decoded beam = 3 are "Mi, She, Tang". Respectively use these three words as the input of the decoder for the next decoding. Taking "Mi" as the input, we get "Ya, Tang, Fan"; taking "She" as the input, we get "De, Shi, Men"; taking "Tang" as the input, we get "Mu, Cai, Shui". Comprehensively compare the probabilities of "Mia, Rice soup, Rice, Her, She is, They, Tom, Soup with vegetables, Soup", and select the top 3 words "Mia, Tom, Her" as the root nodes to continue the next decoding. The finally generated dialogue abstract is:

[0252] "Mia doesn't have a credit card now. Tom will use his card to buy a ticket for her."

[0253] Obviously, the generated dialogue abstract accurately reflects the above dialogue content.

[0254] In some embodiments, the functional unit modules described above may be implemented as general-purpose processors, programmable logic controllers (PLCs), digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, or any suitable combination thereof for performing the functions described herein.

[0255] Those skilled in the art will understand that all or part of the steps of the above embodiments can be implemented by hardware or by a program instructing related hardware. The program can be stored in a computer-readable storage medium, such as a read-only memory, a disk, or an optical disk.

[0256] The description in this disclosure is provided for illustrative and descriptive purposes only and is not intended to be exhaustive or to limit the disclosure to its forms. Many modifications and variations will be apparent to those skilled in the art. The embodiments were chosen and described in order to better illustrate the principles and practical application of this disclosure and to enable those skilled in the art to understand this disclosure and to design various embodiments with various modifications suitable for a particular purpose.

Claims

1. A method for training a machine learning model, comprising: Each incomplete statement in the dialogue sample is supplemented by the preceding statement to generate the corresponding first rewritten statement. The dialogue sample is updated using the first rewrite statement to obtain the first dialogue text; The first dialogue text is processed using a machine learning model to generate a first summary; A first loss function is determined based on the first summary and the summary sample of the dialogue sample; The machine learning model is trained based on the first loss function; The step of supplementing each incomplete statement with the preceding statement in the dialogue sample includes: Extract all the sentences to be supplemented from the dialogue sample, as well as the set of preceding sentences corresponding to each sentence to be supplemented; The content of the tth statement to be supplemented is supplemented by using the set of preceding statements corresponding to the tth statement to be supplemented, to obtain a first rewritten statement corresponding to the tth statement to be supplemented, wherein T is the total number of the total statements to be supplemented. The process of supplementing the content of the t-th statement to be supplemented includes: Generate a first word sequence using all preceding statements in the set of preceding statements corresponding to the t-th statement to be supplemented; The second word sequence is generated using the t-th sentence to be supplemented; Extract the vectors of each word in the first word sequence to generate a first vector set, and extract the vectors of each word in the second word sequence to generate a second vector set; Each vector in the first vector set and each vector in the second vector set are processed using multiple distinct similarity functions to generate a feature mapping matrix; The feature mapping matrix is ​​processed using a skip connection layer to generate an edit matrix M; Based on the elements in the edit matrix M The numerical value is used to edit the l-th word of the second word sequence using the k-th word of the first word sequence. K is the total number of words in the first word sequence. L is the total number of words in the second word sequence.

2. The method according to claim 1, wherein, The step of extracting all the sentences to be supplemented in the dialogue sample, and the set of preceding sentences corresponding to each sentence to be supplemented, includes: In the dialogue sample, the m-th statement is designated as the statement to be supplemented. M is the total number of statements in the dialogue sample; The first to the (m-1)th statements are taken as the set of preceding statements of the m-th statement.

3. The method according to claim 1, wherein, The step of editing the l-th word of the second word sequence using the k-th word of the first word sequence includes: If the element If the value is in the neighborhood of the first value, insert the k-th word in the first word sequence after the l-th word in the second word sequence.

4. The method according to claim 3, wherein, The step of editing the l-th word of the second word sequence using the k-th word of the first word sequence includes: If the element The value of the first word sequence is in the neighborhood of the second value, and the k-th word in the first word sequence is inserted before the l-th word in the second word sequence, wherein the second value is less than the first value.

5. The method according to claim 4, wherein, The step of editing the l-th word of the second word sequence using the k-th word of the first word sequence includes: If the element The value is within the neighborhood of the third value, and the k-th word in the second word sequence is replaced by the k-th word in the first word sequence, wherein the third value is less than the second value.

6. The method according to claim 5, wherein, The step of editing the l-th word of the second word sequence using the k-th word of the first word sequence includes: If the element If the value is in the neighborhood of the fourth value, no operation is performed, where the fourth value is less than the third value.

7. The method according to claim 1, wherein, The generated feature mapping matrix includes: The k-th vector in the first vector set and the l-th vector in the second vector set are processed using n distinct similarity functions respectively to obtain n relevance scores of the k-th word in the first word sequence and the l-th word in the second word sequence, where n is a natural number greater than 1; The relevance between the k-th word in the first word sequence and the l-th word in the second word sequence is obtained based on the n relevance scores. A feature mapping matrix is ​​generated based on the correlation between each word in the first word sequence and each word in the second word sequence.

8. The method according to claim 7, wherein, The value of n is 3; The similarity functions include the dot product function, the cosine similarity calculation function, and the bilinear similarity function.

9. The method according to claim 1, wherein, The step of extracting the vectors of each word in the first word sequence to generate a first vector set, and extracting the vectors of each word in the second word sequence to generate a second vector set, includes: The first word sequence and the second word sequence are merged to obtain the third word sequence; The third word sequence is processed using a bidirectional long short-term memory model to generate a first vector set including the word vectors of the first word sequence and a second vector set including the word vectors of the second word sequence.

10. The method according to any one of claims 1-9, further comprising: Using the context of the dialogue sample, each pronoun in the dialogue sample is replaced with the corresponding name information to generate the corresponding second rewritten statement; The dialogue sample is updated using the second rewrite statement to obtain the second dialogue text; The first dialogue text and the second dialogue text are combined to generate the third dialogue text; The third dialogue text is processed using the machine learning model to generate a second summary; A second loss function is determined based on the second summary and the summary sample of the dialogue sample; The machine learning model is trained according to the second loss function.

11. The method according to claim 10, wherein, Replacing each pronoun in the dialogue sample with its corresponding name information includes: Collect all name information from the dialogue sample; Extract all coreference chains in the dialogue sample using a pre-defined coreference resolution system; If the number of pronouns J in the i-th coreference chain is not less than a preset threshold, then the j-th pronoun in the i-th coreference chain is randomly replaced with one of the names in the entire set of names. I represents the total number of common-pointing chains. ; If the replacement confidence of the j-th pronoun output by the coreference parsing system is not less than the first preset confidence threshold, then the j-th pronoun is replaced with the currently used name information.

12. The method according to claim 11, wherein, Replacing each pronoun in the dialogue sample with its corresponding name information also includes: If the number J of pronouns in the i-th coreference chain is greater than 0 and less than the quantity threshold, then the j-th pronoun is randomly replaced with one of the entire name information; If the replacement confidence of the j-th pronoun output by the coreference parsing system is not less than the second preset confidence threshold, then the j-th pronoun is replaced with the currently used name information, wherein the second preset confidence threshold is greater than the first preset confidence threshold.

13. The method according to claim 12, wherein, Replacing each pronoun in the dialogue sample with its corresponding name information also includes: If the number of pronouns in the i-th coreference chain is 0, then the i-th coreference chain is deleted.

14. A machine learning model training device, comprising: The first training module is configured to supplement each incomplete statement in the dialogue sample using the preceding statements, thereby generating a corresponding first rewritten statement. The dialogue sample is then updated using the first rewritten statement to obtain a first dialogue text. Specifically, all statements to be supplemented in the dialogue sample, and a set of preceding statements corresponding to each statement to be supplemented, are extracted. The t-th statement to be supplemented is supplemented using the set of preceding statements corresponding to the t-th statement to obtain the first rewritten statement corresponding to the t-th statement to be supplemented. T represents the total number of sentences to be supplemented. A first word sequence is generated using all preceding sentences in the set of preceding sentences corresponding to the t-th sentence to be supplemented. A second word sequence is generated using the t-th sentence to be supplemented. Vectors of each word in the first word sequence are extracted to generate a first vector set. Vectors of each word in the second word sequence are extracted to generate a second vector set. Multiple distinct similarity functions are used to process each vector in the first and second vector sets to generate a feature mapping matrix. A skip connection layer is used to process the feature mapping matrix to generate an edit matrix M. The elements of the edit matrix M are then used to generate the edit matrix M. The numerical value is used to edit the l-th word of the second word sequence using the k-th word of the first word sequence. K is the total number of words in the first word sequence. L is the total number of words in the second word sequence; The second training module is configured to process the first dialogue text using a machine learning model to generate a first summary; The third training module is configured to determine a first loss function based on the first summary and the summary samples of the dialogue samples, and to train the machine learning model based on the first loss function.

15. A machine learning model training device, comprising: The memory is configured to store instructions; A processor, coupled to a memory, configured to implement the method as described in any one of claims 1-13 based on memory-stored instruction execution.

16. A method for generating a summary, comprising: Each incomplete statement in the dialogue to be processed is supplemented by the preceding statement to generate the corresponding first rewritten statement. The dialogue to be processed is updated using the first rewrite statement to obtain the first dialogue text; The first dialogue text is processed using a machine learning model to generate a first summary, wherein the machine learning model is trained using the machine learning model training method of any one of claims 1-13.

17. The method of claim 16, further comprising: Using the context of the dialogue to be processed, each pronoun in the dialogue to be processed is replaced with the corresponding name information to generate the corresponding second rewritten statement; The dialogue to be processed is updated using the second rewrite statement to obtain the second dialogue text; The first dialogue text and the second dialogue text are combined to generate the third dialogue text; The third dialogue text is processed using the machine learning model to generate a second summary.

18. A summary generation apparatus, comprising: The first processing module is configured to supplement each incomplete statement in the dialogue to be processed by using the preceding statement of each incomplete statement to generate a corresponding first rewritten statement, and update the dialogue to be processed by using the first rewritten statement to obtain the first dialogue text. The second processing module is configured to process the first dialogue text using a machine learning model to generate a first summary, wherein the machine learning model is trained using the machine learning model training method of any one of claims 1-13.

19. A summary generation apparatus, comprising: The memory is configured to store instructions; A processor, coupled to a memory, is configured to implement the method of claim 16 or 17 based on memory-stored instruction execution.

20. A computer-readable storage medium, wherein, The computer-readable storage medium stores computer instructions that, when executed by a processor, implement the method as described in any one of claims 1-13, 16, and 17.

Images