A dialogue summarization method for specific roles based on contrastive learning

Abstract

The application provides a specific role-oriented dialogue summary method based on contrast learning, and relates to the technical field of specific role-oriented dialogue summary. The method first acquires dialogue summary data sets and performs preprocessing; then constructs a specific role-oriented dialogue summary model to generate summaries of different roles; then constructs an interactive perception-based contrast learning task to assist the dialogue summary model in coding the dialogue; adopts a multi-task learning manner to train the dialogue summary model by taking the interactive perception task as an auxiliary task and the dialogue summary task; and finally tests the dialogue summary model effect. The method overcomes the problem that the existing dialogue summary method cannot fully capture the interaction between different roles, and uses an interactive perception-based contrast learning target to assist the encoder of the dialogue summary model to understand the interaction mode between different roles, helping the dialogue summary model to generate more accurate and concise summaries.

Description

Technical Field

[0001] This invention relates to the field of dialogue summarization technology for specific roles, and more particularly to a dialogue summarization method for specific roles based on contrastive learning. Background Technology

[0002] With the development of online media technology, people are exposed to massive amounts of dialogue data every day, including casual conversations, meeting dialogues, customer service conversations, and so on. It is difficult to extract key information from this vast amount of data. Dialogue summarization technology is a special text summarization technique that can compress a lengthy dialogue into a concise summary, helping people quickly obtain key information. In many real-world dialogue scenarios (such as customer service conversations), each speaker has a specific role (e.g., user and customer service representative). Therefore, generating dialogue summaries tailored to a specific role is sometimes even more important than generating summaries for the entire dialogue.

[0003] In traditional text summarization tasks such as news summarization and scientific literature summarization, pre-trained seq2seq models like BART have achieved great success. However, due to the unstructured and multi-speaker nature of dialogues, directly applying these models to dialogue summarization tasks is not ideal. Many existing dialogue summarization methods, based on these models, introduce additional dialogue-related information, such as topic information, referential information, discourse relation information, and common sense information, to help the model better understand the structure of the dialogue. However, existing methods usually only focus on summarizing the entire dialogue and cannot meet the needs of certain specific scenarios. Recently, a small number of works have begun to focus on dialogue summarization tasks oriented towards specific roles, hoping to generate a summary for each role while generating a summary of the entire dialogue, reflecting each role's purpose and perspective.

[0004] Chinese patent CN115905513B designs a dialogue summarization method based on denoising question-answering for role-specific dialogue summarization tasks. This patent first uses a user summarization module to generate user summaries, then concatenates these user summaries with the dialogue as questions and sends them to a customer service summarization module. This question-answering modeling method generates customer service summaries. When generating customer service summaries, this patent fully considers the relationship between customer service summaries and user summaries, making the generated customer service summaries more closely matched to the user summaries.

[0005] However, this invention requires training two models to generate user summaries and customer service summaries separately, resulting in greater computational overhead and failing to fully share information from summaries of different roles. Secondly, the invention only considers the impact of user summaries on customer service summaries, but not the impact of customer service summaries on user summaries. In fact, customer service summaries can also aid in user summary generation. Furthermore, the invention cannot adequately capture the interactions between different roles. When generating a summary for a particular role, the model tends to focus only on that role's statements, ignoring key information from other roles, hindering the generation of concise and accurate summaries. Summary of the Invention

[0006] The technical problem to be solved by the present invention is to address the shortcomings of the prior art by providing a dialogue summarization method based on contrastive learning for specific roles, which fully captures the interaction information between different roles and generates concise and accurate summaries for the entire dialogue and each role in the dialogue.

[0007] To solve the above-mentioned technical problems, the technical solution adopted by the present invention is: a dialogue summarization method based on contrastive learning for specific roles, comprising the following steps:

[0008] Step 1: Acquisition and preprocessing of dialogue summary dataset;

[0009] Step 1.1: Obtain the dialogue summary dataset: Obtain the customer service dialogue summary dataset and divide it into training set, validation set and test set; each dialogue in the dataset contains two roles: user and customer service; in addition to the overall dialogue summary, the dataset also provides a summary for the user and a summary for the customer service.

[0010] Step 1.2: Dataset preprocessing: Preprocess all dialogue samples in the training set, validation set, and test set;

[0011] Step 1.2.1: Training Sample Preprocessing: Each dialogue contains three summaries: user summary, customer service summary, and overall summary; [The text then abruptly shifts to a different topic:] ...and use three special symbols... <ubos> ， <abos> ， <fbos>Append them to the beginning of the user summary, customer service summary and overall summary respectively as the start symbol for decoding; for the training set data, combine each dialogue in the training set with the corresponding user summary, customer service summary and overall summary to form a dialogue-summary pair, and increase the original number of training samples to three times the original number.

[0012] Step 1.2.2: Dialogue Preprocessing: First, remove URL links and garbled characters from the dialogue; then, merge multiple consecutive statements from one speaker into one statement, and prepend each speaker's role to the beginning of their corresponding statement to distinguish between different roles; next, concatenate all the statements in the dialogue together, using a special symbol [SEP] to separate different statements; use Bert Tokenizer to segment the concatenated statement sequence, mapping each character to a corresponding index using a predefined vocabulary; if the length of the statement sequence exceeds a set threshold, it is truncated, thus obtaining the dialogue sequence X = {x1,…,x...} n }, where x n The nth word in the dialogue sequence;

[0013] Step 1.2.3: Summary Preprocessing: Use Bert Tokenizer to segment the summary, mapping each character to its corresponding index using a predefined vocabulary; if the length of the resulting summary sequence exceeds a set threshold, it is truncated to obtain the reference summary sequence Y = {y1,…,y}. t };

[0014] Step 2: Construct a dialogue summarization model for specific roles to generate summaries for different roles; the dialogue summarization model is based on a pre-trained BART model, including a dialogue encoder and a role-specific summarization decoder, and the specific steps are as follows;

[0015] Step 2.1: Feed the preprocessed dialogue sequence X into the dialogue encoder based on the BART model, and obtain the semantic vector representation H of the dialogue context through self-attention and feedforward neural network;

[0016] Step 2.2: The reference summary sequence Y obtained after summary preprocessing is fed into the summary decoder based on the BART model. Different decoder start symbols are used to control the decoder to generate summaries with different roles. The hidden vector representation of the decoder is obtained through self-attention, cross-attention and feedforward neural network, and then the probability distribution of generated words is obtained through a linear layer. The training loss is calculated using the cross-entropy function.

[0017] Step 3: Construct an interaction-aware contrastive learning task to assist the dialogue summarization model in encoding dialogue; based on the dialogue vector representation output by the encoder, an interaction-aware contrastive learning objective is designed to assist the dialogue summarization model in capturing the interactions between different roles. The specific method is as follows:

[0018] Step 3.1: Separate the self-vector representation of each role from the dialogue semantic vector representation output by the encoder; first, define two binary role mask vectors, the user role mask vector m u The mask vector m for the customer service role a If a character comes from a user statement, then m u Set the element at the corresponding position in the m column to 1, otherwise set it to 0; if a character comes from a customer service statement, then set m... a The element at the corresponding position is set to 1, otherwise it is set to 0; next, the self-vector representation of each character is separated from the dialogue semantic vector representation output by the encoder, as shown in the following formula:

[0019]

[0020]

[0021] in, For the user's self-vector representation, The self-vector representation of customer service, where ⊙ represents element-wise product;

[0022] Step 3.2: Calculate the cross vector representation for each character;

[0023] First, calculate the attention score matrix for users and customer service representatives using the following formula;

[0024]

[0025]

[0026] Among them, A u2a A a2u These are the attention score matrices for users and customer service representatives, respectively; d represents the dimension of the model's hidden vector representation, and M... u2a and M a2u These are two attention mask matrices, as shown in the following formula:

[0027]

[0028]

[0029] in, and Representing m respectively u and m a The j-th element, I i and I j These represent the indices of the statements containing the i-th and j-th characters, respectively. w is a hyperparameter used to mask out attention between statements that are more than a set threshold apart.

[0030] Next, a cross vector representation is obtained for each character according to the following formula:

[0031]

[0032]

[0033] Among them, C u C a These represent the cross vectors of the user and customer service representatives, respectively.

[0034] Step 3.3: Calculate the interaction perception vector representation for each character;

[0035] The self-vector representation of each character obtained in step 3.1 and the cross-vector representation of each character obtained in step 3.2 are fused together through a linear layer to obtain an interaction-aware vector representation for each character, as shown in the following formula:

[0036]

[0037]

[0038] in, These represent the interaction perception vectors of the user and customer service representatives, respectively. W represents splicing. u and W a Represents the learnable weight matrix;

[0039] Step 3.4: Construct positive and negative examples; use the original dialogue as a positive example; disrupt the interaction between different roles in the dialogue to construct negative examples, specifically: first, fix all statements of the user role in the dialogue, and replace all statements of the customer service role with statements from other customer service roles in the dialogue; repeat this process to construct K negative examples for each positive example; fix the statements of the customer service role and replace the statements of the user role, and obtain K more negative examples; finally, a total of 2K+1 examples are obtained, with the first example representing a positive example and the remaining examples representing negative examples;

[0040] Step 3.5: Calculate the contrastive learning loss based on positive and negative examples; use the InfoNCE loss as the training target for contrastive learning;

[0041] Step 4: Using a multi-task learning approach, the interaction perception task is used as an auxiliary task and trained together with the dialogue summarization task to train the dialogue summarization model;

[0042] Step 4.1: Calculate the training loss for multi-task learning:

[0043] The training loss for multi-task learning is a weighted sum of cross-entropy loss and contrastive learning loss;

[0044] Step 4.2: Set training parameters:

[0045] The AdamW optimizer was used to optimize the dialogue summarization model with a learning rate of 3e-5. A warmup strategy was used to dynamically adjust the learning rate, and the number of warmup steps, batch size, and training epochs were set. The model's performance was validated using a validation set, which was validated every other epoch. The best-performing dialogue summarization model on the validation set was saved for subsequent testing.

[0046] Step 5: Test the performance of the dialogue summarization model: Test the trained dialogue summarization model using a test set;

[0047] Step 5.1: Generate summaries for the dialogues in the test set; feed the dialogues in the test set into the encoder of the dialogue summarization model to obtain the encoded representation of the dialogues; then use... <ubos> 、 <abos> 、 <fbos>As the start symbol of the decoder, it controls the dialogue summarization model to decode the user summary, customer service summary, and overall summary; the BeamSearch algorithm is used as the decoding strategy, and special symbols are encountered. <eos>This will prematurely terminate the decoding process;

[0048] Step 5.2: Evaluate model performance: Use ROUGE-1, ROUGE-2, and ROUGE-L to evaluate the model performance.

[0049] The beneficial effects of adopting the above technical solution are as follows: This invention provides a role-specific dialogue summarization method based on contrastive learning, taking into account the interactions between different roles and the connections between summaries of different roles. A role-interaction-aware contrastive learning objective is constructed based on the dialogue representation encoded by the encoder, assisting the encoder of the dialogue summarization model in understanding the interaction patterns between different roles. This helps the summarization model track key information scattered among the statements of different roles, improving the accuracy of the generated summaries. Simultaneously, this invention utilizes the start symbol at the decoder to control the model's decoding, requiring only one model to be trained to generate summaries for different roles. This reduces computational overhead while fully considering the connections between summaries of different roles.

[0050] (1) The method of the present invention overcomes the problem that existing dialogue summarization methods cannot fully capture the interaction between different roles. It uses a contrastive learning objective of role interaction perception to assist the encoder of the dialogue summarization model in understanding the interaction patterns between different roles, helping the model to locate key content in the dialogue, track key information scattered in the statements of different roles, and thus help the dialogue summarization model generate more accurate and concise summaries.

[0051] (2) The method of this invention overcomes the shortcomings of existing methods that require training multiple models or multiple decoders. The method of this invention uses different special symbols as start symbols for the decoder to control the dialogue summarization model to generate different summaries. Only one model needs to be trained to generate summaries for different roles, which is simple to implement, has low computational overhead, and makes full use of the relationship between the summaries of different roles.

[0052] (3) The method of this invention does not require the introduction of additional common sense, topics, discourse, or other information, thus reducing the cost of data annotation. Furthermore, the contrastive learning method designed in this invention only operates during the training phase and does not introduce any additional structure during the inference phase, resulting in faster inference speed.

[0053] (4) The method of the present invention can effectively utilize the interaction information between different roles to generate an accurate and concise summary for each role. The results on the customer service dialogue summary dataset CSDS significantly outperform existing methods. Attached Figure Description

[0054] Figure 1 A flowchart illustrating a role-specific dialogue summarization method based on contrastive learning, provided as an embodiment of the present invention;

[0055] Figure 2 A schematic diagram of the structure of a role-specific dialogue summarization model provided in an embodiment of the present invention;

[0056] Figure 3 This is a schematic diagram illustrating the comparative learning process of interactive perception provided in an embodiment of the present invention. Detailed Implementation

[0057] The specific embodiments of the present invention will be described in further detail below with reference to the accompanying drawings and examples. The following examples are for illustrative purposes only and are not intended to limit the scope of the invention.

[0058] In this embodiment, a role-specific dialogue summarization method based on contrastive learning is described, such as... Figure 1 As shown, it includes the following steps:

[0059] Step 1: Acquisition and preprocessing of dialogue summary dataset;

[0060] Step 1.1: Obtain the Dialogue Summary Dataset: Obtain the open-source Customer Service Dialogue Summary Dataset CSDS and divide it into training, validation, and test sets; the training, validation, and test sets contain 9001, 800, and 800 dialogues respectively. Each dialogue in the dataset includes two roles: user and customer service representative; in addition to the overall dialogue summary, the dataset also provides a user-oriented summary and a customer service-oriented summary.

[0061] In this embodiment, the specific partitioning results of the dialogue summary dataset are shown in Table 1:

[0062] Table 1. Dialogue Summary Dataset Partition Results

[0063] Data 9101 800 800 Average number of characters in a conversation 321.9 317.8 325.5 Average character count of user summary 37.3 38.5 37.9 Average number of characters in customer service summary 48.1 50.2 52.0 Average number of characters in the overall summary 83.2 85.7 87.1

[0064] Step 1.2: Dataset Preprocessing: Preprocess all dialogue samples from the training, validation, and test sets to obtain the format required by the dialogue summarization model; the specific method is as follows:

[0065] Step 1.2.1: Training Sample Preprocessing: Each dialogue contains three summaries: user summary, customer service summary, and overall summary. (The remaining text appears to be incomplete and possibly contains errors. A more accurate translation would require the full context.) <ubos> ， <abos> ， <fbos>These are appended to the beginning of the user summary, customer service summary, and overall summary, respectively, as start symbols for decoding, used to control the generation of different summaries by the entire model. For the training set data, each dialogue in the training set is combined with the corresponding user summary, customer service summary, and overall summary to form dialogue-summary pairs, tripling the original number of training samples;

[0066] Step 1.2.2: Dialogue Preprocessing: First, remove URL links and garbled characters from the dialogue; then, merge multiple consecutive statements from one speaker into one statement, and prepend each speaker's role to the beginning of their corresponding statement to distinguish between different roles; next, concatenate all the statements in the dialogue together, using a special symbol [SEP] to separate different statements; use Bert Tokenizer to segment the concatenated statement sequence, mapping each character to a corresponding index using a predefined vocabulary; if the length of the statement sequence exceeds a set threshold of 512, it is truncated, thus obtaining the dialogue sequence X = {x1, ..., x...} n }, where x n The nth word in the dialogue sequence;

[0067] Step 1.2.3: Summary Preprocessing: The summary is segmented using Bert Tokenizer, and each character is mapped to its corresponding index using a predefined vocabulary. If the length of the resulting summary sequence exceeds a set threshold of 200, it is truncated to obtain the reference summary sequence Y = {y1,…,y}. t };

[0068] Step 2: Construct a dialogue summarization model tailored to specific roles, generating summaries for different roles; such as... Figure 2 As shown, the dialogue summarization model is based on a pre-trained BART model, including a dialogue encoder and a role-specific summary decoder. The encoder is responsible for encoding the input dialogue and includes an embedding layer and six Transformer layers. The decoder is responsible for generating role-specific summaries, also including an embedding layer and six Transformer layers. The specific steps are as follows;

[0069] Step 2.1: Feed the preprocessed dialogue sequence X into the BART-based dialogue encoder, and obtain the semantic vector representation H of the dialogue context through self-attention and feedforward neural networks, as shown in the following formula:

[0070] H = {h1,…,h} n } = Encoder({x1,…,x n })

[0071] Step 2.2: The reference summary sequence Y obtained after summary preprocessing is fed into the summary decoder based on the BART model. Different decoder start symbols are used to control the decoder to generate summaries for different roles. The hidden vector representation of the decoder is obtained through self-attention, cross-attention and feedforward neural network, and then the probability distribution of generated words is obtained through a linear layer. The training loss is calculated using the cross-entropy function, as shown in the following formula, where X represents the input dialogue sequence.

[0072]

[0073] Step 3: Construct an interaction-aware contrastive learning task to assist the dialogue summarization model in encoding the dialogue; based on the dialogue vector representation output by the encoder, an interaction-aware contrastive learning objective is designed to assist the dialogue summarization model in capturing the interactions between different roles; in Step 2, the encoder input is a flat sequence, which cannot fully understand the interaction patterns between different roles, and the model tends to focus only on the information of a certain role itself, ignoring interaction-related information. For example... Figure 2 As shown, this invention designs an interaction-aware contrastive learning objective based on the dialogue vector representation output by the encoder, which helps the model capture the interaction between different roles. Figure 3 The detailed process of the contrastive learning task for interactive perception is demonstrated, specifically as follows:

[0074] Step 3.1: Separate the self-vector representation of each role from the dialogue semantic vector representation output by the encoder; first, define two binary role mask vectors, the user role mask vector m u The mask vector m for the customer service role a If a character comes from a user statement, then m u Set the element at the corresponding position in the m column to 1, otherwise set it to 0; if a character comes from a customer service statement, then set m... a The element at the corresponding position is set to 1, otherwise it is set to 0; next, the self-vector representation of each character is separated from the dialogue semantic vector representation output by the encoder, as shown in the following formula:

[0075]

[0076]

[0077] in, For the user's self-vector representation, The self-vector representation of customer service, where ⊙ represents element-wise product;

[0078] Step 3.2: Calculate the cross vector representation for each character;

[0079] First, calculate the attention score matrix for users and customer service representatives using the following formula;

[0080]

[0081]

[0082] Among them, A u2a A a2u These are the attention score matrices for users and customer service representatives, respectively; d represents the dimension of the model's hidden vector representation, and M... u2a and M a2u These are two attention mask matrices, as shown in the following formula:

[0083]

[0084]

[0085] in, and Representing m respectively u and m a The j-th element, I i and I j These represent the indices of the statements containing the i-th and j-th characters, respectively. w is a hyperparameter used to mask out attention between statements that are more than a set threshold apart.

[0086] Next, a cross vector representation is obtained for each character according to the following formula:

[0087]

[0088]

[0089] Among them, C u C a These represent the cross vectors of the user and customer service representatives, respectively.

[0090] Step 3.3: Calculate the interaction perception vector representation for each character;

[0091] The self-vector representation of each character obtained in step 3.1 and the cross-vector representation of each character obtained in step 3.2 are fused together through a linear layer to obtain an interaction-aware vector representation for each character, as shown in the following formula:

[0092]

[0093]

[0094] in, These represent the interaction perception vectors of the user and customer service representatives, respectively. W represents splicing. u and W a Represents the learnable weight matrix;

[0095] Step 3.4: Constructing Positive and Negative Examples; To train the contrastive learning objective of interactive perception, positive and negative examples need to be constructed. The original dialogue is used as a positive example; negative examples are constructed by disrupting the interaction relationships between different roles in the dialogue. Specifically: First, all statements from the user role in the dialogue are fixed, and all statements from the customer service role are replaced with statements from other customer service roles in the dialogue; this process is repeated to construct K negative examples for each positive example; similarly, the customer service role's statements are fixed, and the user role's statements are replaced, resulting in another K negative examples; finally, a total of 2K+1 examples are obtained, with the first example representing a positive example and the remaining examples representing negative examples; based on steps 3.1 to 3.4, each example is... and

[0096] Step 3.5: Calculate the contrastive learning loss based on positive and negative examples; use the InfoNCE loss as the training objective for contrastive learning, as shown in the following formula:

[0097]

[0098]

[0099]

[0100] in, These represent user loss, customer loss, and overall loss, respectively. τ is a hyperparameter representing the temperature of the contrastive learning, and sim represents similarity calculation, which first pools each vector and then calculates the cosine similarity between vectors. By optimizing the contrastive learning loss, positive examples are made as close as possible, and negative examples are made as far apart as possible.

[0101] Step 4: Using a multi-task learning approach, the interaction perception task is used as an auxiliary task and trained together with the dialogue summarization task to train the dialogue summarization model;

[0102] Interaction awareness tasks can help dialogue summarization models capture the interactions between different roles in a dialogue, helping the model generate more accurate summaries.

[0103] Step 4.1: Calculate the training loss for multi-task learning:

[0104] The training loss for multi-task learning is a weighted sum of the cross-entropy loss and the contrastive learning loss, as shown in the following formula:

[0105]

[0106] Here, γ is a hyperparameter representing the weight of contrastive learning. The value of γ is selected based on the results on the test set. If the value is too small, it will not help the model fully capture the interaction between different roles. If it is too large, it will cause the model to focus too much on the auxiliary task and ignore the main task.

[0107] Step 4.2: Set training parameters:

[0108] The AdamW optimizer was used to optimize the dialogue summarization model with a learning rate of 3e-5. A warmup strategy was used to dynamically adjust the learning rate with 800 warmup steps. The batch size was set to 4, and the training was conducted for 5 epochs. The model's performance was validated using a validation set, which was validated every other epoch. The best-performing dialogue summarization model on the validation set was saved for subsequent testing.

[0109] Step 5: Test the performance of the dialogue summarization model: Test the trained dialogue summarization model using a test set;

[0110] Step 5.1: Generate summaries for the dialogues in the test set; feed the dialogues in the test set into the encoder of the dialogue summarization model to obtain the encoded representation of the dialogues; then use... <ubos> 、 <abos> 、 <fbos>As the start symbol of the decoder, it controls the dialogue summarization model to decode the user summary, customer service summary, and overall summary. The BeamSearch algorithm is used as the decoding strategy; at each time step, it retains not only the output with the highest score, but also beam_size outputs. In this embodiment, beam_size is set to 3, and the maximum decoding length is set to 200. Special symbols are encountered... <eos>This will prematurely terminate the decoding process;

[0111] Step 5.2: Evaluate the model performance: Use ROUGE-1, ROUGE-2, and ROUGE-L to evaluate the model performance and compare it with existing role-specific dialogue summarization methods.

[0112] This embodiment also selects baseline models for comparison: To verify the effectiveness of the dialogue summarization model of this invention, this embodiment selects several strong baselines from previous studies for comparison: BART, training the BART model three times to generate user summaries, customer service summaries, and overall summaries respectively. BART-Multi, using two decoders to generate user summaries and customer service summaries respectively, and jointly training the model using a multi-task learning approach. BART-Both, adding a self-attention and a cross-attention layer on the decoder side to BART-Multi, modeling the interaction between different roles. BART-RAC, using discrete prompts to control the model to generate summaries for different roles, and using uncertain scores to measure the importance of each statement. BART-GLC, built on BART-RAC, considering the saliency of statements as well as the saliency of subtopics.

[0113] Table 2 shows the comparison results of the dialogue summarization model of the present invention with other baselines. The three values ​​in each cell represent the overall summary, user summary, and customer service summary results from left to right. BART-CIAM represents the experimental results of the present invention. It can be seen that the results of the present invention are significantly improved compared to the basic BART model, proving the effectiveness of the method. Furthermore, the method of the present invention outperforms existing methods on most metrics. In addition, ablation experiments were conducted. w / oIACL represents the experimental results after absolving the contrastive learning objective of interaction perception. It can be seen that the model's performance significantly declined on all metrics, proving that the contrastive learning objective constructed in the present invention can indeed effectively assist the dialogue summarization model in capturing key information in the dialogue. Moreover, even after removing the contrastive learning objective, the model's performance is still significantly better than the basic BART model, proving that using different decoder start symbols to control the model to generate summaries for different roles can fully utilize the connections between the summaries of different roles and improve the quality of the generated summaries.

[0114] Table 2 Comparison of the Dialogue Summarization Model of the Present Invention with Other Baselines

[0115] BART 58.66 / 60.35 / 54.13 43.35 / 45.09 / 39.37 56.60 / 58.13 / 51.18 BART-Multi 58.90 / 60.18 / 54.05 43.72 / 45.21 / 40.00 56.90 / 58.00 / 51.72 BART-both 59.21 / 60.53 / 54.22 43.88 / 45.39 / 40.32 57.32 / 58.28 / 51.90 BART-RLC 59.86 / 61.67 / 54.83 44.42 / 46.14 / 40.29 57.86 / 59.45 / 52.43 BART-GLC 60.07 / 61.72 / 54.82 44.55 / 46.21 / 40.11 58.06 / 59.51 / 52.46 BART-CIAM 60.27 / 62.21 / 55.04 44.63 / 46.35 / 40.46 58.20 / 59.88 / 52.69 w / o IACL 59.39 / 61.69 / 54.68 43.85 / 46.13 / 40.12 57.34 / 59.37 / 52.39

[0116] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit them. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope defined by the claims of the present invention.< / eos> < / fbos> < / abos> < / ubos> < / fbos> < / abos> < / ubos> < / eos> < / fbos> < / abos> < / ubos> < / fbos> < / abos> < / ubos>

Claims

1. A role-specific dialogue summarization method based on contrastive learning, characterized in that: Includes the following steps: Step 1: Acquisition and preprocessing of dialogue summary dataset; Each dialogue in the acquired dialogue summary dataset contains two roles: user and customer service representative. In addition to a summary of the overall dialogue, the dataset also provides a user-oriented summary and a customer service-oriented summary; Step 2: Build a dialogue summarization model for specific roles to generate summaries for different roles; The dialogue summarization model is based on a pre-trained BART model, which includes a dialogue encoder and a role-specific summarization decoder. Step 3: Construct an interaction-aware contrastive learning task to assist the dialogue summarization model in encoding dialogues; Based on the dialogue vector representation output by the encoder, an interaction-aware contrastive learning objective is designed to assist the dialogue summarization model in capturing the interactions between different roles. Step 3.1: Separate the self-vector representation of each role from the dialogue semantic vector representation output by the encoder; First, define two binary role mask vectors: the user role mask vector. and the mask vector of the customer service role ; If a character comes from a user's statement, then The element at the corresponding position is set to 1, otherwise it is set to 0; if a character comes from a customer service statement, then... The element at the corresponding position is set to 1, otherwise it is set to 0; next, the self-vector representation of each character is separated from the dialogue semantic vector representation output by the encoder, as shown in the following formula: ； in, For the user's self-vector representation, The self-vector representation of customer service. This represents element-wise product; Step 3.2: Calculate the attention score matrix for users and customer service representatives, and then calculate the cross vector representation for each role; First, calculate the attention score matrix for users and customer service representatives using the following formula; ； in, These are the attention score matrices for users and customer service representatives, respectively; d represents the dimension of the model's hidden vector representation. and These are two attention mask matrices, as shown in the following formula: ； in, and Represent and The j-th element, and These represent the indices of the statements containing the i-th and j-th characters, respectively. w is a hyperparameter used to mask out attention between statements that are more than a set threshold apart. Next, a cross vector representation is obtained for each character according to the following formula: ； in, These represent the cross vectors of the user and customer service representatives, respectively. Step 3.3: Calculate the interaction perception vector representation for each character; Step 3.4: Construct positive and negative examples; use the original dialogue as a positive example; disrupt the interaction between different characters in the dialogue to construct negative examples; Step 3.5: Calculate the contrastive learning loss based on positive and negative examples; use the InfoNCE loss as the training target for contrastive learning; Step 4: Using a multi-task learning approach, the interaction perception task is used as an auxiliary task and trained together with the dialogue summarization task to train the dialogue summarization model; Step 5: Test the performance of the dialogue summarization model: Test the trained dialogue summarization model using a test set.

2. The role-specific dialogue summarization method based on contrastive learning according to claim 1, characterized in that: The specific method for step 1 is as follows: Step 1.1: Obtain the dialogue summary dataset: Obtain the customer service dialogue summary dataset and divide it into training set, validation set and test set; Step 1.2: Dataset Preprocessing: Preprocess all dialogue samples in the training set, validation set, and test set; Step 1.2.1: Training Sample Preprocessing: Preprocessing the three special symbols... <ubos> ， <abos> ， <fbos> Append them to the beginning of the user summary, customer service summary and overall summary respectively as the start symbol for decoding; for the training set data, combine each dialogue in the training set with the corresponding user summary, customer service summary and overall summary to form a dialogue-summary pair, and increase the original number of training samples to three times the original number.< / fbos> < / abos> < / ubos> Step 1.2.2: Dialogue Preprocessing: First, remove URL links and garbled characters from the dialogue; then, merge multiple consecutive statements from one speaker into one statement, and prepend each speaker's role to the beginning of their corresponding statement to distinguish between different roles; next, concatenate all the statements in the dialogue together, using a special symbol [SEP] to separate different statements; use Bert Tokenizer to segment the concatenated statement sequence, mapping each character to a corresponding index using a predefined vocabulary; if the length of the statement sequence exceeds a set threshold, it is truncated, thus obtaining the dialogue sequence. ,in, The nth word in the dialogue sequence; Step 1.2.3: Summary Preprocessing: The summary is segmented using Bert Tokenizer, and each character is mapped to its corresponding index using a predefined vocabulary. If the length of the processed summary sequence exceeds a set threshold, it is truncated to obtain a reference summary sequence. .

3. The role-specific dialogue summarization method based on contrastive learning according to claim 2, characterized in that: The specific method for step 2 is as follows: Step 2.1: Feed the preprocessed dialogue sequence X into the dialogue encoder based on the BART model, and obtain the semantic vector representation H of the dialogue context through self-attention and feedforward neural network; Step 2.2: The reference summary sequence Y obtained after summary preprocessing is fed into the summary decoder based on the BART model. Different decoder start symbols are used to control the decoder to generate summaries with different roles. The hidden vector representation of the decoder is obtained through self-attention, cross-attention, and feedforward neural networks, and then the probability distribution of generated words is obtained through a linear layer; the training loss is calculated using the cross-entropy function.

4. The role-specific dialogue summarization method based on contrastive learning according to claim 3, characterized in that: The specific method for step 3.3 is as follows: The self-vector representation of each character obtained in step 3.1 and the cross-vector representation of each character obtained in step 3.2 are fused together through a linear layer to obtain an interaction-aware vector representation for each character, as shown in the following formula: ； in, These represent the interaction perception vectors of the user and customer service representatives, respectively. Represents splicing, and This represents the learnable weight matrix.

5. A role-specific dialogue summarization method based on contrastive learning according to claim 4, characterized in that: The specific method for constructing positive and negative samples in step 3.4 is as follows: Specifically, the process is as follows: First, fix all the statements of the user role in the dialogue, and replace all the statements of the customer service role with statements from other customer service roles in the dialogue; repeat this process to construct K negative samples for each positive sample; fix the statements of the customer service role, replace the statements of the user role, and obtain K more negative samples; finally, a total of 2K+1 samples are obtained, with the first sample representing a positive sample and the remaining samples representing negative samples.

6. A role-specific dialogue summarization method based on contrastive learning according to claim 5, characterized in that: The specific method for step 4 is as follows: Step 4.1: Calculate the training loss for multi-task learning: The training loss for multi-task learning is a weighted sum of cross-entropy loss and contrastive learning loss; Step 4.2: Set training parameters: The AdamW optimizer was used to optimize the dialogue summarization model with a learning rate of 3e-5. A warmup strategy was used to dynamically adjust the learning rate, setting the number of warmup steps, batch size, and training epochs. The model's performance was validated using a validation set, which was validated every other epoch. The best-performing dialogue summarization model on the validation set was saved for subsequent testing.

7. A role-specific dialogue summarization method based on contrastive learning according to claim 1, characterized in that: The specific method for step 5 is as follows: Step 5.1: Generate summaries for the dialogues in the test set; feed the dialogues in the test set into the encoder of the dialogue summarization model to obtain the encoded representation of the dialogues; then use... <ubos> 、 <abos> 、 <fbos>As the start symbol of the decoder, it controls the dialogue summarization model to decode the user summary, customer service summary, and overall summary; the BeamSearch algorithm is used as the decoding strategy, and special symbols are encountered. <eos> This will prematurely terminate the decoding process;< / eos> < / fbos> < / abos> < / ubos> Step 5.2: Evaluate model performance: Use ROUGE-1, ROUGE-2, and ROUGE-L to evaluate the model performance.

Images