Dialogue abstracting method and system based on semantic map

Abstract

The invention relates to a dialogue abstract method and system based on a semantic atlas, and the method comprises the steps: obtaining a plurality of user dialogues and corresponding reference abstracts, labeling the user dialogues and reference abstract labels, and constructing a training set; constructing a deep learning network model based on a semantic map, training the model by using the training set, extracting sentence features from the dialogue text and the reference abstract thereof through the deep learning network model, calculating the similarity of sentence pairs, further constructing a semantic correlation map, and analyzing the correlation and coverage between the generated abstract and the reference abstract; in addition, uncovered loss and inconsistent loss functions are designed to optimize the performance of the model, so that the understanding of the model on the semantic relationship between the user dialogue and the user dialogue abstract is improved; and inputting the user dialogue into the trained deep learning network model, and outputting an abstract of the user dialogue. According to the method and the system, the accuracy of the dialogue abstract can be improved, and the generated abstract has higher fact consistency and coverage rate.

Description

Technical Field

[0001] The present invention relates to the technical field of natural language processing, and in particular to a conversation summarization method and system based on semantic graphs. Background Art

[0002] Conversation is one of the most common forms of communication on social networks. Organizing conversation content into relevant summaries or abstracts has become a crucial task. Conversation summarization aims to summarize the utterances of two or more interlocutors to identify the key information considered by a third party. This task not only provides new participants with quick context but also helps people grasp the central idea or search for key points in a conversation, thereby improving efficiency and productivity. In recent years, social networks have become increasingly intertwined with people, enabling them to communicate, make friends, work, learn, and entertain themselves. As the number of users continues to grow, the amount of communication data generated by social networks is also increasing. Conversation summarization techniques can be used to extract the key information from conversations and create summaries, allowing for quick and efficient understanding of relevant information. However, unlike traditional document summarization of news or articles, conversation data has unique characteristics, including diverse language styles, fragmented information, flexible discourse structure, and unclear topic boundaries, which pose numerous challenges for conversation summarization.

[0003] The earliest work on dialogue summarization solved this problem by converting the dialogue into a word graph and selecting appropriate paths in the graph as summary sentences based on complex rules. Template-based methods have also been used, which collect templates from summaries written by humans and generate abstract summaries by selecting appropriate words from the dialogue to fill in the blanks. However, the summaries they generate lack fluency and diversity and are therefore far from practical use. With the emergence of neural encoder-decoder models, it is possible to project the input into a dense semantic representation and generate a summary with new words by gradually sampling from the vocabulary until a special token indicating the end is output. Based on these models, generative text summarization has made significant progress, from non-pre-trained PGN, Fast-Abs and HRED, to pre-trained models BART and Pegasus, which have promoted the development of abstract text summarization.

[0004] Injecting preprocessed features, either through manual annotation or external tagging tools as part of the input, explicitly utilizes additional features from the conversation context. Injecting preprocessed features includes obtaining intra-utterance features and inter-utterance features.

[0005] Intra-discourse features can be divided into word-level, phrase-level and discourse-level, where word-level intra-discourse features include TF-IDF weights, part-of-speech (POS) tags and named entity tags. Phrase-level intra-discourse features include key phrases / keywords, which emphasize prominent n-grams in the original conversation, which helps to overcome the challenge of information dispersion and thus produce a more detailed summary. Discourse-level intra-discourse features enhance the summary through features such as the speaker or role, user intent and dialogue behavior. Lin et al. proposed two interactive methods to obtain key information of other roles to enhance role-oriented dialogue summaries. Kim et al. used the COMET and PARACOMET common sense knowledge models to extract common sense knowledge from conversations and injected common sense knowledge into state-of-the-art language models (such as BART) to perform abstract dialogue summaries.

[0006] Inter-discourse features can be divided into partitioning and graph methods. Partitioning refers to extracting or dividing the entire conversation into relatively independent segments. The information in each segment is more concentrated and interferes less with summary generation. At the same time, these features reduce the requirements for GPU memory with shorter input lengths, which is especially suitable for long conversation summaries. Krishna et al.'s Cluster2Sent extracts important discourse, clusters related discourse together, generates a summary sentence for each cluster, and obtains a semi-structured summary suitable for clinical conversations. Except for Shang et al. who used K-means for discourse clustering in an unsupervised manner, the supervised training of the extractors or classifiers of most methods requires labeled oracle partitions. Typically, the partitions are concatenated as input to the abstract summary model, or the summaries generated for each segment are concatenated or sorted to form the final summary. Summary of the Invention

[0007] The purpose of the present invention is to provide a conversation summarization method and system based on semantic graphs, which are conducive to improving the accuracy of conversation summaries, and the generated summaries have higher factual consistency and coverage.

[0008] To achieve the above objectives, the present invention adopts a technical solution: a conversation summarization method based on speaker perception enhancement, comprising the following steps:

[0009] Step A: Obtain multiple user conversations and their corresponding reference summaries, annotate the user conversations and reference summaries, construct training samples, and then build a training set DS;

[0010] Step B: Construct a deep learning network model M based on the semantic graph and train it using the training set DS. The deep learning network model M extracts sentence features from the conversation text and its reference summary, calculates sentence pair similarities, and then constructs a semantic relevance graph to analyze the relevance and coverage between the generated summary and the reference summary. Furthermore, uncovered loss and inconsistency loss functions are designed to optimize model performance, effectively improving the model's understanding of the semantic relationship between user conversations and user conversation summaries, ensuring that the generated summary has higher factual consistency and coverage.

[0011] Step C: Input the user conversation into the trained deep learning network model M and output the summary of the user conversation.

[0012] Furthermore, the step B specifically includes the following steps:

[0013] Step B1: Initially process all training samples in the training dataset DS to obtain the dialogue sequence D and reference summary S;

[0014] Step B2: The dialogue sequence D obtained in step B1 is passed through the encoder and decoder of the BART model to obtain the dialogue output representation S', which is normalized by the softmax function and the loss value is calculated using cross entropy as the loss function to obtain the dialogue summary main task loss. main ;

[0015] Step B3: The dialogue output representation S' obtained in step B2 is decoded by the tokenizer of the BART model to generate the summary S * ; Calculate reference summary S and generate summary S * Build a semantic correlation graph based on the text similarity of

[0016] Step B4: By analyzing the semantic relevance graph obtained in step B3, the uncovered loss value loss is calculated using the negative log-likelihood as the loss function. uncovered And use InfoNCE as the loss function to calculate the loss value of the fact consistency auxiliary task loss Contrastive ;

[0017] Step B5: The dialogue summary loss obtained in step B2 is used as the main task loss main and the uncovered loss loss obtained in step B5 un covered And the loss value of the fact consistency auxiliary task loss Contrastive The final loss of the model is obtained by weighting, the gradient of each parameter in the deep learning network model is calculated by the back propagation method, and the parameters are updated using the stochastic gradient descent method;

[0018] Step B6: When the loss value change generated by each iteration of the deep learning network model is less than a given threshold, or the maximum number of iterations is reached, the training process of the deep learning network model M is terminated.

[0019] Furthermore, the step B1 specifically includes the following steps:

[0020] Step B11: Traverse the training set DS, perform word segmentation and remove stop words on the dialogue texts. Each training sample in DS is represented as ds = (D, S), where D is the dialogue sequence and S is the reference summary.

[0021] Dialogue D is represented as:

[0022] D={t1,t2,…,t i ,…,t n}

[0023] Among them, t i is the i-th speaker-utterance pair in dialogue D, i = 1, 2, …, n, and n is the number of speaker-utterance pairs in dialogue D;

[0024] speaker-utterance dialogue i Expressed as:

[0025] t i =(s i ,u i )

[0026]

[0027] Among them, s i For the i-th utterance u i The corresponding speaker, u i is the i-th utterance, is the jth word in the i-th utterance, i = 1, 2, ..., n, j = 1, 2, ..., l i , l i is the text length of the i-th utterance;

[0028] The reference summary S is represented as:

[0029] S={s1,s2,…,s i ,…,s N}

[0030] Among them, s i is the i-th summary sentence in the reference summary S, i = 1, 2, ..., N, and N is the number of sentences contained in the reference summary S.

[0031] Furthermore, the step B2 specifically includes the following steps:

[0032] Step B21: Pass D through the BART model tokenizer to perform padding and length truncation, and obtain the token sequence T of the dialogue sequence D as the BART model encoder input;

[0033] The token sequence T of the conversation sequence D is calculated as follows:

[0034] T=BART_Tokenizer(D)

[0035] Among them, BART_Tokenizer() represents the tokenizer of the BART model;

[0036] The encoder output is calculated as follows:

[0037] H=[H1,H2,…,H i ,…,H n ]=BART_Encoder(T)

[0038]

[0039] Among them, BART_Encoder() represents the encoder of the BART model, is the jth word in the i-th utterance The corresponding word vector, I i is the text length of the i-th utterance, and d is the dimension of the feature vector;

[0040] Step B22: The encoder output H obtained in step B21 is used as the input to the BART model decoder. It is decoded through the self-attention mechanism, layer normalization, cross attention, and feedforward neural network to obtain the dialogue output representation S'.

[0041] H′=SelfAttention(LayNorm(H))

[0042] H″=CrossAttention(LayNorm(H′+H))

[0043] S'=FFN(LayNorm(H'+H'''))

[0044] Among them, LayNorm(·) represents layer normalization, SelfAttention(·) represents self-attention mechanism, CrossAttention(·) represents cross attention, and FFN(·) represents feedforward neural network;

[0045] Step B23: The decoded output representation S' obtained in step B22 is normalized by the softmax function to obtain the score distribution P of the next word generated. The loss value is calculated using the negative log-likelihood as the loss function to obtain the loss of the main task of the dialogue summary. main ;

[0046] The calculation formula of score distribution P is as follows:

[0047] P = softmax(W·S'+b)

[0048] Where P∈[0,1], W is the trainable parameter matrix, and b is the bias vector;

[0049] Conversation summary main task loss loss main The calculation formula is as follows:

[0050]

[0051] Among them, y ＜t =(y1,y2,…,y t-1 ) represents the first t-1 words of the output sequence, S represents the reference summary, D is the dialogue sequence, and θ is the training parameter; logP(y t |y ＜t ,D;θ) is the logarithmic form of the score distribution P, which means that given the first t-1 words y ＜t and dialogue sequence D, generate the i-th word y i probability.

[0052] Furthermore, the specific implementation method of step B3 is:

[0053] Step B31: The dialogue output representation S' obtained in step B2 is decoded by the tokenizer of the BART model to generate the summary S * ;

[0054] Generate summary S * The calculation is as follows:

[0055] S * =BART_Tokenizer(S')

[0056] Among them, BART_Tokenizer() represents the tokenizer decoding using the BART model;

[0057] Generate summary S * Expressed as:

[0058]

[0059] in, Indicates the generation of summary S * The i-th summary sentence in, i=1,2,…,M, M represents the generated summary S * The number of sentences contained in

[0060] Step B32: Compare the reference summary S obtained in step B11 with the generated summary S obtained in step B31 * The token sequence T of the reference summary S is obtained by padding and length truncation through the tokenizer of the BART model. S and generate summary S * Token sequence T S* As input to the BART model encoder;

[0061] Reference digest S's token sequence T S and generate summary S * Token sequence T S* The calculation is as follows:

[0062] T S =BART_Tokenizer(S)

[0063]

[0064] Among them, BART_Tokenizer() represents the tokenizer of the BART model;

[0065] The encoder output is calculated as follows:

[0066] H s =[h1,h2,…,h i ,…,h N ]=BART_Encoder(T s )

[0067]

[0068] Among them, BART_Encoder() represents the encoder of the BART model, h i The vector representation of the i-th sentence in the reference summary S, i = 1, 2, ..., N, where N is the number of sentences contained in the reference summary S; Indicates the generation of summary S * The vector representation of the jth sentence in , j = 1, 2, ..., M, M is the generated summary S * The number of sentences contained in

[0069] Step B33: Calculate the encoder output H of the reference summary S obtained in step B32 S and generate summary S* The encoder output H S* The cosine similarity between each sentence vector representation is used to construct a semantic correlation graph;

[0070] For H S Each item h i , H S* Each of Calculate the cosine similarity. The calculation rules are as follows:

[0071]

[0072] Where cos() represents the cosine similarity calculation, Represents vector h i and The inner product of , which is the sum of the corresponding elements of the two vectors multiplied together; Represents vector h i and The modulus of the cosine similarity; the result of the cosine similarity is a value between -1 and 1. The closer the value is to 1, the more similar the two vectors are. The closer the value is to -1, the less similar the two vectors are. 0 means that the two vectors are orthogonal, that is, completely unrelated.

[0073] Furthermore, the step B4 specifically includes the following steps:

[0074] Step B41: Set a specified threshold value θ, analyze the semantic relevance graph obtained in step B33, stipulate that nodes with cosine similarity between sentences less than the threshold value θ do not establish edge connections, and nodes with cosine similarity between sentences greater than or equal to the threshold value θ establish edge connections, and obtain a semantic relevance matrix Matrix;

[0075]

[0076] Step B42: Traverse and generate summary S * The sentences in the generated summary are connected to the sentences in the reference summary S, and the nodes with edge connections in the generated summary sentences are divided into a sentence set S * ,The nodes in the reference summary sentence that have not established edge connections are divided into a sentence set S, and S is merged * and S is

[0077]

[0078] S=[h1,h2,…,h j ,…,h q ]

[0079]

[0080] in, Represents the set S * The vector representation of the i-th sentence in , i = 1, 2, ..., k, k is the number of sentences with edge connections in the generated summary sentence, h j The vector representation of the j-th sentence in the set Sˉ, j = 1, 2, ..., q, where q is the number of sentences in the reference summary that have no edge connections;

[0081] Step B43: The sentence set obtained in step B42 Calculate uncovered loss uncovered And the sentence set S obtained in step B42 * , calculate the loss of the fact consistency auxiliary task loss Contrastive ;

[0082] Uncovered loss loss uncovered The calculation formula is as follows:

[0083]

[0084] Among them, y ＜t =(y1,y2,…,y t-1 ) represents the first t-1 words of the output sequence, D is the dialogue sequence, and θ is the training parameter; logP(y t |y <t ,D;θ) is the logarithmic form of the score distribution P, which means that given the first t-1 words y <t and dialogue sequence D, generate the i-th word y i probability;

[0085] Loss of the fact consistency auxiliary task Contrastive The calculation is as follows:

[0086]

[0087] Among them, exp() represents the exponential function operation, cos(h i ,h S ) represents the sentence vector representation h i and h S The cosine similarity, S * Indicates the generated summary, S * Indicates that the nodes with edge connections in the generated summary sentence are merged into a sentence set; the meaning of the formula is expressed as the sentence s that is consistent with the facts i ∈S * As positive samples, sentences that are inconsistent with the facts As a negative sample; through contrast loss loss Contrastive To train the model, the similarity between consistent sentences and reference summaries is maximized, while the similarity between inconsistent sentences and reference summaries is minimized.

[0088] Furthermore, the step B5 specifically includes the following steps:

[0089] Step B51: The conversation summary main task loss obtained in step B2 is main and the uncovered loss loss obtained in step B4 uncovered And the loss value of the fact consistency auxiliary task loss Contrastive Weighted to get the final loss of the model;

[0090] The final loss function of the model is calculated as follows:

[0091] loss=loss main +αloss iuncovered +βloss Contrastive

[0092] Among them, α and β are hyperparameters used to balance the impact of auxiliary tasks on the model;

[0093] Step B52: Calculate the gradient of each parameter in the deep learning network model through the back propagation method, and update each parameter using the stochastic gradient descent method.

[0094] The present invention also provides a conversation summary system based on a semantic graph, comprising a memory, a processor, and computer program instructions stored in the memory and capable of being executed by the processor. When the processor executes the computer program instructions, the above-mentioned method steps can be implemented.

[0095] Compared with the prior art, the present invention has the following beneficial effects:

[0096] 1) By designing uncovered and inconsistent loss functions, this model optimizes the semantic relationship between the generated summary and the reference summary. This design not only focuses on literal text similarity but also effectively analyzes the semantic differences and information coverage between the generated and reference summaries, ensuring that the generated summaries are more comprehensive and accurate, thereby improving summary quality.

[0097] 2) This paper analyzes the relationship between user conversations and reference summaries by constructing a correlation graph. This graph modeling approach helps to deeply understand the semantic structure of conversations and capture the complex connections between texts. Compared with traditional methods, semantic graphs enhance the model's expressiveness and understanding depth when processing complex conversation data, thereby improving the model's sensitivity to context and semantic relationships.

[0098] 3) This invention uses a sentence-level contrastive loss optimization strategy to provide a more fine-grained training signal, rather than training only at the summary level. This fine-grained training signal helps capture subtle semantic differences and information omissions during summary generation, thereby ensuring the accuracy and consistency of the generated summaries. This advantage enables the generated summaries to better reflect the factual content of the original conversation. BRIEF DESCRIPTION OF THE DRAWINGS

[0099] Figure 1 This is a flowchart of an implementation of a conversation summarization method based on a semantic graph provided by an embodiment of the present invention;

[0100] Figure 2 It is an architectural diagram of the deep learning network model M based on semantic graph in an embodiment of the present invention. DETAILED DESCRIPTION

[0101] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0102] It should be noted that the following detailed descriptions are exemplary and are intended to provide further explanation of the present application. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which the present application belongs.

[0103] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit the exemplary embodiments according to the present application. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that when the terms "comprise" and / or "include" are used in this specification, they indicate the presence of features, steps, operations, devices, components and / or combinations thereof.

[0104] like Figure 1 As shown, this embodiment provides a conversation summarization method based on semantic graph, including the following steps:

[0105] Step A: Obtain multiple user conversations and their corresponding reference summaries, annotate the user conversations and reference summaries, construct training samples, and then build a training set DS;

[0106] Step B: Construct a deep learning network model M based on the semantic graph and train it using the training set DS. The deep learning network model M extracts sentence features from the conversation text and its reference summary, calculates sentence pair similarities, and then constructs a semantic relevance graph to analyze the relevance and coverage between the generated summary and the reference summary. Furthermore, uncovered loss and inconsistency loss functions are designed to optimize model performance, effectively improving the model's understanding of the semantic relationship between user conversations and user conversation summaries, ensuring that the generated summary has higher factual consistency and coverage.

[0107] Step C: Input the user conversation into the trained deep learning network model M and output the summary of the user conversation.

[0108] Figure 2 : is an architecture diagram of the deep learning network model G based on context deconstruction in this embodiment. In this embodiment, the specific implementation steps of step B are as follows.

[0109] Step B1: Perform initial processing on all training samples in the training dataset DS to obtain the dialogue sequence D and reference summary S.

[0110] In this embodiment, step B1 specifically includes the following steps:

[0111] Step B11: Traverse the training set DS, perform word segmentation and remove stop words on the dialogue texts. Each training sample in DS is represented as ds = (D, S), where D is the dialogue sequence and S is the reference summary.

[0112] Dialogue D is represented as:

[0113] D={t1,t2,…,t i ,…,t n}

[0114] Among them, t i is the i-th speaker-utterance pair in dialogue D, i = 1, 2, …, n, and n is the number of speaker-utterance pairs contained in dialogue D.

[0115] speaker-utterance dialogue i Expressed as:

[0116] t i =(s i ,u i )

[0117]

[0118] Among them, s i For the i-th utterance u i The corresponding speaker, u iis the i-th utterance, is the jth word in the i-th utterance, i = 1, 2, ..., n, j = 1, 2, ..., l i , l i is the text length of the i-th utterance.

[0119] The reference summary S is represented as:

[0120] S={s1,s2,…,s i ,…,s N}

[0121] Among them, s i is the i-th summary sentence in the reference summary S, i = 1, 2, ..., N, and N is the number of sentences contained in the reference summary S.

[0122] Step B2: The dialogue sequence D obtained in step B1 is passed through the encoder and decoder of the BART model to obtain the dialogue output representation S', which is normalized by the softmax function and the loss value is calculated using cross entropy as the loss function to obtain the dialogue summary main task loss. main .

[0123] In this embodiment, step B2 specifically includes the following steps:

[0124] Step B21: Pass D through the BART model tokenizer to perform padding and length truncation to obtain the token sequence T of the dialogue sequence D as the input of the BART model encoder.

[0125] The token sequence T of the conversation sequence D is calculated as follows:

[0126] T=BART_Tokenizer(D)

[0127] Among them, BART_Tokenizer() represents the tokenizer of the BART model.

[0128] The encoder output is calculated as follows:

[0129] H=[H1,H2,…,H i ,…,H n ]=BART_Encoder(T)

[0130]

[0131] Among them, BART_Encoder() represents the encoder of the BART model, is the jth word in the i-th utterance The corresponding word vector, li is the text length of the i-th utterance, and d is the dimension of the feature vector.

[0132] Step B22: Use the encoder output H obtained in step B21 as the BART model decoder input, and decode it through the self-attention mechanism, layer normalization, cross attention, and feedforward neural network to obtain the dialogue output representation S'.

[0133] H′=SelfAttention(LayNorm(H))

[0134] H″=CrossAttention(LayNorm(H′+H))

[0135] S'=FFN(LayNorm(H'+H""))

[0136] Among them, LayNorm(·) represents layer normalization, SelfAttention(·) represents self-attention mechanism, CrossAttention(·) represents cross attention, and FFM(·) represents feedforward neural network.

[0137] Step B23: The decoded output representation S' obtained in step B22 is normalized by the softmax function to obtain the score distribution P of the next word generated. The loss value is calculated using the negative log-likelihood as the loss function to obtain the loss of the main task of the dialogue summary. main .

[0138] The calculation formula of score distribution P is as follows:

[0139] P = softmax(W·S'+b)

[0140] Among them, P∈[0,1], W is the trainable parameter matrix, and b is the bias vector.

[0141] Conversation summary main task loss loss main The calculation formula is as follows:

[0142]

[0143] Among them, y ＜t =(y1,y2,…,y t-1 ) represents the first t-1 words of the output sequence, S represents the reference summary, D is the dialogue sequence, and θ is the training parameter; logP(y t |y ＜t ,D;θ) is the logarithmic form of the score distribution P, which means that given the first t-1 words y ＜t and dialogue sequence D, generate the i-th word y i probability.

[0144] Step B3: The dialogue output representation S' obtained in step B2 is decoded by the tokenizer of the BART model to generate the summary S * ; Calculate reference summary S and generate summary S * , and build a semantic relevance graph.

[0145] In this embodiment, the specific implementation method of step B3 is:

[0146] Step B31: The dialogue output representation S' obtained in step B2 is decoded by the tokenizer of the BART model to generate the summary S * .

[0147] Generate summary S * The calculation is as follows:

[0148] S * =BART_Tokenizer(S')

[0149] Among them, BART_Tokenizer() represents the tokenizer decoding using the BART model.

[0150] Generate summary S * Expressed as:

[0151]

[0152] in, Indicates the generation of summary S * The i-th summary sentence in, i=1,2,…,M, M represents the generated summary S * The number of sentences contained in .

[0153] Step B32: Compare the reference summary S obtained in step B11 with the generated summary S obtained in step B31 * The token sequence T of the reference summary S is obtained by padding and length truncation through the tokenizer of the BART model. S and generate summary S * Token sequence Serves as input to the BART model encoder.

[0154] Reference digest S's token sequence T S and generate summary S * Token sequence T S* The calculation is as follows:

[0155] T S =BART_Tokenizer(S)

[0156]

[0157] Among them, BART_TOkenizer() represents the tokenizer of the BART model.

[0158] The encoder output is calculated as follows:

[0159] H S =[h1,h2,…,h i ,…,h N ]=BART_Encoder(T s )

[0160]

[0161] Among them, BART_Enecoder() represents the encoder of the BART model, h i The vector representation of the i-th sentence in the reference summary S, i = 1, 2, ..., N, where N is the number of sentences contained in the reference summary S; Indicates the generation of summary S * The vector representation of the jth sentence in , j = 1, 2, ..., M, M is the generated summary S * The number of sentences contained in .

[0162] Step B33: Calculate the encoder output H of the reference summary S obtained in step B32 S and generate summary S * The encoder output H S* The cosine similarity between the vector representations of each sentence is used to construct a semantic correlation graph.

[0163] For H S Each item h i , H S* Each of Calculate the cosine similarity. The calculation rules are as follows:

[0164]

[0165] Where cos() represents the cosine similarity calculation, Represents vector h i and The inner product of , which is the sum of the corresponding elements of the two vectors multiplied together; Represents vector h i and The modulus of the cosine similarity; the result of the cosine similarity is a value between -1 and 1. The closer the value is to 1, the more similar the two vectors are. The closer the value is to -1, the less similar the two vectors are. 0 means that the two vectors are orthogonal, that is, completely unrelated.

[0166] Step B4: By analyzing the semantic relevance graph obtained in step B3, the uncovered loss value loss is calculated using the negative log-likelihood as the loss function. uncovered And use InfoNCE as the loss function to calculate the loss value of the fact consistency auxiliary task loss Contrastive .

[0167] In this embodiment, step B4 specifically includes the following steps:

[0168] Step B41: Set a specified threshold value θ, analyze the semantic relevance graph obtained in step B33, stipulate that nodes with cosine similarity between sentences less than the threshold value θ do not establish edge connections, and nodes with cosine similarity between sentences greater than or equal to the threshold value θ establish edge connections, and obtain a semantic relevance matrix Matrix;

[0169]

[0170] Step B42: Traverse and generate summary S * The sentences in the generated summary are connected to the sentences in the reference summary S, and the nodes with edge connections in the generated summary sentences are divided into a sentence set S * ,The nodes in the reference summary sentence that have not established edge connections are divided into a sentence set S, and S is merged * and S is

[0171]

[0172] S=[h1,h2,…,h j ,…,h q ]

[0173]

[0174] in, Represents the set S * The vector representation of the i-th sentence in , i = 1, 2, ..., k, k is the number of sentences with edge connections in the generated summary sentence, h j The vector representation of the j-th sentence in the set S, j = 1, 2, ..., q, where q is the number of sentences in the reference summary that have no edge connections.

[0175] Step B43: The sentence set obtained in step B42 Calculate uncovered loss uncovered And the sentence set S obtained in step B42 * , calculate the loss of the fact consistency auxiliary task loss Contrastive .

[0176] Uncovered loss loss uncovered The calculation formula is as follows:

[0177]

[0178] Among them, y ＜t =(y1,y2,…,y t-1 ) represents the first t-1 words of the output sequence, D is the dialogue sequence, and θ is the training parameter; logP(y t |y <t ,D;θ) is the logarithmic form of the score distribution P, which means that given the first t-1 words y <t and dialogue sequence D, generate the i-th word y i probability.

[0179] Loss of the fact consistency auxiliary task Contrastive The calculation is as follows:

[0180]

[0181] Among them, exp() represents the exponential function operation, cos(h i ,h S ) represents the sentence vector representation h i and h S The cosine similarity, S * Indicates the generated summary, S * Indicates that the nodes with edge connections in the generated summary sentence are merged into a sentence set; the meaning of the formula is expressed as the sentence s that is consistent with the facts i ∈S * As positive samples, sentences that are inconsistent with the facts As a negative sample; through contrast loss loss Contrastive To train the model, the similarity between consistent sentences and reference summaries is maximized, while the similarity between inconsistent sentences and reference summaries is minimized.

[0182] Step B5: The dialogue summary loss obtained in step B2 is used as the main task loss main and the uncovered loss loss obtained in step B5 un covered And the loss value of the fact consistency auxiliary task loss Contrastive The final loss of the model is obtained by weighting, the gradient of each parameter in the deep learning network model is calculated by the back propagation method, and the parameters are updated using the stochastic gradient descent method.

[0183] In this embodiment, step B5 specifically includes the following steps:

[0184] Step B51: The conversation summary main task loss obtained in step B2 is main and the uncovered loss loss obtained in step B4 uncovered And the loss value of the fact consistency auxiliary task loss Contrastive Weighted to get the final loss of the model;

[0185] The final loss function of the model is calculated as follows:

[0186] loss=loss main +αloss uncovered +βloss Contrastive

[0187] Among them, α and β are hyperparameters used to balance the impact of auxiliary tasks on the model;

[0188] Step B52: Calculate the gradient of each parameter in the deep learning network model by back propagation method, and use random

[0189] gradient

[0190] The descent method updates the parameters.

[0191] Step B6: When the loss value change generated by each iteration of the deep learning network model is less than a given threshold, or the maximum number of iterations is reached, the training process of the deep learning network model M is terminated.

[0192] In this embodiment, the performance of the deep learning network model M provided by this method is compared with other models on the SAMSum and DialogSum datasets, and the results are shown in Table 1. As can be seen from Table 1, the performance of the model provided by this method is better than that of other models.

[0193] Table 1 Performance comparison of this method model and other models on SAMSum and DialogSum datasets

[0194]

[0195] This embodiment also provides a conversation summary system based on a semantic graph, including a memory, a processor, and computer program instructions stored in the memory and capable of being executed by the processor. When the processor executes the computer program instructions, the above-mentioned method steps can be implemented.

[0196] Those skilled in the art will appreciate that the embodiments of the present application can be provided as methods, systems, or computer program products. Therefore, the present application can adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application can adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0197] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the steps in the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0198] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0199] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0200] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention in any other manner. Any person skilled in the art may utilize the above-disclosed technical content to modify or modify the present invention into equivalent embodiments. However, any simple modifications, equivalent variations, and modifications to the above embodiments that do not depart from the technical content of the present invention and are based on the technical essence of the present invention remain within the scope of protection of the present invention.

Claims

1. A conversation summarization method based on semantic graph, characterized in that: The following steps are involved: Step A: Obtain multiple user conversations and their corresponding reference summaries, annotate the user conversations and reference summaries, construct training samples, and then build a training set DS; Step B: Construct a deep learning network model M based on the semantic graph and train it using the training set DS. The deep learning network model M extracts sentence features from the conversation text and its reference summary, calculates sentence pair similarities, and then constructs a semantic relevance graph to analyze the relevance and coverage between the generated summary and the reference summary. Furthermore, uncovered loss and inconsistency loss functions are designed to optimize model performance, effectively improving the model's understanding of the semantic relationship between user conversations and user conversation summaries, ensuring that the generated summary has higher factual consistency and coverage. Step C: Input the user conversation into the trained deep learning network model M and output the summary of the user conversation.

2. The conversation summarization method based on semantic graph according to claim 1 is characterized in that: The step B specifically comprises the following steps: Step B1: Initially process all training samples in the training dataset DS to obtain the dialogue sequence D and reference summary S; Step B2: Pass the dialogue sequence D obtained in step B1 through the encoder and decoder of the BART model to obtain the dialogue output representation S ‘ , normalized by the softmax function, and using cross entropy as the loss function to calculate the loss value, the loss of the main task of the dialogue summary is calculated main ; Step B3: The dialogue output obtained in step B2 is represented as S ‘ , the generated summary S is obtained by decoding the tokenizer of the BART model * ; Calculate reference summary S and generate summary S * Build a semantic correlation graph based on the text similarity of Step B4: By analyzing the semantic relevance graph obtained in step B3, the uncovered loss value loss is calculated using the negative log-likelihood as the loss function. uncovered And use InfoNCE as the loss function to calculate the loss value of the fact consistency auxiliary task loss Contrastive ; Step B5: The dialogue summary loss obtained in step B2 is used as the main task loss main and the uncovered loss loss obtained in step B5 uncovered And the loss value of the fact consistency auxiliary task loss Contrastive The final loss of the model is obtained by weighting, the gradient of each parameter in the deep learning network model is calculated by the back propagation method, and the parameters are updated using the stochastic gradient descent method; Step B6: When the loss value change generated by each iteration of the deep learning network model is less than a given threshold, or the maximum number of iterations is reached, the training process of the deep learning network model M is terminated.

3. The conversation summarization method based on semantic graph according to claim 2 is characterized in that: The step B1 specifically includes the following steps: Step B11: Traverse the training set DS, perform word segmentation and remove stop words on the dialogue texts. Each training sample in DS is represented as ds = (D, S), where D is the dialogue sequence and S is the reference summary. Dialogue D is represented as: D={t1,t2,…,t i ,…,t n } Among them, t i is the i-th speaker-utterance pair in dialogue D, i = 1, 2, …, n, and n is the number of speaker-utterance pairs in dialogue D; speaker-utterance dialogue i Expressed as: t i =(s i ,u i ) Among them, s i For the i-th utterance u i The corresponding speaker, u i is the i-th utterance, is the jth word in the i-th utterance, i = 1, 2, ..., n, j = 1, 2, ..., l i , l i is the text length of the i-th utterance; The reference summary S is represented as: S={s1,s2,…,s i ,…,s N } Among them, s i is the i-th summary sentence in the reference summary S, i = 1, 2, ..., N, and N is the number of sentences contained in the reference summary S.

4. The conversation summarization method based on semantic graph according to claim 3 is characterized in that: The step B2 specifically includes the following steps: Step B21: Pass D through the BART model tokenizer to perform padding and length truncation, and obtain the token sequence T of the dialogue sequence D as the input of the BART model encoder; The token sequence T of the conversation sequence D is calculated as follows: T=BART_Tokenizer(D) Among them, BART_Tokenizer() represents the tokenizer of the BART model; The encoder output is calculated as follows: H=[H1,H2,…,H i ,…,H n ]=BART_Encoder(T) Among them, BART_Encoder() represents the encoder of the BART model, is the jth word in the i-th utterance The corresponding word vector, l i is the text length of the i-th utterance, and d is the dimension of the feature vector; Step B22: The encoder output H obtained in step B21 is used as the input of the BART model decoder, and decoded through the self-attention mechanism, layer normalization, cross attention, and feedforward neural network to obtain the dialogue output representation S ‘ ; H′=SelfAttention(LayNorm(H)) H″=CrossAttention(LayNorm(H′+H)) S′=FFN(LayNorm(H′+H″)) Among them, LayNorm(·) represents layer normalization, SelfAttention(·) represents self-attention mechanism, CrossAttention(·) represents cross attention, and FFN(·) represents feedforward neural network; Step B23: The decoded output representation S' obtained in step B22 is normalized by the softmax function to obtain the score distribution P of the next word generated. The loss value is calculated using the negative log-likelihood as the loss function to obtain the loss of the main task of the dialogue summary. main ; The calculation formula of score distribution P is as follows: P = softmax(W·S'+b) Where P∈[0,1], W is the trainable parameter matrix, and b is the bias vector; Conversation summary main task loss loss main The calculation formula is as follows: Among them, y ＜t =(y1,y2,…,y t-1 ) represents the first t-1 words of the output sequence, S represents the reference summary, D is the dialogue sequence, and θ is the training parameter; logP(y t |y <t ,D;θ) is the logarithmic form of the score distribution P, which means that given the first t-1 words y ＜t and dialogue sequence D, generate the i-th word y i probability.

5. The conversation summarization method based on semantic graph according to claim 4 is characterized in that: The specific implementation method of step B3 is: Step B31: The dialogue output representation S' obtained in step B2 is decoded by the tokenizer of the BART model to generate the summary S * ; Generate summary S * The calculation is as follows: S * =BART_Tokenizer(S‘) Among them, BART_Tokenizer() represents the tokenizer decoding using the BART model; Generate summary S * Expressed as: in, Indicates the generation of summary S * The i-th summary sentence in, i=1,2,…,M, M represents the generated summary S * The number of sentences contained in Step B32: Compare the reference summary S obtained in step B11 with the generated summary S obtained in step B31 * The token sequence T of the reference summary S is obtained by padding and length truncation through the tokenizer of the BART model. S and generate summary S * Token sequence T S* As input to the BART model encoder; Reference digest S's token sequence T S and generate summary S * Token sequence T S* The calculation of is as follows: T S =BART_Tokenizer(S) T S* =BART_Tokenizer(S * ) Among them, BART_Tokenizer() represents the tokenizer of the BART model; The encoder output is calculated as follows: H S =[h1,h2,…,h i ,…,h N ]=BART_Encoder(T s ) Among them, BART_Encoder() represents the encoder of the BART model, h i The vector representation of the i-th sentence in the reference summary S, i = 1, 2, ..., N, where N is the number of sentences contained in the reference summary S; Indicates the generation of summary S * The vector representation of the jth sentence in , j = 1, 2, ..., M, M is the generated summary S * The number of sentences contained in Step B33: Calculate the encoder output H of the reference summary S obtained in step B32 S and generate summary S * The encoder output H S* The cosine similarity between each sentence vector representation is used to construct a semantic correlation graph; For H S Each item h i , Each of Calculate the cosine similarity. The calculation rules are as follows: Where cos() represents the cosine similarity calculation, Represents vector h i and The inner product of , which is the sum of the corresponding elements of the two vectors multiplied together; Represents vector h i and The modulus of the cosine similarity; the result of the cosine similarity is a value between -1 and 1. The closer the value is to 1, the more similar the two vectors are. The closer the value is to -1, the less similar the two vectors are. 0 means that the two vectors are orthogonal, that is, completely unrelated.

6. The conversation summarization method based on semantic graph according to claim 5 is characterized in that: The step B4 specifically includes the following steps: Step B41: Set a specified threshold value θ, analyze the semantic relevance graph obtained in step B33, stipulate that nodes with cosine similarity between sentences less than the threshold value θ do not establish edge connections, and nodes with cosine similarity between sentences greater than or equal to the threshold value θ establish edge connections, and obtain a semantic relevance matrix Matrix; Step B42: Traverse and generate summary S * The sentences in the generated summary are connected to the sentences in the reference summary S, and the nodes with edge connections in the generated summary sentences are divided into sentence sets The nodes without edge connections in the reference summary sentences are divided into sentence sets merge and for in, Representing a collection The vector representation of the i-th sentence in , i = 1, 2, ..., k, k is the number of sentences with edge connections in the generated summary sentence, h j Representing a collection The vector representation of the j-th sentence in , j = 1, 2, ..., q, where q is the number of sentences in the reference summary that have no edge connections; Step B43: The sentence set obtained in step B42 Calculate uncovered loss uncovered And the sentence set obtained in step B42 Calculate the loss of the fact consistency auxiliary task Contrastive ; Uncovered loss loss uncovered The calculation formula is as follows: Among them, y ＜t =(y1,y2,…,y t-1 ) represents the first t-1 words of the output sequence, D is the dialogue sequence, and θ is the training parameter; logP(y t |y ＜t ,D;θ) is the logarithmic form of the score distribution P, which means that given the first t-1 words y tt and dialogue sequence D, generate the i-th word y i probability; Loss of the fact consistency auxiliary task Contrastive The calculation is as follows: Among them, exp() represents the exponential function operation, cos(h i ,h s ) represents the sentence vector representation h i and h S The cosine similarity, S * Indicates the generation of a summary. Indicates that the nodes with edge connections in the generated summary sentence are merged into a sentence set; the meaning of the formula is expressed as the sentences with consistent facts As positive samples, sentences that are inconsistent with the facts As a negative sample; through contrast loss loss Contrastive To train the model, the similarity between consistent sentences and reference summaries is maximized, while the similarity between inconsistent sentences and reference summaries is minimized.

7. The conversation summarization method based on semantic graph according to claim 6 is characterized in that: The step B5 specifically includes the following steps: Step B51: The conversation summary main task loss obtained in step B2 is main and the uncovered loss loss obtained in step B4 uncovered And the loss value of the fact consistency auxiliary task loss Contrastive Weighted to get the final loss of the model; The final loss function of the model is calculated as follows: loss=loss main +αloss uncovered +βloss Contrastive Among them, α and β are hyperparameters used to balance the impact of auxiliary tasks on the model; Step B52: Calculate the gradient of each parameter in the deep learning network model through the back propagation method, and update each parameter using the stochastic gradient descent method.

8. A conversation summarization system based on semantic graph, characterized by: The method comprises a memory, a processor, and computer program instructions stored in the memory and capable of being executed by the processor. When the processor executes the computer program instructions, the method according to any one of claims 1 to 7 can be implemented.

Images