A summary generation system and method based on three-channel contrastive learning framework

Abstract

This invention relates to a summarization system and method based on a three-channel contrastive learning framework. The system includes a pre-trained model BART, a summarization quality classification unit, and a three-channel contrastive learning unit connected sequentially. The method includes: constructing the pre-trained model BART to generate an initial summary; scoring and ranking the initial summary to classify it into high-quality and low-quality summaries; performing three-channel contrastive learning based on the high-quality and low-quality summaries; training the optimized three-channel contrastive learning module by weighted summarization of the contrastive learning loss functions of the three channels; inputting the current document into the pre-trained model BART to generate a corresponding initial summary, then scoring and ranking it to classify it into high-quality and low-quality summaries, and inputting these summaries into the three-channel contrastive learning module to generate the final summary result. Compared with existing technologies, this invention can effectively handle complex semantic relationships and significantly improve the stability and accuracy of the generated summary.

Description

Technical Field

[0001] This invention relates to the field of natural language processing technology, and in particular to a summary generation system and method based on a three-channel contrastive learning framework. Background Technology

[0002] Generative summarization is a Natural Language Processing (NLP) technique that aims to generate concise and coherent summaries by understanding the semantics of input text, rather than directly extracting sentences from the original text. Unlike traditional extractive summarization, generative summarization can produce new expressions that better summarize the core content of the original text.

[0003] Generative summarization technology plays a crucial role in practical applications, helping users efficiently process massive amounts of text information and improve content comprehension and decision-making efficiency. For example, in the news industry, generative summarization can automatically extract the core content of lengthy reports, allowing readers to quickly access key information. In the finance and legal industries, this technology can be used to generate concise summaries of contracts, reports, or regulations, improving the work efficiency of professionals. Furthermore, intelligent customer service and meeting recording systems also rely on generative summarization to provide accurate information summaries and optimize user experience.

[0004] In recent years, contrastive learning has provided significant assistance in generative summarization tasks. By constructing positive and negative sample pairs, it enhances the model's ability to capture semantic information during summary generation, thereby improving the reliability of the generated summaries. For example, Chinese patent CN115374884A proposes a summary generation method based on contrastive learning. This scheme utilizes contrastive learning to reduce factual errors in summary generation while constructing positive and negative examples from the model-generated summary and the reference summary. It also comprehensively considers fidelity and generalization to score the generated summary and the reference summary separately, making it difficult to distinguish between positive and negative examples. Positive examples possess both fidelity and generalization, thus solving the problems of factual errors and weak generalization in the generated summaries.

[0005] While contrastive learning has made some progress in generative summarization tasks, it still has several limitations. First, existing generative summarization models are susceptible to outliers during sampling, especially when data distribution is biased or there are many low-quality samples, which may lead to unstable summaries. This instability not only affects the readability and information integrity of the summaries but may also lead to semantic misunderstandings, thus weakening the model's reliability and generalization ability. Second, traditional frameworks rely too heavily on simple positive and negative sample pairs, which may result in insufficient ability to handle complex semantic relationships. Finally, existing generative summarization methods have low sensitivity to subtle data changes and struggle to adapt to different text styles or domain characteristics, potentially affecting the applicability and consistency of the summaries and leading to unsatisfactory performance. Summary of the Invention

[0006] The purpose of this invention is to overcome the shortcomings of the existing technology by providing a summary generation system and method based on a three-channel contrastive learning framework, which can effectively handle complex semantic relationships and improve the stability and accuracy of the generated summary.

[0007] The objective of this invention can be achieved through the following technical solution: a summary generation system based on a three-channel contrastive learning framework, comprising a pre-trained model BART (Bidirectional and Auto-Regressive Transformers, a Transformer that combines bidirectional language modeling and autoregressive mechanisms), a summary quality division unit, and a three-channel contrastive learning unit connected in sequence. The pre-trained model BART is used to receive the input document and generate an initial summary.

[0008] The abstract quality classification unit is used to score and rank the generated initial abstracts, and to classify them into high-quality abstracts and low-quality abstracts.

[0009] The three-channel contrastive learning unit performs outlier detection for both high-quality and low-quality summaries, mines multi-level correlation information between documents, summaries, and scores, and performs contrastive learning on summaries with the same score range and summaries with different score ranges to generate the final summary.

[0010] Furthermore, the three-channel contrastive learning unit includes an encouraged contrastive learning module, a re-scoring heterogeneous graph contrastive learning module, and a similar scoring edge contrastive learning module. The input of the encouraged contrastive learning module is a high-quality summary, which is used to divide the high-quality summary into high-scoring summaries and low-scoring summaries. By comparing the high-scoring and low-scoring summaries, an outlier detection algorithm is used to detect outliers.

[0011] The input to the re-scoring heterogeneous graph comparison learning module is a high-quality summary and a low-quality summary. It is used to construct a special heterogeneous graph containing document, summary and score nodes, and to design the edge types of document-summary and summary-score to mine multi-level association information between documents, summaries and scores.

[0012] The input to the similar rating edge comparison learning module is a high-quality summary and a low-quality summary. It is used to compare and learn summaries of the same rating interval and different rating intervals to capture rating differences.

[0013] A summary generation method based on a three-channel contrastive learning framework includes the following steps:

[0014] S1. Construct a pre-trained model BART, input the document samples into BART, and generate the corresponding initial summary.

[0015] S2. Score and rank the initial abstracts to distinguish between high-quality and low-quality abstracts;

[0016] S3. Based on high-quality and low-quality summaries, perform three-channel comparative learning:

[0017] For high-quality summaries, we further divide them into high-scoring and low-scoring summaries, and then sample them using a self-resampled algorithm. We then dynamically adjust the margins using the BADK (Bootstrap Aggregating and Dynamic Knowledge) algorithm to detect outliers in the sampling.

[0018] To address high-quality and low-quality summaries, a heterogeneous document-summary-scoring graph is designed. The node representations are updated through a GCN (Graph Convolutional Network), and comparative learning at the graph level is performed based on the heterogeneous graph.

[0019] For high-quality and low-quality abstracts, they are divided into multiple intervals according to the score. The similarity between abstracts from the same interval and the similarity between abstracts from different intervals are compared and learned.

[0020] S4. The contrast learning loss functions of the three channels are weighted and accumulated before training to obtain the optimized three-channel contrast learning module.

[0021] S5. Input the current document into the pre-trained model BART to generate the corresponding initial summary. Then, score and sort the documents to distinguish between high-quality and low-quality summaries. Input the summaries into the three-channel contrastive learning module to generate the final summary result.

[0022] Furthermore, the pre-trained model BART specifically introduces the Pfeiffer adapter on the basis of BART. The Pfeiffer adapter optimizes the model's adaptability to specific tasks without significantly increasing computational overhead by adding a set of lightweight trainable parameters to BART.

[0023] Furthermore, in step S3, the BADK algorithm mitigates the impact of outliers by introducing distance-based outlier detection. BADK calculates the Euclidean distance between samples and uses the D-kNN (k-neighbors) method to determine whether outliers exist during the sampling process. If a sample is too far from other samples (i.e., exceeds a certain threshold), it will be considered an outlier by the model and excluded. The specific formula is as follows:

[0024] ,

[0025] in, and These are the feature vectors of two high-quality summary samples. and These are their values ​​on the m-th feature, where N is the dimension of the feature. for and The distance between them, if d ( , If )>δ, where δ is a preset threshold, then the sample It was identified as an outlier.

[0026] Furthermore, the heterogeneous graph of document-summary-rating in step S3 includes heterogeneous graph nodes and heterogeneous graph edges. There are three types of heterogeneous graph nodes: documents in the dataset are used as document nodes, summaries output by the pre-trained model are used as summary nodes, and the ratings corresponding to the summaries are used as rating nodes.

[0027] The heterogeneous graph edges are of two types: edges from document nodes to summary nodes, and edges from summary nodes to rating nodes.

[0028] The process of updating the node representation via GCN in step S3 is as follows:

[0029] For the two types of edges in heterogeneous graphs, graph convolution operations are used to process them in order to update the edges of both types.

[0030] For heterogeneous Figure 3 The document node, summary node, and rating node are updated sequentially using graph convolution.

[0031] Furthermore, the update process for the two types of edges is as follows:

[0032] For document node v d and summary node v s Information propagation occurs through the edge from document to summary, and the edge is updated using the following formula:

[0033] ,

[0034] Where, h′ s It is the summary node v s The update indicates that N(v) s ) is related to the summary node v s The set of adjacent document nodes, W DA It is the convolution weight matrix from document to summary, h d It is a document node v d The feature representation of , where σ() is the activation function;

[0035] For summary nodes and rating nodes, information is propagated through the edge from summary to rating, and the update formula for this edge is as follows:

[0036] ,

[0037] Where, h′ c It is the rating node v c The update indicates that N(v) c ) is related to the scoring node v c The set of adjacent summary nodes, W SC It is the convolution weight matrix from summary to score, h s It is the summary node v s The characteristics are represented.

[0038] Furthermore, the update process for the three types of nodes is as follows:

[0039] In graph convolutional networks, document nodes update their own features by interacting with the features of neighboring summary nodes. The update process considers the edge weights between document and summary nodes, and information from summary nodes is passed to document nodes through graph convolution. The update formula for document nodes is:

[0040] ,

[0041] Among them, h doc It is the feature vector of the document node. It is the feature vector of the updated document node. W is the feature vector of the i-th summary node. doc and W su is the weight matrix in graph convolution, representing the edge weights between document nodes and summary nodes, and σ() is the activation function;

[0042] Next, the summary node is updated. The update process considers feature information from the document node and the rating node. The convolution process weights and converges the features of the document node and the rating node to update the features of the summary node. The update formula for the summary node is:

[0043] ,

[0044] Among them, h su It is the feature vector of the summary node. It is the feature vector of the updated summary node. h is the feature vector of the i-th document node. sc W is the feature vector of the scoring node. su、 W doc and W sc is the weight matrix in graph convolution, representing the edge weights between summary nodes and document nodes, and rating nodes, respectively; σ() is the activation function.

[0045] Finally, the scoring nodes are updated. The update process of the scoring nodes mainly relies on the features of the summary nodes. The scoring nodes interact with the information of the summary nodes through graph convolutional layers to update their own features. The formula for updating the scoring nodes is as follows:

[0046] ,

[0047] Among them, h sc It is the feature vector of the scoring node. It is the feature vector of the updated scoring node. W is the feature vector of the i-th summary node. sc and W su is the weight matrix in graph convolution, representing the edge weights between the scoring node and the summary node, respectively, and σ() is the activation function.

[0048] Furthermore, in step S3, graph-level contrastive learning is performed based on the heterogeneous graph. The goal of this contrastive learning is to maximize the similarity between positive samples, i.e., related nodes, and minimize the similarity between negative samples, i.e., unrelated nodes.

[0049] This comparative learning includes: comparative learning between documents and summaries, and comparative learning between summaries and ratings.

[0050] Furthermore, in step S3, the similarity between summaries from the same interval and the similarity between summaries from different intervals are compared and learned. The goal of this comparison and learning is to maximize the similarity between summaries from the same interval, i.e., minimize the distance, and minimize the similarity between summaries from different intervals, i.e., maximize the distance.

[0051] This contrastive learning employs a contrastive loss function to calculate the distance difference between positive and negative samples:

[0052] ,

[0053] in, For the interval corresponding to abstract i, For the interval corresponding to abstract j, e i Let e ​​be the embedding vector of the abstract i. j Let d(e be the embedding vector of summary j) i e j ) for e i and e j The Euclidean distance between them;

[0054] This indicates that abstracts i and j belong to the same interval. This indicates that they belong to different intervals, and m is a predefined margin used to ensure that there is sufficient distance between the summaries of different intervals.

[0055] Compared with the prior art, the present invention has the following advantages:

[0056] This invention designs a pre-trained model BART, a summary quality classification unit, and a three-channel contrastive learning unit connected sequentially. The BART model generates an initial summary, which is then scored and ranked by the summary quality classification unit to identify high-quality and low-quality summaries. The three-channel contrastive learning unit performs outlier detection on both high-quality and low-quality summaries, mines multi-level correlations between documents, summaries, and scores, and performs comparative learning on summaries with the same and different score ranges to generate the final summary. This effectively handles complex semantic relationships and significantly improves the stability and accuracy of the generated summary.

[0057] In this invention, the three-channel contrastive learning unit includes an encouraged contrastive learning module, a re-scoring heterogeneous graph contrastive learning module, and a similar scoring edge contrastive learning module. First, the encouraged contrastive learning module compares high-scoring and low-scoring summaries and employs an outlier detection algorithm to reduce the impact of outliers on the model's sampling process, thereby improving the stability and overall quality of the generated content. Second, the re-scoring heterogeneous graph contrastive learning module constructs a special heterogeneous graph containing document, summary, and score nodes, and designs document-summary and summary-scoring edge types to deeply explore the multi-level association information between documents, summaries, and scores, effectively improving the model's ability to model complex relationships. Finally, the similar scoring edge contrastive learning module refines the model's ability to capture score differences by comparing summaries with the same score range and summaries with different score ranges, enhancing the model's performance in scoring accuracy. Experiments have verified that this three-channel contrastive learning unit can effectively improve the performance of generative text summarization.

[0058] This invention constructs a pre-trained model BART based on the Pfeiffer adapter. An adapter is introduced on the basis of BART. The adapter optimizes the model’s adaptability to specific tasks without significantly increasing computational overhead by adding a set of lightweight trainable parameters to the model. These newly added trainable parameters are gradually adjusted during training, thereby improving the model’s performance in the summarization task and ultimately improving the quality of the generated summaries.

[0059] In encouraging contrastive learning, this invention considers that conventional bootstrap resampling algorithms are susceptible to outliers. Outliers typically negatively impact margin adjustment, causing the model to overemphasize unrepresentative summaries during optimization, thus affecting its learning ability. This is particularly problematic in summary quality discrimination tasks, where outliers may prevent the model from correctly assigning probabilities of high-quality and low-quality summaries in the early stages of training. Therefore, this invention employs the BADK algorithm to mitigate the impact of outliers, aiming to dynamically adjust margins and enhance the ability to discriminate summary quality. The BADK algorithm effectively mitigates the effects of outliers by introducing distance-based outlier detection.

[0060] This invention designs a document-summary-rating heterogeneous graph for comparative learning of re-rating heterogeneous graphs. Unlike traditional methods that establish entity node and edge types based on datasets, this invention designs a heterogeneous graph with three node types: documents from the dataset as document nodes, summaries output by the pre-trained model as summary nodes, and summaries corresponding to ratings as rating nodes. The heterogeneous graph also has two edge types: edges from document nodes to summary nodes and edges from summary nodes to rating nodes. Using this heterogeneous graph, the model can better dynamically understand the relationships between the three elements, enhancing its ability to capture complex semantic relationships.

[0061] This invention uses a Graph Convolutional Network (GCN) to update node representations. After each graph convolution operation, the node representation is updated. This invention effectively aggregates information from neighboring nodes using graph convolution operations, thereby updating the feature representation of the target node. Ultimately, this results in nodes that reflect the deep semantic relationships between documents, summaries, and ratings, and the model can learn even deeper node representations. Furthermore, through multiple rounds of graph convolution operations, the model progressively optimizes node representations, fully capturing the relationships between documents, summaries, and ratings. The core of information propagation lies in the aggregation and updating of information between nodes. Through these operations, the model can more accurately understand the complex semantic relationships among the three, thereby improving the accuracy of summary generation and rating analysis.

[0062] This invention compares the similarity between summaries from the same interval with the similarity between summaries from different intervals. The ultimate goal of this comparative learning is to maximize the similarity between summaries from the same interval and minimize the similarity between summaries from different intervals. This invention designs a comparative loss function to calculate the distance difference between positive and negative samples. By adjusting the model parameters, the loss function minimizes the distance between summaries from the same interval and maximizes the distance between summaries from different intervals, thereby improving the model's sensitivity to subtle semantic differences and its recognition ability. Attached Figure Description

[0063] Figure 1 This is a schematic diagram of the system structure of the present invention;

[0064] Figure 2 This is a schematic diagram of the method flow of the present invention;

[0065] Figure 3 This is a schematic diagram of the application framework for an example.

[0066] Figure 4 This is a diagram illustrating the document node update process.

[0067] Figure 5 This is a schematic diagram of the summary node update process;

[0068] Figure 6 This is a diagram illustrating the scoring node update process.

[0069] The markings in the figure are as follows: 1. Pre-trained model BART, 2. Summary quality division unit, 3. Three-channel contrastive learning unit, 301. Encouraging contrastive learning module, 302. Re-scoring heterogeneous graph contrastive learning module, 303. Similarity scoring edge contrastive learning module. Detailed Implementation

[0070] The present invention will now be described in detail with reference to the accompanying drawings and specific embodiments.

[0071] Example 1

[0072] like Figure 1 As shown, a summary generation system based on a three-channel contrastive learning framework includes a pre-trained model BART1, a summary quality division unit 2, and a three-channel contrastive learning unit 3 connected in sequence. The pre-trained model BART is used to receive the input document and generate an initial summary.

[0073] Abstract quality classification unit 2 is used to score and rank the generated initial abstracts, and to classify them into high-quality abstracts and low-quality abstracts;

[0074] The three-channel contrastive learning unit 3 performs outlier detection for both high-quality and low-quality summaries, mines multi-level correlation information between documents, summaries and scores, and performs contrastive learning on summaries with the same score range and summaries with different score ranges to generate the final summary.

[0075] The three-channel contrast learning unit 3 includes an encouragement contrast learning module 301, a re-scoring heterogeneous graph contrast learning module 302, and a similar scoring edge contrast learning module 303. The input of the encouragement contrast learning module 301 is a high-quality summary, which is used to divide the high-quality summary into high-scoring summaries and low-scoring summaries. By comparing the high-scoring and low-scoring summaries, outlier detection is performed using an outlier detection algorithm.

[0076] The input to the re-scoring heterogeneous graph contrastive learning module 302 is a high-quality summary and a low-quality summary. It is used to construct a special heterogeneous graph containing document, summary and score nodes, and to design the document-summary and summary-score edge types to mine multi-level association information between documents, summaries and scores.

[0077] The input to the similar rating edge comparison learning module 303 is a high-quality summary and a low-quality summary. It is used to compare and learn summaries of the same rating interval and different rating intervals to capture rating differences.

[0078] In the three-channel contrastive learning unit 3 designed in this scheme, firstly, the contrastive learning module 301 improves the stability and overall quality of the generated data by comparing high-scoring and low-scoring summaries and using appropriate outlier detection algorithms to reduce the impact of outliers on the model's sampling process. Secondly, the re-scoring heterogeneous graph contrastive learning module 302 constructs a special heterogeneous graph containing document, summary, and score nodes, designs document-summary and summary-score edge types, and deeply explores the multi-level correlation information between documents, summaries, and scores, effectively improving the model's ability to model complex correlations. Finally, the similar score edge contrastive learning module 303 refines the model's ability to capture score differences by comparing summaries with the same score range and summaries with different score ranges, enhancing the model's performance in scoring accuracy.

[0079] Example 2

[0080] Based on the summary generation system proposed in Embodiment 1, a summary generation method based on a three-channel contrastive learning framework is implemented, such as... Figure 2 As shown, it includes the following steps:

[0081] S1. Construct a pre-trained model BART, input the document samples into BART, and generate the corresponding initial summary.

[0082] S2. Score and rank the initial abstracts to distinguish between high-quality and low-quality abstracts;

[0083] S3. Based on high-quality and low-quality summaries, perform three-channel comparative learning:

[0084] For high-quality abstracts, they are further divided into high-scoring and low-scoring abstracts, and then sampled using a self-resampled resampling algorithm. The margins are dynamically adjusted using the BADK algorithm to detect outliers in the sampling.

[0085] To address high-quality and low-quality abstracts, a document-abstract-scoring heterogeneous graph is designed. The node representations are updated using GCN, and graph-level comparative learning is performed based on the heterogeneous graph.

[0086] For high-quality and low-quality abstracts, they are divided into multiple intervals according to the score. The similarity between abstracts from the same interval and the similarity between abstracts from different intervals are compared and learned.

[0087] S4. The contrast learning loss functions of the three channels are weighted and accumulated before training to obtain the optimized three-channel contrast learning module.

[0088] S5. Input the current document into the pre-trained model BART to generate the corresponding initial summary. Then, score and sort the documents to distinguish between high-quality and low-quality summaries. Input the summaries into the three-channel contrastive learning module to generate the final summary result.

[0089] This embodiment applies the above-described solution to build such a system. Figure 3 The model framework shown, and its main application process, includes:

[0090] 1. Constructing the pre-trained model BART based on the Pfeiffer adapter: To better adapt the pre-trained model BART to generative text summarization tasks, this embodiment introduces an adapter module on top of BART. The adapter optimizes the model's adaptability to specific tasks without significantly increasing computational overhead by adding a set of lightweight trainable parameters to the model. These new parameters are gradually adjusted during training, thereby improving the model's performance in summarization tasks and ultimately enhancing the quality of the generated summaries.

[0091] 2. The summaries generated by the pre-trained model are scored, and then sorted and divided into high-quality and low-quality summaries according to the scores.

[0092] 3. After segmenting high-quality summaries, use a bootstrap resampling algorithm and dynamically adjust the margins using the BADK algorithm: After selecting high-quality summaries, dynamically adjust the margins using a bootstrap resampling algorithm to adapt to the model's needs at different training stages. In the early stages of training, the model's probability distribution may be very coarse, requiring larger margins to force the model to pay more attention to differences in summary quality. As training progresses and the model can more accurately distinguish between high-quality and low-quality summaries, the margins can be appropriately reduced.

[0093] However, conventional bootstrap resampling algorithms are susceptible to outliers, which typically negatively impact margin adjustment. This causes the model to overemphasize unrepresentative summaries during optimization, thus affecting its learning ability. This is particularly problematic in summary quality assessment tasks, where outliers may prevent the model from correctly assigning probabilities of high-quality and low-quality summaries during early training, hindering its full potential. Therefore, the BADK algorithm is used to mitigate the impact of outliers, aiming to dynamically adjust margins and enhance the ability to discriminate summary quality. The BADK algorithm effectively mitigates the effects of outliers by introducing distance-based outlier detection.

[0094] Specifically, BADK calculates the Euclidean distance between samples and uses the D-kNN (k-neighbors) method to determine if outliers exist during the sampling process. If a sample's distance from other samples is too large (i.e., exceeds a certain threshold), it is considered an outlier and excluded by the model. In this way, the BADK algorithm effectively mitigates the impact of outliers on the training process, ensuring that the model learns only from valid samples, indirectly improving the robustness of training. The specific formula is as follows:

[0095] (1)

[0096] in, and These are the feature vectors of two high-quality summary samples. and These are their values ​​on the m-th feature, where N is the dimension of the feature. If d( , If )>δ, where δ is a preset threshold, then the sample It was identified as an outlier.

[0097] Finally, the calculated boundary value Margin t This is applied to the loss function, encouraging the model to more accurately distinguish the score differences between high-quality and low-quality summaries. In this way, the loss function incentivizes the model to pay more attention to the score differences between high-quality and low-quality summaries during training, thereby improving the quality of the generated summaries.

[0098] Design a document-summary-rating heterogeneous graph: To enhance the model's ability to capture complex semantic relationships, heterogeneous graphs can help the model better dynamically understand the connections between the three. Unlike traditional methods of establishing entity node types and edge types based on the dataset, this scheme designs three types of heterogeneous graph nodes: documents in the dataset as document nodes, summaries output by the pre-trained model as summary nodes, and summaries corresponding to ratings as rating nodes. There are two types of heterogeneous graph edges: edges from document nodes to summary nodes, and edges from summary nodes to rating nodes.

[0099] After the initial training, a secondary training mechanism is introduced, allowing the model to further optimize from the learned node representations. During this secondary training phase, the model can more accurately capture the potential impact of scoring, thereby enhancing its understanding of semantic relationships. This secondary training mechanism can be viewed as "deep optimization," gradually improving the model's ability to model complex semantic relationships through multiple iterative updates.

[0100] 5. Updating Node Representations via GCN: The main idea is to update node representations by aggregating information from neighboring nodes. Specifically, the features of the three node types are first initialized based on their content, and then the node representation is updated using GCN (Graph Convolutional Network). Graph convolution operations update the target node's features by aggregating information from neighboring nodes. For heterogeneous graphs, different graph convolution operations are used to handle different types of edges.

[0101] For document node v d and summary node v s Information propagation occurs through the edge from document to summary, and the edge is updated using the following formula:

[0102] , (2)

[0103] Where, h′ s It is the summary node v s The update indicates that N(v) s ) is related to the summary node v s The set of adjacent document nodes, W DA It is the convolution weight matrix from document to summary, h d It is a document node v d The feature representation of σ() is the activation function.

[0104] For summary nodes and rating nodes, information is propagated through the edge from summary to rating, and the update formula for this edge is as follows:

[0105] , (3)

[0106] Where, h′ c It is the rating node v c The update indicates that N(v) c ) is related to the scoring node v c The set of adjacent summary nodes, W SC It is the convolution weight matrix from summary to score, h s It is the summary node v s The characteristics are represented.

[0107] After each graph convolution operation, the node representation is updated. The graph convolution operation used effectively aggregates information from neighboring nodes, thereby updating the feature representation of the target node. The final result is a node that reflects the deep semantic relationships between the document, summary, and rating, and the model can learn even deeper node representations.

[0108] Through multi-round graph convolution operations, the model progressively optimizes the node representations, enabling a full capture of the relationships between documents, summaries, and scores. The core of information dissemination lies in the aggregation and updating of information between nodes. Through these operations, the model can more accurately understand the complex semantic relationships among the three, thereby improving the accuracy of summary generation and score analysis.

[0109] In Graph Convolutional Networks (GCNs), document nodes update their own features by interacting with the features of neighboring summary nodes. The update process takes into account the edge weights between document nodes and summary nodes, and passes information from summary nodes to document nodes through graph convolution. Figure 4 This describes the update process for document nodes. The update formula for document nodes is:

[0110] , (4)

[0111] Among them, h doc It is the feature vector of the document node. W is the feature vector of the i-th summary node. doc and W su is the weight matrix in graph convolution, representing the edge weights between document nodes and summary nodes, and σ() is the activation function;

[0112] Next, the summary node is updated, taking into account the feature information from the document node and the rating node. Figure 5 This describes the update process for the summary node. Convolutional processing weights and aggregates the features of the document node and the rating node to update the summary node's features. The update formula for the summary node is:

[0113] , (5)

[0114] Among them, h su It is the feature vector of the summary node. h is the feature vector of the i-th document node. sc W is the feature vector of the scoring node. su、 W doc and W sc is the weight matrix in graph convolution, representing the edge weights between summary nodes and document nodes, and rating nodes, respectively; σ() is the activation function.

[0115] Finally, the scoring nodes are updated, and this update process mainly relies on the features of the summary nodes. The scoring nodes interact with the information from the summary nodes through graph convolutional layers to update their own features. Figure 6 For the update process of the scoring nodes, the update formula for the scoring nodes is:

[0116] , (6)

[0117] Among them, h sc It is the feature vector of the scoring node. W is the feature vector of the i-th summary node. sc and W su is the weight matrix in graph convolution, representing the edge weights between the scoring node and the summary node, respectively, and σ() is the activation function.

[0118] 6. Graph-level contrastive learning based on heterogeneous graphs: Within the framework of contrastive learning, the goal is to optimize the model's learning process by maximizing the similarity between positive samples (relevant nodes) and minimizing the similarity between negative samples (irrelevant nodes). Specifically, it consists of two parts: contrastive learning between documents and summaries, and contrastive learning between summaries and ratings.

[0119] The document-summary similarity matrix is ​​obtained by calculating the inner product (dot product) of document node features and summary node features. To account for the correlation between documents and summaries, edge weights are used to adjust the similarity matrix. Within the contrastive learning framework, the model is optimized by minimizing the similarity between positive samples (relevant nodes) and maximizing the similarity between negative samples (irrelevant nodes).

[0120] 7. Divide the abstract into multiple intervals according to the score.

[0121] 8. Comparative learning is performed on the similarity between summaries from the same interval and the similarity between summaries from different intervals: The ultimate goal of comparative learning is to improve the model's sensitivity to subtle semantic differences by maximizing the similarity between summaries from the same interval and minimizing the similarity between summaries from different intervals. This scheme designs a comparative loss function to optimize this objective. The key idea of ​​this loss function is: for each summary i, calculate the distance between it and summaries from the same interval (positive samples) and summaries from different intervals (negative samples), and then optimize the result.

[0122] The objective is to maximize the similarity between summaries of the same interval (i.e., minimize the distance) and minimize the similarity between summaries of different intervals (i.e., maximize the distance). To achieve this objective, the following contrastive loss function L is designed. con The loss function calculates the distance difference between positive and negative samples:

[0123] , (7)

[0124] in, This indicates that abstracts i and j belong to the same interval (positive samples). This indicates that they belong to different intervals (negative samples), and m is a predefined margin used to ensure that there is sufficient distance between the summaries of different intervals.

[0125] This loss function enhances the model's ability to recognize subtle semantic differences by adjusting the model parameters to minimize the distance between summaries of the same interval and maximize the distance between summaries of different intervals.

[0126] 9. After weighted summation of the contrastive learning loss functions for the three channels, training is performed:

[0127] To dynamically ensure the model's generation and evaluation capabilities, the aforementioned multiple objectives are combined with weight values ​​into a general loss function.

[0128] In summary, existing generative summarization models are susceptible to outliers during the sampling process, especially when data distribution is biased or there are many low-quality samples. This can lead to unstable summaries. Such instability not only affects readability and information integrity but can also cause semantic misunderstandings, potentially weakening the model's reliability and generalization ability. Therefore, this solution compares high-scoring and low-scoring summaries and uses an appropriate outlier detection algorithm to reduce the impact of outliers on the model's sampling process, thereby improving the stability and overall quality of the generated summaries.

[0129] Furthermore, traditional frameworks rely too heavily on simple positive and negative sample pairs, which may lead to insufficient ability to handle complex semantic relationships. To address this, this solution constructs a special heterogeneous graph containing document, summary, and rating nodes, designs edge types for document-summary and summary-rating, and delves into the multi-level association information between documents, summaries, and ratings, effectively improving the model's ability to model complex relationships.

[0130] Furthermore, existing generative summarization methods are less sensitive to subtle data changes and struggle to adapt to different text styles or domain characteristics, potentially affecting the applicability and consistency of the summaries and resulting in unsatisfactory performance. To address this, this approach refines the model's ability to capture score differences by comparing summaries from the same and different score ranges, thereby enhancing the model's performance in scoring accuracy.

Claims

1. A summary generation system based on a three-channel contrastive learning framework, characterized in that, It includes a pre-trained model BART (1), a summary quality division unit (2), and a three-channel contrast learning unit (3) connected in sequence. The pre-trained model BART (1) is used to receive the input document and generate an initial summary. The abstract quality classification unit (2) is used to score and rank the generated initial abstracts, and to classify them into high-quality abstracts and low-quality abstracts. The three-channel contrast learning unit (3) performs outlier detection for high-quality and low-quality summaries, mines multi-level correlation information between documents, summaries and scores, and performs contrast learning on summaries with the same score range and summaries with different score ranges to generate the final summary. The three-channel contrast learning unit (3) includes an encouraged contrast learning module (301), a re-scoring heterogeneous graph contrast learning module (302), and a similar scoring edge contrast learning module (303). The input of the encouraged contrast learning module (301) is a high-quality summary, which is used to divide the high-quality summary into high-scoring summaries and low-scoring summaries. By comparing the high-scoring and low-scoring summaries, an outlier detection algorithm is used to detect outliers. The input to the re-scoring heterogeneous graph comparison learning module (302) is a high-quality summary and a low-quality summary. It is used to construct a special heterogeneous graph containing document, summary and score nodes, and to design the edge types of document-summary and summary-score to mine multi-level association information between documents, summaries and scores. The input to the similar rating edge comparison learning module (303) is a high-quality summary and a low-quality summary, which are used to compare and learn summaries of the same rating interval and different rating intervals to capture rating differences.

2. A summarization method based on a three-channel contrastive learning framework, characterized in that, Includes the following steps: S1. Construct a pre-trained model BART, input the document samples into BART, and generate the corresponding initial summary. S2. Score and rank the initial abstracts to distinguish between high-quality and low-quality abstracts; S3. Based on high-quality and low-quality summaries, perform three-channel comparative learning: For high-quality abstracts, they are further divided into high-scoring and low-scoring abstracts, and then sampled using a self-resampled resampling algorithm. The margins are dynamically adjusted using the BADK algorithm to detect outliers in the sampling. To address high-quality and low-quality summaries, a heterogeneous document-summary-scoring graph is designed. The representation of nodes is updated through a graph convolutional network (GCN), and comparative learning at the graph level is performed based on the heterogeneous graph. For high-quality and low-quality abstracts, they are divided into multiple intervals according to the score. The similarity between abstracts from the same interval and the similarity between abstracts from different intervals are compared and learned. S4. The contrast learning loss functions of the three channels are weighted and accumulated before training to obtain the optimized three-channel contrast learning module. S5. Input the current document into the pre-trained model BART to generate the corresponding initial summary. Then, score and sort the documents to distinguish between high-quality and low-quality summaries. Input the summaries into the three-channel contrastive learning module to generate the final summary result.

3. The summarization method based on a three-channel contrastive learning framework according to claim 2, characterized in that, The pre-trained model BART specifically introduces the Pfeiffer adapter on top of BART. The Pfeiffer adapter optimizes the model's adaptability to specific tasks without significantly increasing computational overhead by adding a set of lightweight trainable parameters to BART.

4. The summarization method based on a three-channel contrastive learning framework according to claim 2, characterized in that, In step S3, the BADK algorithm mitigates the impact of outliers by introducing distance-based outlier detection. BADK calculates the Euclidean distance between samples and uses the D-kNN method to determine if outliers exist during the sampling process. If the distance between a sample and other samples exceeds a preset distance threshold, it will be considered an outlier and excluded by the model. The specific formula is as follows: , in, and These are the feature vectors of two high-quality summary samples. and These are their values ​​on the m-th feature, where N is the dimension of the feature. for and The distance between them, if d ( , If )>δ, where δ is a preset threshold, then the sample It was identified as an outlier.

5. The summarization method based on a three-channel contrastive learning framework according to claim 2, characterized in that, In step S3, the heterogeneous graph of document-summary-rating includes heterogeneous graph nodes and heterogeneous graph edges. There are three types of heterogeneous graph nodes: documents in the dataset are used as document nodes, summaries output by the pre-trained model are used as summary nodes, and the summaries corresponding to the ratings are used as rating nodes. The heterogeneous graph edges are of two types: edges from document nodes to summary nodes, and edges from summary nodes to rating nodes. The process of updating the node representation via GCN in step S3 is as follows: For the two types of edges in heterogeneous graphs, graph convolution operations are used to process them in order to update the edges of both types. For the three types of nodes in the heterogeneous graph, the document node, summary node and rating node are updated sequentially by graph convolution.

6. The summarization method based on a three-channel contrastive learning framework according to claim 5, characterized in that, The update process for the two types of edges is as follows: For document node v d and summary node v s Information propagation occurs through the edge from document to summary, and the edge is updated using the following formula: ， Where, h′ s It is the summary node v s The update indicates that N(v) s ) is related to the summary node v s The set of adjacent document nodes, W DA It is the convolution weight matrix from document to summary, h d It is a document node v d The feature representation of , where σ() is the activation function; For summary nodes and rating nodes, information is propagated through the edge from summary to rating, and the update formula for this edge is as follows: ， Where, h′ c It is the rating node v c The update indicates that N(v) c ) is related to the scoring node v c The set of adjacent summary nodes, W SC It is the convolution weight matrix from summary to score, h s It is the summary node v s The characteristics are represented.

7. The summarization method based on a three-channel contrastive learning framework according to claim 6, characterized in that, The update process for the three types of nodes is as follows: In graph convolutional networks, document nodes update their own features by interacting with the features of neighboring summary nodes. The update process considers the edge weights between document and summary nodes, and information from summary nodes is passed to document nodes through graph convolution. The update formula for document nodes is: , Among them, h doc It is the feature vector of the document node. It is the feature vector of the updated document node. W is the feature vector of the i-th summary node. doc and W su is the weight matrix in graph convolution, representing the edge weights between document nodes and summary nodes, and σ() is the activation function; Next, the summary node is updated. The update process considers feature information from the document node and the rating node. The convolution process weights and converges the features of the document node and the rating node to update the features of the summary node. The update formula for the summary node is: , Among them, h su It is the feature vector of the summary node. It is the feature vector of the updated summary node. h is the feature vector of the i-th document node. sc W is the feature vector of the scoring node. su、 W doc and W sc is the weight matrix in graph convolution, representing the edge weights between summary nodes and document nodes, and rating nodes, respectively; σ() is the activation function. Finally, the scoring nodes are updated. The update process of the scoring nodes mainly relies on the features of the summary nodes. The scoring nodes interact with the information of the summary nodes through graph convolutional layers to update their own features. The formula for updating the scoring nodes is as follows: , Among them, h sc It is the feature vector of the scoring node. It is the feature vector of the updated scoring node. W is the feature vector of the i-th summary node. sc and W su is the weight matrix in graph convolution, representing the edge weights between the scoring node and the summary node, respectively, and σ() is the activation function.

8. The summarization method based on a three-channel contrastive learning framework according to claim 2, characterized in that, In step S3, graph-level contrastive learning is performed based on heterogeneous graphs. The goal of this contrastive learning is to maximize the similarity between positive samples and minimize the similarity between negative samples. This comparative learning includes: comparative learning between documents and summaries, and comparative learning between summaries and ratings.

9. A summarization method based on a three-channel contrastive learning framework according to claim 2, characterized in that, In step S3, the similarity between summaries from the same interval and the similarity between summaries from different intervals are compared and learned. The goal of this comparison and learning is to maximize the similarity between summaries from the same interval, i.e., minimize the distance; and to minimize the similarity between summaries from different intervals, i.e., maximize the distance. This contrastive learning employs a contrastive loss function to calculate the distance difference between positive and negative samples: ， in, For the interval corresponding to abstract i, For the interval corresponding to abstract j, e i Let e ​​be the embedding vector of the abstract i. j Let d(e be the embedding vector of summary j) i e j ) for e i and e j The Euclidean distance between them; This indicates that abstracts i and j belong to the same interval. This indicates that they belong to different intervals, and m is a predefined margin used to ensure that there is sufficient distance between the summaries of different intervals.

Images