Abstract generation system and method based on three-channel comparative learning framework

Abstract

The invention relates to an abstract generation system and method based on a three-channel comparative learning framework, the system comprises a pre-training model BART, an abstract quality division unit and a three-channel comparative learning unit which are connected in sequence, and the method comprises the following steps: constructing the pre-training model BART, and generating a primary abstract; scoring and sorting the primary abstract, and dividing a high-quality abstract and a low-quality abstract; performing three-channel comparison learning based on the high-quality abstract and the low-quality abstract; carrying out weighted accumulation on the contrast learning loss functions of the three channels and then carrying out training to obtain an optimized three-channel contrast learning module; a current document is input into a pre-training model BART, a corresponding initial abstract is generated, then scoring and sorting are carried out, a corresponding high-quality abstract and a corresponding low-quality abstract are divided, and the high-quality abstract and the low-quality abstract are input into a three-channel comparison learning module to generate a final abstract result. Compared with the prior art, the method has the advantages that complex semantic relations can be effectively processed, and the abstract generation stability and precision are remarkably improved.

Description

Technical Field

[0001] The present invention relates to the technical field of natural language processing, and in particular to a summary generation system and method based on a three-channel contrastive learning framework. Background Art

[0002] Generative summarization is a natural language processing (NLP) technique that aims to generate concise and coherent summaries by understanding the semantics of input text, rather than directly extracting sentences from the original text. Unlike traditional extractive summarization, generative summarization can generate new expressions to better summarize the core content of the original text.

[0003] Generative summarization technology plays a vital role in practical applications, helping users efficiently process massive amounts of textual information, improving content comprehension and decision-making. For example, in the news industry, generative summarization can automatically extract the core content of long reports, allowing readers to quickly access key information. In the financial and legal industries, this technology can be used to generate concise summaries of contracts, reports, or regulations, improving professional efficiency. Furthermore, intelligent customer service and meeting record-keeping systems also rely on generative summaries to provide accurate information summaries and optimize the user experience.

[0004] In recent years, contrastive learning has provided significant assistance in generative summarization tasks. By constructing positive and negative sample pairs, it enhances the model's ability to capture semantic information when generating summaries, thereby improving the reliability of summary generation. For example, Chinese patent CN115374884A proposes a summary generation method based on contrastive learning. This scheme uses contrastive learning to reduce factual errors in summary generation while constructing positive and negative examples from model-generated summaries and reference summaries. It also comprehensively considers fidelity and generalization to score the generated summaries and reference summaries respectively. This makes it difficult to distinguish between the constructed positive and negative examples, while the positive examples are both faithful and generalizable, thus solving the problem of factual errors and weak generalization in the generated summaries.

[0005] Although contrastive learning has made some progress in generative summarization tasks, it still has several limitations. First, existing generative summarization models are easily affected by outliers during the sampling process. Especially when there is a bias in the data distribution or there are many low-quality samples, the model may generate unstable summary content. This instability not only affects the readability and information completeness of the summary, but may also lead to semantic misunderstandings, which may weaken the reliability and generalization ability of the model. Secondly, traditional frameworks rely too much on simple positive and negative sample pairs, which may lead to their insufficient ability to handle complex semantic relationships. Finally, existing generative summarization methods are less sensitive to subtle data changes and have difficulty adapting to different text styles or domain characteristics, which may affect the applicability and consistency of the summary, resulting in unsatisfactory performance of the generated summary. Summary of the Invention

[0006] The purpose of the present invention is to overcome the defects of the above-mentioned prior art and provide a summary generation system and method based on a three-channel contrastive learning framework, which can effectively handle complex semantic relationships and improve the stability and accuracy of generated summaries.

[0007] The objectives of the present invention can be achieved by the following technical solutions: a summary generation system based on a three-channel contrastive learning framework, comprising a pre-trained BART (Bidirectional and Auto-Regressive Transformers, a Transformer with both bidirectional language modeling and autoregressive mechanisms), a summary quality classification unit, and a three-channel contrastive learning unit connected in sequence, wherein the pre-trained BART model is used to receive an input document and generate an initial summary; The summary quality classification unit is used to score and sort the generated initial summaries, and classify them into high-quality summaries and low-quality summaries; The three-channel comparative learning unit performs outlier detection on high-quality summaries and low-quality summaries, mines multi-level correlation information between documents, summaries and scores, and performs comparative learning on summaries in the same score interval and those in different score intervals to generate the final summary.

[0008] Furthermore, the three-channel contrastive learning unit includes an encouraged contrastive learning module, a rescoring heterogeneous graph contrastive learning module, and a similarity scoring edge contrastive learning module. The encouraged contrastive learning module receives high-quality summaries as input and is used to classify high-quality summaries into high-scoring summaries and low-scoring summaries. Outlier detection is performed by comparing the high-scoring and low-scoring summaries and using an outlier detection algorithm. The input of the rescoring heterogeneous graph comparative learning module is high-quality summaries and low-quality summaries. It is used to construct a special heterogeneous graph containing documents, summaries and score nodes, and design document-summary and summary-score edge types to mine multi-level association information between documents, summaries and scores. The similarity score edge comparison learning module takes high-quality summaries and low-quality summaries as input, and is used to perform comparison learning on summaries in the same score interval and those in different score intervals to capture the score differences.

[0009] A summary generation method based on a three-channel contrastive learning framework includes the following steps: S1. Build a pre-trained model BART, input document samples into BART, and generate the corresponding initial summary; S2. Score and sort the initial abstracts to identify high-quality and low-quality abstracts; S3. Perform three-channel comparative learning based on high-quality and low-quality summaries: For high-quality abstracts, we further divide them into high-scoring abstracts and low-scoring abstracts, and use the self-resample algorithm to sample them. The BADK (Bootstrap Aggregating and Dynamic Knowledge) algorithm dynamically adjusts the margins to detect outliers in the sampling. For high-quality and low-quality summaries, we designed a heterogeneous document-summary-rating graph, updated the node representations using a GCN (Graph Convolutional Network), and performed graph-level comparative learning based on the heterogeneous graph. For high-quality and low-quality abstracts, we divide them into multiple intervals according to their score, and compare the similarity between abstracts from the same interval with the similarity between abstracts from different intervals. S4, performing weighted accumulation of the contrastive learning loss functions of the three channels and then training to obtain an optimized three-channel contrastive learning module; S5. Input the current document into the pre-trained model BART to generate the corresponding initial summary, then score and sort it, divide the corresponding high-quality summary and low-quality summary, and input it into the three-channel comparative learning module to generate the final summary result.

[0010] Furthermore, the pre-trained model BART specifically introduces a Pfeiffer adapter based on BART. The Pfeiffer adapter optimizes the model's adaptability to specific tasks without significantly increasing computational overhead by adding a set of lightweight trainable parameters to BART.

[0011] Furthermore, in step S3, the BADK algorithm mitigates the impact of outliers by introducing distance-based outlier detection. BADK calculates the Euclidean distance between samples and uses the D-kNN (k-neighbors of distance) method to determine whether there are outliers in the sampling process. If the distance between a sample and other samples is too large (i.e., exceeds a certain threshold), it will be regarded as an outlier by the model and excluded. The specific formula is as follows: , in, and are the feature vectors of two high-quality summary samples, and are their values ​​on the mth feature, N is the dimension of the feature, for and If the distance between d ( , )>δ, δ is the preset threshold, then the sample Determined to be an outlier.

[0012] Furthermore, the heterogeneous graph of document-summary-score in step S3 includes heterogeneous graph nodes and heterogeneous graph edges, and the heterogeneous graph nodes are of three types: documents in the dataset as document nodes, summaries output by the pre-trained model as summary nodes, and scores corresponding to the summaries as score nodes; There are two types of heterogeneous graph edges: edges from document nodes to summary nodes, and edges from summary nodes to rating nodes; The process of updating the node representation by GCN in step S3 is as follows: For the two types of edges in the heterogeneous graph, graph convolution operations are used to process them to complete the update of the two types of edges; For heterogeneous Figure 3 The document nodes, summary nodes and score nodes are updated in turn through graph convolution.

[0013] Furthermore, the update process of the two types of edges is specifically as follows: For document node v d and summary node v s , information is propagated through the edge from document to summary, and the edge update formula is as follows: , Where h′ s is the summary node v s The update representation is, N(v s ) is associated with the summary node v s The set of adjacent document nodes, WDA is the document-to-summary convolution weight matrix, h d Is the document node v d The feature representation of , σ() is the activation function; For summary nodes and scoring nodes, information propagation is performed through the edge from summary to scoring. The edge update formula is as follows: , Where h′ c is the scoring node v c The update representation is, N(v c ) is the same as the scoring node v c The set of adjacent summary nodes, W SC is the convolution weight matrix from summary to score, h s is the summary node v s feature representation.

[0014] Furthermore, the update process of the three types of nodes is specifically as follows: In graph convolutional networks, document nodes update their own features by interacting with the features of adjacent summary nodes. The update process considers the edge weights between document nodes and summary nodes, and transfers the information of summary nodes to document nodes through graph convolution. The update formula of document nodes is: , Among them, h doc is the feature vector of the document node, is the feature vector of the updated document node, is the feature vector of the i-th summary node, W doc and W su is the weight matrix in the graph convolution, representing the edge weights between the document node and the summary node, and σ() is the activation function; Then, the summary node is updated. The update process of the summary node considers the feature information from the document node and the score node. The convolution updates the features of the summary node after weighted aggregation of the features of the document node and the score node. The update formula of the summary node is: , Among them, h su is the feature vector of the summary node, is the feature vector of the updated summary node, is the feature vector of the i-th document node, h sc is the feature vector of the scoring node, W su、 W doc and W scis the weight matrix in the graph convolution, which represents the edge weights between the summary node and the document node and the score node respectively, and σ() is the activation function; Finally, the scoring node is updated. The updating process of the scoring node mainly depends on the features of the summary node. The scoring node interacts with the information of the summary node through the graph convolution layer to update its own features. The scoring node update formula is: , Among them, h sc is the feature vector of the scoring node, is the updated feature vector of the scoring node, is the feature vector of the i-th summary node, W sc and W su is the weight matrix in the graph convolution, representing the edge weights between the score node and the summary node, and σ() is the activation function.

[0015] Furthermore, in step S3, graph-level contrastive learning is performed based on the heterogeneous graph, and the goal of the contrastive learning is to maximize the similarity between positive samples, i.e., related nodes, and minimize the similarity between negative samples, i.e., unrelated nodes; The comparative learning includes: comparative learning between documents and summaries, and comparative learning between summaries and ratings.

[0016] Furthermore, in step S3, the similarities between the abstracts from the same interval and the similarities between the abstracts from different intervals are compared and learned. The goal of the comparative learning is to maximize the similarity between the abstracts in the same interval, that is, to minimize the distance, and to minimize the similarity between the abstracts from different intervals, that is, to maximize the distance. The contrastive learning adopts the contrastive loss function to calculate the distance difference between positive samples and negative samples: , in, is the interval corresponding to summary i, is the interval corresponding to summary j, e i is the embedding vector of summary i, e j is the embedding vector of summary j, d(e i , e j ) is e i and e j The Euclidean distance between means that summaries i and j belong to the same interval, Indicates that they belong to different intervals, and m is a predefined margin used to ensure that there is enough distance between the summaries of different intervals.

[0017] Compared with the prior art, the present invention has the following advantages: The present invention designs a pre-trained model BART, a summary quality classification unit, and a three-channel comparative learning unit connected in sequence. The pre-trained model BART is used to generate an initial summary. The summary quality classification unit is used to score and rank the generated initial summary to obtain high-quality and low-quality summaries. The three-channel comparative learning unit performs outlier detection on the high-quality and low-quality summaries, mines multi-level association information between documents, summaries, and scores, and conducts comparative learning between summaries in the same scoring interval and those in different scoring intervals to generate the final summary. This can effectively handle complex semantic relationships and greatly improve the stability and accuracy of generated summaries.

[0018] In this invention, the three-channel contrastive learning unit includes an encouraging contrastive learning module, a rescoring heterogeneous graph contrastive learning module, and a similarity score edge contrastive learning module. First, the encouraging contrastive learning module compares high-scoring and low-scoring summaries and uses an outlier detection algorithm to make the model sampling process less susceptible to outliers, thereby improving the stability and overall quality of the generated data. Second, the rescoring heterogeneous graph contrastive learning module constructs a special heterogeneous graph containing documents, summaries, and score nodes, and designs document-summary and summary-score edge types. This deeply explores the multi-level association information between documents, summaries, and scores, effectively improving the model's ability to model complex associations. Finally, the similarity score edge contrastive learning module refines the model's ability to capture score differences by comparing summaries in the same score interval with those in different score intervals, thereby enhancing the model's performance in scoring accuracy. Experimental verification shows that this three-channel contrastive learning unit can effectively improve the performance of generative text summarization.

[0019] The present invention constructs a pre-trained model BART based on the Pfeiffer adapter and introduces the adapter on the basis of BART. By adding a set of lightweight trainable parameters to the model, the adapter can optimize the model's adaptability to specific tasks without significantly increasing computational overhead. These newly added trainable parameters are gradually adjusted during the training process, thereby improving the model's performance in the summary generation task and ultimately improving the quality of the generated summaries.

[0020] When performing encouraged contrastive learning, the present invention takes into account that conventional bootstrap resampling algorithms are susceptible to outliers. Outliers often have an adverse effect on margin adjustment, causing the model to over-focus on unrepresentative summaries during optimization, thereby affecting the model's learning ability. In particular, in the task of distinguishing summary quality, the presence of outliers may prevent the model from correctly assigning the probabilities of high-quality and low-quality summaries in the early stages of training. Therefore, the present invention adopts the BADK algorithm to reduce the impact of outliers, aiming to dynamically adjust margins and enhance the ability to distinguish summary quality. The BADK algorithm effectively mitigates the impact of outliers by introducing distance-based outlier detection.

[0021] When performing comparative learning on a rescoring heterogeneous graph, the present invention designs a document-summary-score heterogeneous graph. Unlike traditional dataset-based approaches to establishing entity node and edge types, the present invention employs three node types: documents in the dataset as document nodes, summaries output by pretrained models as summary nodes, and scores corresponding to summaries as score nodes. The present invention also employs two edge types: edges from document nodes to summary nodes, and edges from summary nodes to score nodes. This heterogeneous graph design helps the model better dynamically understand the relationships between the three, enhancing its ability to capture complex semantic relationships.

[0022] The present invention updates the representation of nodes through the graph convolutional network (GCN). The representation of the node will be updated after each graph convolution operation. The present invention uses the graph convolution operation to effectively aggregate the information of neighboring nodes, thereby updating the feature representation of the target node, and finally obtaining nodes that can reflect the deep semantic relationship between documents, summaries and scores, and the model can learn deeper node representations; in addition, through multiple rounds of graph convolution operations, the model gradually optimizes the representation of nodes, so that the relationship between documents, summaries and scores can be fully captured. The core of information dissemination is the information aggregation and update between nodes. Through these operations, the model can more accurately understand the complex semantic relationship between the three, thereby improving the accuracy of summary generation and score analysis.

[0023] The present invention conducts comparative learning on the similarity between abstracts from the same interval and the similarity between abstracts from different intervals: when performing comparative learning, the ultimate goal is to maximize the similarity between abstracts in the same interval and minimize the similarity between abstracts in different intervals. The present invention designs a comparative loss function to calculate the distance difference between positive samples and negative samples. The loss function minimizes the distance between abstracts in the same interval and maximizes the distance between abstracts in different intervals by adjusting the model parameters, thereby improving the model's sensitivity to subtle semantic differences and its recognition ability. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 Schematic diagram of the system structure of the present invention; Figure 2 Schematic diagram of the method flow of the present invention; Figure 3 Schematic diagram of the application framework of the embodiment; Figure 4 This is a schematic diagram of the document node update process; Figure 5 Schematic diagram of the summary node update process; Figure 6 Schematic diagram of the scoring node update process; Explanation of the marks in the figure: 1. Pre-trained model BART, 2. Summary quality division unit, 3. Three-channel contrastive learning unit, 301. Encouraged contrastive learning module, 302. Re-scoring heterogeneous graph contrastive learning module, 303. Similarity scoring edge contrastive learning module. DETAILED DESCRIPTION

[0025] The present invention will be described in detail below with reference to the accompanying drawings and specific embodiments.

[0026] Example 1

[0027] like Figure 1 As shown, a summary generation system based on a three-channel contrastive learning framework includes a pre-trained model BART1, a summary quality classification unit 2, and a three-channel contrastive learning unit 3 connected in sequence, wherein the pre-trained model BART is used to receive an input document and generate an initial summary; The summary quality classification unit 2 is used to score and sort the generated initial summaries, and classify them into high-quality summaries and low-quality summaries; The three-channel comparative learning unit 3 performs outlier detection on high-quality and low-quality summaries, mines multi-level correlation information between documents, summaries, and scores, and performs comparative learning on summaries in the same and different score intervals to generate the final summary. The three-channel contrastive learning unit 3 includes an encouraging contrastive learning module 301, a rescoring heterogeneous graph contrastive learning module 302, and a similar scoring edge contrastive learning module 303. The encouraging contrastive learning module 301 receives high-quality summaries as input and is used to classify high-quality summaries into high-scoring summaries and low-scoring summaries. The outlier detection algorithm is used to detect outliers by comparing the high-scoring and low-scoring summaries. The input of the rescoring heterogeneous graph comparative learning module 302 is high-quality summaries and low-quality summaries. It is used to construct a special heterogeneous graph containing documents, summaries, and score nodes, and to design document-summary and summary-score edge types to mine multi-level association information between documents, summaries, and scores. The input of the similarity score edge comparison learning module 303 is a high-quality summary and a low-quality summary, and is used to perform comparative learning on summaries of the same score interval and summaries of different score intervals to capture the score differences.

[0028] In the three-channel comparative learning unit 3 designed in this scheme, first, the comparative learning module 301 is encouraged to compare high-scoring and low-scoring summaries, and through the use of an applicable outlier detection algorithm, the model sampling process is less affected by outliers, thereby improving the stability and overall quality of the generated data. Secondly, the rescoring heterogeneous graph comparative learning module 302 constructs a special heterogeneous graph containing documents, summaries, and scoring nodes, designs document-summary and summary-scoring edge types, and deeply explores the multi-level correlation information between documents, summaries, and scores, effectively improving the model's ability to model complex correlation relationships. Finally, the similarity score edge comparative learning module 303 refines the model's ability to capture score differences and enhances the model's performance in scoring accuracy by comparative learning of summaries with the same scoring interval and different scoring intervals.

[0029] Example 2

[0030] Based on the summary generation system proposed in Example 1, a summary generation method based on a three-channel contrastive learning framework is implemented, such as Figure 2 As shown, the following steps are included: S1. Build a pre-trained model BART, input document samples into BART, and generate the corresponding initial summary; S2. Score and sort the initial abstracts to identify high-quality and low-quality abstracts; S3. Perform three-channel comparative learning based on high-quality and low-quality summaries: For high-quality abstracts, we further divide them into high-scoring abstracts and low-scoring abstracts, and use the self-resample algorithm to perform sampling. The BADK algorithm dynamically adjusts the margin to detect sampled outliers. For high-quality and low-quality summaries, we design a document-summary-rating heterogeneous graph, update the node representation through GCN, and perform graph-level comparative learning based on the heterogeneous graph. For high-quality and low-quality abstracts, we divide them into multiple intervals according to their score, and compare the similarity between abstracts from the same interval with the similarity between abstracts from different intervals. S4, performing weighted accumulation of the contrastive learning loss functions of the three channels and then training to obtain an optimized three-channel contrastive learning module; S5. Input the current document into the pre-trained model BART to generate the corresponding initial summary, then score and sort it, divide the corresponding high-quality summary and low-quality summary, and input it into the three-channel comparative learning module to generate the final summary result.

[0031] This embodiment applies the above solution to build Figure 3 The model framework shown in the figure mainly includes the following application processes: 1. Building a Pre-Trained BART Model Based on Pfeiffer Adapters: To better adapt the pre-trained BART model to generative text summarization tasks, this example introduces an adapter module based on BART. The adapter adds a set of lightweight, trainable parameters to the model, optimizing its adaptability to specific tasks without significantly increasing computational overhead. These newly added parameters are gradually adjusted during training, improving the model's performance in summarization tasks and ultimately enhancing the quality of generated summaries.

[0032] 2. Score the summaries generated by the pre-trained model, sort them by score, and divide them into high-quality summaries and low-quality summaries.

[0033] 3. The high-quality summaries are then sorted through a bootstrap resampling algorithm, with the margins dynamically adjusted using the BADK algorithm. After selecting high-quality summaries, the margins are dynamically adjusted using a bootstrap resampling algorithm to adapt to the needs of the model at different training stages. In the early stages of training, the model's probability distribution may be very rough, so a larger margin is needed to force the model to pay more attention to differences in summary quality. As training progresses, the margin can be reduced as the model becomes more accurate in distinguishing between high- and low-quality summaries.

[0034] However, conventional bootstrap resampling algorithms are susceptible to outliers, which often adversely affect margin adjustments, causing the model to overemphasize unrepresentative summaries during optimization, thereby affecting the model's learning ability. In particular, in the task of distinguishing summary quality, the presence of outliers can cause the model to incorrectly assign probabilities between high-quality and low-quality summaries during early training, thus underutilizing the algorithm. Therefore, the BADK algorithm is used to mitigate the impact of outliers, aiming to dynamically adjust margins and enhance the ability to distinguish summary quality. The BADK algorithm effectively mitigates the impact of outliers by introducing distance-based outlier detection.

[0035] Specifically, BADK calculates the Euclidean distance between samples and uses the D-kNN (distance k neighbors) method to determine whether there are outliers in the sampling process. If the distance between a sample and other samples is too large (i.e., exceeds a certain threshold), it will be considered an outlier by the model and excluded. In this way, the BADK algorithm strategy can effectively mitigate the impact of outliers on the training process, ensuring that the model learns only from valid samples, indirectly improving the robustness of training. The specific formula is as follows: (1) in, and are the feature vectors of two high-quality summary samples, and are their values ​​on the mth feature, N is the dimension of the feature, if d( , )>δ, δ is the preset threshold, then the sample Determined to be an outlier.

[0036] Finally, the calculated margin value Margin t Applied to the loss function, this can encourage the model to more accurately distinguish the score gap between high-quality and low-quality summaries. In this way, the loss function encourages the model to pay more attention to the score difference between high-quality and low-quality summaries during training, thereby improving the quality of summary generation.

[0037] Design a heterogeneous graph of documents, summaries, and ratings: To enhance the model's ability to capture complex semantic relationships, heterogeneous graphs can help the model better dynamically understand the connections between the three. Unlike traditional dataset-based approaches to establishing entity node and edge types, this solution uses three heterogeneous graph node types: documents in the dataset as document nodes, summaries output by the pre-trained model as summary nodes, and scores corresponding to summaries as rating nodes. There are also two types of heterogeneous graph edges: edges from document nodes to summary nodes, and edges from summary nodes to rating nodes.

[0038] After the initial training phase, a secondary training mechanism is introduced to further optimize the model based on the learned node representations. During this secondary training phase, the model can more accurately capture the potential impact of ratings, thereby enhancing its understanding of semantic relationships. This secondary training mechanism can be considered "deep optimization," gradually improving the model's ability to model complex semantic relationships through multiple iterative updates.

[0039] 5. Updating node representations using GCN: The key idea is to update node representations by aggregating information from neighboring nodes. Specifically, the features of the three node types are initialized based on their content. Then, the node representations are updated using GCN (Graph Convolutional Network). The graph convolution operation updates the features of the target node by aggregating information from neighboring nodes. For heterogeneous graphs, different graph convolution operations are used to process different edge types.

[0040] For document node v d and summary node v s , information is propagated through the edge from document to summary, and the edge update formula is as follows: , (2) Where h′ s is the summary node v s The update representation is, N(v s ) is associated with the summary node v s The set of adjacent document nodes, W DA is the document-to-summary convolution weight matrix, h d Is the document node v d The feature representation of , σ() is the activation function.

[0041] For summary nodes and scoring nodes, information propagation is performed through the edge from summary to scoring. The edge update formula is as follows: , (3) Where h′ c is the scoring node v c The update representation is, N(v c ) is the same as the scoring node v c The set of adjacent summary nodes, W SC is the convolution weight matrix from summary to score, h s is the summary node v s feature representation.

[0042] After each graph convolution operation, the node representation is updated. The graph convolution operation effectively aggregates information from neighboring nodes to update the feature representation of the target node. Ultimately, the resulting nodes reflect the deep semantic relationships between documents, summaries, and ratings, allowing the model to learn deeper node representations.

[0043] Through multiple rounds of graph convolution, the model gradually optimizes node representations, fully capturing the relationships between documents, summaries, and ratings. The core of information dissemination is the aggregation and updating of information between nodes. Through these operations, the model can more accurately understand the complex semantic relationships between the three, thereby improving the accuracy of summary generation and rating analysis.

[0044] In a graph convolutional network (GCN), document nodes update their own features by interacting with the features of adjacent summary nodes. The update process takes into account the edge weights between document nodes and summary nodes, and transfers the information of summary nodes to document nodes through graph convolution. Figure 4 This is the update process of the document node. The update formula of the document node is: , (4) Among them, h doc is the feature vector of the document node, is the feature vector of the i-th summary node, W doc and Wsu is the weight matrix in the graph convolution, representing the edge weights between the document node and the summary node, and σ() is the activation function; Then the summary node is updated, and the updating process of the summary node takes into account the feature information from the document node and the score node. Figure 5 This is the update process of the summary node. Convolution updates the features of the summary node after weighted aggregation of the features of the document node and the score node. The update formula of the summary node is: , (5) Among them, h su is the feature vector of the summary node, is the feature vector of the i-th document node, h sc is the feature vector of the scoring node, W su、 W doc and W sc is the weight matrix in the graph convolution, which represents the edge weights between the summary node and the document node and the score node respectively, and σ() is the activation function; Finally, the scoring node is updated. The updating process of the scoring node mainly depends on the features of the summary node. The scoring node interacts with the information of the summary node through the graph convolution layer to update its own features. Figure 6 This is the update process of the scoring node. The scoring node update formula is: , (6) Among them, h sc is the feature vector of the scoring node, is the feature vector of the i-th summary node, W sc and W su is the weight matrix in the graph convolution, representing the edge weights between the score node and the summary node, and σ() is the activation function.

[0045] 6. Graph-level contrastive learning based on heterogeneous graphs: Within the contrastive learning framework, the goal is to optimize the model learning process by maximizing the similarity between positive samples (related nodes) and minimizing the similarity between negative samples (irrelevant nodes). Specifically, contrastive learning is divided into two parts: contrastive learning between documents and summaries, and contrastive learning between summaries and ratings.

[0046] The document-summary similarity matrix is ​​calculated by calculating the inner product (dot product) of document node features and summary node features. To account for the associations between documents and summaries, edge weights are used to adjust the similarity matrix. Within the contrastive learning framework, the model is optimized by minimizing the similarity between positive samples (relevant nodes) and maximizing the similarity between negative samples (irrelevant nodes).

[0047] 7. Divide the summary into multiple intervals according to the score size.

[0048] 8. Comparative learning of the similarity between summaries from the same interval and those from different intervals: The ultimate goal of contrastive learning is to maximize the similarity between summaries from the same interval and minimize the similarity between summaries from different intervals, thereby improving the model's sensitivity to subtle semantic differences. This solution designs a contrastive loss function to optimize this objective. The key idea of ​​this loss function is to calculate the distance between each summary i and the summaries from the same interval (positive samples) and different intervals (negative samples), and then optimize the distance.

[0049] According to the goal, we hope to maximize the similarity between summaries in the same interval (i.e., minimize the distance) and minimize the similarity between summaries in different intervals (i.e., maximize the distance). To achieve this goal, the following contrast loss function L is designed con , this loss function calculates the distance difference between positive and negative samples: , (7) in, Indicates that summaries i and j belong to the same interval (positive sample), Indicates that they belong to different intervals (negative samples), and m is a predefined margin to ensure that there is enough distance between the summaries of different intervals.

[0050] This loss function adjusts the model parameters to minimize the distance between summaries in the same interval and maximize the distance between summaries in different intervals, thereby enhancing the model's ability to recognize subtle semantic differences.

[0051] 9. Perform training by weighted accumulation of the contrastive learning loss functions of the three channels: In order to dynamically ensure the generation and evaluation capabilities of the model, the above multiple objectives are added with weight values ​​and combined into a universal loss function.

[0052] In summary, existing generative summarization models are susceptible to outliers during the sampling process. This is especially true when the data distribution is skewed or there are a large number of low-quality samples, which can lead to unstable summaries. This instability not only affects the readability and information integrity of the summaries, but can also lead to semantic misunderstandings, which in turn can weaken the model's reliability and generalization capabilities. To this end, this solution compares high- and low-scoring summaries and uses an appropriate outlier detection algorithm to make the model sampling process less susceptible to outliers, thereby improving the stability and overall quality of the summaries.

[0053] Furthermore, traditional frameworks rely too heavily on simple positive and negative sample pairs, which can lead to insufficient processing of complex semantic relationships. To address this, our solution constructs a special heterogeneous graph containing document, summary, and rating nodes, and designs document-summary and summary-rating edge types. This deeply explores the multi-level associations between documents, summaries, and ratings, effectively improving the model's ability to model complex relationships.

[0054] Furthermore, existing generative summarization methods are less sensitive to subtle data changes and struggle to adapt to diverse text styles or domain characteristics, which can affect the applicability and consistency of summaries and lead to unsatisfactory performance. To address this, this solution refines the model's ability to capture rating differences by comparing summaries within the same and different rating intervals, thereby enhancing the model's performance in rating accuracy.

Claims

1. A summary generation system based on a three-channel contrastive learning framework, characterized by: The system comprises a pre-training model BART (1), a summary quality classification unit (2), and a three-channel contrastive learning unit (3) connected in sequence, wherein the pre-training model BART (1) is used to receive an input document and generate an initial summary; The summary quality classification unit (2) is used to score and sort the generated initial summaries, and classify them into high-quality summaries and low-quality summaries; The three-channel comparative learning unit (3) performs outlier detection on high-quality summaries and low-quality summaries, mines multi-level correlation information between documents, summaries and scores, and performs comparative learning on summaries in the same score interval and those in different score intervals to generate a final summary; The three-channel contrastive learning unit (3) includes an encouraging contrastive learning module (301), a rescoring heterogeneous graph contrastive learning module (302), and a similarity score edge contrastive learning module (303). The input of the encouraging contrastive learning module (301) is a high-quality summary, which is used to divide the high-quality summary into a high-scoring summary and a low-scoring summary, and perform outlier detection by comparing the high-scoring summary with the low-scoring summary and using an outlier detection algorithm; The input of the re-scoring heterogeneous graph comparative learning module (302) is a high-quality summary and a low-quality summary, which is used to construct a special heterogeneous graph containing documents, summaries and score nodes, and design document-summary and summary-score edge types to mine multi-level association information between documents, summaries and scores; The input of the similarity score edge comparison learning module (303) is a high-quality summary and a low-quality summary, and is used to perform comparison learning on summaries of the same score interval and summaries of different score intervals to capture the score differences.

2. A summary generation method based on a three-channel contrastive learning framework, characterized in that: The following steps are involved: S1. Build a pre-trained model BART, input document samples into BART, and generate the corresponding initial summary; S2. Score and sort the initial abstracts to identify high-quality and low-quality abstracts; S3. Perform three-channel comparative learning based on high-quality and low-quality summaries: For high-quality abstracts, we further divide them into high-scoring abstracts and low-scoring abstracts, and use the self-resample algorithm to perform sampling. The BADK algorithm dynamically adjusts the margin to detect sampled outliers. For high-quality and low-quality summaries, we design a heterogeneous document-summary-rating graph, update the node representations through a graph convolutional network (GCN), and perform graph-level comparative learning based on the heterogeneous graph. For high-quality and low-quality abstracts, we divide them into multiple intervals according to their score, and compare the similarity between abstracts from the same interval with the similarity between abstracts from different intervals. S4, performing weighted accumulation of the contrastive learning loss functions of the three channels and then training to obtain an optimized three-channel contrastive learning module; S5. Input the current document into the pre-trained model BART to generate the corresponding initial summary, then score and sort it, divide the corresponding high-quality summary and low-quality summary, and input it into the three-channel comparative learning module to generate the final summary result.

3. The summary generation method based on the three-channel contrastive learning framework according to claim 2 is characterized in that: The pre-trained model BART specifically introduces the Pfeiffer adapter on the basis of BART. The Pfeiffer adapter optimizes the adaptability of the model to specific tasks without significantly increasing the computational overhead by adding a set of lightweight trainable parameters to BART.

4. The summary generation method based on the three-channel contrastive learning framework according to claim 2, characterized in that: In step S3, the BADK algorithm mitigates the impact of outliers by introducing distance-based outlier detection. BADK calculates the Euclidean distance between samples and uses the D-kNN method to determine whether there are outliers in the sampling process. If the distance between a sample and other samples exceeds a preset distance threshold, it will be regarded as an outlier by the model and excluded. The specific formula is as follows: , in, and are the feature vectors of two high-quality summary samples, and are their values ​​on the mth feature, N is the dimension of the feature, for and If the distance between d ( , )>δ, δ is the preset threshold, then the sample Determined to be an outlier.

5. The summary generation method based on the three-channel contrastive learning framework according to claim 2, characterized in that: The heterogeneous graph of document-summary-score in step S3 includes heterogeneous graph nodes and heterogeneous graph edges. There are three types of heterogeneous graph nodes: documents in the dataset as document nodes, summaries output by the pre-trained model as summary nodes, and scores corresponding to the summaries as score nodes; There are two types of heterogeneous graph edges: edges from document nodes to summary nodes, and edges from summary nodes to rating nodes; The process of updating the node representation by GCN in step S3 is as follows: For the two types of edges in the heterogeneous graph, graph convolution operations are used to process them to complete the update of the two types of edges; For the three types of nodes in the heterogeneous graph, the document node, summary node and score node are updated in turn through graph convolution.

6. The method for generating a summary based on a three-channel contrastive learning framework according to claim 5, characterized in that: The update process of the two types of edges is specifically as follows: For document node v d and summary node v s , information is propagated through the edge from document to summary, and the edge update formula is as follows: ， Where h′ s is the summary node v s The update representation is, N(v s ) is associated with the summary node v s The set of adjacent document nodes, W DA is the document-to-summary convolution weight matrix, h d Is the document node v d The feature representation of , σ() is the activation function; For summary nodes and scoring nodes, information is propagated through the edge from summary to scoring. The edge update formula is as follows: ， Where h′ c is the scoring node v c The update representation is, N(v c ) is the same as the scoring node v c The set of adjacent summary nodes, W SC is the convolution weight matrix from summary to score, h s is the summary node v s feature representation.

7. The method for generating a summary based on a three-channel contrastive learning framework according to claim 6, characterized in that: The update process of the three types of nodes is specifically as follows: In graph convolutional networks, document nodes update their own features by interacting with the features of adjacent summary nodes. The update process considers the edge weights between document nodes and summary nodes, and transfers the information of summary nodes to document nodes through graph convolution. The update formula of document nodes is: , Among them, h doc is the feature vector of the document node, is the feature vector of the updated document node, is the feature vector of the i-th summary node, W doc and W su is the weight matrix in the graph convolution, representing the edge weights between the document node and the summary node, and σ() is the activation function; Then, the summary node is updated. The update process of the summary node considers the feature information from the document node and the score node. The convolution updates the features of the summary node after weighted aggregation of the features of the document node and the score node. The update formula of the summary node is: , Among them, h su is the feature vector of the summary node, is the feature vector of the updated summary node, is the feature vector of the i-th document node, h sc is the feature vector of the scoring node, W su、 W doc and W sc is the weight matrix in the graph convolution, which represents the edge weights between the summary node and the document node and the score node respectively, and σ() is the activation function; Finally, the scoring node is updated. The updating process of the scoring node mainly depends on the features of the summary node. The scoring node interacts with the information of the summary node through the graph convolution layer to update its own features. The scoring node update formula is: , Among them, h sc is the feature vector of the scoring node, is the updated feature vector of the scoring node, is the feature vector of the i-th summary node, W sc and W su is the weight matrix in the graph convolution, representing the edge weights between the score node and the summary node, and σ() is the activation function.

8. The method for generating a summary based on a three-channel contrastive learning framework according to claim 2, wherein: In step S3, graph-level contrastive learning is performed based on the heterogeneous graph, where the goal of the contrastive learning is to maximize the similarity between positive samples and minimize the similarity between negative samples; The comparative learning includes comparative learning between documents and summaries, and comparative learning between summaries and ratings.

9. The method for generating a summary based on a three-channel contrastive learning framework according to claim 2, wherein: In step S3, the similarity between the summaries from the same interval and the similarity between the summaries from different intervals are compared and learned. The goal of the comparative learning is to maximize the similarity between the summaries in the same interval, that is, to minimize the distance; and to minimize the similarity between the summaries in different intervals, that is, to maximize the distance; The contrastive learning adopts the contrastive loss function to calculate the distance difference between positive samples and negative samples: ， in, is the interval corresponding to summary i, is the interval corresponding to summary j, e i is the embedding vector of summary i, e j is the embedding vector of summary j, d(e i , e j ) is e i and e j The Euclidean distance between means that summaries i and j belong to the same interval, Indicates that they belong to different intervals, and m is a predefined margin used to ensure that there is enough distance between the summaries of different intervals.

Images