Summary Generation Model Training with Contrastive Loss
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing summary generation models trained with maximum likelihood probability functions struggle to accurately reflect semantic relationships, leading to reduced accuracy in generating coherent and faithful summaries.
Innovation Solution
The introduction of a total contrastive loss function based on document and summary representations, including positive and negative samples, to train the summary generation model, ensuring semantically relevant summaries are generated by keeping relevant samples close and irrelevant samples far apart.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of manufacture
If maximum likelihood probability function is used to train the summary generation model, then the training process is simple, but the accuracy of summary generation is reduced
Solution Approach 1:
The patent changes the loss function parameter from maximum likelihood probability to contrastive loss function, which fundamentally alters the training objective to optimize semantic representation alignment between documents and summaries, thereby improving generation accuracy
Solution Approach 2:
The patent introduces a new dimension of semantic representation by constructing contrastive loss based on embedding spaces, adding a semantic alignment dimension to the training process that goes beyond traditional probability-based approaches
2Manufacturing precision
If contrastive loss function is used to train the summary generation model, then the accuracy of summary generation is improved, but the device complexity increases
Solution Approach 1:
The patent segments the contrastive loss function into distinct components: positive sample construction, negative sample construction, and loss calculation, making the complex training process more structured and manageable
Solution Approach 2:
The patent introduces summary representations as intermediary elements that bridge the document input and the loss function, facilitating the contrastive learning process by creating explicit semantic intermediates for comparison
Data Source
AI summary
The present disclosure provides a summary generation model training method and apparatus, a device and a storage medium, and relates to the field of computer technologies, and in particular, to the field of artificial intelligence such as natural language processing and deep learning. The summary generation model training method includes: acquiring a document representation corresponding to a document sample; constructing, based on the document representation, a summary representation corresponding to the document representation, the summary representation including a positive summary representation and a negative summary representation; and constructing a total contrastive loss function based on the document representation, the positive summary representation and the negative summary representation, and training a summary generation model based on the total contrastive loss function. The present disclosure may improve accuracy of the summary generation model.