Machine Learning Model Training for Document Summarization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for training machine learning models to summarize documents are subjective and inflexible, relying on human annotation, which introduces bias and limits the model's ability to extract varying numbers of important sentences, and are time- and cost-intensive due to the need for labeled data.
Innovation Solution
An apparatus and method that preprocesses documents into token sequences, trains a machine learning model to predict sentence order and maximize document similarity, using a loss function that combines sentence order prediction and document similarity maximization, allowing the model to extract a variable number of important sentences without human subjectivity.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If human annotation is used to label correct answers for training, then the model can learn from labeled data, but human subjectivity introduces bias and limits flexibility in selecting the number of important sentences
Solution Approach 1:
The system uses self-training where the model trains itself by predicting sentence orders and using those predictions as training data, eliminating the need for human annotation. The model generates its own training labels through automated sentence ordering predictions, thereby removing human subjectivity while maintaining scalability and flexibility.
Solution Approach 2:
The system performs preliminary sentence ordering predictions to generate training data before actual model training. By pre-generating sentence order predictions and using them as training labels, the system prepares training data without human intervention, enabling flexible adaptation to different document types and sentence counts.
2Reliability
If human annotation is used to create training datasets, then the model can be trained with labeled data, but it takes a lot of time and cost
Solution Approach 1:
The system implements self-training by automatically generating training labels through sentence ordering predictions. The model predicts sentence orders, uses these predictions as training labels, and iteratively improves itself without human annotation, dramatically reducing time and cost while maintaining training data quality.
Solution Approach 2:
The system creates synthetic training data by copying and reordering sentences from existing documents. By generating multiple permutations of sentence orders and using them as training examples, the system creates abundant training data without human labor, reducing both time and cost requirements.
3Device complexity
If the model is trained to extract only the most important sentence, then the training is simple, but the model cannot flexibly select varying numbers of important sentences
Solution Approach 1:
The system dynamically adjusts the number of sentences to extract by modifying the training objective to predict sentence orders rather than selecting a fixed number. The model can flexibly determine the number of important sentences based on the predicted order and confidence scores, adapting to different document types and requirements without increasing training complexity.
Data Source
AI summary
An apparatus of training a machine learning model includes a preprocessing module segmenting a document into each sentence and performing tokenization to generate a token sequence for the document, wherein a document representative token representing the document and representative sentence tokens representing each sentence are included in the token sequence for the document, a first training module training the machine learning model to predict an order of sentences in the document, based on the token sequence for the document, and a second training module training the machine learning model to perform document similarity maximization based on the token sequence for the document.