Document Summarization Using Attention Vectors for Accuracy
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current deep learning models for text summarization face challenges due to the need for large amounts of labeled data and the complexity of text meaning changes with minor alterations.
Innovation Solution
A document summarization method and apparatus that includes an encoding unit to convert document data into a feature vector, an extraction summary unit to calculate probability values for each sentence and generate attention vectors, and a decoding unit to generate abstract summary data based on the feature and attention vectors.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If deep learning models are trained using traditional methods with labeled data, then model performance can be improved, but the requirement for large amounts of labeled data increases training complexity and cost
Solution Approach 1:
The model performs self-training by generating its own training data through the summarization process. The system uses the trained model to generate summaries, which then serve as training data for further model improvement, eliminating the need for external labeled data sources.
Solution Approach 2:
The system performs preliminary training with available data to establish a baseline model capability, then uses this model to generate additional training data before final deployment. This preliminary action allows the system to bootstrap the training process without requiring extensive pre-labeled data.
2Measurement precision
If text summarization focuses on maintaining exact meaning, then accuracy is improved, but the flexibility to handle various text types and lengths decreases
Solution Approach 1:
The attention mechanism applies different weights to different parts of the input text based on their importance. Rather than treating all text uniformly, the system identifies and focuses on key sentences and tokens that carry the most meaningful information, allowing accurate summarization across diverse text types.
Solution Approach 2:
The model dynamically adjusts its processing based on the characteristics of the input text. The attention weights are computed adaptively for each input, allowing the system to handle various text types, lengths, and structures while maintaining summary quality through flexible, context-dependent processing.
3Loss of information
If the model processes all sentences and tokens equally, then comprehensive coverage is achieved, but the computational efficiency and summary quality decrease
Solution Approach 1:
The attention mechanism extracts and focuses on the most important sentences and tokens from the input text, rather than processing all elements equally. By identifying and extracting key information carriers, the system achieves high-quality summaries with improved computational efficiency.
Solution Approach 2:
The model changes the processing parameters dynamically by assigning different attention weights to different input elements. This parameter change allows the system to prioritize important information while reducing processing of less critical content, balancing coverage and efficiency.
Data Source
AI summary
A document summarizing apparatus includes an encoding unit receiving document data comprised of one or more sentences and converts the document data into a token defined in a predetermined unit to generate a feature vector, an extraction summary unit receiving the feature vector and calculating a probability value that each sentence corresponds to a summary, with respect to each one or more sentences constituting the document data, and generating an attention vector for each token weight based on the probability value, and a decoding unit receiving the feature vector and the attention vector and generating abstract summary data.