Document Summarization via Pertinence and Coverage Scoring
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing document summarization techniques often produce low-quality summaries by either selecting common words that carry little semantic information or favoring excessively rare words, failing to balance relevance across multiple documents and specificity to the collection.
Innovation Solution
A method that computes pertinence scores for features within a document collection, ranks them based on relevance, and selects a set of features using coverage scores to ensure both relevance and specificity, with re-ranking to optimize the summary.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If word frequency selection techniques are used, then common words are selected, but these words carry little semantic information
Solution Approach 1:
The patent changes the selection parameter from simple frequency count to a composite scoring mechanism that incorporates multiple factors including frequency, document collection specificity, and semantic weight. This transforms the selection criterion to balance commonality with information value, resolving the contradiction between selecting enough features and maintaining semantic quality
Solution Approach 2:
The patent introduces an intermediary scoring system that mediates between frequency-based selection and semantic quality. The score combines frequency metrics with collection-specificity metrics, acting as a bridge that prevents both common words and excessively rare words from being selected, thereby maintaining semantic information while achieving adequate coverage
2Reliability
If word frequency comparison techniques are used, then words over-represented in the collection are selected, but excessively rare words are favored
Solution Approach 1:
The patent modifies the selection parameter from raw frequency comparison to a normalized scoring system that accounts for both collection-specificity and overall frequency. This prevents rare words with high collection-specificity scores from being selected when their actual information value is low, thereby maintaining both representativeness and information quality
Solution Approach 2:
The patent implements a feedback mechanism where the scoring system continuously evaluates both frequency and collection-specificity, adjusting feature selection based on the interplay between these two factors. This feedback loop ensures that selected features maintain appropriate representativeness without devolving into rare, non-informative words
3Device complexity
If a single feature selection method is used, then selection is simple, but quality of summary is poor
Solution Approach 1:
The patent segments the feature selection process into distinct stages: initial frequency-based filtering, scoring based on multiple criteria, and final selection based on coverage thresholds. This segmentation allows each stage to address specific requirements, achieving high summary quality while keeping individual steps manageable
Solution Approach 2:
The patent employs a dynamic, multi-stage selection process that adapts to the characteristics of the document collection. The scoring mechanism dynamically weights different factors based on collection properties, and the selection threshold adjusts based on coverage requirements, thereby achieving high reliability without requiring excessive static complexity
Data Source
AI summary
Provided is a process including: obtaining a corpus having a plurality of document collections, each of which is associated with features; for a given document collection, computing a pertinence score for each feature; ranking the features based on the features' pertinence scores; selecting a first set of features based on a first coverage score thereof and a threshold; re-ranking the first set of features based on the features' relevance to the document collection; and selecting a second set of features from the first set of features based on a second coverage score thereof and the threshold, the second set of features being used for summarizing the document collection.