Text Summarization Using Multi-Level Sentence Similarity Scoring
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Text summarization in natural language processing is challenging due to the complexity of determining the relative importance of sentences, leading to inaccurate selection of key information from vast textual data.
Innovation Solution
An NLP-based method that extracts sentences by generating word, n-grams, and phrase-n vector sets, computing similarity scores, and combining them to determine edge scores for selecting text features, thereby generating a summary that handles different types of text and word ordering effects.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If traditional text summarization methods are used to select important sentences, then the summarization process can be completed, but the accuracy of sentence selection is low due to the complexity of determining relative importance
Solution Approach 1:
The patent segments the sentence importance determination process into three distinct components: word-level similarity scoring, n-gram level similarity scoring, and phrase-n level similarity scoring. Each component evaluates different aspects of sentence similarity independently, and their scores are combined to produce the final edge score. This segmentation allows the system to capture multiple dimensions of sentence similarity without overwhelming complexity in any single component.
Solution Approach 2:
The patent changes the parameters used for sentence comparison by employing multiple levels of linguistic abstraction (words, n-grams, and phrases) rather than relying on a single parameter. Each level provides different granularities of similarity measurement, allowing the system to adaptively capture various aspects of sentence similarity. The edge score combines these different parameter measurements to achieve more accurate sentence selection.
2Adaptability or versatility
If multiple types of text (same wording and synonyms) are handled using traditional methods, then text processing can proceed, but the handling of synonyms and different text types reduces summarization accuracy
Solution Approach 1:
The patent creates a universal similarity evaluation framework that handles multiple types of text representations (exact wording, synonyms, paraphrases) through a unified multi-level approach. The word-level, n-gram level, and phrase-n level scorers work together to evaluate similarity across different text types, making the system versatile while maintaining precision. This universal framework can process various text forms without requiring separate specialized methods for each type.
3Manufacturing precision
If word ordering effects and extra words are not considered, then the summarization process is simpler, but the quality of summary output decreases
Solution Approach 1:
The patent addresses word ordering effects by evaluating similarity at multiple dimensional levels: individual words (1-gram), sequences of words (n-grams), and phrase structures (phrase-n). This dimensional approach allows the system to detect both the presence of key terms and their contextual arrangements. Extra words are naturally handled as the multi-level scoring identifies meaningful patterns regardless of additional irrelevant content, maintaining summary quality without requiring explicit complex handling mechanisms.
Data Source
AI summary
A method for summarizing text with sentence extraction including steps as follows. Sentences are extracted from a document including text by a natural language processing (NLP) based feature extractor. A word vector set with respect to each of the sentences is generated by a processor. The word vector set with respect to each of the sentences is used to generate a n-grams vector set and a phrase-n vector set with respect to each of the sentences. A word score representing similarity between the word vector sets, a n-grams score representing similarity between the n-grams vector sets, and a phrase-n score representing similarity between the phrase-n vector sets are computed. The word, n-grams, and phrase-n scores are combined to compute an edge score. Text features are selected from the sentences using the edge scores of the sentences, so as to output a summary of the document.