Key Phrase Extraction Using PMI and Dual Phrase Streams
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current key phrase extraction (KPE) methods struggle with emerging and domain-specific phrases, requiring significant labeling effort and suffer from informativeness and readability issues, failing to capture the meaningful connections between words.
Innovation Solution
A method utilizing a large language model (LLM) to identify segment-level and sentence-level phrases, combined with pointwise mutual information (PMI) and fuzzy matching, to extract informative and readable phrases from documents, determining similarity and relevance through transformer-based encoders and POS tagging.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of manufacture
If unsupervised KPE approaches (e.g., LDA) are used, then labeling effort is reduced, but informativeness and readability of extracted phrases deteriorate
Solution Approach 1:
The patent segments the document processing into two distinct pathways: one for extracting informative phrases (removing stop words) and another for extracting readable phrases (retaining stop words). This segmentation allows each pathway to optimize for its specific goal while avoiding the trade-off present in traditional unsupervised approaches.
Solution Approach 2:
The patent introduces PMI (pointwise mutual information) scoring as an intermediary mechanism to evaluate and combine results from both extraction pathways. The PMI scorer acts as a mediator that identifies meaningful connections between words while preserving readability, resolving the contradiction between informativeness and readability.
2Loss of information
If supervised KPE approaches are used, then informativeness of extracted phrases is improved, but labeling effort and device complexity increase
Solution Approach 1:
The system performs self-service by automatically computing PMI scores and using them to evaluate phrase quality without requiring external labeling. The model learns from the data itself through the PMI-based evaluation mechanism, eliminating the need for supervised labeling while maintaining informativeness.
Solution Approach 2:
The patent changes the evaluation parameter from traditional supervised metrics to PMI scoring, which measures the statistical association between words. This parameter change allows the system to achieve high informativeness without the complexity of supervised learning pipelines.
3Loss of information
If stop words are removed from segments, then informativeness of extracted phrases is improved, but readability deteriorates
Solution Approach 1:
The patent creates two separate processing streams: one that removes stop words to maximize informativeness, and another that retains stop words to preserve readability. Both streams process the same input segments independently, allowing each to optimize for its specific objective without compromising the other.
Solution Approach 2:
The patent merges the results from both processing streams by combining informative phrases (without stop words) and readable phrases (with stop words). The final output includes both types of phrases, allowing users to benefit from both informativeness and readability simultaneously.
Data Source
AI summary
A method for extracting key phrases based on extracting both informative and readable phrases from documents. The method includes separating documents into sentences to extract sentence-level phrases and separating the documents into segments of a fixed length to extract segment-level phrases. The method further includes using a PMI and entropy model to also extract readable phrases from the documents. The method includes comparing the sentence-level phrases, the segment-level phrases, and the readable phrases to generate key phrases for the documents.