Context-Based Keyphrase Extraction Using Transformer Attention
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current keyphrase extraction techniques lack efficiency and accuracy, especially in genealogical research, where large collections of records require automated and objective methods to identify keyphrases accurately due to the impracticality of manual tagging and subjective human evaluation.
Innovation Solution
A computer-implemented method using a transformer-based machine learning model, such as BERT, generates attention matrices to identify context-based relationships between tokens and words, forming clusters and selecting keyphrases based on contextual relationships without requiring training data or external statistics.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If manual keyphrase extraction is used, then keyphrases can be identified with human understanding, but the process is laborious, subjective, and impractical for large collections
Solution Approach 1:
The patent replaces the manual mechanical process of human keyphrase extraction with an automated computational system using machine learning models (TF-IDF, TextRank, and neural network-based approaches). This substitution enables processing of large genealogical collections at scale while maintaining objective, consistent keyphrase identification through algorithmic analysis of term frequency, inverse document frequency, and contextual relationships.
2Productivity
If current automatic keyphrase extraction techniques are used, then productivity is improved, but accuracy and reliability are insufficient for genealogical research
Solution Approach 1:
The patent employs a composite approach combining multiple keyphrase extraction methodologies (TF-IDF for statistical term importance, TextRank for graph-based ranking, and neural network models for contextual understanding) rather than relying on a single technique. This composite system leverages the strengths of each method to achieve both high productivity and improved accuracy, with the neural network component specifically addressing contextual nuances in genealogical texts.
Solution Approach 2:
The patent introduces attention mechanisms as an intermediary layer between input text and keyphrase output, allowing the system to weigh and prioritize contextual relationships between words. This intermediary processing step enables more accurate identification of meaningful keyphrases by capturing dependencies and relationships that simpler methods miss, while still maintaining automated high-speed processing.
3Reliability
If manual keyphrase extraction is used, then contextual understanding can be applied, but the process is subjective and prone to errors and biases
Solution Approach 1:
The patent replaces subjective human evaluation with objective computational algorithms that consistently apply the same criteria across all genealogical records. The systematic use of TF-IDF scoring, TextRank calculations, and neural network predictions eliminates human bias and variability, ensuring reliable and reproducible keyphrase extraction across the entire collection without manual intervention.
4Quantity of substance
If large collections of genealogical records are processed, then comprehensive coverage is achieved, but manual tagging becomes impractical
Solution Approach 1:
The patent segments the complex task of keyphrase extraction into distinct computational components: preprocessing stage (tokenization, stopword removal), feature extraction stage (TF-IDF calculation, TextRank computation), and classification stage (neural network prediction). This segmentation allows each component to be optimized independently and processed efficiently across large numbers of genealogical records, making the overall system manageable despite the large volume of data.
Data Source
AI summary
Described herein are systems, methods, and other techniques for extracting one or more keyphrases from an input text. The input text may include a plurality of words. A plurality of token-level attention matrices may be generated using a transformer-based machine learning model. The plurality of token-level attention matrices may be converted into a plurality of word-level attention matrices. A set of candidate phrases may be identified from the plurality of words based on the plurality of word-level attention matrices. The one or more keyphrases may be selected from the set of candidate phrases.