Ensemble Keyword Extraction Pipeline for Support Ticket Analysis
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current keyword extraction methods in Natural Language Processing, such as TF-IDF, TextRank, RAKE, LDA, and Large Language Models, are computationally expensive, require extensive manual labeling, or are less effective for domain-specific content, failing to efficiently extract relevant keywords and phrases from customer data in ticketing systems.
Innovation Solution
A machine-learning pipeline that combines multiple inter-changeable keyword extraction models, including statistical, rule-based, and Large Language Models, with a voting system to achieve consensus on extracted keywords, and applies these to customer support data to identify relevant keywords and phrases, thereby overcoming the limitations of individual models.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If traditional keyword extraction methods (TF-IDF, TextRank, RAKE, LDA) are used, then computational costs are high and manual labeling is required, but they fail to efficiently extract relevant keywords from customer data
Solution Approach 1:
The patent combines multiple keyword extraction models (statistical models like TF-IDF, rule-based models like RAKE, and machine learning models like LDA) into an ensemble system. This merging allows the system to leverage the strengths of each individual model while compensating for their weaknesses, achieving both high accuracy and computational efficiency in extracting keywords from customer support data.
Solution Approach 2:
The patent creates a simplified representation of customer support tickets by extracting only the relevant keywords and phrases using the ensemble model. This copying approach transforms the complex, unstructured ticket data into a condensed keyword set that maintains the essential information while reducing computational requirements for subsequent processing and analysis.
2Measurement precision
If Large Language Models are used for keyword extraction, then extraction accuracy improves, but computational expenses increase
Solution Approach 1:
The patent merges Large Language Models with traditional statistical and rule-based extraction methods in an ensemble architecture. The LLMs provide high accuracy for understanding context and extracting nuanced keywords, while the traditional models handle straightforward extraction tasks more efficiently, thereby reducing overall computational expenses while maintaining high accuracy.
Solution Approach 2:
The patent applies LLMs selectively only to portions of the data that require their advanced capabilities, such as ambiguous or context-dependent keyword extraction, while using simpler, less computationally expensive models for straightforward extraction tasks. This partial application of LLMs maintains high accuracy where needed while significantly reducing overall computational expenses.
3Measurement precision
If domain-specific keyword extraction is performed, then relevance improves, but general model effectiveness decreases
Solution Approach 1:
The patent applies local quality by tailoring the keyword extraction approach to the specific domain of customer support tickets. The ensemble model is fine-tuned and evaluated specifically on support data, allowing each component model to be optimized for this domain while maintaining the ability to handle various ticket types and customer scenarios effectively.
Solution Approach 2:
The patent creates a universal ensemble model that can effectively extract keywords across different customer support domains and scenarios. By combining multiple models with different strengths, the system achieves both domain-specific relevance and broad adaptability, capable of handling diverse ticket types, products, and customer inquiries without requiring separate specialized models for each domain.
Data Source
AI summary
A machine-learning pipeline for ontology generation via large language models is described. A system receives historical communications between support agents and customers, and multiple types of machine-learning models extract historical keywords from the historical communications. The system selects historical keywords which were identified by at least a specific number of the multiple types of machine-learning models. The system identifies some of the selected keywords from communications between support agents and customers, in response to receiving the communications. The system applies the identified keywords to recognizing skills required by a support agent to handle an open case, a trend in cases related to a product and/or a skill, and/or identifying skills for which support agents require additional training.