Automatic Topic Detection Using Word Embedding Salience Scoring
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current topic detection systems in contact centers are prone to noise, detecting irrelevant topics due to recurring phrases not related to the organization's business, and struggle to handle new or unexpected terms effectively, leading to inefficient analysis of customer interactions.
Innovation Solution
A method and system for automatic assessment and filtering of topic labels using word embedding neural networks to calculate salience and relevance scores, filtering out low-quality phrases and irrelevant concepts, and integrating these into a multi-tenant, cloud-based infrastructure to identify domain-specific topics of interest.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If manual allow-list and deny-list filtering is used to filter recurring phrases, then relevant topics can be identified, but the system cannot handle new or unexpected terms effectively
Solution Approach 1:
The system automatically learns and adapts to new terms through machine learning models that analyze patterns in the data itself, rather than requiring manual updates to allow-lists and deny-lists. The model self-adjusts by identifying recurring phrases and their contextual relevance automatically.
Solution Approach 2:
The system transitions from static filtering parameters (manual lists) to dynamic parameters that adapt based on learned patterns. The machine learning model continuously adjusts its understanding of what constitutes relevant topics based on incoming data patterns.
2Adaptability or versatility
If topic detection systems detect all recurring phrases, then comprehensive topic coverage is achieved, but noise increases with irrelevant topics
Solution Approach 1:
The system applies different evaluation criteria to different phrases based on their contextual characteristics. Instead of uniform detection, it assesses each phrase's relevance to the specific business domain and interaction type, filtering out contextually inappropriate topics while maintaining comprehensive coverage of relevant ones.
Solution Approach 2:
The system uses feedback from salience scoring and relevance assessment to continuously refine topic detection. Phrases that score low on salience or relevance are filtered out, while high-scoring phrases are retained and used to improve future detection accuracy.
3Measurement precision
If manual allow-list and deny-list are continuously updated, then filtering accuracy improves, but system complexity and maintenance effort increase
Solution Approach 1:
The system automatically maintains its filtering accuracy through continuous learning from data patterns, eliminating the need for manual updates to allow-lists and deny-lists. The machine learning model self-updates its understanding of relevant topics based on incoming interactions.
Solution Approach 2:
The patent replaces the mechanical system of manual list maintenance with an intelligent system based on machine learning. Instead of manually adding and removing phrases from lists, the system uses algorithms to automatically identify and classify relevant topics.
Data Source
AI summary
A method and system for automatic topic detection in text may include receiving a text document of a corpus of documents and extracting one or more phrases from the document, based on one or more syntactic patterns. For each phrase, embodiments of the invention may: apply a word embedding neural network on one or more words of the phrase, to obtain one or more respective word embedding vectors; calculate a weighted phrase embedding vector, and compute a phrase saliency score, based on the weighted phrase embedding vector. Embodiments of the invention may subsequently produce one or more topic labels, representing one or more respective topics in the document, based on the computed phrase saliency scores, and may select one or more topic labels according to their relevance to the business domain of the corpus.