AI Topic Detection via LDA Term Subtraction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Enterprise organizations face difficulties in tracking and identifying emerging topics in customer service interactions, particularly in large volumes of customer service data, where new or increasing topics may go unnoticed.
Innovation Solution
The implementation of artificial intelligence and machine learning techniques to analyze customer issue data, converting voice data to text, generating bodies of terms for different time periods, and using Latent Dirichlet Allocation (LDA) to identify and label emerging topics, which are then stored and displayed for further analysis.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Difficulty of detecting and measuring
If manual tracking and identification methods are used for customer service topics, then operational simplicity is maintained, but the ability to detect emerging topics in large volumes of data deteriorates
Solution Approach 1:
The patent replaces manual tracking and identification methods with an automated machine learning system that uses natural language processing and topic modeling algorithms to detect emerging topics in customer service data, thereby improving detection capability while accepting increased system complexity
Solution Approach 2:
The patent introduces intermediate processing layers including text preprocessing, term frequency analysis, and LDA topic modeling that act as mediators between raw customer service data and final topic identification, enabling automated detection of emerging topics through structured analytical steps
2Difficulty of detecting and measuring
If automated AI and machine learning techniques are implemented to analyze customer issue data, then topic detection capability is improved, but processing time and computational resources increase
Solution Approach 1:
The patent performs preliminary text preprocessing operations including converting text to lowercase, removing special characters, and eliminating stop words before main topic analysis, which streamlines subsequent processing and reduces computational overhead for the core LDA topic modeling
Solution Approach 2:
The patent segments the analysis process into distinct phases: data preprocessing, term frequency calculation, LDA topic modeling, and emerging topic identification. This segmentation allows each phase to be optimized independently and enables parallel processing where applicable, reducing overall processing time
Data Source
AI summary
Arrangements for providing emerging topic detection are provided. In some aspects, customer issue data may be received. For instance, voice data from customer and customer service associate interactions may be received. The customer issue data may be converted to text data and the text data may be further analyzed. For instance, a first body of terms for a first time period and a second body of terms for a second time period after the first time period may be generated. The generated first body of terms may be subtracted from the second body of terms to generate filtered terms. The filtered terms may then be further analyzed using, for instance, Latent Dirichlet Allocation (LDA) to identify emerging terms that may then be categorized and stored in one or more databases. In some examples, emerging terms and/or associated categories may be transmitted to a computing device for display on a dashboard.