Topic Modeling System for Removing Scripted Call Content
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Topic models used in machine learning to identify topics from call transcripts often include overinclusive and scripted topics, making it difficult to extract useful information, as they incorrectly identify broad or scripted content, which is challenging to manually remove and negatively impacts the accuracy of topic modeling.
Innovation Solution
An improved topic modeling system that segregates parties in conversations and uses a Pitman-Yor process to model probabilities, allowing for the identification and removal of broad or scripted topics by computing vocabulary, consistency, and distance values to distinguish them from conversational topics, thereby refining the output.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If a topic model is built to identify a topic for every word in the input dataset, then the model can comprehensively cover all words including grammatical words, but the model becomes overinclusive and produces broad topics that provide little useful information
Solution Approach 1:
The patent extracts and removes broad topics from the topic model output. A broad topic identification module specifically targets and extracts topics that are too general (containing common words like 'go', 'get', 'say') or scripted content, separating them from the main topic results to prevent them from diluting the useful information.
Solution Approach 2:
The patent applies different quality standards to different topics by identifying characteristics specific to broad topics (high entropy, presence of common words, scripted patterns) and treating them differently from substantive topics. This allows the system to maintain comprehensive coverage while filtering out low-quality broad topics.
2Productivity
If recorded messages or scripted content are included in the topic model input, then the model processes all call content uniformly, but the model incorrectly identifies topics in scripted calls, making it appear like every call included a discussion of goods or services
Solution Approach 1:
The patent performs preliminary identification of scripted calls before topic modeling. By detecting scripted content patterns in advance and flagging these calls, the system can then apply different processing or filtering to their topic outputs, preventing scripted topics from contaminating the overall topic analysis.
Solution Approach 2:
The patent introduces an intermediary broad topic identification module that acts as a mediator between the topic model and the final results. This module detects and filters out scripted topics by analyzing topic characteristics, serving as a buffer that prevents inaccurate scripted topic identification from affecting the overall measurement precision.
3Measurement precision
If manual identification and removal of broad topics is performed, then the accuracy of topic modeling can be improved, but the process becomes unfeasible and runs counter to the purpose of automated topic modeling
Solution Approach 1:
The patent enables the topic model system to automatically identify and remove its own broad topics through algorithmic detection of topic characteristics (entropy, word frequency patterns, scripted content detection). The system serves itself by autonomously filtering out low-quality topics without requiring manual intervention, thus maintaining high automation while improving accuracy.
Solution Approach 2:
The patent implements feedback mechanisms where the topic model output is analyzed to identify broad topics, which are then removed and fed back into the system. This closed-loop feedback allows the automated system to continuously refine its output quality by detecting and correcting its own overinclusion problems.
4Loss of information
If the output of a topic model includes all identified topics, then the model provides complete topic data, but viewing and understanding the results becomes extremely difficult due to the presence of broad topics
Solution Approach 1:
The patent extracts broad topics from the complete topic output and separates them into a distinct category or removes them entirely from the main results. This extraction maintains the completeness of the original topic data for analysis purposes while improving interpretability by presenting only high-quality substantive topics to users.
Solution Approach 2:
The patent segments the topic output into different categories (broad topics vs. substantive topics) based on identified characteristics. This segmentation allows the system to preserve complete topic data for computational purposes while presenting segmented, filtered results that are easier for users to interpret and understand.
Data Source
AI summary
Systems and methods for improving machine learning systems used to model topics on a plurality of calls are described herein. In an embodiment, a server computer receives plurality of digitally stored call transcripts that have been prepared from digitally recorded voice calls. The server computer uses a topic model of an artificial intelligence machine learning system, the topic model modeling words of a call as a function of one or more word distributions for each topic of a plurality of topics, to generate an output of the topic model which identifies the plurality of topics represented in the plurality of call transcripts. The server computer computes, for a particular topic of the plurality of topics a first value representing a vocabulary of the particular topic and a second value representing a consistency of the particular topic in two more call transcripts of the plurality of call transcripts which include the particular topic. Based, at least in part, on one or more of the first value or the second value, the server computer determines that the particular topic meets a particular criterion and, in response, updates the output of the topic model to remove the particular topic or distinguish the particular topic from other topics of the plurality of topics which do not meet the particular criterion.