Dynamic Threshold Topic Labeling for Supervised Learning

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing topic discovery models struggle to accurately convert their probabilistic outputs into binary labels required for supervised machine learning systems, leading to mislabeling and inefficiencies in training data due to intolerance for uncertainty and errors, especially when dealing with brief discussions of topics in long calls.

Innovation Solution

A method is developed to generate an input training dataset for supervised machine learning systems by programmatically determining whether call transcripts include or exclude specific topics based on word probabilities and call length thresholds, segregating transcripts into 'true', 'false', and 'unknown' categories for training purposes.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Ease of manufacture

If a basic threshold of 75% probability is used to label call transcripts from a discovery model, then the supervised machine learning system can be trained with binary labels, but 25% of the call transcripts may be mislabeled causing incorrect training data

Engineering Contradiction:

Improveease of training data generationVSAvoidaccuracy of binary labels

Core Design Contradiction:

Ease of manufactureVSReliability

Solution Approach 1:

The patent changes the parameter from a fixed 75% probability threshold to a dynamic threshold calculated as (a priori proportion × call length). This allows the threshold to adapt to different call durations and topic prevalence, reducing mislabeling while maintaining binary label generation capability for training supervised machine learning systems

Inventive Principle:

Principle #35Parameter changes

Solution Approach 2:

The patent performs preliminary calculation of topic proportions across the entire dataset before generating binary labels. By pre-comparing each call's topic proportion against the calculated threshold, the system prepares accurate training data in advance, preventing mislabeling issues before training begins

Inventive Principle:

Principle #10Preliminary action

2Productivity

If a fixed proportion threshold is used to determine topic inclusion, then binary labels can be generated efficiently, but brief discussions of topics in long calls may be incorrectly excluded

Engineering Contradiction:

Improveefficiency of training data generationVSAvoidaccuracy of topic detection

Core Design Contradiction:

ProductivityVSMeasurement precision

Solution Approach 1:

The patent transforms the static fixed proportion threshold into a dynamic threshold that scales with call length. The formula (a priori proportion × call length) ensures that longer calls have higher absolute thresholds, preventing brief topic mentions from being incorrectly excluded while maintaining efficient binary label generation

Inventive Principle:

Principle #15Dynamics

Solution Approach 2:

The patent applies different threshold criteria based on local characteristics of each call, specifically its length. By comparing each call's topic proportion against a threshold adjusted for its duration, the system achieves accurate topic detection tailored to each call's specific context

Inventive Principle:

Principle #3Local quality

3Loss of information

If all topics identified by the discovery model are processed, then comprehensive topic coverage is achieved, but processing power is wasted on topics of limited use

Engineering Contradiction:

Improvecompleteness of topic identificationVSAvoidcomputational power consumption

Core Design Contradiction:

Loss of informationVSLoss of energy

Solution Approach 1:

The patent applies partial action by selectively processing only those topics that meet the dynamic threshold criterion for each call. Instead of processing all discovered topics uniformly, the system focuses computational resources on topics with sufficient prevalence, eliminating waste on low-value topics while maintaining coverage of important topics

Inventive Principle:

Principle #16Partial or excessive action

Data Source

PatentUS11804216B2Generating training datasets for a supervised learning topic model from outputs of a discovery topic model

Publication Date: 2023.10.31 INVOCA INC

AI summary

Systems and methods for generating training data for a supervised topic modeling system from outputs of a topic discovery model are described herein. In an embodiment, a system receives a plurality of digitally stored call transcripts and, using a topic model, generates an output which identifies a plurality of topics represented in the plurality of digitally stored call transcripts. Using the output of the topic model, the system generates an input dataset for a supervised learning model by identify a first subset of the plurality of digitally stored call transcripts that include the particular topic, storing a positive value for the first subset, identifying a second subset that do not include the particular topic, and storing a negative value for the second subset. The input training dataset is then used to train a supervised learning model.