Anomaly-Based Training Data Curation for Dialogue Systems

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Modern virtual assistants implemented with a rules-based approach are inflexible and unable to effectively address queries or commands that exceed the scope of predetermined rules, and existing machine learning models for conversational systems face inefficiencies in training and classification tasks.

Innovation Solution

A machine learning-based dialogue system that sources raw training data, generates vector representations, identifies anomalous instances, and curates a joint corpus of training data using re-seeding, enabling efficient training and evolution beyond fixed rules for conversational interactions.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Reliability

If a rules-based approach is used to implement virtual assistants, then the system can provide deterministic responses to specific queries, but the system lacks flexibility to address queries beyond the scope of predetermined rules

Engineering Contradiction:

Improvedeterministic response capabilityVSAvoidflexibility to address varied queries

Core Design Contradiction:

ReliabilityVSAdaptability or versatility

Solution Approach 1:

The system transitions from static rules to dynamic machine learning models that can adapt to new query patterns. The virtual assistant uses trained ML models to generate responses for queries beyond predetermined rules, enabling the system to evolve and handle diverse conversational inputs while maintaining reliability through continuous learning and training.

Inventive Principle:

Principle #15Dynamics

Solution Approach 2:

The system changes the operational parameters from fixed rule sets to probabilistic machine learning predictions. By using trained models that process input queries through vector representations and anomaly detection, the system can dynamically adjust its response generation based on learned patterns, balancing deterministic behavior for known queries with adaptive responses for novel queries.

Inventive Principle:

Principle #35Parameter changes

2Adaptability or versatility

If machine learning models are used for conversational systems, then the system gains flexibility to handle varied queries, but training inefficiencies persist

Engineering Contradiction:

Improvecapability to address varied queriesVSAvoidtraining efficiency

Core Design Contradiction:

Adaptability or versatilityVSProductivity

Solution Approach 1:

The system performs preliminary actions by proactively identifying and curating anomalous training instances before full model training. The anomaly detection mechanism pre-processes training data to isolate valuable edge cases, and the re-seeding process pre-prepares enhanced training corpora, reducing overall training time and improving efficiency while maintaining model adaptability.

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The system extracts anomalous instances from the training corpus using anomaly detection on vector representations. By separating these valuable edge cases from the general training data and creating targeted re-seeded training sets, the system improves training efficiency by focusing computational resources on the most informative samples rather than processing entire corpora uniformly.

Inventive Principle:

Principle #2Taking out (Extraction)

3Device complexity

If a finite set of rules is used to drive virtual assistant responses, then the system structure is simple and manageable, but the system cannot effectively interact with users for queries exceeding the rule scope

Engineering Contradiction:

Improvesystem structure simplicityVSAvoidconversational interaction capability

Core Design Contradiction:

Device complexityVSAdaptability or versatility

Solution Approach 1:

The system replaces static rule structures with dynamic machine learning models that automatically adapt to conversational patterns. The ML-based dialogue system processes user inputs through trained models that learn from training data, enabling the virtual assistant to handle diverse queries without requiring manual rule creation for each scenario, thus maintaining manageable complexity while dramatically improving conversational capability.

Inventive Principle:

Principle #15Dynamics

Data Source

PatentUS20200193331A1Systems and methods for automatically configuring training data for training machine learning models of a machine learning-based dialogue system including seeding training samples or curating a corpus of training data based on instances of training data identifiedas anomalous

Publication Date: 2020.06.18 CLINC INC

AI summary

A system and method for improving a machine learning-based dialogue system includes: sourcing a corpus of raw machine learning training data from sources of training data based on a plurality of seed training samples, wherein the corpus of raw machine learning training data comprises a plurality of distinct instances of training data; generating a vector representation for each distinct instance of training data; identifying statistical characteristics of the corpus of raw machine learning training data based on a mapping of the vector representation for each distinct instance of training data; identifying anomalous instances of the plurality of distinct instances of training data of the corpus of raw machine learning training data based on the identified statistical characteristics of the corpus; and curating the corpus of raw machine learning training data based on each of the instances of training data identified as anomalous instances.