Entity-Aware Data Augmentation for Robust Intent Prediction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Machine learning models for intent prediction in chatbots are sensitive to entity value changes, leading to inconsistent outputs and erroneous predictions due to entity value variations in user utterances.
Innovation Solution
Implementing a combination of positive and negative entity-aware data augmentation techniques, including random replacement, deletion, and insertion of named entities, to enhance the robustness of intent prediction models.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If traditional training data is used without entity-aware augmentation, then the training process is simple, but the model is sensitive to entity value changes and produces inconsistent outputs
Solution Approach 1:
The patent applies preliminary action by performing entity-aware data augmentation before model training. The system identifies named entities in training utterances and generates augmented versions by replacing entities with synonyms or random values of the same type. This preprocessing step creates a more robust training dataset that accounts for entity variations, enabling the model to learn entity-agnostic representations and produce consistent outputs regardless of specific entity values.
Solution Approach 2:
The patent applies copying by creating multiple copies of training utterances with modified entity values. For each original training example, the system generates augmented copies by substituting entities with synonyms or random alternatives. These copied and modified examples are added to the training set, allowing the model to learn from varied entity representations without changing the underlying intent structure, thereby improving reliability across different entity instances.
2Reliability
If entity-aware data augmentation is applied, then the model robustness to entity value changes is improved, but the training data processing time and complexity increase
Solution Approach 1:
The patent applies partial action by selectively applying entity-aware augmentation to only the named entities present in training utterances, rather than processing all text uniformly. The system identifies specific entity spans and applies augmentation only to those portions, leaving the rest of the utterance structure intact. This selective approach achieves robustness improvement while minimizing the additional processing time compared to comprehensive text transformation.
3Measurement precision
If the model focuses on individual entity values, then it can capture specific details, but it becomes sensitive to entity value changes and produces erroneous predictions
Solution Approach 1:
The patent applies parameter changes by systematically varying the entity value parameter in training examples while keeping the intent and context constant. The entity-aware augmentation process replaces entities with synonyms or random values of the same type, creating training examples that differ only in entity parameters. This teaches the model to recognize that variations in entity parameters should not affect intent prediction, thereby improving generalization across different entity values while maintaining the ability to recognize entities when present.
Data Source
AI summary
Novel techniques are described for data augmentation using a two-stage entity-aware augmentation to improve model robustness to entity value changes for intent prediction. In some embodiments, a method comprises accessing a first set of training data for a machine learning model; applying one or more data augmentation techniques to the first set of training data to result in a second set of training data; applying an additional augmentation technique to augment the second set of training data to create a post-processed augmented training data where the additional augmentation technique comprises replacing at least one or more entity values of the named entities within the second set of training data with random values of same entity type; and combining the first set of training data and the post-processed augmented training data to generate expanded training data; and training the machine learning model using the expanded training data.