Conversational Intent Training Using Example Forgetting
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing virtual assistant applications face challenges in accurately determining user intent due to unrecognized or incomprehensible user messages, leading to user dissatisfaction and inefficiencies, and the process of creating and validating training data for intent recognition models is time-consuming and inefficient.
Innovation Solution
A system and method utilizing example forgetting techniques to analyze training data for conversational interfaces, identifying and refining utterances with low prediction accuracy, and generating an updated training dataset to improve intent prediction models.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If the intent recognition model is re-trained to improve accuracy, then the model's intent prediction performance improves, but the time required for data creation, validation, and re-training increases
Solution Approach 1:
The patent extracts and removes utterances from the training data that do not contribute to improving model accuracy. By identifying and excluding ineffective training examples, the system reduces the time needed for data creation and validation while maintaining intent prediction accuracy.
Solution Approach 2:
The patent implements a feedback mechanism where the intent recognition model's predictions are used to evaluate training data effectiveness. The system uses prediction accuracy as feedback to determine which utterances should be included or excluded from training data, creating an iterative optimization process that reduces time loss.
2Quantity of substance
If all utterances are included in training data, then the training data corpus is comprehensive, but the model training becomes less efficient due to inclusion of non-contributing utterances
Solution Approach 1:
The patent extracts and removes utterances from the training data that do not contribute to improving model accuracy. By identifying and excluding ineffective training examples, the system reduces the time needed for data creation and validation while maintaining intent prediction accuracy.
Solution Approach 2:
The patent changes the parameter of training data composition by dynamically adjusting which utterances are included based on their contribution to model accuracy. This allows the training corpus to evolve from a static comprehensive set to a dynamic optimized set that balances quantity with training efficiency.
Data Source
AI summary
Methods and apparatuses for optimizing performance of conversational interface applications using example forgetting include a server that retrieves training data comprising utterances each mapped to one or more known intents. The server determines a forgetting count for each utterance and selects utterances from the training data that have a forgetting count above a predetermined threshold. The server identifies whether the predicted intent associated with each utterance is accurate. The server generates updated training data comprising the selected utterances and corresponding predicted intents, and trains conversational interface applications using the updated training data. The server validates performance of the trained conversational interface applications and saves the updated training data.