Composed Variational NLG for Few-Shot Intent Training
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing intelligent dialogue systems face challenges in intent detection, particularly with few-shot intents, due to imbalanced training data and the need for large amounts of labeled examples, leading to unsatisfactory performance and limited adaptability.
Innovation Solution
A composed variational natural language generation (CLANG) model that uses a transformer-based conditional variational autoencoder to encode intents as combinations of domain and action, generating training samples for few-shot intents by transferring knowledge from many-shot intents, and employs contrastive regularization to improve learning.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If supervised learning with large amounts of labeled examples is used for intent detection, then classification accuracy is improved, but training cost and time consumption increase significantly
Solution Approach 1:
The system performs preliminary action by using the CLANG model to generate synthetic training data for few-shot intents before the main training process. This pre-generated data is then used to train the intent detection model, eliminating the need to manually collect and label large amounts of data for each intent class, thus significantly reducing training time while maintaining detection accuracy.
Solution Approach 2:
The system applies copying by using the CLANG model to generate synthetic examples that replicate the characteristics of real user utterances for few-shot intents. These copied synthetic samples are then used as training data, allowing the system to achieve accurate intent detection without requiring extensive manual data collection and labeling efforts.
2Reliability
If supervised learning with balanced training data is used for intent detection, then performance on few-shot intents is improved, but data collection and labeling cost increase
Solution Approach 1:
The system uses the CLANG model to generate synthetic training data that copies the essential characteristics of real user utterances for few-shot intents. This allows the system to achieve reliable few-shot intent detection without needing to collect and label large volumes of real training data, thereby reducing data collection and labeling costs while maintaining performance.
Solution Approach 2:
The system applies parameter changes by transforming the approach from using real labeled data to using synthetically generated data. The CLANG model changes the data source parameter from manual collection to automated generation, maintaining data quality and reliability while significantly reducing the quantity of data that needs to be manually labeled.
3Productivity
If existing training data is used for intent detection, then training speed is improved, but adaptability to unseen classes deteriorates
Solution Approach 1:
The system performs preliminary action by generating synthetic training data for unseen few-shot intent classes before training the model. This pre-generated data enables the model to quickly adapt to new classes without requiring extensive retraining or data collection, thus maintaining high training speed while improving adaptability to unseen classes.
Solution Approach 2:
The CLANG model serves a universal function by being able to generate training data for any intent class, whether seen or unseen. This multi-functional capability allows the system to maintain fast training speeds while simultaneously improving adaptability to new unseen classes, as the same model can generate data for any intent without requiring class-specific adjustments.
Data Source
AI summary
Embodiments described herein provide a composed variational natural language generation (CLANG) model that is configured to generate training samples for few-shot intents. Specifically, the CLANG model may build connections between existing training samples of many-shot intents and new training samples of few-shot intents by modeling an intent as a combination of a domain and an action. In this way, the CLANG model transfers knowledge from existing many-shot intents to few-shot intents in natural language generation by learning how to compose utterances with many-shot intents and transferring such knowledge to few-shot intents.