Generative Language Model Data Augmentation for Intent Classification
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Creating an intent classification dataset with a sufficient number of intent samples is challenging, leading to difficulties in training and fine-tuning intent classifiers for natural language understanding applications.
Innovation Solution
A data augmentation system and method that uses a large, pre-trained autoregressive generative language model, such as GPT-3, to generate new intent samples from an initial dataset without fine-tuning, thereby augmenting the dataset for training intent classifiers.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If manual intent labeling is performed to create intent classification datasets, then the quality and accuracy of intent labels are improved, but the time and cost required to generate datasets increase significantly
Solution Approach 1:
The system performs preliminary actions by using the generative language model to pre-generate candidate intent labels and sample utterances before manual review. This preliminary generation reduces the overall time required while maintaining quality, as human annotators only need to review and validate pre-generated content rather than create everything from scratch
Solution Approach 2:
A generative language model is introduced as an intermediary between the data collection process and final intent classification. The model generates candidate labels and utterances that are then refined through human review, acting as a mediator that reduces manual workload while maintaining accuracy standards
2Reliability
If a large number of intent samples are collected manually to train intent classifiers, then the classifier performance is improved, but the cost and complexity of data collection increase
Solution Approach 1:
The system creates copies of existing intent samples by using the generative language model to synthesize new sample utterances based on template patterns. This allows multiplication of training data without proportional increase in manual collection effort, improving classifier performance while controlling complexity
Solution Approach 2:
The system changes parameters of existing intent samples by generating variations with different entities, contexts, and phrasings using the generative model. This transforms a limited set of manual samples into a diverse, large-scale dataset through parameter variation rather than manual collection
3Measurement precision
If fine-tuning is applied to pre-trained language models for specific intent classification tasks, then the task-specific performance is improved, but the loss of model generality occurs
Solution Approach 1:
Instead of fully fine-tuning the model which would compromise generality, the system applies partial adaptation by using the model for data generation while keeping the base model intact. This excessive use of the pre-trained model's capabilities for a specific task maintains generality while achieving task performance through generated data
4Reliability
If diverse intent samples are collected to cover various scenarios, then the robustness of intent classification is improved, but the difficulty of ensuring comprehensive coverage increases
Solution Approach 1:
The generative language model serves multiple functions simultaneously: it generates diverse samples, ensures coverage of edge cases, maintains consistency with existing data, and can adapt to different intent types. This multi-functionality achieves comprehensive coverage without increasing the difficulty of measurement or validation
Data Source
AI summary
The present disclosure relates to a data augmentation system and method that uses a large pre-trained encoder language model to generate new, useful intent samples from existing intent samples without fine-tuning. In certain embodiments, for a given class (intent), a limited number of sample utterances of a seed intent classification dataset may be concatenated and provided as input to the encoder language model, which may generate new sample utterances for the given class (intent). Additionally, when the augmented dataset is used to fine-tune an encoder language model of an intent classifier, this technique improves the performance of the intent classifier.