Intent Classification via Curriculum Learning and Sentence Augmentation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Large language models face challenges in accurately distinguishing intent behind user inputs due to limited initial datasets, leading to misclassification of similar phrases with different wording, and require extensive manual effort for data collection and training.
Innovation Solution
The method involves obtaining an initial dataset, training a first intent classifier, generating augmented phrases through sentence augmentation, classifying these phrases into subsets based on confidence levels, and training a second intent classifier using curriculum learning to improve intent recognition within dialogues.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If manual data collection is used to train intent classifiers, then training data quality can be ensured, but the time and labor cost increases significantly
Solution Approach 1:
The system performs preliminary sentence augmentation on existing training data before actual training begins. By pre-generating augmented phrases and classifying them into subsets, the system prepares enriched training material in advance, reducing the need for extensive manual data collection later while maintaining training quality.
Solution Approach 2:
The system creates copies of existing training phrases through sentence augmentation, generating multiple variations of each phrase. These augmented copies are then classified and used as additional training data, effectively multiplying the value of the original manual data without requiring proportional increases in manual collection effort.
2Productivity
If a simple intent classifier is trained with limited initial dataset, then training speed is fast, but classification accuracy deteriorates for similar phrases with different wording
Solution Approach 1:
The augmented phrases are segmented into different subsets based on their confidence levels and similarity to original phrases. This segmentation allows the training process to progressively incorporate different types of augmented data, first with high-confidence matches and then with more challenging variations, improving accuracy without overwhelming the training process.
Solution Approach 2:
The training process is made dynamic through curriculum learning, where the difficulty of training data is gradually increased. The system starts with easier examples (original phrases and high-confidence augmentations) and progressively introduces more challenging examples (lower-confidence augmentations), allowing the classifier to adapt its capacity dynamically rather than being overwhelmed from the start.
3Measurement precision
If extensive manual data collection is performed to improve classification accuracy, then intent recognition precision improves, but the complexity and cost of the system increases
Solution Approach 1:
The system performs self-service by automatically generating augmented training data through sentence augmentation techniques. Instead of requiring external manual data collection, the system uses its own existing training data as the basis for generating additional training examples, effectively serving its own data needs and reducing external complexity.
Solution Approach 2:
The system changes parameters of existing phrases through sentence augmentation, creating variations in wording, structure, and expression while maintaining the core meaning. This parameter transformation allows the system to explore the phrase space more effectively and improve recognition precision without requiring proportional increases in manual data collection complexity.
Data Source
AI summary
A language processing method includes following steps. An initial dataset including initial phrases and initial intent labels about the initial phrases is obtained. A first intent classifier is trained with the initial dataset. Augmented phrases are produced corresponding to the initial phrases by sentence augmentation. First predicted intent labels about the augmented phrases and first confidence levels of the first predicted intent labels are generated by the first intent classifier. The augmented phrases are classified into augmentation subsets according to comparisons between the first predicted intent labels and the initial intent labels and according to the first confidence levels. A second intent classifier is trained according to a part of the augmentation subsets by curriculum learning. The second intent classifier is configured to distinguish an intent of an input phrase within a dialogue.