LLM Utterance Augmentation for Scalable Intent Detection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current intent detection mechanisms in voice-based systems rely on rule-based regular expressions or supervised machine learning with heavy feature engineering, which are tedious, time-consuming, and not scalable, requiring manual tagging and extensive labeled datasets, and fail to adapt to unknown utterances.
Innovation Solution
A paradigm shift from 'pre-train and fine-tune' to 'pre-train, prompt, and predict' using a pre-trained large language model (LLM) to generate semantically related phrase variations, allowing for reduced manual input and improved flexibility in interpreting utterances and extracting intents and slots.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If rule-based regular expressions or supervised machine learning techniques are used for intent detection, then the system can detect known intents with reasonable accuracy, but the process becomes tedious, time-consuming, and not scalable due to manual feature engineering and extensive labeled datasets
Solution Approach 1:
The patent uses a pre-trained large language model to generate synthetic training data by copying and adapting patterns from its pre-training corpus. Instead of manually creating labeled datasets, the system leverages the LLM's existing language understanding to generate utterance-intent-slot triplets that mimic real-world data distributions, thereby eliminating time-consuming manual annotation while maintaining detection accuracy
Solution Approach 2:
The patent performs preliminary action by pre-training the large language model on a general corpus before fine-tuning it for specific intent detection tasks. This pre-training phase pre-computes language understanding capabilities, semantic relationships, and contextual patterns that would otherwise require extensive manual feature engineering and labeled data preparation, thereby reducing the time and effort needed for subsequent task-specific adaptation
2Adaptability or versatility
If extensive labeled datasets with exhaustive utterance collections are curated for each intent, then the system achieves better coverage of possible user inputs, but the complexity and time required for data preparation increases significantly
Solution Approach 1:
The patent implements self-service by enabling the large language model to generate its own task-specific training data from its pre-existing knowledge. The model autonomously creates utterance-intent-slot triplets without requiring external manual curation, thereby achieving comprehensive utterance coverage while eliminating the complex and time-consuming data preparation process. The system serves itself by leveraging its pre-trained capabilities to produce the training data it needs
Solution Approach 2:
The patent applies universality by using a single pre-trained large language model to handle multiple intents and generate training data for various domains. Instead of curating separate exhaustive datasets for each intent, the model's general language understanding capabilities are leveraged to adaptively generate relevant training examples across different intents and contexts, thereby achieving broad coverage with a unified approach
3Adaptability or versatility
If traditional pre-train and fine-tune paradigm is used, then the model can adapt to specific tasks, but the process remains time-consuming and requires extensive manual intervention for feature engineering
Solution Approach 1:
The patent uses copying by having the large language model generate synthetic training examples that replicate the structure and semantics of real-world data. Instead of manually engineering features and curating datasets for each task, the model copies language patterns and semantic relationships from its pre-training corpus to create task-specific training data, thereby achieving effective adaptation while dramatically improving operational efficiency by eliminating manual intervention
Solution Approach 2:
The patent implements self-service in the fine-tuning phase by enabling the model to autonomously generate its own training data from its pre-trained knowledge. Rather than requiring external teams to perform manual feature engineering and data annotation, the system self-generates the training corpus needed for task adaptation, thereby maintaining high adaptability while significantly boosting productivity by eliminating bottlenecks in the fine-tune process
Data Source
AI summary
Techniques for causing an LLM to generate semantically related phrase variations for an identified phrase are disclosed. An LLM that is generally pre-trained on an arbitrary corpus of language training data is accessed. Seed data is fed as input to the LLM. The seed data includes multiple phrases that are semantically related and that describe a command. When any one of the phrases is received as utterance input, the utterance input triggers execution of the command. The LLM generates multiple phrase variations based on the phrases, where each phrase variation is semantically related to the other phrases. When any one of the phrase variations is received as new utterance input, the new utterance input also triggers execution of the command. The phrases and phrase variations are then stored together in a data storage.