Contextual Intent and Slot Extraction with LLM Prompting
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Traditional intent and slot extraction techniques in voice-based systems are labor-intensive, prone to errors, and not scalable, requiring extensive manual labeling and struggling with unknown utterances.
Innovation Solution
A pre-train, prompt, and predict paradigm using a large language model (LLM) that generalizes from a limited number of semantically related phrases, allowing for flexible and context-aware intent and slot extraction.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If traditional supervised machine learning techniques with heavy feature engineering are used for intent detection, then the system can achieve accurate intent classification, but the process requires extensive manual labeling and massive curated datasets making it very tedious and time-consuming
Solution Approach 1:
The patent applies preliminary action by pre-training the language model on vast corpora of text data before deployment. This pre-training phase automatically acquires semantic understanding and intent recognition capabilities without requiring manual labeling during the actual intent detection process, thereby resolving the contradiction between accuracy and time consumption
Solution Approach 2:
The system employs self-service through automated semantic analysis where the pre-trained language model independently performs intent classification without human intervention. The model automatically processes utterances, identifies intents, and extracts slots without requiring manual curating of datasets or labeling, eliminating the tedious manual work while maintaining high accuracy
2Ease of manufacture
If rule-based regular expressions are used for intent detection, then the system can operate with simple implementation, but the system struggles to handle unknown utterances and lacks flexibility in interpreting varied user expressions
Solution Approach 1:
The patent applies parameter changes by transitioning from fixed rule-based parameters to dynamic semantic parameters. The pre-trained language model uses contextual embeddings and attention mechanisms that adaptively weigh different words and phrases based on their semantic relationships, allowing the system to handle diverse and unknown utterances while maintaining implementation simplicity through a unified model architecture
Solution Approach 2:
The system achieves universality through a single pre-trained language model that can handle multiple intent detection tasks across different domains. This multi-functional model replaces numerous domain-specific rule sets, providing both implementation simplicity and high adaptability to various utterance types and domains simultaneously
3Measurement precision
If the pre-train and then fine-tune paradigm is used for slot extraction, then the system can achieve domain-specific accuracy, but the process is not scalable and requires extensive curated datasets for each domain
Solution Approach 1:
The patent applies preliminary action by performing comprehensive pre-training on diverse, large-scale corpora that cover multiple domains and linguistic patterns. This pre-training establishes a robust foundation for slot extraction that can be directly applied to new domains without requiring domain-specific fine-tuning, thereby achieving both domain-specific accuracy and scalability
Solution Approach 2:
The system achieves universality through a single pre-trained model that generalizes slot extraction capabilities across multiple domains. The model learns transferable semantic patterns during pre-training that enable it to accurately extract slots in various domains without requiring separate fine-tuning processes, thus resolving the contradiction between domain-specific accuracy and scalability
Data Source
AI summary
Techniques for performing contextualized intent and slot extraction using a large language model (LLM) are disclosed. The LLM is generally pre-trained on an arbitrary corpus of language training data. A prompt is provided to the LLM. This prompt includes a limited number of prompt phrases. The prompt phrases share a semantic relationship with one another. A spoken utterance is recorded and then converted to text, resulting in generation of a transcription. The transcription is provided to the LLM. The LLM extracts, from the transcription, an extracted intent and an extracted slot. The extracted intent is determined to be related to a prompt-described intent that was included in the prompt. The prompt is supplemented by adding the extracted intent and the extracted slot to the prompt, resulting in the extracted intent being identified as sharing the semantic relationship with the other prompt phrases in the prompt.