LSTM Networks for Open Intent Extraction and Domain Adaptation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional systems for natural language understanding of text inputs face challenges in accuracy, flexibility, and efficiency due to their rigidity in predefined intent categories, limited ability to identify multiple intents, and inefficient resource utilization.
Innovation Solution
The use of recurrent neural networks, specifically LSTM neural networks, to determine the existence and extract open intents from text inputs without prior knowledge of intent categories, employing an unsupervised domain adaptation approach to transfer across conversational domains and reduce the need for labeled training data.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If conventional systems use predefined intent categories to classify text inputs, then the system structure is simple and easy to implement, but the system lacks flexibility and cannot accurately identify intents that do not conform to preexisting classifications
Solution Approach 1:
The patent changes the fundamental parameter of intent classification from categorical (predefined buckets) to continuous (intent existence probability and extraction). By using LSTM neural networks to output intent existence probabilities and extract verb-object pairs directly from text, the system adapts to any intent type without being constrained by predefined categories, thereby achieving flexibility without proportionally increasing complexity.
Solution Approach 2:
The patent replaces the mechanical categorical classification system with a neural network-based semantic understanding system. Instead of manually defining intent categories and mapping text to them, the system uses LSTM networks to learn semantic patterns and extract intents programmatically, enabling the system to handle unseen intent types while maintaining reasonable computational complexity.
2Measurement precision
If conventional systems are limited to extracting only one intent from a text input, then the processing is simple and fast, but the accuracy decreases when multiple intents exist in the text
Solution Approach 1:
The patent segments the intent extraction process into two distinct stages: (1) intent existence detection using an LSTM network that outputs probability scores for each potential intent, and (2) intent extraction using another LSTM network that identifies verb-object pairs. This segmentation allows the system to accurately identify multiple intents when they exist while maintaining efficiency by processing each stage independently with specialized networks.
3Measurement precision
If conventional systems expend significant resources to identify labeled training data and train computer models, then the model can achieve reasonable accuracy on training domains, but the system becomes inefficient and resource-intensive
Solution Approach 1:
The patent enables the system to self-adapt to new domains through unsupervised domain adaptation. The LSTM networks are trained to learn general semantic patterns and intent structures that transfer across domains without requiring extensive retraining on labeled data from each new domain. This allows the system to maintain accuracy across different conversational domains while significantly reducing the computational resources needed for training and adaptation.
4Extent of automation
If conventional systems inaccurately classify intent from digital text inputs, then additional language understanding tasks can be performed, but computational resources are wasted on tasks based on incorrect intents
Solution Approach 1:
The patent introduces feedback mechanisms through intent existence probability scores that indicate the confidence level of detected intents. The system can use this feedback to determine whether to proceed with additional language understanding tasks based on the confidence threshold, thereby avoiding wasteful computation on low-confidence or incorrect intent classifications while still enabling automation for high-confidence cases.
Data Source
AI summary
The present disclosure relates to systems, non-transitory computer-readable media, and methods that utilize recurrent neural networks to determine the existence of one or more open intents in a text input, and then extract the one or more open intents from the text input. In particular, in one or more embodiments, the disclosed systems utilize a trained intent existence neural network to determine the existence of an actionable intent within a text input. In response to verifying the existence of an actionable intent, the disclosed systems can apply a trained intent extraction neural network to extract the actionable intent from the text input. Furthermore, in one or more embodiments, the disclosed systems can generate a digital response based on the intent identified from the text input.