Multi-model Structure for Simultaneous Intent Recognition
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional Spoken Language Understanding (SLU) systems struggle to identify multiple intents from a single utterance across different domains, as they operate independently and lack the ability to share information, leading to incomplete intent recognition.
Innovation Solution
A multi-model structure is implemented, where multiple domain-specific models are trained simultaneously using local and global loss metrics, generating convex coefficients to combine outputs and determine intents, allowing for the identification of multiple intents from a single utterance by leveraging collaborative information.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If multiple domain-specific models operate independently, then each model can be trained separately, but the system cannot identify multiple intents from a single utterance across different domains
Solution Approach 1:
The patent merges multiple independent domain-specific models into a unified multi-model structure that shares information across domains. The models are connected through a common input layer and coordinated through loss function optimization, enabling them to jointly identify multiple intents from a single utterance while maintaining domain-specific expertise.
Solution Approach 2:
The multi-model structure achieves universality by enabling each domain-specific model to contribute to intent recognition across multiple domains. The shared architecture allows models to perform both their specific domain functions and collaborative intent determination, making the system multi-functional rather than isolated specialized components.
2Productivity
If models are trained separately, then training process is simpler, but training speed and predictive accuracy are reduced
Solution Approach 1:
The patent combines multiple separate training processes into a unified training framework where domain-specific models are trained simultaneously using a shared loss function. This coordinated training approach enables parallel processing of multiple domains while maintaining optimization coherence, thereby increasing training speed and predictive accuracy compared to sequential separate training.
Solution Approach 2:
The training process incorporates feedback mechanisms through the loss function that continuously monitors and adjusts model outputs across all domains. The shared loss function provides feedback that coordinates training across multiple models, enabling them to learn from each other and converge faster toward optimal performance than isolated models would achieve separately.
3Reliability
If independent models are used, then implementation is easier, but the system is susceptible to vanishing gradient problem
Solution Approach 1:
The patent merges multiple independent models into a connected multi-model structure where gradients can flow through shared layers. This architecture enables gradient propagation across domains while maintaining the stability benefits of domain-specific training, reducing susceptibility to vanishing gradients through coordinated optimization rather than isolated training.
Solution Approach 2:
The shared loss function provides feedback that stabilizes training by coordinating gradient updates across all domain-specific models. This feedback mechanism ensures that gradients are properly propagated and adjusted throughout the network, preventing vanishing gradient problems while maintaining training stability and reliability.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
Intent determination based on one or more multi-model structures can include generating an output from each of a plurality of domain-specific models in response to a received input. The domain-specific models can comprise simultaneously trained machine learning models that are trained using a corresponding local loss metric for each domain-specific model and a global loss metric for the plurality of domain-specific models. The presence or absence of an intent corresponding to one or more domain-specific models can be determined by classifying the output of each domain-specific model.