Intent Matching via Transfer Learning BiLSTM
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
As digital assistants and conversational agents become more sophisticated, the complexity of understanding user intents increases with the number of possible intents and domains, making it difficult for existing rule-based and machine learning approaches to effectively detect and handle user inputs, especially due to the need for extensive linguistic rules and large training datasets.
Innovation Solution
A deep learning model that uses a matching approach instead of classification, leveraging transfer learning and pretrained models to reduce the need for domain-specific training and linguistic rules, and identifies user intents by comparing input requests to candidate intent predicates using a bi-directional Long Short-Term Memory (BiLSTM) neural network and knowledge embeddings.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If rule-based and machine learning approaches are used to detect user intents, then intent detection capability is improved, but the complexity of the system increases due to extensive linguistic rules and large training datasets
Solution Approach 1:
The patent replaces traditional rule-based mechanical systems with a neural network-based semantic matching system. Instead of using extensive linguistic rules and classification models that require large training datasets, the system uses pre-trained word embeddings and a neural network to compute semantic similarity between user inputs and intent predicates, significantly reducing system complexity while maintaining or improving intent detection capability.
2Adaptability or versatility
If the number of possible intents a user can express increases, then the system becomes more sophisticated and capable, but the complexity of providing a system that understands all possible intents increases
Solution Approach 1:
The patent implements a universal semantic matching mechanism that can handle any number of intents without requiring separate processing logic for each intent. The neural network computes semantic similarity between user inputs and intent predicates in a domain-agnostic manner, allowing the system to scale to accommodate increasing numbers of intents while maintaining consistent performance and avoiding exponential growth in system complexity.
3Measurement precision
If domain-specific training is performed for each domain, then intent detection accuracy for that domain is improved, but the time and resources required for training increase
Solution Approach 1:
The patent performs preliminary action by pre-training word embeddings on large corpora before deployment. These pre-trained embeddings capture general semantic relationships that are transferable across domains. When the system needs to detect intents in new or specific domains, it can directly utilize these pre-trained embeddings without requiring extensive domain-specific training, thereby maintaining high accuracy while minimizing training time and computational resources.
Data Source
AI summary
Described herein is a mechanism to identify user intent in requests submitted to a system such as a digital assistant or question-answer systems. Embodiments utilize a match methodology instead of a classification methodology. Features derived from a subgraph retrieved from a knowledge base based on the request are concatenated with pretrained word embeddings for both the request and a candidate predicate. The concatenated inputs for both the request and predicate are encoded using two independent LSTM networks and then a matching score is calculated using a match LSTM network. The result is identified based on the matching scores for a plurality of candidate predicates. The pretrained word embeddings allow for knowledge transfer since pretrained word embeddings in one intent domain can apply to another intent domain without retraining.