Hierarchical Domain Routing for Virtual Assistant Intent Classification
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Task-oriented dialogue systems face challenges in accurately interpreting user queries due to ambiguous or out-of-scope utterances, leading to frustrating user experiences as they struggle to determine the correct intent and domain, often requiring users to rephrase their questions without clear guidance.
Innovation Solution
Implementing a hierarchical domain routing and intent classification method using an AI model that analyzes user queries, applies domain classification when confidence is low, and prompts users for additional input, with the AI model trained using techniques like keyboard perturbation, back-translation, and paraphrasing to enhance intent determination.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If a single model directly processes natural language input (end-to-end approach), then the system complexity is reduced, but the accuracy in determining user intent and domain decreases
Solution Approach 1:
The system divides the intent determination process into two independent models: a domain classification model that first identifies the domain, and an intent classification model that then determines the specific intent within that domain. This segmentation allows each model to specialize in its specific task, improving overall accuracy while maintaining manageable system complexity through modular architecture.
2Reliability
If the confidence level threshold is set high to ensure accurate intent classification, then the reliability of intent determination improves, but the number of queries requiring additional user input increases
Solution Approach 1:
The system implements a confidence-based feedback mechanism where the intent classification model outputs a confidence score for each predicted intent. When the confidence score exceeds a threshold, the system automatically accepts the prediction without further user interaction. When the confidence is below the threshold, the system provides feedback to the user with multiple choice options, allowing the user to select the correct intent. This feedback loop ensures high reliability while minimizing unnecessary user interactions.
3Measurement precision
If the system requests additional user input for low-confidence queries to improve intent accuracy, then the measurement precision of intent classification improves, but the ease of operation deteriorates
Solution Approach 1:
The system introduces an intermediary multiple-choice interface that mediates between the system's uncertain intent classification and the user's actual intent. Instead of directly asking users to rephrase their queries (which is complex), the system provides pre-generated candidate intents based on the domain classification and confidence scoring, allowing users to simply select from predefined options. This intermediary mechanism maintains high intent classification accuracy while preserving ease of operation.
Data Source
AI summary
A system and a method for performing hierarchical domain routing and intent classification on user queries in order to improve accuracy in responding to such queries and to create smoother conversations between virtual voice assistants and users are provided. The method includes: receiving an utterance from a user; analyzing the utterance in order to make an initial determination of user intent and a confidence level that relates thereto; when the confidence level is less than a threshold, applying an artificial intelligence (AI) model that is configured to assign the received utterance to one or more domains; outputting, based on the assigned domain(s), information that prompts the user to provide additional input that relates to the user intent; receiving the additional input; and secondarily determining, based on the additional input, the user intent.