Hybrid Speech NLU Routing for Ambiguous Multi-Turn Utterances
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional speech recognition systems face challenges in processing multi-turn dialogues due to co-reference resolution and ambiguity issues, and the integration of large language models is costly and resource-intensive.
Innovation Solution
A hybrid approach combining a rule-based natural language-understanding engine with a machine learning-based engine, augmented by a large language model agent, to process utterances, resolving ambiguities and leveraging context for effective dialog management.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If a rule-based natural language-understanding engine is used to process utterances, then processing speed and accuracy for predefined commands are improved, but the system fails to handle ambiguous utterances and co-reference resolution in multi-turn dialogues
Solution Approach 1:
The system segments the processing of ambiguous utterances by first attempting rule-based NLU, and only when that fails does it invoke the LLM agent. This segmentation allows the system to maintain high-speed rule-based processing for clear commands while selectively applying the more versatile LLM only when needed for ambiguous cases.
Solution Approach 2:
The LLM agent acts as an intermediary component that bridges the gap between the rule-based NLU engine and the speech recognition system. When the rule-based engine cannot process an utterance, the LLM agent intervenes to provide contextual understanding and resolve ambiguities, then passes the processed result back to the system.
2Adaptability or versatility
If a large language model is used to process all utterances, then the system can handle ambiguous utterances and multi-turn dialogues effectively, but the computational cost and resource consumption increase significantly
Solution Approach 1:
Instead of applying the LLM agent to all utterances (excessive action), the system applies it only partially - specifically to utterances that the rule-based NLU engine cannot process. This partial application reduces computational resource consumption while still providing LLM capabilities when needed for ambiguous or complex multi-turn dialogue cases.
Solution Approach 2:
The processing pipeline is segmented into two stages: first the rule-based NLU engine processes straightforward commands, and only the remaining ambiguous cases are passed to the LLM agent. This segmentation optimizes resource usage by avoiding unnecessary LLM computation for simple, clear commands.
3Adaptability or versatility
If conventional out-of-domain algorithms are used to identify and handle ambiguous utterances, then the system can detect co-reference resolution problems, but the system requires three types of exception handling that complicate the architecture and reduce efficiency
Solution Approach 1:
The system merges the co-reference resolution and ambiguity handling capabilities into the unified LLM agent component. Instead of maintaining separate exception handling modules for misclassification, incomplete recognition, and unsupported features, the LLM agent consolidates these functions, simplifying the overall system architecture while maintaining comprehensive handling capabilities.
Data Source
AI summary
A method and apparatus process input utterances by a speech recognition system. The method and apparatus are implemented by a computer of a speech recognition system to process an utterance that is received as an input. The method includes processing the utterance by a rule-based natural language-understanding engine. The method further includes, when the rule-based natural language-understanding engine fails to process the utterance, converting a representation of the utterance and allowing a machine learning-based natural language-understanding engine to process the utterance by using a large language model (LLM) agent. The method further includes processing the utterance with a converted representation by the machine learning-based natural language-understanding engine.