Targeted Clarification in Speech Recognition via Concept Scoring
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current natural language dialog systems inefficiently handle errors by employing generic clarification strategies, failing to mimic human-like targeted clarification methods that provide contextual feedback, leading to inefficient communication and user experience in spoken dialog systems.
Innovation Solution
Implementing a system with localized error detection (LED) and targeted clarification using two separate models for concept presence and correctness prediction, allowing the system to generate targeted clarification questions that specifically address misrecognized segments in speech recognition outputs, thereby improving error recovery and user interaction.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If generic clarification strategies are used in speech recognition systems, then the system can handle errors, but the communication efficiency and user experience deteriorate due to lack of contextual feedback
Solution Approach 1:
The system segments the speech recognition output into individual speech acts and identifies errors at the sentence level rather than treating the entire utterance as a single unit. This allows targeted clarification of specific segments while preserving correct portions, improving communication efficiency while maintaining error handling capability
Solution Approach 2:
The system implements feedback by providing contextual information about what was correctly recognized and what requires clarification. This feedback mechanism enables the system to ask targeted clarification questions that reference correct speech acts, improving both user experience and communication efficiency
2Reliability
If generic clarification strategies are used, then error recovery is possible, but user experience deteriorates due to inefficiency in repeating entire utterances
Solution Approach 1:
The system extracts and identifies only the specific sentence or speech act containing the error, separating it from the rest of the correct utterance. This extraction allows the user to provide minimal corrections rather than repeating entire utterances, reducing time loss while maintaining error recovery
Solution Approach 2:
Instead of requiring complete repetition of the entire utterance, the system applies partial action by targeting only the specific erroneous segment for clarification. This partial clarification approach reduces the user's time investment while ensuring accurate error recovery
3Measurement precision
If targeted clarification questions are generated using concept presence and correctness scores, then error detection precision improves, but system complexity increases due to multiple models
Solution Approach 1:
The system segments the error detection task into two distinct models: concept presence prediction and concept correctness prediction. This segmentation allows each model to specialize in one aspect, improving measurement precision while managing complexity through modular design
Solution Approach 2:
The two separate models (presence and correctness) serve universal functions across different speech acts and contexts. This multi-functionality allows the system to handle various types of errors consistently, improving precision without proportionally increasing complexity through reuse of the same model architecture
4Reliability
If the system asks users to repeat entire utterances, then error recovery is achieved, but information loss occurs due to redundant repetition of correct segments
Solution Approach 1:
The system extracts only the erroneous speech act or sentence from the complete utterance for clarification. By taking out only the problematic segment, the system eliminates redundant repetition of correct information while maintaining effective error recovery
Solution Approach 2:
The system applies partial clarification by focusing only on the specific portion of the utterance that contains errors. This partial action approach prevents excessive repetition of correct segments, reducing information loss while achieving complete error recovery
Data Source
AI summary
A system, method and computer-readable storage devices are disclosed for using targeted clarification (TC) questions in dialog systems in a multimodal virtual agent system (MVA) providing access to information about movies, restaurants, and musical events. In contrast with open-domain spoken systems, the MVA application covers a domain with a fixed set of concepts and uses a natural language understanding (NLU) component to mark concepts in automatically recognized speech. Instead of identifying an error segment, localized error detection (LED) identifies which of the concepts are likely to be present and correct using domain knowledge, automatic speech recognition (ASR), and NLU tags and scores. If at least concept is identified to be present but not correct, the TC component uses this information to generate a targeted clarification question. This approach computes probability distributions of concept presence and correctness for each user utterance, which can apply to automatic learning for clarification policies.