Sentiment-Aware Voice Interface Frustration Handling
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Spoken language understanding systems face friction in user interactions due to errors in automatic speech recognition and natural language understanding, leading to misinterpretation of user inputs and frustration, especially when audio quality is poor or user sentiment is negative.
Innovation Solution
The system detects user frustration and poor audio quality, prompting the user to confirm actions or rephrase inputs, and handles negative sentiment by generating appropriate dialogues to reduce user frustration and improve interaction quality.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If speech recognition and natural language understanding processing techniques are used to interpret user input, then the system can perform tasks based on spoken commands, but errors in recognition and understanding occur leading to misinterpretation of user inputs
Solution Approach 1:
The system implements a confirmation mechanism where it detects user frustration through sentiment analysis and audio quality assessment, then prompts the user to confirm actions or rephrase inputs. This feedback loop resolves misinterpretations by verifying user intent before execution, directly addressing the reliability issue while maintaining ease of voice-based operation.
Solution Approach 2:
The system performs preliminary checks on audio quality and user sentiment before executing commands. By assessing these factors in advance, the system can identify potential misinterpretations early and take corrective action (prompting confirmation) before carrying out actions, thus preventing errors rather than correcting them after the fact.
2Reliability
If the system continuously monitors audio quality and user sentiment, then interaction quality improves, but system complexity increases
Solution Approach 1:
The system divides the monitoring function into separate modular components: one for audio quality assessment and another for sentiment analysis. This segmentation allows each component to be independently optimized and managed, reducing overall system complexity while maintaining comprehensive monitoring capabilities for improved interaction quality.
Solution Approach 2:
The system introduces an intermediary layer that sits between the speech recognition module and the command execution module. This intermediary assesses audio quality and sentiment, and decides whether to proceed with execution or prompt for confirmation. This mediator simplifies the architecture by centralizing the complexity in a single decision-making layer rather than distributing it throughout the system.
3Reliability
If the system prompts users to confirm actions or rephrase inputs, then misinterpretation errors decrease, but interaction time increases
Solution Approach 1:
The system applies preliminary anti-action by detecting user frustration and poor audio quality before executing commands, and preemptively prompts for confirmation. This prevents the more time-consuming cycle of executing wrong actions and then correcting them, thereby reducing overall interaction time while maintaining high accuracy.
Solution Approach 2:
The system selectively prompts for confirmation only in specific conditions (detected frustration or poor audio quality) rather than for every command. This partial application of the confirmation mechanism balances reliability improvement with time efficiency, avoiding excessive interactions that would unnecessarily extend dialogue time.
Data Source
AI summary
Described herein is a system for responding to a frustrated user with a response determined based on spoken language understanding (SLU) processing of a user input. The system detects user frustration and responds to a repeated user input by confirming an action to be performed or presenting an alternative action, instead of performing the action responsive to the user input. The system also detects poor audio quality of the captured user input, and responds by requesting the user to repeat the user input. The system processes sentiment data and signal quality data to respond to user inputs.