Vehicle Voice Assistant Error Detection With Behavioral Feedback
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing voice assistant systems face issues with incorrect classification of spoken utterances as system-directed or non-system directed, leading to user frustration due to false accepts and false rejects, particularly in environments where manual triggers are impractical.
Innovation Solution
A vehicle system that uses microphones and sensors to detect acoustic and user behavior data, classifying utterances accurately by analyzing subsequent user actions and adjusting classifications based on observed behavior to mitigate errors.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If voice triggers are used to eliminate manual actions, then ease of operation is improved, but measurement precision of utterance direction deteriorates due to false accepts and false rejects
Solution Approach 1:
The system monitors user behavior data (eye tracking, gestures, head position) after an utterance is classified to determine if the classification was correct. This feedback loop allows the system to detect false accepts and false rejects, then apply mitigating adjustments to improve future classification accuracy while maintaining hands-free operation.
Solution Approach 2:
The patent introduces intermediate sensors (eye trackers, gesture detectors, head position sensors) that mediate between the voice utterance and the classification decision. These intermediaries provide additional context about user intent, helping to resolve ambiguities in voice-only classification and reduce false accepts and false rejects.
2Measurement precision
If the system processes all acoustic input, then measurement precision is improved, but use of energy increases and device complexity worsens
Solution Approach 1:
The system performs partial processing by initially classifying only utterances that meet certain criteria (e.g., contain wake words, come from expected directions) as system-directed. Full processing is applied only to these candidate utterances, while other acoustic input receives minimal or no processing, reducing energy consumption while maintaining detection accuracy for relevant utterances.
Solution Approach 2:
The processing pipeline is segmented into multiple stages: initial acoustic event detection, wake word detection, utterance classification, and behavioral verification. Each stage processes only the necessary subset of acoustic input, avoiding full-system processing of all acoustic events and thereby reducing energy usage while preserving detection precision.
3Measurement precision
If wake-up words are required to activate the system, then measurement precision is improved, but ease of operation deteriorates due to unnecessary user effort
Solution Approach 1:
The system dynamically adjusts its activation requirements based on context. In certain situations (e.g., when the vehicle is already in a specific state, or when the user has previously indicated they want hands-free operation), the system can activate without requiring a wake-up word, reducing unnecessary user effort while maintaining accurate intent recognition through contextual analysis and behavioral verification.
Data Source
AI summary
A vehicle system for classifying spoken utterance within a vehicle cabin as one of system-directed and non-system directed, the system may include at least one microphone configured to detect at least one acoustic utterance from at least one occupant of a vehicle, at least one sensor to detect user behavior data indicative of user behavior, and a processor programmed to: receive the acoustic utterance, classify the acoustic utterance as one of a system-directed utterance and a non-system directed utterance, determine whether the acoustic utterance was properly classified based on user behavior observed via data received from the sensor after the classification, and apply a mitigating adjustment to classifications of subsequent acoustic utterances based on an improper classification.