Assistant Utterance Fusion for Correcting Misheard Voice Queries
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing speech recognition systems often misinterpret repeated user queries due to accent, dialect, and background noise, failing to utilize the context of multiple utterances and lacking acoustic similarity determination, leading to inaccurate transcription and user frustration.
Innovation Solution
Jointly process multiple spoken utterances within a threshold duration to determine acoustic similarity, using machine learning models to average or weight probabilities, and retroactively correct previous misinterpretations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If conventional speech recognition systems process each utterance independently, then the system structure remains simple, but speech recognition accuracy deteriorates due to inability to utilize contextual information from repeated queries
Solution Approach 1:
The patent merges multiple independent utterance processing operations into a unified joint processing framework. The system combines acoustic features, spectral mappings, and machine learning model outputs from multiple utterances to determine a single accurate interpretation, rather than treating each utterance as an independent task.
Solution Approach 2:
The system implements feedback by using the output of one utterance processing operation to inform subsequent processing. When a first utterance is misinterpreted, the system uses the user's corrective second utterance as feedback to retroactively correct the previous misinterpretation and improve future recognition accuracy.
2Measurement precision
If the system processes multiple repeated utterances separately, then computational resources are reduced, but transcription accuracy deteriorates due to overlooking semantic relationships between repeated queries
Solution Approach 1:
The system applies partial processing by selectively joint-processing only certain utterances that meet specific criteria (acoustic similarity, temporal proximity, semantic relationship) rather than all utterances. This avoids excessive computational action while still capturing the necessary contextual information for improved accuracy.
Solution Approach 2:
The system performs preliminary filtering and analysis to identify candidate utterances for joint processing before executing the full joint processing algorithm. This includes preliminary acoustic similarity determination and temporal proximity checking to pre-select which utterances warrant combined processing.
3Measurement precision
If the system does not determine acoustic similarity between utterances, then the processing method remains simple, but the ability to identify repeated commands and improve accuracy deteriorates
Solution Approach 1:
The patent replaces traditional mechanical or rule-based speech recognition methods with machine learning-based acoustic similarity determination. The system uses trained models to compare spectral mappings and acoustic features, substituting complex computational mechanisms for simpler but less accurate traditional approaches.
4Reliability
If the system treats repeated queries as independent tasks, then processing speed is maintained, but user experience deteriorates due to failure to recognize and correct misinterpretations
Solution Approach 1:
The system maintains continuity by keeping contextual information from previous utterances active and available for subsequent processing. Rather than resetting the processing state for each new utterance, the system continuously accumulates and utilizes contextual information to maintain accurate interpretation across the interaction sequence.
Data Source
AI summary
Implementations set forth herein relate to an automated assistant that can jointly process spoken utterances that can include at least one corrective utterance and one utterance that may have been misinterpreted by the automated assistant. In this way, the automated assistant does not need to exclusively rely on the corrective utterance to determine a request of a user but, rather, can utilize data capturing any prior spoken utterance to more accurately determine the request. The automated assistant can process utterances jointly after determining that they are acoustically similar and/or received within a threshold duration of time. The joint processing can involve generating a single input, or multiple inputs, to characterize the series of utterances and processing the single input, or multiple inputs, with one or more neural network models and/or machine learning models. The automated assistant can then proactively correct any previous responses to misinterpreted utterances.