Voice Command Auto-Correction Using Audio-Text Attention
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing voice assistants lack the ability to accurately understand and correct errors in ongoing speech commands, leading to inefficient processing, lack of context focus, and incorrect responses due to missing context and sentence correction intent, resulting in low performance and user frustration.
Innovation Solution
A system and method utilizing multi-modal fusion and self-attention on extracted text and acoustic features to determine cue words and correction elements, which are fed into an on-the-fly decoder for correcting errors in real-time, enabling accurate and context-aware speech command auto-correction.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If traditional speech recognition systems are used, then the system is simple and fast, but it cannot detect or correct errors in ongoing speech commands
Solution Approach 1:
The patent implements nested processing by embedding multiple levels of analysis within the speech recognition system. The acoustic model is nested within the speech recognition pipeline, which is further nested within the correction system. The system nests disfluency detection, error identification, and correction generation within a unified architecture, allowing complex error correction capabilities while maintaining a structured, manageable system organization.
Solution Approach 2:
The patent introduces intermediary components including a disfluency detector that acts as a mediator between raw speech input and the recognition engine, and a correction generator that mediates between identified errors and the final corrected output. These intermediary elements enable error correction without requiring complete system redesign, bridging the gap between simple recognition and intelligent correction.
2Measurement precision
If context-based spelling correction is applied, then some errors are corrected, but the system lacks focus on correction context and processes all queries through the same path
Solution Approach 1:
The patent applies local quality by implementing specialized processing paths for different query types. The system identifies correction contexts and applies focused correction algorithms specifically to those regions, rather than uniformly processing all queries. This allows precise error correction in disfluency-prone areas while maintaining efficient standard processing for clear commands, improving both accuracy and efficiency.
Solution Approach 2:
The patent segments the speech processing pipeline into distinct functional modules: disfluency detection, error identification, correction generation, and output synthesis. This segmentation allows each module to specialize in its specific task, with the disfluency detector focusing on identifying correction contexts and the correction generator focusing on producing accurate corrections, thereby improving overall system efficiency and precision.
3Reliability
If the system waits for complete commands before processing, then accuracy can be verified, but user interaction becomes slower and less natural
Solution Approach 1:
The patent implements preliminary action by performing disfluency detection and error identification during the ongoing speech recognition process, before the complete command is finalized. The system proactively identifies potential errors and generates corrections in real-time, allowing verification and correction to occur preliminarily rather than requiring waiting for complete command intake, thus reducing response time while maintaining accuracy.
Solution Approach 2:
The patent incorporates feedback mechanisms where the disfluency detector continuously monitors the recognition output and provides feedback about detected errors. This real-time feedback allows the system to adjust and generate corrections during the speech stream, enabling dynamic error correction without waiting for the complete command, thereby maintaining both accuracy and responsiveness.
Data Source
AI summary
The system includes a voice assistant receiving a voice command as input from user. The speech to text convertor converts the voice command into a text. A feature extractor extracts acoustic features from raw waveform of voice command and textual features from converted text for determining nearby context tokens. A multi modal unified attention sequence tagger determines a connection between the audio and the text based on an individual contextual embedding and a fused contextual embedding at context tokens level. It further tags replacement, cue and correction words sequentially based on determined connection between the audio and the text. An on-the-fly decoder decodes revised text on-the-fly based on tagged replacement, cue and correction words, to display the decoded revised text on user interface and sends the decoded revised text to NLP to generate a response corresponding to the input speech.