Continuous Voice Command Detection for Open-Mic Assistants
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing voice assistants face challenges in continuous conversations due to the inability to distinguish between device-directed commands and background noise, leading to poor user experience and privacy concerns when the microphone remains open, processing unintended audio.
Innovation Solution
A command detection model using autoencoders and joint layers is employed to differentiate between intended speech for the virtual assistant and background noise, ensuring seamless conversation flow and preserving user privacy by accurately identifying device-directed commands.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Duration of action of moving object
If the microphone is kept open for continuous listening, then the virtual assistant can maintain conversation flow, but it processes unintended audio such as background noise or utterances
Solution Approach 1:
The patent introduces an intermediary classification system between the continuous audio input and the virtual assistant processing. This classifier acts as a mediator that filters audio segments, determining whether they contain genuine user commands or background noise before passing them to the assistant, thereby resolving the contradiction between continuous listening and accurate processing
Solution Approach 2:
The patent extracts and isolates the command detection function from the main virtual assistant processing pipeline. By separating the classification of intended speech from background noise into a distinct preprocessing stage, the system can maintain continuous microphone operation while ensuring only relevant audio segments are processed by the assistant
2Productivity
If the virtual assistant processes all audio continuously, then it can respond to user commands, but it raises privacy concerns when processing unintended audio
Solution Approach 1:
The patent implements preliminary action by performing audio classification before full virtual assistant processing. The system pre-evaluates audio segments to identify those containing genuine user commands, ensuring that privacy-sensitive processing is only applied to relevant audio segments while maintaining the ability to respond to commands
3Measurement precision
If a simple audio processing system is used, then device resources are conserved, but it cannot distinguish between device-directed commands and background noise
Solution Approach 1:
The patent applies segmentation by dividing the audio processing system into distinct functional modules: audio input, feature extraction, classification layer, and virtual assistant processing. This modular segmentation enables accurate distinction between commands and background noise while allowing each component to be optimized independently for resource efficiency
Data Source
AI summary
A method includes receiving a user utterance. The method also includes providing the user utterance to a first convolutional recurrent neural network (RNN) classifier and a second convolutional RNN classifier to process the user utterance and provide outputs to a first joint layer. The method also includes providing the user utterance to an automated speech recognition (ASR) model to process the user utterance and provide a text transcript to a text classifier. The method also includes combining the outputs from the first convolutional RNN classifier and the second convolutional RNN classifier using the first joint layer. The method also includes combining outputs from the first joint layer and the text classifier using a second joint layer. The method also includes determining an audio class based on a result from the second joint layer, wherein the audio class indicates whether the user utterance includes speech intended for further processing.