Device-Directed Speech Detection Without Wakeword
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing speech processing systems struggle to detect and process device-directed speech effectively, especially when a wakeword is not explicitly spoken, leading to inefficiencies and user frustration.
Innovation Solution
The system employs an on-device interrupt architecture that includes an interrupt detector and a device-directed classifier to detect device-directed speech with low latency and high accuracy, allowing for seamless processing and user interaction without requiring a wakeword.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If the system requires a wakeword to be spoken for speech processing, then false activations are reduced, but user interaction becomes less seamless and more frustrating
Solution Approach 1:
The system segments the speech detection function into two parts: a lightweight on-device interrupt detector that continuously monitors for device-directed speech, and a full speech processing pipeline that activates only when the interrupt detector confirms device-directed speech. This segmentation allows the system to maintain high reliability by filtering out non-device-directed speech locally while enabling seamless user interaction through continuous monitoring.
Solution Approach 2:
The on-device interrupt detector acts as an intermediary between the continuous audio input and the full speech processing system. It pre-filters audio streams to identify device-directed speech before triggering the main speech processing pipeline, thereby reducing false activations while maintaining seamless user experience.
2Ease of operation
If the system continuously monitors for device-directed speech without wakeword, then user interaction becomes more seamless, but computational resources and energy consumption increase
Solution Approach 1:
The system performs partial speech processing continuously through the lightweight interrupt detector on the device, and only activates the full speech processing pipeline when device-directed speech is detected. This partial action approach enables seamless user interaction while minimizing energy consumption by avoiding continuous full-scale processing.
Solution Approach 2:
The speech processing workload is segmented into a continuous lightweight monitoring phase on-device and an intermittent full processing phase when needed. This segmentation reduces overall energy consumption while maintaining seamless user experience.
3Measurement precision
If the system processes all detected speech through full speech processing pipeline, then speech recognition accuracy is maximized, but processing latency increases
Solution Approach 1:
The on-device interrupt detector performs preliminary filtering of audio streams to identify device-directed speech before triggering the full speech processing pipeline. This preliminary action reduces processing latency by avoiding full pipeline activation for non-device-directed speech while maintaining accuracy for relevant inputs.
Solution Approach 2:
The system skips the full speech processing pipeline for audio that the interrupt detector determines is not device-directed speech, rushing through the filtering step quickly. This skipping mechanism reduces overall processing latency while maintaining recognition accuracy for device-directed speech.
Data Source
AI summary
A speech interface device is configured to detect an interrupt event and process a voice command without detecting a wakeword. The device includes on-device interrupt architecture configured to detect when device-directed speech is present and send audio data to a remote system for speech processing. This architecture includes an interrupt detector that detects an interrupt event (e.g., device-directed speech) with low latency, enabling the device to quickly lower a volume of output audio and/or perform other actions in response to a potential voice command. In addition, the architecture includes a device directed classifier that processes an entire utterance and corresponding semantic information and detects device-directed speech with high accuracy. Using the device directed classifier, the device may reject the interrupt event and increase a volume of the output audio or may accept the interrupt event, causing the output audio to end and performing speech processing on the audio data.