Always-On DSP Follow-On Query Detection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing speech-enabled environments require users to repeatedly speak hotwords to initiate follow-on queries, which is tiresome and unnatural, and also consume significant computational resources and battery power by keeping the device in a high power state for an extended period.
Innovation Solution
Implementing an always-on digital signal processor (DSP) that operates in a follow-on query detection mode after an initial query is responded to, allowing it to detect voice activity and verify the speaker without requiring hotwords, thus reducing power consumption and enhancing user experience.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If the device keeps the processor in a high power state for an extended period to detect follow-on queries, then the user experience is improved by enabling continuous conversation, but the power consumption increases significantly
Solution Approach 1:
The system segments the query detection process into two phases: an initial hotword detection phase using a lightweight model, and a follow-on query detection phase using a more sophisticated model. This segmentation allows the system to use computational resources only when necessary, reducing overall power consumption while maintaining good user experience for follow-on queries.
Solution Approach 2:
The system performs preliminary hotword detection before full query processing. By using a lightweight hotword detection model first, the system can filter out non-queries early, then only activate the more power-intensive follow-on query detection model when needed, thus reducing average power consumption while maintaining conversational capability.
2Use of energy by moving object
If the device requires users to speak hotwords for each follow-on query, then the power consumption is reduced by keeping the processor in a low power state, but the ease of operation deteriorates due to repetitive input requirements
Solution Approach 1:
The system uses feedback from the initial query processing to enable follow-on query detection without requiring repeated hotwords. After the initial query is processed, the system enters a state where it can detect follow-on queries directly, providing feedback that improves ease of operation for subsequent interactions while managing power consumption through selective activation of detection modes.
3Use of energy by moving object
If the device maintains a low power state after initial query response, then the power consumption is minimized, but the ability to detect follow-on queries without hotwords is lost
Solution Approach 1:
The system dynamically adjusts its detection capability based on the interaction state. After the initial query is processed, the system transitions to a follow-on query detection mode that is more adaptable to continuous conversation. This dynamic state change allows the system to maintain low power consumption during idle periods while providing enhanced follow-on query detection capability when activated.
Data Source
AI summary
A method includes instructing an always-on first processor to operate in a follow-on query detection mode, and while the always-on first processor operates in the follow-on query detection mode: receiving follow-on audio data captured by the assistant-enabled device; determining, using a voice activity detection (VAD) model executing on the always-on first processor, whether or not the VAD model detects voice activity in the follow-on audio data; performing, using a speaker identification (SID) model executing on the always-on first processor, speaker verification on the follow-on audio data to determine whether the follow-on audio data includes an utterance spoken by the same user. The method also includes initiating a wake-up process on a second processor to determine whether the utterance includes a follow-on query.