Hotword Detection Threshold Adaptation for False Trigger Control
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing automated assistants face issues with false negatives and false positives in hotword detection, leading to prolonged interactions, resource wastage, and privacy concerns due to unnecessary processing of audio data.
Innovation Solution
Implementing a system that automatically adjusts thresholds for hotword detection models based on local decision corrections and generates gradients for federated learning to update machine learning models, reducing false negatives and positives.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If automated assistants continuously process all audio data to ensure no hotword is missed, then detection reliability improves, but resource consumption and privacy risks increase
Solution Approach 1:
The system segments the audio processing pipeline into multiple stages: a lightweight hotword detection model that continuously monitors audio with low computational cost, and a more comprehensive speech recognition model that processes only segments containing detected hotwords. This segmentation allows the system to maintain high detection reliability while minimizing resource consumption by activating intensive processing only when necessary.
2Reliability
If automated assistants continuously process all audio data to ensure no hotword is missed, then detection reliability improves, but privacy risks increase
Solution Approach 1:
The system segments the audio processing pipeline into multiple stages: a lightweight hotword detection model that continuously monitors audio with low computational cost, and a more comprehensive speech recognition model that processes only segments containing detected hotwords. This segmentation allows the system to maintain high detection reliability while minimizing resource consumption by activating intensive processing only when necessary.
3Measurement precision
If automated assistants use a high threshold for hotword detection, then false positives decrease, but false negatives increase leading to prolonged interactions
Solution Approach 1:
The system dynamically adjusts the detection threshold based on contextual factors such as speaker identification confidence, audio environment characteristics, and conversation state. Rather than using a fixed high threshold, the threshold adapts in real-time to balance false positives and false negatives, reducing unnecessary interaction prolongation while maintaining detection accuracy.
Solution Approach 2:
The system incorporates feedback mechanisms where detection results and interaction outcomes are used to refine future detection decisions. When false negatives occur, the system learns to adjust its detection sensitivity, and when false positives occur, it adjusts accordingly. This feedback loop optimizes the balance between detection accuracy and interaction efficiency over time.
4Reliability
If automated assistants use a low threshold for hotword detection, then false negatives decrease, but false positives increase causing resource wastage
Solution Approach 1:
The system segments the audio processing pipeline into multiple stages: a lightweight hotword detection model that continuously monitors audio with low computational cost, and a more comprehensive speech recognition model that processes only segments containing detected hotwords. This segmentation allows the system to maintain high detection reliability while minimizing resource consumption by activating intensive processing only when necessary.
Solution Approach 2:
The system dynamically adjusts the detection threshold based on contextual factors such as speaker identification confidence, audio environment characteristics, and conversation state. Rather than using a fixed low threshold, the threshold adapts in real-time to balance false positives and false negatives, reducing unnecessary interaction prolongation while maintaining detection accuracy.
Data Source
Figure 1A
Figure 1B
Figure 2
AI summary
Techniques are described herein for improving performance of machine learning model(s) and thresholds utilized in determining whether automated assistant function(s) are to be initiated. A method includes: receiving, via one or more microphones of a client device, audio data that captures a spoken utterance of a user; processing the audio data using a machine learning model to generate a predicted output that indicates a probability of one or more hotwords being present in the audio data; determining that the predicted output satisfies a secondary threshold that is less indicative of the one or more hotwords being present in the audio data than is a primary threshold; in response to determining that the predicted output satisfies the secondary threshold, prompting the user to indicate whether or not the spoken utterance includes a hotword; receiving, from the user, a response to the prompting; and adjusting the primary threshold based on the response.