Hotword Detection Threshold Tuning With User Feedback
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing automated assistants face issues with false negative and false positive determinations in hotword detection, leading to prolonged interactions, resource wastage, and privacy concerns due to unnecessary activation of functions.
Innovation Solution
Implementing a system that automatically adjusts thresholds for initiating automated assistant functions based on user feedback and environmental conditions, using local and remote machine learning model updates to improve performance and reduce false positives and negatives.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If a fixed threshold is used for hotword detection, then the system operation is simple, but false positives and false negatives occur frequently
Solution Approach 1:
The patent implements dynamic threshold adjustment by transitioning from a fixed threshold to a movable threshold that adapts based on environmental noise levels. The threshold is continuously updated according to the detected background noise, allowing the system to maintain high detection accuracy across varying acoustic conditions without requiring complex manual configuration.
Solution Approach 2:
The system performs self-adjustment of the detection threshold by automatically monitoring environmental noise and adapting the threshold accordingly. This self-service mechanism eliminates the need for external intervention or complex configuration processes, while still achieving improved detection precision through adaptive thresholding.
2Reliability
If the automated assistant function is activated frequently to ensure no valid input is missed, then detection reliability improves, but resource consumption increases
Solution Approach 1:
The patent replaces the mechanical approach of continuously activating the automated assistant with an acoustic field-based pre-filtering mechanism. The voice activity detection component analyzes acoustic characteristics before triggering full assistant activation, substituting heavy computational processing with lighter acoustic analysis to reduce energy consumption while maintaining reliable detection.
Solution Approach 2:
The system applies partial action by implementing a two-stage detection process: first using voice activity detection for preliminary filtering, then using hotword detection only when necessary. This avoids excessive activation of the full automated assistant function while ensuring no valid inputs are missed, thereby reducing energy consumption without compromising reliability.
3Reliability
If all audio data is processed to ensure privacy, then user privacy is protected, but computational resources are wasted
Solution Approach 1:
The patent segments the audio processing pipeline into distinct stages: voice activity detection, hotword detection, and full audio processing. This segmentation allows the system to apply different processing intensities to different portions of audio data, processing only relevant segments with full analysis while discarding irrelevant portions, thereby protecting privacy without wasting computational resources on unnecessary data.
Solution Approach 2:
The system applies local quality by using different processing strategies for different parts of the audio stream. High-privacy-protection processing is applied only to segments that contain potential hotwords or valid commands, while other segments receive minimal or no processing. This localized approach maintains privacy protection reliability where needed while improving overall processing efficiency.
Data Source
Figure 1A
Figure 1B
Figure 2
AI summary
Techniques are described herein for improving performance of machine learning model(s) and thresholds utilized in determining whether automated assistant function(s) are to be initiated. A method includes: receiving, via one or more microphones of a client device, audio data that captures a spoken utterance of a user; processing the audio data using a machine learning model to generate a predicted output that indicates a probability of one or more hotwords being present in the audio data; determining that the predicted output satisfies a secondary threshold that is less indicative of the one or more hotwords being present in the audio data than is a primary threshold; in response to determining that the predicted output satisfies the secondary threshold, prompting the user to indicate whether or not the spoken utterance includes a hotword; receiving, from the user, a response to the prompting; and adjusting the primary threshold based on the response.