Multi-Stage Keyword Detection to Cut Speech Recognition Latency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing voice recognition systems, particularly in battery-powered devices, suffer from high power consumption and latency due to always-on functionality, which reduces battery life and increases system processing demands.
Innovation Solution
Implementing a multi-stage keyword detection system that dynamically skips one or more stages based on detection score information, using a first low-complexity model to trigger a second high-complexity model only when necessary, thereby reducing end-to-end latency and power consumption.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If always-on speech recognition function is implemented, then user convenience and responsiveness are improved, but power consumption increases
Solution Approach 1:
The speech recognition system is divided into multiple stages with different complexity levels. The first stage uses a simpler keyword detection model for continuous monitoring, while the second stage uses a more complex model for detailed recognition. This segmentation allows the system to maintain always-on functionality with lower power consumption by using the simpler model continuously and only activating the complex model when necessary.
Solution Approach 2:
The system dynamically adjusts its processing stages based on detection scores from the first keyword detection model. When the detection score exceeds a threshold, the system transitions to the second keyword detection model. This dynamic adjustment allows the system to optimize between power consumption and recognition accuracy in real-time, maintaining responsiveness while reducing overall power usage.
2Measurement precision
If multi-stage keyword detection is performed, then recognition accuracy is improved, but end-to-end latency increases
Solution Approach 1:
The system skips the second keyword detection model when the detection score from the first model exceeds a predetermined threshold. This skipping mechanism allows the system to maintain high recognition accuracy for clear speech patterns while significantly reducing latency by avoiding unnecessary processing steps in those cases.
Solution Approach 2:
The first keyword detection model performs preliminary processing continuously to filter and prioritize audio frames. By preparing the audio data in advance and identifying high-confidence detections beforehand, the system reduces the overall processing time when the second model is activated, thereby reducing end-to-end latency while maintaining accuracy.
3Speed
If always-on keyword detection is performed, then system responsiveness is improved, but processing capacity consumption increases
Solution Approach 1:
The processing workload is segmented between two keyword detection models with different computational requirements. The first model handles continuous monitoring with lower processing capacity demands, while the second model handles only the necessary frames that require enhanced analysis. This segmentation maintains system responsiveness while reducing overall processing capacity consumption.
Solution Approach 2:
The system applies different processing quality levels to different audio frames based on their detection scores. High-confidence frames from the first model are processed with lower computational quality, while only frames that exceed the threshold receive high-quality processing from the second model. This local quality adjustment maintains responsiveness for critical detections while conserving processing capacity.
Data Source
AI summary
Systems and techniques are provided for processing one or more audio samples. For example, a process can include receiving one or more audio samples in a first audio frame and determining, using a first keyword detection model, a first keyword detection score for the first audio frame. One or more audio samples can be received in additional audio frames. Based on the first keyword detection score exceeding a first threshold, the first keyword detection model can be used to determine a keyword detection score for each audio frame of the additional audio frames. The respective keyword detection score for each audio frame of the additional audio frames can be compared to a second threshold that is greater than the first threshold. Based on the respective keyword detection score exceeding the second threshold, using a second keyword detection model to process the first audio frame and the additional audio frames can be skipped.