Dual-Mode Speech Recognition Model for Low-Latency Accuracy
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing speech recognition technologies face challenges in balancing low-latency streaming recognition accuracy and fully contextual recognition accuracy, with separate models required for each mode, leading to increased computational costs and complexity.
Innovation Solution
A dual-mode machine-learned speech recognition model that can switch between streaming and contextual recognition modes, utilizing a contextual recognition mode to enhance the accuracy and reduce latency of streaming recognition through distillation training.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If separate models are used for streaming and contextual recognition, then each mode can be optimized independently, but device complexity and computational costs increase
Solution Approach 1:
The patent merges streaming and contextual recognition capabilities into a single unified model. The model architecture integrates both recognition modes, allowing the system to switch between them dynamically without requiring separate model instances, thereby reducing device complexity while maintaining optimization for both modes
Solution Approach 2:
The unified model is designed with multi-functionality to perform both streaming and contextual recognition tasks. By incorporating universal components that can operate in different modes, the model achieves versatility without requiring separate specialized models, resolving the contradiction between reliability and device complexity
2Loss of time
If streaming recognition is used, then latency is reduced, but prediction accuracy deteriorates
Solution Approach 1:
The patent implements dynamic mode switching within the unified model, allowing the system to adapt between streaming and contextual recognition modes based on task requirements. This dynamic capability enables the system to achieve low latency when needed while maintaining the option to switch to higher accuracy contextual mode, resolving the contradiction between speed and precision
Solution Approach 2:
The model utilizes parameter changes to transition between streaming and contextual recognition modes. By adjusting internal parameters and configuration settings, the same model can optimize for either low latency or high accuracy depending on the specific recognition task, thereby resolving the trade-off between recognition speed and prediction accuracy
3Reliability
If multiple separate models are deployed, then specific recognition tasks can be optimized, but resource overhead increases
Solution Approach 1:
The patent combines multiple recognition capabilities into a single unified model that shares computational resources. By merging streaming and contextual recognition functions into one model with shared parameters and architecture, the system reduces resource overhead while maintaining optimized performance for different recognition tasks through dynamic mode selection
Data Source
AI summary
Systems and methods of the present disclosure are directed to a computing system. including one or more processors and a machine-learned multi-mode speech recognition model configured to operate in a streaming recognition mode or a contextual recognition mode. The computing system can perform operations including obtaining speech data and a ground truth label and processing the speech data using the contextual recognition mode to obtain contextual prediction data. The operations can include evaluating a difference between the contextual prediction data and the ground truth label and processing the speech data using the streaming recognition mode to obtain streaming prediction data. The operations can include evaluating a difference between the streaming prediction data and the ground truth label and the contextual and streaming prediction data. The operations can include adjusting parameters of the speech recognition model.