Latency-Constrained Acoustic Model Training
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Acoustic modeling systems face challenges in reducing latency in speech recognition, particularly in live-streaming applications, where long and variable delays in classifying acoustic frames hinder real-time processing and user experience.
Innovation Solution
Implementing latency constraints during the training of recurrent neural networks by penalizing or disallowing output delays beyond a maximum threshold, ensuring phone labels are assigned within a defined time frame, thereby restricting the search paths in the forward-backward algorithm and reducing computational latency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If acoustic models allow variable delays in classifying acoustic frames to improve classification accuracy, then recognition performance is improved, but latency in real-time speech recognition increases
Solution Approach 1:
The system dynamically adjusts the latency constraint parameter during training and operation, allowing the model to learn optimal timing behaviors for different acoustic contexts while maintaining bounded latency through the constraint mechanism
Solution Approach 2:
The invention changes the parameter space by introducing a latency constraint parameter that bounds the delay between acoustic frame input and classification output, transforming the unbounded variable delay into a controlled parameter with defined limits
2Measurement precision
If acoustic models use extended search paths in the forward-backward algorithm to improve phone label accuracy, then recognition performance is improved, but computational time and latency increase
Solution Approach 1:
The system performs preliminary action by pre-defining latency constraints during the training phase, which guides the forward-backward algorithm to focus search within bounded time windows, reducing computational exploration of excessively long paths while maintaining accuracy
Solution Approach 2:
The invention applies partial action by restricting the search paths to only those that satisfy the latency constraint, eliminating excessive computation on paths that would produce acceptable but overly delayed results
Data Source
AI summary
Methods, systems, and apparatus, including computer programs encoded on computer storage media for acoustic modeling of audio data. One method includes receiving audio data representing a portion of an utterance, providing the audio data to a trained recurrent neural network that has been trained to indicate the occurrence of a phone at any of multiple time frames within a maximum delay of receiving audio data corresponding to the phone, receiving, within the predetermined maximum delay of providing the audio data to the trained recurrent neural network, output of the trained neural network indicating a phone corresponding to the provided audio data using output of the trained neural network to determine a transcription for the utterance, and providing the transcription for the utterance.