Speech Recognition Model Parallel Processing Speed
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
End-to-end speech recognition systems using auto-regressive models are inefficient due to the sequential recognition of characters, leading to slower speech recognition speeds, especially with long input speech data.
Innovation Solution
A speech recognition method that involves obtaining an acoustic representation of speech, determining character probabilities for each frame vector, predicting the number of characters and frame boundaries, extracting vector representations of character speech, and using a non-auto-regressive decoder to obtain recognition results, thereby reducing the number of model calls.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If an auto-regressive model is used to sequentially recognize characters, then the speech recognition system can achieve joint optimization of acoustic and language models, but the recognition speed becomes slow when input speech data is long
Solution Approach 1:
The patent segments the speech recognition task into two independent parts: (1) a predictor that predicts the number of characters and their frame boundaries from acoustic representations, and (2) a decoder that extracts and recognizes character speech segments based on these predictions. This segmentation allows parallel processing of multiple character recognitions simultaneously, eliminating the sequential bottleneck of auto-regressive models while maintaining joint optimization of acoustic and language models.
Solution Approach 2:
The patent applies preliminary action by having the predictor first determine the number of characters and their temporal boundaries before the actual recognition process. This prediction step enables the decoder to extract and process multiple character speech segments in parallel, preparing the recognition task structure in advance and allowing simultaneous processing rather than sequential character-by-character recognition.
2Loss of information
If an auto-regressive decoder sequentially recognizes characters, then the model can maintain context awareness, but the number of model calls increases significantly for long speech data
Solution Approach 1:
The patent merges multiple character recognition tasks into a single parallel processing step. Instead of making separate model calls for each character sequentially, the system makes one model call to process the entire speech input, extracting multiple character speech segments simultaneously. This combining approach maintains context awareness by processing all characters together while dramatically reducing the total number of model calls and time consumption.
Solution Approach 2:
The predictor performs preliminary action by predicting the number of characters and their frame boundaries before recognition. This preparation enables the system to extract and process multiple character segments in a single parallel model call, eliminating the need for multiple sequential model calls while maintaining comprehensive context awareness across all characters.
3Ease of operation
If sequential character recognition is used, then the system can process speech step-by-step, but the recognition time increases linearly with speech length
Solution Approach 1:
The patent segments the recognition process into independent parallel tasks: the predictor analyzes the entire acoustic representation to determine character counts and boundaries, while the decoder simultaneously extracts and processes all character speech segments. This segmentation transforms the linear sequential process into a parallel process, maintaining ease of operation through structured processing while reducing recognition time from linear to constant complexity.
Solution Approach 2:
The predictor performs preliminary analysis of the entire speech input to determine the number of characters and their temporal distributions before extraction. This preliminary action enables the decoder to efficiently extract and process all character segments in parallel, transforming the step-by-step sequential processing into a batch parallel processing approach that maintains operational simplicity while dramatically reducing total processing time.
Data Source
AI summary
Embodiments of the present application provide a speech recognition method, a speech recognition model, an electronic device and a storage medium. The speech recognition method includes: obtaining an acoustic representation of to-be-recognized speech; determining a character probability corresponding to each frame vector in the acoustic representation, where the character probability is used to indicate a probability of recognizing corresponding character speech based on a current frame vector; predicting, according to the character probability corresponding to each frame vector, the number of characters included in the to-be-recognized speech and a frame boundary of each character to obtain a prediction result; extracting a vector representation of each piece of character speech from the acoustic representation according to the prediction result; obtaining a recognition result of the to-be-recognized speech according to the vector representation of each piece of character speech. This solution can improve the speed of speech recognition.