Streaming End-to-End Speech Recognition With Block Activation Prediction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing streaming end-to-end speech recognition systems face challenges in robustness to noise and accuracy due to mismatches between training and testing, particularly in determining activation points for real-time speech recognition.
Innovation Solution
The method involves dividing encoded frames into blocks and predicting the number of activation points within each block, determining the position of these points based on prediction results, and using a decoder to output recognition results at these positions, rather than relying on threshold comparisons.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If threshold comparison method is used to determine activation points, then the system can operate in streaming mode, but the accuracy of activation point positioning deteriorates
Solution Approach 1:
The patent applies preliminary action by pre-training a prediction model on complete sentence data to learn activation point patterns. During streaming recognition, this pre-trained model predicts the number of activation points in advance, allowing the system to maintain both streaming capability and high positioning accuracy without relying on simple threshold comparison.
2Measurement precision
If Attention model is trained on complete sentences, then training accuracy is improved, but testing robustness to noise deteriorates due to training-testing mismatch
Solution Approach 1:
The patent applies segmentation by dividing the encoded frames into blocks and predicting the number of activation points within each block separately. This block-based approach reduces the training-testing mismatch because the model learns to handle localized patterns rather than requiring complete sentence context, thereby improving noise robustness while maintaining training accuracy.
3Measurement precision
If future voice frames are considered for activation point determination, then positioning accuracy may be improved, but real-time processing capability deteriorates
Solution Approach 1:
The patent applies self-service by designing a prediction model that determines activation points based solely on current and past information without requiring future voice frames. The model predicts the number of activation points in each block using only available historical data, enabling accurate real-time processing without looking ahead to future frames.
Data Source
AI summary
A method, an apparatus, and an electronic device for streaming end-to-end speech recognition are described. The method includes: extracting and encoding speech acoustic features of a received voice stream in units of frames; performing block processing, and predicting a number of activation points included in a same block that need to be encoded and outputted; determining position(s) of activation point(s) that need(s) to be decoded and outputted according to a prediction result, to a decoder to perform decoding at the position(s) of the activation point(s) and output a recognition result. Through the embodiments of the present disclosure, the robustness of a streaming end-to-end speech recognition system to noise can be improved, thereby improving the performance and the accuracy of the system.