Streaming Speech Recognition with Historical Feature Abstraction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing speech recognition systems face challenges in achieving high accuracy and efficient real-time processing of long audio data with large vocabularies, particularly in scenarios requiring real-time presentation of recognition results.
Innovation Solution
The proposed method involves encoding current audio features with historical feature abstraction information using a Conformer-based Streaming Multi-Layer Truncated Attention (SMLTA) model, which integrates a causal Conformer encoder and transformer decoder to improve recognition accuracy and efficiency by leveraging historical feature abstraction and synchronous decoding.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If end-to-end speech recognition is used with large vocabulary (more than 1000 frames), then recognition accuracy is improved, but recognition efficiency deteriorates
Solution Approach 1:
The patent divides the long audio feature sequence into multiple feature fragments (e.g., first feature fragment, second feature fragment, etc.). Each fragment is processed separately by the recognition model, allowing parallel processing and reducing the computational burden of processing the entire sequence at once. This segmentation enables real-time recognition while maintaining accuracy for large vocabulary speech recognition.
Solution Approach 2:
The patent performs feature abstraction on historical feature fragments before processing current fragments. By pre-processing and compressing historical features into abstract representations, the system reduces the amount of data that needs to be processed in real-time, improving recognition efficiency while preserving important contextual information for accuracy.
2Measurement precision
If all historical feature fragments are processed in detail, then recognition accuracy is improved, but computational resource consumption increases
Solution Approach 1:
The patent extracts only the essential information from historical feature fragments through feature abstraction, creating compressed representations that capture key contextual patterns. This extraction process removes redundant detailed information while preserving the most important features, reducing computational resource consumption while maintaining recognition accuracy.
Solution Approach 2:
By performing feature abstraction in advance on historical fragments, the system prepares compressed feature representations that can be quickly integrated with current fragments during real-time processing. This preliminary processing reduces the computational load during actual recognition while preserving accuracy-critical information.
3Productivity
If real-time processing of feature fragments is implemented, then recognition efficiency is improved, but processing completeness may deteriorate
Solution Approach 1:
The patent implements continuous processing where current feature fragments are processed in real-time while being combined with abstracted historical fragments. This continuous approach ensures that processing is never interrupted, maintaining real-time efficiency while the integration of historical context ensures processing completeness is preserved through the accumulation of fragmented information over time.
Solution Approach 2:
Historical feature fragments are pre-processed into abstract representations that capture essential contextual information. This preliminary action ensures that when current fragments are processed in real-time, they are supplemented with relevant historical context, maintaining processing completeness without compromising real-time efficiency.
Data Source
Figure 1
Figure 2~3
Figure 4
AI summary
The present disclosure provides speech recognition and codec methods and apparatuses, an electronic device and a storage medium, and relates to the field of artificial intelligence such as intelligent speech, deep learning and natural language processing. The speech recognition method may include: acquiring an audio feature of to-be-recognized speech; encoding the audio feature to obtain an encoding feature; truncating the encoding feature to obtain continuous N feature fragments, N being a positive integer greater than one; and acquiring, for any one of the feature segments, corresponding historical feature abstraction information, encoding the feature segment in combination with the historical feature abstraction information, and decoding an encoding result to obtain a recognition result corresponding to the feature segment, wherein the historical feature abstraction information is information obtained by feature abstraction of recognized historical feature fragments. By use of the solution of the present disclosure, the accuracy of the recognition result and the efficiency of recognition can be improved.