ASR Model Fusion via Text Alignment and Re-scoring
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current speech recognition systems struggle to efficiently fuse multiple automatic speech recognition (ASR) models with different structures or parameters, leading to suboptimal recognition rates due to limitations in existing fusion methods.
Innovation Solution
A text output system comprising an ASR model group, a text alignment model, and a re-scoring model, where the ASR model group converts input audio data into multiple texts, the text alignment model aligns these texts to equal lengths, and the re-scoring model selects the highest scored words/terms at each alignment position to produce a final output text, effectively improving recognition accuracy by leveraging complementarity between models.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If multiple ASR models are used to improve recognition accuracy, then recognition rate improves, but the fusion of models becomes complex and difficult to implement efficiently
Solution Approach 1:
The patent segments the model fusion process into distinct functional modules: an ASR model group that generates multiple candidate texts, a text alignment model that aligns these texts, and a re-scoring model that selects the best words/terms. This segmentation transforms the complex fusion problem into manageable sequential steps, improving implementation efficiency while maintaining high recognition accuracy.
Solution Approach 2:
The text alignment model serves as an intermediary component between the ASR model group and the re-scoring model. It aligns the candidate texts generated by different ASR models to equal lengths, creating a standardized format that enables efficient processing by the re-scoring model. This intermediary step simplifies the overall fusion process by providing a structured intermediate representation.
2Ease of manufacture
If traditional fusion methods are used, then implementation is simpler, but word error rate remains high and recognition accuracy is suboptimal
Solution Approach 1:
The patent replaces traditional mechanical voting or simple averaging fusion methods with a neural network-based re-scoring model. This re-scoring model processes the aligned candidate texts and assigns scores to select the most accurate words/terms, achieving superior word error rate reduction compared to simple mechanical fusion methods, while maintaining reasonable implementation complexity.
3Reliability
If different acoustic model structures are trained to achieve complementarity, then recognition rate improves, but the system complexity and training difficulty increase
Solution Approach 1:
The patent designs a universal text alignment model and re-scoring model that can process outputs from various different ASR model structures. These downstream components serve multiple functions: they can handle aligned texts from any number of ASR models with different architectures, making the overall system adaptable to different model configurations without requiring separate fusion mechanisms for each case.
Data Source
AI summary
Embodiments of the present application provide a text output method and system, a storage medium, and an electronic device. The system includes at least an automatic speech recognition ASR model group, a text alignment model, and a re-scoring model that are sequentially connected, where the ASR model group includes a plurality of ASR models each configured to convert input audio data into respective first texts; the text alignment model is configured to perform alignment for a plurality of first texts, to obtain a plurality of target texts, where lengths of the plurality of target texts are all equal; and the re-scoring model is configured to score words/terms at each alignment position of the plurality of target texts, to obtain a word/term with the highest score at each alignment position, as a target word/term, and determine the target word/terms, as an output text, by the respective alignment positions.