Streaming And Non-Streaming Voice Recognition Feature Alignment
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing end-to-end automatic speech recognition (ASR) systems face challenges in achieving high accuracy, with streaming ASR processing voice frames in real time but at lower accuracy, and non-streaming ASR requiring complete speech data for higher accuracy.
Innovation Solution
A voice recognition device and method that combines streaming and non-streaming ASR by using Knowledge Distillation to align and optimize features and outputs between streaming and non-streaming encoders and decoders, enhancing accuracy through a learning unit that performs Knowledge Distillation between these components.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If streaming ASR is used to process voice frames in real time, then processing speed is improved, but recognition accuracy deteriorates
Solution Approach 1:
The patent merges streaming ASR and non-streaming ASR into a unified system where both modes operate simultaneously on the same voice stream. The streaming encoder processes frames in real-time while the non-streaming encoder processes accumulated frames, and their outputs are combined through knowledge distillation to achieve both real-time processing and high accuracy.
Solution Approach 2:
The patent introduces a knowledge distillation module as an intermediary that bridges the streaming and non-streaming ASR systems. This mediator transfers knowledge from the non-streaming encoder to the streaming encoder, allowing the streaming system to benefit from the accuracy of non-streaming processing without sacrificing real-time performance.
2Measurement precision
If non-streaming ASR is used to achieve higher speech recognition accuracy, then recognition accuracy is improved, but processing time increases
Solution Approach 1:
The patent segments the voice stream processing into two parallel paths: one for streaming ASR that processes frames immediately as they arrive, and another for non-streaming ASR that accumulates and processes complete speech segments. Both paths operate simultaneously, allowing the system to achieve high accuracy through non-streaming processing while maintaining real-time responsiveness through streaming processing.
Solution Approach 2:
The system performs preliminary processing using the streaming encoder to generate initial transcriptions and features in real-time. These preliminary results are then refined by the non-streaming encoder which processes the same data with higher accuracy, effectively preparing the data in advance for both immediate output and refined processing.
Data Source
AI summary
A voice recognition device includes an acquisition unit which acquires a frame per unit time of a voice stream, a streaming feature generation unit which generates a first feature from the frame using a streaming encoder, a streaming character generation unit which generates a first character from the first feature using a streaming decoder, a non-streaming feature generation unit which generates a second feature sequence from a first feature sequence obtained by joining the first feature of each of the plurality of frames using a non-streaming encoder, a streaming character generation unit which generates a second character string from the second feature sequence using a plurality of non-streaming decoders, and a learning unit which performs Knowledge Distillation between the streaming encoder and the non-streaming encoder on the basis of the first feature sequence and the second feature sequence.