End-to-End ASR Encoder-Decoder Split for Speech Recognition
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current automatic speech recognition (ASR) systems face challenges in accurately recognizing user speech due to limitations in processing and decoding speech inputs across devices and servers, particularly in integrating end-to-end ASR models effectively.
Innovation Solution
A system and method where a device encodes user speech using an end-to-end ASR model and transmits the output to a server, which decodes it using corresponding decoders associated with specific domains to provide an accurate text string, enabling improved speech recognition and natural language understanding.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If speech recognition is performed entirely on a device using a pre-trained ASR model, then processing speed is improved, but recognition accuracy deteriorates due to limited computational resources and model size
Solution Approach 1:
The ASR model is segmented into encoder and decoder components, with the encoder running on the device and the decoder running on the server. This allows the device to perform initial processing quickly while the server handles the computationally intensive decoding to achieve high accuracy.
Solution Approach 2:
The encoder output from the device acts as an intermediary representation that is transmitted to the server. This intermediate form contains essential speech features that can be efficiently processed by the server's decoder to achieve accurate recognition without requiring the entire model to run on the device.
2Measurement precision
If a complete end-to-end ASR model is deployed on the server, then recognition accuracy is improved, but device-server communication overhead and processing time increase
Solution Approach 1:
The encoder processes the speech input on the device before transmission, performing preliminary feature extraction and encoding. This preliminary action reduces the amount of data that needs to be transmitted to the server and prepares it in a format optimized for decoding, thereby reducing communication overhead and total processing time.
3Measurement precision
If domain-specific decoders are used on the server, then speech recognition accuracy for specific domains is improved, but system complexity increases
Solution Approach 1:
Different domain-specific decoders are deployed on the server to handle different types of speech inputs (e.g., commands, conversations, specialized terminology). Each decoder is optimized for its specific domain, providing high accuracy for that domain while the overall system maintains modularity to manage complexity.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
Provided is a system and method for recognizing a user's speech. A method, performed by a server, of providing a text string for a speech signal input to a device includes: receiving, from the device, an encoder output value derived from an encoder of an end-to-end automatic speech recognition (ASR) model included in the device; identifying a domain corresponding to the received encoder output value; selecting a decoder corresponding to the identified domain from among a plurality of decoders of an end-to-end ASR model included in the server; obtaining a text string from the received encoder output value using the selected decoder; and providing the obtained text string to the device.