Multi-Encoder ASR System with Dynamic Selection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional end-to-end automatic speech recognition (ASR) systems face reduced accuracy when handling mixed close-talk and far-talk data or when input devices change, as they are not easily adaptable to switch components at runtime, leading to lower recognition performance compared to matched condition training.
Innovation Solution
An end-to-end ASR system with multiple encoders for close-talk and far-talk inputs, incorporating an encoder selection neural network to dynamically switch between encoders and calculate weighted averages, along with a neural or traditional beamforming block to enhance signal processing, allowing for improved accuracy across different input devices.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If a single neural network model is used for end-to-end ASR, then the system achieves unified acoustic, pronunciation and language modeling, but the system cannot easily switch components at runtime to adapt to different input conditions
Solution Approach 1:
The patent divides the single neural network model into multiple specialized encoder networks, each trained for specific input conditions (e.g., far-field, close-talk, noisy environments). This segmentation allows the system to select and switch between appropriate encoders at runtime based on the input condition, thereby improving adaptability while maintaining manageable complexity through modular architecture.
Solution Approach 2:
The patent introduces dynamic component selection by implementing an encoder selection mechanism that can switch between different encoder networks based on runtime conditions. This dynamic adaptation allows the system to optimize performance for varying input conditions without requiring a complete system redesign, resolving the contradiction between adaptability and structural complexity.
2Adaptability or versatility
If the ASR system is trained with mixed close-talk data and far-talk data, then it achieves similar accuracy for both conditions, but the accuracy level is lower than matched condition training
Solution Approach 1:
The patent applies local quality by training each encoder network with specialized data corresponding to its intended input condition (e.g., far-field encoder trained with far-talk data, close-talk encoder trained with close-talk data). This specialized training allows each encoder to achieve high accuracy for its specific condition, avoiding the performance dilution that occurs when a single model is trained on mixed data.
Solution Approach 2:
The patent creates multiple encoder network copies, each specialized for different input conditions. Instead of one general-purpose encoder, the system maintains separate encoder instances that are copied and adapted for specific scenarios, allowing each to achieve matched-condition training accuracy for its designated input type while the system as a whole handles multiple conditions.
3Measurement precision
If multiple encoders are implemented for different input devices, then the system achieves matched condition accuracy on both close-talk and far-talk inputs, but the system complexity increases
Solution Approach 1:
The patent implements universality by designing multiple encoders that share a common architecture and training methodology, allowing them to be used across different input conditions. Each encoder is multi-functional in that it can process its specialized input type while contributing to a unified ASR system, reducing the effective complexity compared to completely separate systems.
Solution Approach 2:
The patent introduces an encoder selection mechanism that acts as an intermediary between the multiple specialized encoders and the rest of the ASR system. This mediator selects and routes appropriate encoder outputs based on input conditions, managing the complexity of having multiple encoders while maintaining high accuracy for each input type.
Data Source
AI summary
An end-to-end automatic speech recognition (ASR) system includes: a first encoder configured for close-talk input captured by a close-talk input mechanism; a second encoder configured for far-talk input captured by a far-talk input mechanism; and an encoder selection layer configured to select at least one of the first and second encoders for use in producing ASR output. The selection is made based on at least one of short-time Fourier transform (STFT), Mel-frequency Cepstral Coefficient (MFCC) and filter bank derived from at least one of the close-talk input and the far-talk input. If signals from both the close-talk input mechanism and the far-talk input mechanism are present for a speech segment, the encoder selection layer dynamically selects between the close-talk encoder and the far-talk encoder to select the encoder that better recognizes the speech segment. An encoder-decoder model is used to produce the ASR output.