Acoustic Model Noise Representation via Auto-Encoder Multi-Task Learning
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing speech recognition systems face performance deterioration due to noise disparities between training and real environments, necessitating a robust acoustic model generation method that can effectively handle various noises.
Innovation Solution
The method involves generating an acoustic model using a multi-task learning approach, which combines speech and noise data to create a noise representation through an auto-encoder-based noise model, allowing the model to perform phoneme classification and noise representation simultaneously, thereby enhancing robustness against ambient noise.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If traditional speech recognition models are trained in controlled environments, then the model training process is simple and data requirements are manageable, but the model performance deteriorates when deployed in noisy real-world environments
Solution Approach 1:
The patent applies preliminary action by pre-training auto-encoder networks on noise data before the main acoustic model training. The auto-encoders are first trained to denoise speech signals, creating a noise representation capability that is then integrated into the acoustic model. This preliminary preparation enables the model to handle noisy environments without requiring complex real-time noise processing during inference.
Solution Approach 2:
The patent uses an intermediary approach by introducing noise representations as an additional input dimension to the acoustic model. Instead of directly processing raw noisy audio signals, the model receives both the original signal and a derived noise representation from the auto-encoder. This intermediary noise representation serves as a mediator that enables the model to separate speech from noise more effectively, improving recognition accuracy in noisy conditions.
2Adaptability or versatility
If the acoustic model is trained to handle multiple noise types simultaneously, then the model robustness improves, but the training data requirements and computational complexity increase
Solution Approach 1:
The patent applies universality by training the auto-encoder network to perform multiple functions: denoising various types of ambient noise, generating noise representations, and working with diverse speech scenarios. The same auto-encoder architecture handles different noise conditions (traffic, restaurant, outdoor noise, etc.) through a unified approach, eliminating the need for separate specialized models for each noise type and reducing overall training data requirements.
Solution Approach 2:
The patent utilizes parameter changes by adjusting the auto-encoder network parameters during training to adapt to different noise conditions. The model dynamically learns optimal representations for various noise types through parameter optimization, allowing it to generalize across different acoustic environments. This parameter adaptation enables the model to handle diverse noise scenarios without requiring proportional increases in training data for each noise type.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
Described are an apparatus and method for generating to generate an acoustic model. The apparatus and method include a processor a processor configured to calculate a noise representation that represents noise data by using a noise model, and generate the acoustic model through training using training noisy speech data, which comprises speech data and the noise data, a string of phonemes corresponding to the speech data, and the noise representation.