Data Augmentation via Latent Variable Pretraining
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
There are limitations in collecting a large amount of speech data for speech recognition services, particularly in non-native speech recognition and call-center recording scenarios, due to limited speaker populations and personal information security concerns.
Innovation Solution
A method for data augmentation using end-to-end learning for attribute disentanglement-generation, which involves pretraining latent variables using target speech and general speech data, training data augmentation parameters, and augmenting target data through a variational autoencoder structure with a generative adversarial network.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If speech data is collected from limited domains (non-native speakers, call-center recordings), then the data collection process is feasible under privacy constraints, but the quantity and diversity of training data is insufficient
Solution Approach 1:
The patent uses variational autoencoders to learn latent representations from available speech data and generates synthetic speech data by sampling from the learned latent space. This copying approach creates artificial speech samples that mimic the statistical properties of real speech without requiring additional real-world data collection, thereby increasing training data quantity while maintaining domain characteristics
Solution Approach 2:
The patent manipulates latent variables in the encoded representation space to generate diverse speech variations. By changing parameters in the latent space (such as adding controlled noise, interpolating between representations, or sampling different regions), the system generates speech data with varied acoustic properties while preserving the underlying linguistic content, thus improving domain versatility
2Reliability
If traditional data augmentation methods are used, then the implementation is simple, but the acoustic characteristics and naturalness of generated speech are insufficient
Solution Approach 1:
The patent introduces latent variables as an intermediary representation between raw speech data and generated speech. The variational autoencoder learns a probabilistic latent space that captures acoustic characteristics, and speech generation occurs through controlled manipulation of these latent variables. This intermediary approach preserves acoustic fidelity while enabling systematic data augmentation, achieving high reliability without excessive complexity
Solution Approach 2:
The patent replaces traditional mechanical signal processing methods (such as additive noise, time-stretching, or pitch-shifting) with a learned generative model. Instead of applying fixed transformations in the time or frequency domain, the system uses neural networks to learn and apply transformations in the latent space, producing more natural-speech with better acoustic characteristics
3Reliability
If domain-specific speech data is collected (e.g., non-native speakers, call-center recordings), then the data matches service requirements, but personal information security concerns arise
Solution Approach 1:
The patent generates synthetic speech data that copies the statistical and acoustic properties of domain-specific speech without using actual personal recordings. By training the variational autoencoder on domain data and then generating new samples from the learned distribution, the system creates artificial speech that matches service requirements (non-native accents, call-center characteristics) while containing no real personal information, thereby eliminating privacy risks
Data Source
AI summary
Disclosed herein is a method for data augmentation, which includes pretraining latent variables using first data corresponding to target speech and second data corresponding to general speech, training data augmentation parameters by receiving the first data and the second data as input, and augmenting target data using the first data and the second data through the pretrained latent variables and the trained parameters.