Voice Modification Autoencoder Preserving Speaker Identity
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current machine learning systems for speech processing, such as autoencoders, struggle to modify voice accents while maintaining the speaker's unique characteristics and prosody, leading to unintelligible outputs for target audiences and loss of individuality in speech conversion.
Innovation Solution
A bottleneck-based autoencoder system with competing discriminators is employed, using a reconstruction error-based loss function and additional adversarial loss functions to train decoders for accent modification, ensuring the output maintains the speaker's speech characteristics and prosody, while being intelligible to a target audience.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If autoencoders are used to modify voice accents, then accent modification capability is improved, but the speaker's unique characteristics and prosody are lost
Solution Approach 1:
The patent segments the speech representation into two independent parts: accent information and speaker identity information. The encoder separates these components, allowing the decoder to reconstruct speech with modified accent while preserving speaker characteristics. This segmentation enables independent control of accent modification without losing speaker identity.
Solution Approach 2:
The patent introduces an intermediary representation space between the input speech and output speech. The encoder transforms the input speech into this intermediate representation, which contains both accent and speaker information in a separated manner. The decoder then uses this intermediate representation to generate output speech with desired accent modification while preserving speaker characteristics.
2Reliability
If accent modification is applied to improve intelligibility, then intelligibility for target audience is improved, but the speaker's identity becomes less recognizable
Solution Approach 1:
The patent applies local quality modification by selectively transforming only the accent portion of the speech representation while leaving the speaker identity portion unchanged. The decoder reconstructs speech with modified accent characteristics in specific local regions of the spectral domain, while preserving the speaker's unique vocal characteristics and identity information.
3Manufacturing precision
If multiple loss functions are used to train the decoder, then the balance between accent modification and speaker characteristic preservation is improved, but the system complexity increases
Solution Approach 1:
The patent merges multiple loss functions into a unified training objective for the decoder. The total loss combines accent modification loss (to improve intelligibility) and speaker identity preservation loss (to maintain recognizability) into a single combined objective function. This merging approach allows simultaneous optimization of both competing requirements through a single training process.
Data Source
AI summary
A computing system that receives an audio waveform representing speech from an individual and produces as output a modified version of the audio waveform that maintains the speaker's speech characteristics as well as prosody for specific utterances (e.g., voice timbre, intonation, timing, intensity). The system uses a bottleneck-based autoencoder with speech spectrograms as input and output. To produce the output audio waveform, the system includes a reconstruction error-based loss function with two additional loss functions. The second loss function is speaker “real vs fake” discriminator that penalizes for the output not sounding like the speaker. The third loss function is a speech intelligibility scorer that penalizes the output for speech that is difficult for the target population to understand. The produced modified audio waveform is an enhanced speech output that delivers speech m a target accent without sacrificing the personality of the speaker.