TTS Audio Output Model Preventing Speech Neutralization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Multi-speaker based text-to-speech (TTS) systems often result in speech neutralization, leading to unsatisfactory synthesis effects, as they fail to effectively distinguish and maintain the unique characteristics of individual speakers.
Innovation Solution
A neural network-based audio output method that includes an identity recognition model and a speech synthesis model, where the identity recognition model extracts and maps identity features using a residual network, and the speech synthesis model generates phoneme features, which are then combined and processed to produce synthetic audio that accurately represents the target speaker's style, avoiding speech neutralization.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If speaker embedding is used to encode features of each speaker in multiple-speaker based TTS method, then the system can distinguish between each speaker's distinctive characteristics, but speech neutralization occurs where the two styles interact with each other so that the final synthesized speech is neutralized
Solution Approach 1:
The patent segments the speaker embedding vector into multiple independent style vectors, each corresponding to a specific speaker. This segmentation prevents the interaction and neutralization that occurs when using a single combined embedding, as each style vector maintains distinct speaker characteristics independently.
Solution Approach 2:
The patent applies local quality by assigning different weights to different style vectors based on their relevance to the target speaker. This allows the system to emphasize specific speaker characteristics locally rather than treating all speakers equally, thereby preserving distinctive speech styles while avoiding neutralization.
2Adaptability or versatility
If multiple speaker embeddings are combined to achieve multi-speaker capability, then the system gains versatility, but the interaction between styles causes neutralization and unsatisfactory synthesis effect
Solution Approach 1:
The patent introduces dynamic weighting mechanisms that allow the system to adaptively adjust the contribution of each speaker embedding based on the synthesis requirements. This dynamic approach maintains versatility while preventing neutralization by flexibly controlling style interactions.
Solution Approach 2:
The patent creates a composite speaker representation by combining multiple weighted style vectors rather than using simple averaging or concatenation. This composite approach preserves the distinctive characteristics of individual speakers while achieving multi-speaker capability, avoiding the neutralization effect.
Data Source
AI summary
This application provide a synthetic audio output method and apparatus, a storage medium, and an electronic device. The method includes: inputting input text and a specified target identity identifier into an audio output model; extracting an identity feature sequence of a target identity by an identity recognition model; extracting a phoneme feature sequence corresponding to the input text by an encoding layer of a speech synthesis model; superimposing and inputting the identity feature sequence of the target identity and the phoneme feature sequence into a variable adapter of the speech synthesis model; and after duration prediction and alignment, energy prediction, and pitch prediction are performed on the phoneme feature sequence by the variable adapter, outputting a target Mel-frequency spectrum feature corresponding to the input text through a decoding layer of the speech synthesis model; and inputting the target Mel-frequency spectrum feature into a vocoder to output synthetic audio.