Voice Conversion Model Using Vector Pools for Many-to-Many Synthesis
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current voice conversion technologies based on deep learning, such as Bi-LSTM models, are limited to one-to-one conversions, making it impractical and costly to achieve many-to-many voice conversions with a single voice model, as multiple models need to be trained for each target speaker, leading to storage inefficiencies.
Innovation Solution
A method that extracts acoustic features from a to-be-converted voice and uses source and target vector pools to enable a single voice conversion model to perform many-to-many conversions by selecting appropriate source and target vectors, allowing the model to convert the voice features into a target voice using a vocoder.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If multiple one-to-one models are trained for each target speaker, then voice conversion accuracy and similarity are improved, but device complexity and storage requirements increase significantly
Solution Approach 1:
The patent applies universality by training a single voice conversion model that can handle multiple target speakers instead of training separate one-to-one models for each speaker pair. The model is trained on diverse speaker data and uses speaker embedding vectors to identify and convert different speakers, enabling one model to perform the function of multiple specialized models would otherwise be needed.
Solution Approach 2:
The patent changes the parameter approach by moving from fixed speaker-specific parameters in one-to-one models to a unified model with speaker embedding vectors as input parameters. The model takes acoustic features and speaker embeddings as inputs and generates target speaker acoustic features, allowing flexible adaptation to different speaker pairs without retraining.
2Reliability
If multiple one-to-one models are trained for each target speaker, then conversion quality is improved, but storage cost increases
Solution Approach 1:
The patent implements universality by creating a single multi-functional voice conversion model that can convert audio from any source speaker to any target speaker within the trained dataset. This eliminates the need to store and manage multiple separate models, significantly reducing storage requirements while maintaining high conversion quality through the unified model's comprehensive training.
3Quantity of substance
If a single voice model is used for many-to-many conversions, then storage cost is reduced, but conversion flexibility and adaptability worsen
Solution Approach 1:
The patent uses parameter changes by incorporating speaker embedding vectors as dynamic input parameters to the unified model. These embedding vectors capture speaker-specific characteristics and are provided as input along with the acoustic features, allowing the single model to adapt its behavior to different speaker pairs without requiring separate models for each combination.
Solution Approach 2:
The patent introduces speaker embedding vectors as an intermediary mechanism that bridges the gap between the unified model and speaker-specific requirements. These embeddings act as mediators that carry speaker identity information into the model, enabling the single model to maintain flexibility and adaptability across different speaker pairs while using a single stored model.
Data Source
AI summary
The present disclosure discloses a voice conversion method. The method includes: obtaining a to-be-converted voice, and extracting acoustic features of the to-be-converted voice; obtaining a source vector corresponding to the to-be-converted voice from a source vector pool, and selecting a target vector corresponding to the target voice from the target vector pool; obtaining acoustic features of the target voice output by the voice conversion model by using the acoustic features of the to-be-converted voice, the source vector corresponding to the to-be-converted voice, and the target vector corresponding to the target voice as an input of the voice conversion model; and obtaining the target voice by converting the acoustic features of the target voice using a vocoder. In addition, a voice conversion apparatus and a storage medium are also provided.