Audio Style Transfer via Phoneme and Speaker Feature Segmentation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing audio style transfer methods suffer from poor accuracy in imitating audio styles, such as tones and volume, due to inadequate separation of phoneme and speaker features, leading to unsatisfactory transfer results.
Innovation Solution
The method involves obtaining target feature vectors for phoneme, style prosody, and speaker features from original and source audio, performing spectrogram decoding, and converting these features into target audio, using a trained spectrogram decoding network model to achieve accurate audio style transfer.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If audio style transfer is implemented under supervision using related solutions, then the transfer process can be executed, but the transfer result is unsatisfactory since audio style such as tones and volume cannot be accurately imitated
Solution Approach 1:
The patent segments the audio feature representation into three distinct components: phoneme features, style prosody features, and speaker features. This is achieved through separate encoding modules that extract and process each feature type independently, allowing for precise control and accurate imitation of audio styles such as tones and volume without interference from other features.
2Device complexity
If phoneme and speaker features are not adequately separated, then the processing is simpler, but the audio style transfer accuracy deteriorates
Solution Approach 1:
The patent divides the feature extraction and processing system into distinct modules: a phoneme encoding module, a style prosody encoding module, and a speaker encoding module. Each module independently processes its specific feature type, ensuring clean separation between phoneme and speaker features while maintaining manageable system complexity through modular design.
Solution Approach 2:
The patent introduces a style prosody encoding module as an intermediary component that specifically handles style-related features (tones, volume) separately from phoneme and speaker features. This intermediary module acts as a mediator that enables accurate style transfer by isolating and processing style characteristics independently.
Data Source
Figure 1~2
Figure 3
Figure 4
AI summary
Provided are a method and an apparatus for processing audio, an electronic device and a storage medium. A specific implementation solution includes: obtaining a first target feature vector from original audio, where the first target feature vector is used for representing a phoneme feature of the original audio; obtaining a second target feature vector and a third target feature vector from audio to be transferred, where the second target feature vector is used for representing a style prosody feature of the audio to be transferred, and the third target feature vector is used for representing a speaker feature of the audio to be transferred; performing spectrogram decoding on the first target feature vector, the second target feature vector and the third target feature vector to obtain a target spectrogram feature; and converting the target spectrogram feature into target audio.