Singing Voice Conversion with Source Separation Noise Control
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing voice quality conversion technologies struggle to accurately convert a singing voice to a specific singer's voice quality in real-time, especially in karaoke systems, due to the difficulty in handling variations in sound pitch and quality, and the inclusion of noise from sound source separation, which degrades the conversion quality.
Innovation Solution
An information processing apparatus and method that performs sound source separation to isolate vocal and accompaniment signals, using a voice quality conversion unit to adjust the user's voice to match a target singer's voice quality, employing learning models to extract and mix feature amounts such as sound pitch, volume, and speech content, while minimizing noise and enabling real-time processing.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If sound source separation is performed to isolate vocal signal from mixed sound, then voice quality conversion accuracy is improved, but noise is introduced that degrades conversion quality
Solution Approach 1:
The patent extracts only the essential voice quality features (spectral envelope, formant frequencies, harmonics) from the separated vocal signal while discarding the noisy components. This selective extraction approach maintains conversion accuracy by focusing on robust acoustic features that are less susceptible to separation noise.
Solution Approach 2:
The patent introduces an intermediate feature extraction and processing stage between sound source separation and voice quality conversion. This intermediary layer processes the separated signal to enhance quality features and suppress noise before applying conversion, effectively mediating between the noisy separation output and the conversion requirement.
2Productivity
If voice quality conversion is performed on mixed sound signal without sound source separation, then processing speed is improved, but conversion quality deteriorates due to accompaniment interference
Solution Approach 1:
The patent segments the audio signal processing into distinct stages: rapid sound source separation to identify vocal regions, followed by focused voice quality analysis and conversion on those segmented regions. This segmentation enables real-time processing by concentrating computational resources on relevant vocal portions rather than processing the entire mixed signal.
Solution Approach 2:
The patent applies partial action by performing complete sound source separation only on critical segments of the audio signal where voice quality conversion is most needed, while using lighter processing on other segments. This selective approach maintains real-time performance while achieving sufficient conversion quality.
3Measurement precision
If sound source separation and voice quality conversion are both performed, then voice quality conversion accuracy is improved, but processing complexity increases
Solution Approach 1:
The patent employs a multi-functional processing framework where the sound source separation system also performs voice activity detection, and the voice quality conversion system simultaneously handles both conversion and noise suppression. This universality reduces overall system complexity by combining multiple functions into integrated processing modules rather than separate dedicated systems.
Data Source
AI summary
For example, an effective voice quality conversion process is performed.An information processing apparatus includes: a voice quality conversion unit that performs sound source separation of a vocal signal and an accompaniment signal from a mixed sound signal and performs voice quality conversion using a result of the sound source separation.