Voice Quality Conversion from Mixed Audio Without Clean Data
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing voice quality conversion technologies require parallel data or clean data, which are difficult to obtain, making it challenging to easily perform voice quality conversion.
Innovation Solution
A signal processing apparatus and method that converts acoustic data using training data that is not parallel or clean, utilizing sound source separation to separate target and non-target sounds, and training a voice quality converter parameter without requiring clean data or parallel data.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If parallel data or clean data is used for training, then voice quality conversion accuracy is improved, but data acquisition difficulty increases
Solution Approach 1:
The patent introduces a sound source separation unit as an intermediary component that processes mixed sound data to extract target speaker components. This mediator enables the system to work with readily available mixed sound data (easier to obtain) while producing results comparable to using clean data (high accuracy). The sound source separation unit acts as the intermediary that bridges the gap between easy-to-acquire mixed sound data and the quality requirements for accurate voice conversion.
Solution Approach 2:
The patent changes the training data parameter from requiring clean parallel data to accepting mixed sound data. By modifying the data input parameter and introducing sound source separation processing, the system achieves voice quality conversion using parameters (mixed sound data) that are much easier to acquire in practice, while maintaining conversion accuracy through the separation processing.
2Manufacturing precision
If clean data is required for training, then conversion quality is improved, but training data availability decreases
Solution Approach 1:
The sound source separation unit serves as an intermediary that processes mixed sound data to extract clean target speaker components during training. This allows the system to use readily available mixed sound data (high availability) while producing training results equivalent to using clean data (high conversion quality). The intermediary processing resolves the contradiction between data availability and conversion quality.
Solution Approach 2:
The patent segments mixed sound data into target speaker components and other sound components through sound source separation. This segmentation enables the training process to focus on relevant speaker characteristics while filtering out interfering elements, thereby achieving high conversion quality using easily available mixed sound data rather than requiring pre-cleaned data.
3Measurement precision
If vowel section information is required for training, then conversion accuracy is improved, but data processing complexity increases
Solution Approach 1:
The patent merges the sound source separation function with the voice quality conversion training process. Instead of separately identifying vowel sections and separately performing conversion, the sound source separation unit integrates these functions, automatically extracting target speaker characteristics from mixed sound data during the separation process itself, thereby reducing overall processing complexity while maintaining accuracy.
Data Source
AI summary
Provided is a signal processing apparatus that includes a voice quality conversion unit that converts acoustic data of any sound of an input sound source to acoustic data of voice quality of a target sound source different from the input sound source on the basis of a voice quality converter parameter obtained by training using acoustic data for each of one or more sound sources as training data, the acoustic data being different from parallel data or clean data.