U-Net Neural Network for Audio Signal Component Isolation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for automatic singing voice separation in music suffer from poor generalization due to limited training datasets and loss of local detail in processing, particularly in commercial-quality music analysis.
Innovation Solution
A neural network system utilizing a U-Net architecture is trained with large datasets of mixed and instrumental music tracks to separate vocal and instrumental components, employing convolution and deconvolution paths with skip connections to preserve fine-grained details and improve separation accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If spectrograms are compressed through a bottleneck layer and re-expanded, then global patterns are discovered, but local detail is lost
Solution Approach 1:
The patent segments the processing into multiple parallel bottleneck layers instead of a single sequential compression-expansion path. Each bottleneck layer processes different aspects of the spectrogram independently, allowing global patterns to be captured at different hierarchical levels simultaneously without forcing a single compression path that loses local details.
Solution Approach 2:
The patent introduces an additional dimensional aspect by processing spectrograms through multiple parallel bottleneck layers with different compression ratios and characteristics. This multi-dimensional approach allows the system to capture global patterns across different scales while preserving local details through the diversity of processing paths.
2Device complexity
If deep learning models are trained on limited datasets, then model complexity is reduced, but generalization performance deteriorates
Solution Approach 1:
The patent segments the training dataset into multiple distinct components (vocal tracks, instrumental tracks, and mixed tracks) that are processed through separate bottleneck layers. This segmentation allows the model to learn specialized features from each data type while maintaining overall system coherence, improving generalization without requiring excessive model complexity.
Solution Approach 2:
The patent creates a multi-functional bottleneck layer system where each layer can handle different types of audio data (vocal, instrumental, mixed) and can be selectively activated based on the input type. This universality allows the model to generalize better across different music genres and recording qualities without needing separate specialized models for each case.
Data Source
AI summary
A system, method and computer product for training a neural network system. The method comprises inputting an audio signal to the system to generate plural outputs f(X, Θ). The audio signal includes one or more of vocal content and/or musical instrument content, and each output f(X, Θ) corresponds to a respective one of the different content types. The method also comprises comparing individual outputs f(X, Θ) of the neural network system to corresponding target signals. For each compared output f(X, Θ), at least one parameter of the system is adjusted to reduce a result of the comparing performed for the output f(X, Θ), to train the system to estimate the different content types. In one example embodiment, the system comprises a U-Net architecture. After training, the system can estimate various different types of vocal and/or instrument components of an audio signal, depending on which type of component(s) the system is trained to estimate.