Audio Segmentation via Magnitude Spectrogram Deep Learning
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current audio segmentation techniques are inefficient, require human intervention, and are imperfect due to noise, making it difficult to automatically decompose audio signals into component parts, such as isolating vocals and instruments from mixed audio signals.
Innovation Solution
The method involves transforming an audio signal into a complex spectrogram, decomposing it into magnitude and phase spectrograms, using deep neural networks to generate masks for each source, and combining these masks with the original spectrograms to isolate individual audio components, which are then encoded and stored separately.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If common audio segmentation techniques are used, then audio signals can be decomposed into component parts, but the segmentation is inefficient and requires human intervention
Solution Approach 1:
The system uses deep neural networks to automatically perform audio segmentation without human intervention. The neural network model independently processes the mixed audio signal, generates source masks, and decomposes the audio into separate sources, making the system self-sufficient and eliminating the need for manual segmentation operations.
Solution Approach 2:
The patent replaces manual mechanical segmentation processes with an automated deep learning system. Instead of human operators manually separating audio sources, a neural network model processes the audio signal through spectral analysis and mask generation, substituting human mechanical intervention with automated computational processing.
2Reliability
If common audio segmentation techniques are used, then audio decomposition is achieved, but the results are imperfect due to noise
Solution Approach 1:
The system extracts the magnitude information from the complex spectrogram and processes it separately through deep neural networks. By extracting and independently processing the magnitude spectrum, the system can generate clean source masks that are less susceptible to noise contamination, thereby improving segmentation accuracy and reducing noise interference in the final separated audio sources.
Solution Approach 2:
The patent introduces deep neural networks as an intermediary between the spectrogram analysis and final audio separation. The neural networks process the magnitude spectrogram, generate optimized source masks, and guide the separation process, acting as a mediator that filters out noise and improves the reliability of the segmentation results.
3Productivity
If common audio segmentation techniques are used, then audio decomposition is performed, but computational costs are high
Solution Approach 1:
The patent segments the audio processing task into distinct stages: converting audio to spectrogram, extracting magnitude information, processing through neural networks to generate masks, and reconstructing separated sources. This segmentation of the computational process allows for optimized processing at each stage, reducing overall computational costs while maintaining processing speed and accuracy.
Data Source
AI summary
A method, system, and computer readable medium for decomposing an audio signal into different isolated sources. The techniques and mechanisms convert an audio signal into K input spectrogram fragments. The fragments are sent into a deep neural network to isolate for different sources. The isolated fragments are then combined to form full isolated source audio signals.