Phase Reconstruction via Structure Tensor Orientation on Audio Spectrograms
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing harmonic-percussive sound separation methods fail to accurately capture frequency modulated tones, which carry tonal information, due to their reliance on strict horizontal and vertical structure assumptions in spectrograms, leading to leakage into the residual component.
Innovation Solution
The use of the structure tensor to calculate predominant orientation angles in the magnitude spectrogram, allowing for the distinction between harmonic, percussive, and residual signal components, even in cases of frequency modulated signals, by determining local frequency changes and anisotropy measures.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If strict horizontal and vertical structure assumptions are used in spectrogram-based harmonic-percussive separation, then simple classification is achieved, but frequency modulated tones leak into the residual component
Solution Approach 1:
The patent changes the classification parameters from strict horizontal/vertical orientation to orientation angles derived from the structure tensor. This allows flexible classification of spectral structures including frequency modulated tones by calculating predominant orientation angles that can represent any direction, not just horizontal or vertical. The structure tensor parameters (eigenvalues and eigenvectors) are used to determine local orientation, enabling accurate classification of FM tones in the harmonic component.
2Reliability
If diffusion or median filtering is used to enhance spectrogram structures, then harmonic and percussive components are separated, but frequency modulated tones are misclassified as residual
Solution Approach 1:
The patent substitutes the mechanical filtering approaches (diffusion and median filtering) with a tensor-based mathematical approach. Instead of applying convolution or sorting operations to enhance structures, the structure tensor computes local gradient information and orientation angles directly from the spectrogram magnitude. This substitution enables precise detection of frequency changes through orientation angle calculation, correctly identifying FM tones as harmonic rather than residual.
3Manufacturing precision
If non-negative matrix factorization is used to capture non-horizontal structures, then harmonic sounds with vibrato are captured, but computational complexity increases and residual component is omitted
Solution Approach 1:
The patent extracts orientation information from the structure tensor and uses it to create a classification mask that separates harmonic, percussive, and residual components. Instead of using complex NMF decomposition, the method extracts the key feature (predominant orientation angle) from each time-frequency bin and uses threshold-based classification. This extraction approach maintains high accuracy for capturing FM tones while significantly reducing computational complexity and preserving the residual component for non-anisotropic sounds.
Data Source
AI summary
An apparatus for phase reconstruction from a magnitude spectrogram of an audio signal is provided. The apparatus includes a frequency change determiner being configured to determine a change of a frequency for each time-frequency bin of a plurality of time-frequency bins of the magnitude spectrogram of the audio signal depending on the magnitude spectrogram of the audio signal, and a phase reconstructor being configured to generate phase values for the plurality of time-frequency bins depending on the changes of the frequencies determined for the plurality of the time-frequency bins.