Filter-Bank Audio Generation Without Phase Reconstruction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing generative neural network models for audio signal processing face challenges in integrating with frequency-domain tools due to time-domain operation and the need for recursive networks, which limit parallelization and require phase reconstruction algorithms.
Innovation Solution
A generative model operating in the filter-bank domain that learns to handle both magnitude and phase inherently, facilitating easier integration with frequency-domain tools and enabling parallel processing by autoregressively generating probability distributions for filter-bank representations of audio signals.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If generative models operate in the time-domain, then they can process audio signals directly, but they cannot easily integrate with frequency-domain tools and use recursive networks that limit parallelization
Solution Approach 1:
The patent segments the audio signal processing into frequency bands using a filter bank, transforming the monolithic time-domain processing into parallel frequency-domain sub-processing. Each filter bank channel operates independently, enabling parallelization while maintaining compatibility with frequency-domain tools like equalizers.
Solution Approach 2:
The patent transitions from time-domain operation to frequency-domain operation by applying a filter bank transformation. This dimensional change allows the model to work with spectral coefficients directly, enabling integration with frequency-domain signal processing tools and eliminating the need for recursive time-domain networks.
2Measurement precision
If generative models operate on spectrograms, then they can process frequency information, but they do not reconstruct the phase of the audio signal and require phase reconstruction algorithms as post processing
Solution Approach 1:
The patent performs phase reconstruction as a preliminary action during the main generation process rather than as a post-processing step. The phase prediction network generates phase information simultaneously with magnitude prediction, and the inverse filter bank transformation reconstructs the time-domain signal with proper phase coherence before output.
Solution Approach 2:
The patent merges magnitude prediction and phase prediction into a single unified neural network model. Both predictions are generated from the same encoded latent representation, ensuring that magnitude and phase information are coherent and consistent with each other, eliminating the need for separate phase reconstruction algorithms.
3Ease of operation
If generative models use recursive networks, then they can process sequential audio data, but they have limited potential for parallelization
Solution Approach 1:
The patent segments the audio signal into independent frequency band channels using a filter bank. Each channel processes its frequency band independently through the neural network, allowing parallel processing across all frequency channels while still maintaining sequential temporal processing within each channel through the LSTM layers.
Solution Approach 2:
The patent changes the processing dimension from purely temporal (sequential) to spectral-temporal by applying filter bank transformation. This allows parallel processing across the frequency dimension while maintaining sequential processing capability in the time dimension, achieving both objectives simultaneously.
Data Source
AI summary
A neural network system is provided, implementing a generative model for autoregressively generating a distribution for a plurality of current filter-bank samples of an audio signal, wherein the current samples correspond to a current time slot, and each current sample corresponds to a channel of the filter-bank. The system includes a hierarchy of a plurality of neural network processing tiers ordered from a top to a bottom tier, each tier trained to generate conditioning information based on previous filter-bank samples and, for at least each tier but the top tier, also on the conditioning information from a tier higher up in the hierarchy, and an output stage trained to generate the probability distribution based on previous samples for one or more previous time slots and the conditioning information from the lowest processing tier.