Neural Audio Encoding with Split Waveform and Spectral Latents
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing audio encoding and decoding methods using artificial neural networks face challenges such as audio quality degradation, high computational resource requirements, and complex implementations, particularly in Vector Quantized Variational Auto-Encoders (VQVAE), which do not provide optimal performance and efficiency.
Innovation Solution
The approach involves splitting an audio signal into two components, using separate trained artificial neural networks to generate latent space feature sets representing the waveform and spectro-temporal shape of each component, and combining these features to create an encoded audio data signal, which can be efficiently decoded using trained neural networks.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If Vector Quantized Variational Auto-Encoders (VQVAE) are used for audio encoding and decoding, then audio signal processing capability is improved, but audio quality degrades and computational resource requirements increase
Solution Approach 1:
The audio signal is divided into two separate components: a deterministic signal component and a stochastic signal component. Each component is processed by its own dedicated neural network (first encoding trained neural network and second encoding trained neural network), allowing specialized processing that maintains audio quality while reducing computational complexity compared to a unified VQVAE approach.
2Adaptability or versatility
If Vector Quantized Variational Auto-Encoders (VQVAE) are used for audio encoding and decoding, then audio signal processing capability is improved, but device complexity increases
Solution Approach 1:
The complex VQVAE system is segmented into two simpler, specialized neural networks that process different signal components separately. This segmentation reduces implementation complexity by allowing each network to be optimized for its specific function rather than requiring a single complex unified architecture.
Solution Approach 2:
The stochastic component processing is extracted from the deterministic component processing. By separating these functions into distinct neural networks, the system avoids the complexity of integrating both functions within a single VQVAE framework, making the overall system easier to implement and train.
3Reliability
If separate trained artificial neural networks are used for waveform and spectro-temporal shape processing, then audio quality is improved and computational complexity is reduced, but data rate increases
Solution Approach 1:
Different levels of processing quality are applied to different signal components based on their characteristics. The deterministic signal component, which requires precise waveform reconstruction, is processed by a neural network optimized for waveform accuracy. The stochastic component, which is less critical for exact reconstruction, is processed separately, allowing for more efficient compression and reducing the overall data rate requirement.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
An encoder audio apparatus comprises a signal splitter circuit (203) dividing the audio signal into at least a first and second signal component. A first neural network (207) generates a first latent space feature set representing a waveform of the first signal component a second neural network (209) generates a second latent space feature set representing a spectro-temporal shape of the second signal component. An encoded audio data stream comprising the latent space feature sets is transmitted to a decoder audio apparatus which comprises a noise generator (305) and a neural network arrangement (303) which generates an output audio signal. The neural network arrangement (303) has input nodes for receiving the noise signal and receives the latent space features as input features. Typically, the arrangement includes one neural network generating a signal reproducing the waveform of the first signal component and a second neural network generating a signal reproducing the spectro-temporal shape of the second signal component.