Neural Audio Coding via Perceptual Domain Transformation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Neural networks face challenges in encoding and decoding general audio signals due to the need to leverage human auditory system limitations and the high dynamic range and diversity of audio signals, making traditional loss functions like L1 and L2 ineffective.
Innovation Solution
Transform audio signals into a perceptual domain using masking thresholds derived from psychoacoustic models, allowing the use of non-perceptual loss functions and enabling effective training of neural networks.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If traditional loss functions (L1, L2) are used for training neural networks in audio encoding/decoding, then the training process is simple and computationally efficient, but the performance is poor because these functions do not account for human auditory system limitations and the high dynamic range of audio signals
Solution Approach 1:
The patent transforms the audio signal from the original signal domain to the perceptual domain by applying a mask derived from psychoacoustic models. This parameter transformation changes the dynamic range and scaling of the signal, making it suitable for training with traditional loss functions while maintaining perceptual relevance. The mask application modifies the signal parameters to align with human auditory perception characteristics.
Solution Approach 2:
The patent introduces a perceptual domain as an intermediary layer between the original audio signal and the neural network processing. This intermediary domain, created through mask application, serves as a bridge that translates the high dynamic range original signal into a form that is more amenable to standard training approaches while preserving perceptually important information.
2Measurement precision
If audio signals are processed in the original signal domain with high dynamic range, then all audio information is preserved, but neural networks struggle to learn effective representations due to the diversity and scale of the signals
Solution Approach 1:
The patent applies a perceptual mask to transform the audio signal parameters from the original domain to the perceptual domain. This parameter change compresses the dynamic range and re-scales the signal according to human auditory perception characteristics, making the signal more suitable for neural network training while maintaining perceptual fidelity.
Solution Approach 2:
The patent performs preliminary transformation of the audio signal to the perceptual domain before feeding it to the neural network. This preliminary action of applying the mask prepares the signal in advance, converting it into a form that is more amenable to learning while preserving the perceptually relevant information that the network needs to process.
Data Source
AI summary
Described herein is a method of processing an audio signal using a neural network or using a first and a second neural network. Described is further a method of training said neural network or of jointly training a set of said first and said second neural network. Moreover, described is a method of obtaining and transmitting a latent feature space representation of a perceptual domain audio signal using a neural network and a method of obtaining an audio signal from a latent feature space representation of a perceptual domain audio signal using a neural network. Described are also respective apparatuses and computer program products.