Ambisonic Audio Encoding With AI Channel Scaling at Low Bitrates
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing general-purpose codecs do not support efficient encoding and decoding of multi-channel audio, particularly for providing spatial three-dimensional effects, necessitating a method to encode and decode multi-channel audio at low bitrates with high quality.
Innovation Solution
Utilizing deep neural networks (DNNs) to transform audio signals into frequency domains, generate frequency feature signals, and apply AI-based downscaling and upscaling to encode and decode multi-channel audio with general-purpose codecs, incorporating frequency feature signals to compensate for channel changes.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If general-purpose codec is used for encoding multi-channel audio, then device complexity is reduced, but audio quality and spatial three-dimensional effect are deteriorated
Solution Approach 1:
The audio signal processing is segmented into multiple stages: initial encoding with general-purpose codec, extraction of frequency feature signals, AI-based decoding to generate intermediate signals, and final synthesis using spatial parameter information. This segmentation allows each stage to be optimized independently, maintaining high audio quality while using simple general-purpose codecs.
Solution Approach 2:
Frequency feature signals and spatial parameter information serve as intermediaries between the simple general-purpose codec and the final high-quality multi-channel audio output. These intermediate representations carry essential audio characteristics that enable reconstruction of spatial three-dimensional effects without requiring the codec itself to be complex.
2Manufacturing precision
If multi-channel audio is encoded with high quality, then audio quality is improved, but bitrate is increased
Solution Approach 1:
Essential audio characteristics are extracted from the multi-channel audio signal in the form of frequency feature signals and spatial parameter information. Only these extracted features are encoded and transmitted, rather than encoding the complete multi-channel signal, thereby achieving high-quality reconstruction at low bitrate.
Solution Approach 2:
The encoding approach changes from transmitting raw audio samples to transmitting transformed parameter representations (frequency features and spatial parameters). This parameter transformation enables efficient compression while preserving the information necessary for high-quality audio reconstruction with spatial three-dimensional effects.
3Manufacturing precision
If deep neural network processing is applied, then audio quality is improved, but device complexity is increased
Solution Approach 1:
Deep neural networks are trained in advance offline to learn optimal mappings from frequency feature signals and spatial parameters to high-quality multi-channel audio. This preliminary training action transfers complex processing requirements to the training phase, allowing the deployed system to use simpler real-time processing while still achieving high audio quality.
Data Source
AI summary
An audio signal processing apparatus is configured to: transform a first audio signal includes n channels to generate a first audio data in a frequency domain, generate a frequency feature signal for each channel from the first audio data in the frequency domain, based on a first deep neural network (DNN), generate a second audio signal includes m channels from the first audio signal, based on a second DNN, and generate an output audio signal by encoding the second audio signal and the frequency feature signal. The first audio signal is a high order ambisonic signal includes a zeroth order signal and a plurality of first order signals. The second audio signal includes a mono signal or a stereo signal. m is smaller than n.