Decoder Spatial Noise Filling for Low-Bitrate Multi-Channel Audio
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Multi-channel codecs experience noise ambience collapse at low bitrates due to limited downmix channels, coding artifacts, and insufficient decorrelation of channels, leading to a loss of background noise ambience.
Innovation Solution
Implement spatial noise filling techniques in the decoder to regenerate background noise ambience by generating multi-channel noise with desired spatial and spectral shapes using primary downmix channel information and spatial metadata, adding it to the decoder output to fill spatial holes.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If low bitrate is used in multi-channel codec, then bitrate efficiency is improved, but background noise ambience is lost
Solution Approach 1:
The encoder performs preliminary analysis of the audio scene to identify background noise characteristics and spatial information before encoding. This preliminary action allows the decoder to regenerate accurate background noise ambience even at low bitrates by using pre-extracted spatial metadata and noise spectral shape information.
Solution Approach 2:
Spatial metadata and noise spectral shape information act as intermediary elements that carry essential spatial and spectral characteristics of the background noise. These intermediaries enable the decoder to reconstruct the background noise ambience without requiring high bitrate transmission of the full audio signal.
2Adaptability or versatility
If limited downmix channels are used, then device compatibility is improved, but noise ambience collapse occurs
Solution Approach 1:
The audio signal is segmented into different components: downmix channels for compatibility and spatial metadata for ambience preservation. By separating the essential audio information into these segments, the system can use limited downmix channels for device compatibility while maintaining noise ambience through the spatial metadata.
Solution Approach 2:
The patent adds a dimensional layer by encoding spatial information as metadata separate from the audio channels. This additional dimension of spatial metadata allows the system to compensate for the limited number of downmix channels, enabling both device compatibility and noise ambience preservation simultaneously.
3Productivity
If coding artifacts are introduced, then processing speed is improved, but audio quality deteriorates
Solution Approach 1:
The patent converts the harmful effect of coding artifacts into a beneficial process by using the artifacts as clues to regenerate the background noise ambience. The spatial metadata and noise spectral shape information derived from the coded signal are used to synthesize accurate background noise, transforming the artifact problem into an opportunity for enhanced audio quality.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
Embodiments are disclosed for spatial noise filling in multi-channel codecs. In an embodiment, a method of regenerating background noise ambience in a multi-channel codec by generating spatial hole filling noise comprises: computing noise estimates based on a primary downmix channel generated from an input audio signal representing a spatial audio scene with background noise ambience; computing spectral shaping filter coefficients based on the noise estimates; spectrally shaping the multi-channel noise signal using the spectral shaping filter coefficients and a noise distribution, the spectral shaping resulting in a diffused, multi-channel noise signal with uncorrelated channels; spatially shaping the diffused, uncorrelated multi-channel noise signal with uncorrelated channels based on a noise ambience of the spatial audio scene; and adding the spatially and spectrally shaped multi-channel noise to a multi-channel codec output to synthesize the background noise ambience of the spatial audio scene.