Multichannel Audio Upmixing with Covariance-Based Residual Decoding

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing parametric methods for encoding multichannel audio signals, such as MPEG Surround, DirAC, and BCC, face issues with high bitrates, lack of flexibility, and quality degradation when applied to multichannel audio signals.

Innovation Solution

An audio synthesizer that generates a synthesis signal using a first and second mixing matrix, based on covariance matrices and decorrelation, to efficiently encode multichannel audio content at low bitrates, allowing flexibility across various loudspeaker setups.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Manufacturing precision

If discrete coding of multichannel content is used, then encoding quality is maintained, but bit consumption increases significantly

Engineering Contradiction:

Improveencoding qualityVSAvoidbit consumption

Core Design Contradiction:

Manufacturing precisionVSQuantity of substance

Solution Approach 1:

The patent extracts only the essential spatial information (interchannel coherences and channel level differences) from the multichannel signal, separating it from the full waveform data. This allows transmitting a downmixed signal plus compact spatial parameters instead of the complete multichannel audio, significantly reducing bit consumption while preserving spatial quality.

Inventive Principle:

Principle #2Taking out (Extraction)

Solution Approach 2:

Instead of encoding each channel separately (traditional approach), the patent inverts the approach by first downmixing to a mono or stereo signal, then encoding spatial parameters that describe how to reconstruct the original multichannel spatial relationships. This inversion fundamentally changes the encoding strategy from channel-by-channel to spatial-relationship-based.

Inventive Principle:

Principle #13The other way round (Inversion)

2Productivity

If MPEG Surround tree-structure is used, then spatial audio is described efficiently, but flexibility is reduced and quality degradation occurs on certain items

Engineering Contradiction:

Improvespatial audio description efficiencyVSAvoidsystem flexibility

Core Design Contradiction:

ProductivityVSAdaptability or versatility

Solution Approach 1:

The patent creates a universal framework that can handle various multichannel configurations (5.1, 7.1, stereo, etc.) without requiring different tree-structure configurations. The method universally applies interchannel coherence and level difference calculations across any channel pair, making the system adaptable to different loudspeaker setups and signal types without the rigidity of predetermined tree structures.

Inventive Principle:

Principle #6Universality (Multi-functionality)

3Quantity of substance

If DirAC framework is used, then low bitrate encoding is achieved, but the parameters are not well-suited for multichannel audio signals

Engineering Contradiction:

ImprovebitrateVSAvoidoutput quality

Core Design Contradiction:

Quantity of substanceVSManufacturing precision

Solution Approach 1:

The patent changes the parameter set from DirAC's DOA and diffuseness (designed for binaural/surround) to interchannel coherences and channel level differences (designed for multichannel). This parameter transformation adapts the low-bitrate DirAC framework to multichannel signals by using parameters that naturally describe multichannel spatial relationships, thereby maintaining both low bitrate and high output quality.

Inventive Principle:

Principle #35Parameter changes

Data Source

PatentEP4398243B1Parameter encoding and decoding

Publication Date: 2026.01.28 FRAUNHOFER GESELLSCHAFT ZUR FORDERUNG DER ANGEWANDTEN FORSCHUNG EV

Figure 1
Figure 2a
Figure 2b

AI summary

There is disclosed audio synthesizer (300) for generating a synthesis signal (336) from a downmix signal (324, x) having a number of downmix channels, the synthesis signal (336) having a number of synthesis channels, the downmix signal (324, x) being a downmixed version of an original signal (212) having a number of original channels, the audio synthesizer (300) comprising: a first path (610c') including: a first mixing matrix block (600c) configured for synthesizing a first component (336M') of the synthesis signal according to a first mixing matrix (MM) calculated from: a covariance matrix (CYR) associated to the synthesis signal (212); and a covariance matrix (Cx) associated to the downmix signal (324), a second path (610c) for synthesizing a second component (336R') of the synthesis signal, wherein the second component (336R') is a residual component, the second path (610c) including: a prototype signal block (612c) configured for upmixing the downmix signal (324) from the number of downmix channels to the number of synthesis channels; a decorrelator (614c) configured for decorrelating the upmixed prototype signal (613c); a second mixing matrix block (618c) configured for synthesizing the second component (336R') of the synthesis signal according to a second mixing matrix (MR) from the decorrelated version (615c) of the downmix signal (324), the second mixing matrix (MR) being a residual mixing matrix, wherein the audio synthesizer (300) is configured to calculate (618c) the second mixing matrix (MR) from: the residual covariance matrix (Cr) provided by the first mixing matrix block(600c); and an estimate of the covariance matrix of the decorrelated prototype signals (Cy ) obtained from the covariance matrix (Cx) associated to the downmix signal (324), wherein the audio synthesizer (300) further comprises an adder block (620c) for summing the first component (336M') of the synthesis signal with the second component (336R') of the synthesis signal.