Residual Channel Scaling for Misaligned Stereo Audio Coding

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

In audio signal encoding, temporal mismatches between multiple microphones lead to misalignment of audio signals, reducing the efficiency of stereo coding techniques like Mid-Side (MS) and Parametric Stereo (PS), resulting in decreased coding gains due to comparable energies in sum and difference channels.

Innovation Solution

A device and method that perform transform operations on reference and target audio channels to determine inter-channel mismatch values, adjust the target channel to align with the reference channel, and encode the mid and side channels, along with a scaled residual channel, to mitigate temporal misalignment and enhance coding efficiency.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Adaptability or versatility

If multiple microphones are used to receive audio signals, then the audio capture capability is improved, but temporal misalignment between channels occurs due to different distances from sound sources

Engineering Contradiction:

Improveaudio capture capabilityVSAvoidtemporal alignment precision

Core Design Contradiction:

Adaptability or versatilityVSManufacturing precision

Solution Approach 1:

The system performs preliminary temporal alignment by determining an inter-channel mismatch value before performing the down-mix operation. This preliminary correction of temporal misalignment ensures that the mid and side channels are properly synchronized, resolving the contradiction between using multiple microphones and maintaining temporal alignment precision.

Inventive Principle:

Principle #10Preliminary action

2Adaptability or versatility

If temporal misalignment is large between channels, then more microphones can be used for better spatial audio, but analysis and synthesis windows in DFT parameter estimation become mismatched

Engineering Contradiction:

Improvespatial audio capabilityVSAvoidDFT parameter estimation accuracy

Core Design Contradiction:

Adaptability or versatilityVSMeasurement precision

Solution Approach 1:

The system performs preliminary temporal alignment by determining an inter-channel mismatch value and adjusting the target channel before performing the down-mix operation and subsequent DFT parameter estimation. This preliminary correction ensures that analysis and synthesis windows remain properly matched even when using multiple microphones for spatial audio capture.

Inventive Principle:

Principle #10Preliminary action

3Productivity

If stereo coding is performed on misaligned audio signals, then the coding process can proceed, but coding gains decrease due to comparable energies in sum and difference channels

Engineering Contradiction:

Improvecoding efficiencyVSAvoidcoding gain

Core Design Contradiction:

ProductivityVSLoss of information

Solution Approach 1:

The system performs preliminary temporal alignment by determining an inter-channel mismatch value and adjusting the target channel before performing the down-mix operation to generate mid and side channels. This preliminary correction ensures that the mid and side channels have appropriate energy distribution, maximizing coding gains while maintaining coding efficiency.

Inventive Principle:

Principle #10Preliminary action

Data Source

PatentUS10438598B2Coding of multiple audio signals

Publication Date: 2019.10.08 QUALCOMM INC

AI summary

A residual scaling unit is configured to determine a scaling factor for a residual channel based on an inter-channel mismatch value. The inter-channel mismatch value is indicative of a temporal misalignment between a reference channel and a target channel. The residual scaling unit is further configured to scale (e.g., attenuate) the residual channel by the scaling factor to generate a scaled residual channel. A residual channel encoder is configured to encode the scaled residual channel as part of a bitstream.