Audio Coding Residual Scaling for Temporal Mismatch
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In audio signal encoding, temporal mismatches between multiple microphones lead to misalignment of audio signals, affecting coding efficiency and introducing inter-harmonic noise, especially when channels are not properly aligned during the DFT process.
Innovation Solution
A device and method that perform transform operations on reference and target channels to determine and adjust for inter-channel mismatch, generating adjusted frequency-domain channels for down-mixing and residual scaling to minimize noise and improve alignment, thereby encoding mid and side channels efficiently.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If multiple microphones are used to receive audio signals, then the audio capture capability is improved, but temporal misalignment between channels occurs due to different distances from the sound source
Solution Approach 1:
The patent applies preliminary action by estimating the inter-channel time difference (ICTD) before performing the down-mix operation. The ICTD estimation is performed on the original audio signals from multiple microphones, and this estimated time difference is then used to pre-adjust the phase of one channel relative to the other before mixing. This preliminary phase adjustment ensures that the signals are temporally aligned at the moment of down-mixing, thereby resolving the temporal misalignment issue while maintaining the benefit of using multiple microphones for improved audio capture capability.
2Device complexity
If temporal misalignment is not corrected, then the coding process is simpler, but inter-harmonic noise increases and coding efficiency deteriorates
Solution Approach 1:
The patent applies parameter changes by modifying the phase parameter of the audio signals based on the estimated ICTD. Specifically, the phase of one channel is adjusted by a factor derived from the ICTD estimation, which effectively aligns the temporal parameters of the two channels. This parameter adjustment is performed in the frequency domain and integrates seamlessly into the existing coding process, reducing inter-harmonic noise without significantly increasing overall system complexity.
3Adaptability or versatility
If ICTD estimation is performed in the frequency domain, then the processing is more flexible, but the computational complexity increases
Solution Approach 1:
The patent applies mechanics substitution by replacing complex time-domain correlation methods with a more computationally efficient frequency-domain approach. Instead of performing time-consuming cross-correlation operations in the time domain to estimate ICTD, the patent transforms the audio signals into the frequency domain using FFT, where phase differences can be directly measured and used to derive the ICTD. This substitution of the estimation method reduces computational complexity while maintaining processing flexibility and accuracy.
Data Source
Figure 2
Figure 3
Figure 4
AI summary
A residual scaling unit is configured to determine a scaling factor for a residual channel based on an inter-channel mismatch value. The inter-channel mismatch value is indicative of a temporal alignment between a reference channel and a target channel. The residual scaling unit is further configured to scale (e.g., attenuate) the residual channel by the scaling factor to generate a scaled residual channel. A residual channel encoder is configured to encode the scaled residual channel as part of a bitstream.