Adaptive IVAS Downmixing for Low-Channel Spatial Audio
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing immersive voice and audio services (IVAS) codecs face challenges in reconstructing immersive audio scenes with low channel configurations due to imperfect decorrelation and limited quantization range, leading to audio artifacts and spatial distortion.
Innovation Solution
Implementing adaptive downmix strategies that dynamically adjust input downmixing gains based on signal properties and voice activity, using active downmix schemes to minimize prediction errors and ensure better parameter estimation, while preserving audio energy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If passive downmix scheme is used where dominant eigen signal is a delayed version of center channel, then device complexity is reduced, but manufacturing precision of audio reconstruction deteriorates due to imperfect decorrelation and spatial distortion
Solution Approach 1:
The patent implements dynamic downmixing schemes that adapt between passive and active approaches based on voice activity detection. During voice frames, active downmixing with optimized eigen signal computation is applied to maintain spatial precision, while during non-voice frames, passive downmixing is used to reduce complexity. This dynamic adaptation resolves the contradiction by adjusting the processing level according to actual audio content requirements.
Solution Approach 2:
The patent changes key parameters of the downmixing process including downmix gains, eigen signal computation methods, and decorrelation parameters based on detected voice activity. By adjusting these parameters dynamically, the system achieves high reconstruction precision during voice frames while maintaining low complexity during non-voice frames, thus resolving the contradiction between complexity and precision.
2Manufacturing precision
If active downmix scheme is used with eigen signal obtained by scaling and adding channels, then audio reconstruction precision is improved, but device complexity increases due to additional processing requirements
Solution Approach 1:
The system dynamically selects between passive and active downmixing schemes based on voice activity detection. Active downmixing with full eigen signal computation is applied only during voice frames where high precision is critical, while passive downmixing is used during non-voice frames to reduce processing complexity. This dynamic approach resolves the contradiction by applying complex processing only when necessary.
Solution Approach 2:
The audio signal is segmented into voice frames and non-voice frames through voice activity detection. Different downmixing strategies are applied to different segments: active downmixing for voice segments requiring high precision, and passive downmixing for non-voice segments where complexity reduction is prioritized. This segmentation resolves the contradiction by localizing complex processing to specific temporal regions.
3Device complexity
If downmixing is applied to reduce channel configuration, then device complexity is reduced, but reliability of immersive audio rendering deteriorates due to audio artifacts and spatial distortion
Solution Approach 1:
The patent implements dynamic adaptation of downmixing strategies based on voice activity detection. During voice frames, active downmixing with optimized eigen signal computation and decorrelation parameter adjustment is applied to maintain rendering reliability. During non-voice frames, simplified passive downmixing is used to reduce complexity. This dynamic approach ensures reliability is maintained when needed while allowing complexity reduction when possible.
Solution Approach 2:
The system uses voice activity detection as feedback to determine the appropriate downmixing strategy. The detection of voice vs. non-voice conditions provides feedback that drives the selection between active and passive downmixing schemes, ensuring that rendering reliability is maintained during critical voice transmission while allowing complexity reduction during less critical periods.
4Device complexity
If quantization range is limited for downmix parameters, then device complexity is reduced, but measurement precision of audio parameters deteriorates leading to spatial distortion
Solution Approach 1:
The patent dynamically adjusts quantization parameters and precision levels based on voice activity detection. During voice frames, higher precision quantization is applied to downmix parameters and eigen signal coefficients to maintain spatial accuracy. During non-voice frames, coarser quantization is used to reduce complexity. This adaptive parameter adjustment resolves the contradiction by matching precision requirements to actual content needs.
Data Source
AI summary
Disclosed is an audio signal encoding/decoding method that uses an encoding downmix strategy applied at an encoder that is different than a decoding re-mix/upmix strategy applied at a decoder. Based on the type of downmix coding scheme, the method comprises: computing input downmixing gains to be applied to the input audio signal to construct a primary downmix channel; determining downmix scaling gains to scale the primary downmix channel; generating prediction gains based on the input audio signal, the input downmixing gains and the downmix scaling gains; determining residual channel(s) from the side channels by using the primary downmix channel and the prediction gains to generate side channel predictions and subtracting the side channel predictions from the side channels; determining decorrelation gains based on energy in the residual channels; encoding the primary downmix channel, the residual channel(s), the prediction gains and the decorrelation gains; and sending the bitstream to a decoder.