Object-Based Audio Submixing That Preserves Spatial Metadata
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing audio processing algorithms fail to preserve metadata when converting object-based audio signals to channel-based signals, compromising playback performance and immersive audio experiences.
Innovation Solution
A method and system that calculate panning coefficients based on spatial metadata to convert object-based audio signals into submixes, applying audio processing to these submixes and controlling object gains, preserving metadata for accurate and immersive rendering.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If object-based audio signals are converted to channel-based signals for processing, then compatibility with conventional processing algorithms is achieved, but metadata associated with position, velocity, size and other audio object properties is lost
Solution Approach 1:
The patent introduces an intermediary rendering process that converts object-based audio signals to channel-based signals while preserving metadata through side channels. The rendering algorithm acts as a mediator that maintains the connection between the original object-based format and the conventional processing pipeline, allowing metadata to be retained alongside the converted audio signal.
Solution Approach 2:
The patent embeds metadata within the channel-based audio signal structure through nesting. The metadata associated with audio objects is integrated into the downmixed signal in a nested manner, allowing it to travel through the conventional processing pipeline without being lost, while the audio signal itself is processed using standard algorithms.
2Loss of information
If object-based audio signals are directly processed without rendering to channel-based signals, then metadata is preserved, but compatibility with existing audio processing algorithms is lost
Solution Approach 1:
The patent segments the audio processing into distinct components: the audio signal path and the metadata path. This segmentation allows the audio signal to be rendered to channel-based format for compatibility with existing algorithms, while the metadata is separately preserved and maintained alongside the processed signal, enabling both goals to be achieved simultaneously.
3Productivity
If audio objects are downmixed into submixes for processing, then computational complexity is reduced, but spatial precision and immersive audio experience are compromised
Solution Approach 1:
The patent applies partial downmixing where audio objects are grouped into submixes based on their spatial characteristics and processing requirements. Not all objects are downmixed to the same extent - some are processed in submixes for efficiency while others maintain higher spatial precision. This partial action approach balances computational efficiency with spatial rendering quality.
Data Source
AI summary
An audio processing system and method which calculates, based on spatial metadata of the audio object, a panning coefficient for each of the audio objects in relation to each of a plurality of predefined channel coverage zones. Converts the audio signal into submixes in relation to the predefined channel coverage zones based on the calculated panning coefficients and the audio objects. Each of the submixes indicating a sum of components of the plurality of the audio objects in relation to one of the predefined channel coverage zones. Generating a submix gain by applying an audio processing to each of the submix and controls an object gain applied to each of the audio objects. The object gain being as a function of the panning coefficients for each of the audio objects and the submix gains in relation to each of the predefined channel coverage zones.