Height Audio Object Separation for Immersive Multichannel Rendering
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing audio content captured with limited performance microphones, such as those in smartphones, lacks spatial acoustic properties when rendered in immersive multi-channel formats, resulting in a bland or non-immersive experience.
Innovation Solution
A method to extract height audio objects from input audio signals using source separation modules, which are then rendered to height channels in multi-channel presentations, enhancing spatial immersion.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If automatic post-processing techniques (speech separation, noise suppression, EQ-processing) are used to enhance UGC, then perceived intelligibility and quality are improved, but spatial acoustic properties are lost resulting in bland or non-immersive presentation
Solution Approach 1:
The audio signal is segmented into multiple audio objects representing different sound sources (speech, music, effects, background). Each audio object is independently processed for intelligibility enhancement while preserving its spatial characteristics. This allows selective processing of specific sound sources without destroying the overall spatial acoustic scene.
Solution Approach 2:
The system changes processing parameters based on the type of audio object. Different enhancement techniques are applied to different audio objects (e.g., speech separation for dialogue, noise suppression for background), and spatial parameters are maintained or enhanced rather than lost during processing.
2Ease of operation
If mono or stereo audio signals are captured with limited performance microphones, then ease of capture is improved, but spatial immersion is lost when rendered in immersive multi-channel formats
Solution Approach 1:
The system transforms mono or stereo audio signals into immersive multi-channel presentations by adding spatial dimensions. Audio objects are positioned in three-dimensional space with azimuth, elevation, and distance parameters, creating a virtual acoustic environment that provides spatial immersion even when captured with simple microphones.
Solution Approach 2:
The system creates virtual copies of audio sources in the spatial domain by synthesizing spatial acoustic properties that mimic real-world acoustic environments. This allows the reproduction of immersive spatial characteristics without requiring complex physical microphone arrays during capture.
3Measurement precision
If sophisticated manual post-processing by mixing engineers is performed on PGC, then well-balanced audio presentations are achieved, but device complexity and processing requirements increase
Solution Approach 1:
The system performs automatic audio object extraction, classification, and spatial positioning without requiring manual mixing engineering. Machine learning models automatically identify and separate different audio sources, assign them to appropriate audio objects, and position them in spatial space, replacing the need for sophisticated manual post-processing while reducing complexity.
Solution Approach 2:
Manual mixing operations are replaced with automated computational processes. Machine learning algorithms perform the functions that previously required skilled human operators, including source separation, audio object extraction, and spatial positioning, thereby reducing device complexity while maintaining or improving audio quality.
Data Source
AI summary
The present disclosure relates to a method and system for processing audio, as well as a computer program product comprising instructions which, when the program is executed by a computer, causes the computer to carry out the method. The method comprises obtaining an input audio signal and processing the input audio signal to extract a height audio object from the input audio signal, wherein the height audio object is extracted using a source separation module configured to extract an audio object of a predetermined height audio source type. The method further comprises rendering the input audio signal to a multi-channel presentation such that the at least one height audio object is included in at least one height channel of the multi-channel presentation.