Binaural Audio Object Extraction Using Frequency-Dependent Cues
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing audio post-processing systems for binaural audio fail to effectively utilize frequency-dependent level and time differences, leading to suboptimal manipulation and separation of objects in binaural audio signals.
Innovation Solution
A method involving signal transformation, spatial analysis, and object extraction to estimate and process binaural audio signals using frequency-dependent level and time differences, allowing for separate processing of main and residual components.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of manufacture
If linear gains are used for signal decomposition and upmixing, then the processing is simple and works well for channel-based signals, but the processing does not work well for binaural audio which has frequency-dependent level and time differences
Solution Approach 1:
The patent changes the processing parameters from simple linear gains to frequency-dependent transfer functions. The system estimates frequency-dependent level differences and time differences for different frequency bands, then applies these frequency-specific parameters to separate and process binaural audio objects, resolving the contradiction between processing simplicity and effectiveness.
Solution Approach 2:
The patent segments the binaural signal into different frequency bands and processes each band separately with frequency-appropriate parameters. By dividing the signal processing into multiple frequency-specific components, the system achieves both manageable complexity and high effectiveness for binaural audio manipulation.
2Ease of operation
If existing spatial processing methods are applied to the full mix or coherent element only, then the processing is straightforward, but the system cannot separate and process different binaural objects differently
Solution Approach 1:
The patent extracts individual binaural objects from the mixed audio signal by estimating their spatial parameters (level differences and time differences) in different frequency bands. This extraction enables the system to separate multiple objects and apply different processing to each, achieving both operational straightforwardness and processing versatility.
Solution Approach 2:
The patent applies different processing parameters to different spatial locations and frequency bands. By estimating frequency-dependent spatial parameters for each object and applying location-specific processing, the system achieves straightforward operation with high adaptability to handle different binaural objects differently.
Data Source
Figure 1~3A
Figure 3B~7
Figure 6
AI summary
A method of audio processing includes performing spatial analysis on a binaural signal to estimate level differences and phase differences characteristic of a binaural filter of the binaural signal, performing object extraction on the binaural audio signal using the estimated level and phase differences to generate a left/right main component signal and a left/right residual component signal. The system may process the left/right main and left/right residual components differently using different object processing parameters for e.g. repositioning, equalization, compression, upmixing, channel remapping or storage to generate a processed binaural signal that provides an improved listening experience. Repositioning may be based on head tracking sensor data.