Spatial Audio Metadata Interpolation for 6DoF Listener Movement
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing spatial audio capture technologies struggle to provide high-quality spatial audio rendering for 6 degrees of freedom systems, as they require specialized microphone arrays and fail to accurately render sound sources when the listener moves outside the capture area, leading to directional inaccuracies and decreased engagement.
Innovation Solution
A method that uses multiple microphone arrays to determine prominent sound source positions, direct sound contributions, and residual spatial information, allowing for accurate spatial audio rendering even when the listener moves outside the capture area, by combining direct and residual spatial information for precise 6DoF binaural audio reproduction.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If linear spatial audio capture is implemented using high-end microphone arrays (e.g., spherical 32-microphone Eigenmike), then spatial audio rendering quality is improved, but device complexity and cost increase significantly
Solution Approach 1:
The patent replaces expensive, complex high-end microphone arrays with inexpensive, compact microphone arrangements that can be integrated into common devices like VR cameras and mobile phones. This substitution maintains adequate spatial audio capture capability while dramatically reducing device complexity and cost.
Solution Approach 2:
The patent extracts and utilizes prominent sound source positions and direct sound contributions from the captured audio signal, separating them from the residual spatial information. This extraction allows the system to focus computational resources on the most critical spatial components, achieving good rendering quality without requiring complex hardware.
2Device complexity
If parametric spatial audio capture is used with compact microphone arrangements, then device complexity is reduced, but spatial audio accuracy deteriorates when listener moves outside capture area
Solution Approach 1:
The patent performs preliminary analysis during the capture phase to identify and store prominent sound source positions and direct sound contributions. This pre-processing allows the system to later reconstruct accurate spatial audio for listener positions outside the original capture area by leveraging the stored sound source information combined with residual spatial data from multiple microphone arrangements.
Solution Approach 2:
The patent creates a universal spatial audio rendering system that works with various compact microphone arrangements (VR cameras, mobile phones, SLR cameras) rather than requiring specialized high-end arrays. The method adapts to different device configurations while maintaining spatial audio accuracy through the combination of sound source position estimation and residual spatial information.
3Adaptability or versatility
If multiple microphone arrays are deployed to capture spatial audio from different positions, then spatial coverage is improved, but device complexity and data processing requirements increase
Solution Approach 1:
The patent merges data from multiple microphone arrangements by combining direct sound contributions from identified sound sources with residual spatial information from each arrangement. This fusion creates a comprehensive spatial audio representation that covers a broader area while avoiding the need to process and store complete separate audio streams from each microphone array, thus managing data processing requirements efficiently.
Data Source
AI summary
An apparatus including circuitry configured to: obtain two or more audio signal sets, wherein each audio signal set is associated with a position; obtain at least one parameter value for at least two of the audio signal sets; obtain the positions associated with the at least two of the audio signal sets; obtain a listener position; obtain source position information; obtain values related to source energies associated with the source position information; generate at least one audio signal based on at least one audio signal from at least one of the audio signal sets based on the positions associated with the at least two of the audio signal sets and the listener position; generate at least one modified parameter value and a residual value; and process the at least one audio signal to generate a spatial audio output.