Spatial Audio Playback with Auxiliary Elements for 6DoF Continuity
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing spatial audio technologies struggle to provide a seamless audio experience when users perform translational and rotational movements in extended reality environments, particularly in 6 DoF scenarios, as they often result in audio glitches or delays due to incomplete audio data when users zoom or move beyond the perceptual scope of delivered audio sets.
Innovation Solution
The system augments audio elements within defined perceptual scopes by adding auxiliary audio elements that are downmixed versions of elements outside the scope, ensuring continuous audio rendering by pre-creating these elements and seamlessly transitioning to full audio sets as needed.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of energy
If audio elements are delivered only within a defined perceptual scope, then data transmission efficiency is improved, but audio continuity deteriorates when users move or zoom beyond the scope boundaries
Solution Approach 1:
The system pre-creates auxiliary audio elements that are downmixed versions of audio content located outside the current perceptual scope. These auxiliary elements are prepared in advance and stored for quick retrieval, enabling seamless audio transitions when users move or zoom beyond the original audio set boundaries without requiring real-time processing of full audio data
2Reliability
If auxiliary audio elements are pre-created and stored, then audio continuity is improved during user movement, but device complexity increases
Solution Approach 1:
Instead of uniformly processing or storing all audio elements, the system applies different quality levels to different spatial regions. Auxiliary audio elements outside the perceptual scope are stored as downmixed versions with reduced complexity, while elements within the scope maintain full quality. This localized quality differentiation reduces overall system complexity while maintaining audio continuity
3Duration of action of stationary object
If downmixed auxiliary audio elements are used outside perceptual scope, then audio playback continuity is improved, but audio quality deteriorates compared to full audio sets
Solution Approach 1:
The system dynamically switches between different audio element sets based on user position and zoom level. When users are within the perceptual scope, full-quality audio elements are used. When users move or zoom beyond boundaries, the system transitions to downmixed auxiliary elements. This dynamic adaptation maintains playback continuity while optimizing quality based on actual playback needs
Data Source
AI summary
An apparatus including circuitry configured to: obtain at least one location and/or orientation associated with a user; obtain, based on the at least one location and/or orientation, one or more audio element, wherein the one or more audio element at least partially forms an audio scene; obtain, based on the at least one location and/or orientation, at least one auxiliary audio element, the at least one auxiliary audio element being at least one audio element or a combination of audio elements, wherein the at least one auxiliary audio element is associated with at least a part of the audio scene and is located at an outside zone of the audio scene; render the obtained audio element and/or at least one auxiliary audio element.