Mediated Reality Audio Splitting for Coherent Multi-Device Rendering
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing technologies struggle to provide seamless integration of spatial audio rendering across multiple devices, particularly in mediated reality environments, leading to conflicts and suboptimal sound positioning when using hand-held devices and head-mounted audio output devices.
Innovation Solution
A system and method that splits spatial audio content into two parts, with one part rendered by a hand-held device and the other by a head-mounted audio output device, using a rendering control module to adjust sound sources based on user actions and device positions, ensuring first-person perspective-mediated reality.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If spatial audio content is rendered by a single device, then device complexity is reduced, but spatial audio coherence and positioning accuracy deteriorate
Solution Approach 1:
The patent divides spatial audio content into multiple parts and assigns different parts to different devices for rendering. The rendering control module separates audio processing between the hand-held device and head-mounted device, with each device handling specific spatial audio components based on its position and orientation, thereby achieving accurate spatial positioning while distributing system complexity across multiple devices.
2Manufacturing precision
If spatial audio content is split across multiple devices, then spatial audio coherence is improved, but device complexity increases
Solution Approach 1:
The rendering control module acts as an intermediary that coordinates between the hand-held device and head-mounted device. It receives spatial audio content, determines appropriate rendering assignments based on device positions and orientations, and distributes audio parts to each device. This intermediary manages the complexity of multi-device coordination while maintaining spatial audio coherence.
Solution Approach 2:
The system dynamically adjusts which device renders which part of spatial audio content based on real-time device positions and orientations. The rendering control module continuously monitors device states and reassigns audio rendering tasks accordingly, allowing the system to adapt to changing spatial configurations and maintain optimal spatial audio coherence without fixed complex routing.
3Ease of operation
If hand-held device renders all spatial audio, then ease of operation is improved, but spatial audio positioning accuracy deteriorates
Solution Approach 1:
The patent segments spatial audio rendering between the hand-held device and head-mounted device. The hand-held device handles audio components that benefit from its position and orientation, while the head-mounted device handles components requiring proximity to the user's head for accurate spatial perception. This segmentation maintains ease of operation through automatic device coordination while improving sound source positioning accuracy by utilizing each device's spatial advantages.
Data Source
Figure 1A~1F
Figure 2~3C
Figure 4A~6C
AI summary
An apparatus comprising means for: simultaneously controlling content rendered by a hand portable device and content rendered by a spatial audio device; and providing for rendering to a user, in response to an action by the user, of a first part, not a second part, of a spatial audio content via the hand portable device not the spatial audio device.