3D Audio Rendering by Viewing Direction to Cut XR Processing Load
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Providing immersive audio experiences on devices with limited wireless bandwidth, processing resources, or battery capacity is challenging due to the large amount of data required and substantial processing operations needed for audio scene updates.
Innovation Solution
Adjusting the resolution of audio processing based on the user's viewing direction, using higher resolution for audio sources near the gaze direction and lower resolution for those farther away, to reduce processing load, latency, and bandwidth requirements.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If high quality audio rendering is applied to all audio sources, then audio quality is improved, but processing load and bandwidth requirements increase
Solution Approach 1:
The patent applies different rendering qualities to different audio sources based on their spatial position relative to the user's field of view. Audio sources within the central field of view receive high quality rendering, while sources in peripheral or out-of-view regions receive reduced quality rendering. This local differentiation resolves the contradiction by concentrating computational resources where they most impact user experience.
Solution Approach 2:
The system performs full audio scene updates and binauralization only for audio objects within the user's field of view, while using simplified or skipped updates for objects outside the field of view. This partial application of processing actions maintains high quality where needed while reducing overall processing load to manageable levels.
2Reliability
If audio scene updates and binauralization are performed for all audio sources, then immersive audio experience is improved, but battery capacity is depleted faster
Solution Approach 1:
Computational operations are concentrated on audio sources within the user's field of view, while energy-intensive processing is avoided for sources outside the field of view. This localized processing approach maintains immersive experience quality in the critical listening region while significantly reducing overall power consumption.
Solution Approach 2:
The system performs complete audio processing only for necessary audio objects within the field of view, skipping or simplifying processing for unnecessary objects. This partial execution of processing operations maintains user experience reliability while reducing energy consumption to extend battery life.
3Manufacturing precision
If high quality audio data is transmitted for all spatial regions, then audio quality is improved, but wireless bandwidth is consumed faster
Solution Approach 1:
The patent transmits high quality audio data only for spatial regions corresponding to the user's field of view, while using reduced quality or no transmission for regions outside the field of view. This spatially-aware data transmission strategy maintains audio quality where it matters most while optimizing wireless bandwidth utilization.
Solution Approach 2:
The system transmits complete audio data only for audio objects within the user's field of view, while using compressed or skipped transmission for objects outside the field of view. This partial data transmission approach maintains high quality experience in the critical listening region while reducing overall bandwidth consumption.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A device includes a memory configured to store instructions and also includes one or more processors configured to execute the instructions to obtain audio data corresponding to a sound source and metadata indicative of a direction of the sound source. The one or more processors are configured to execute the instructions to obtain direction data indicating a viewing direction associated with a user of a playback device. The one or more processors are configured to execute the instructions to determine a resolution setting based on a similarity between the viewing direction and the direction of the sound source. The one or more processors are also configured to execute the instructions to process the audio data based on the resolution setting to generate processed audio data.