Audio-Guided Image Download for Immersive Video
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing immersive audiovisual content viewing devices face challenges in optimizing bandwidth usage while maintaining high-quality experiences, particularly when viewing panoramic content, as current methods either result in latency or excessive bandwidth usage due to unnecessary data transmission.
Innovation Solution
A method that determines the location of sound sources in spatialized audio content to anticipate user orientation changes, selecting and downloading image parts based on predicted viewing directions, thereby optimizing bandwidth by only transmitting data likely to be viewed.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of energy
If only the parts of the video necessary to display the image corresponding to the user's current orientation are downloaded, then bandwidth usage is reduced, but transmission time of the image parts increases causing delay in the display device
Solution Approach 1:
The patent applies preliminary action by anticipating future user viewing directions based on sound source locations and pre-downloading the corresponding image parts before they are actually needed. This allows the system to prepare content in advance, reducing transmission delays when users turn to look at sound sources while maintaining efficient bandwidth usage by only downloading anticipated regions.
2Loss of time
If additional parts of the image that are not in the user's field of view are downloaded from a low-quality representation, then latency is improved, but bandwidth usage increases to download images that will never be viewed
Solution Approach 1:
The patent applies local quality by differentiating download quality and quantity based on specific spatial regions. Image parts corresponding to sound source locations and anticipated viewing directions are downloaded in high quality, while other regions are either not downloaded or downloaded in lower quality. This selective approach optimizes both latency for important regions and overall bandwidth efficiency.
Solution Approach 2:
The system performs preliminary action by pre-downloading only the specific image regions that are likely to be viewed based on sound source analysis, rather than downloading entire unused portions. This targeted preliminary action reduces wasted bandwidth while ensuring low latency for relevant content.
3Manufacturing precision
If the bitrate of the video portions corresponding to the left side of the image is increased when a voice appears on the left, then quality experience is improved, but overall bandwidth usage increases
Solution Approach 1:
The patent implements local quality by dynamically adjusting video bitrate allocation based on spatial audio events. When a sound source is detected in a specific direction (e.g., left side), the system increases bitrate only for image portions in that direction while maintaining or reducing bitrate for other regions. This ensures high quality experience where needed while optimizing overall bandwidth consumption through spatially-selective quality adjustment.
Data Source
Figure 1a~1b
Figure 2
Figure 3
AI summary
Method for selecting at least one image portion to be downloaded anticipatorily in order to render an audiovisual stream. The invention relates to a method for selecting at least one image portion to be downloaded to render an audiovisual content by means of a rendering device, the method being such that it includes steps of determining the location of a sound source in a spatialized audio component of the audiovisual content, of determining a future observation direction on the basis of the determined location, of selecting at least one portion of the image on the basis of the determined future observation direction, and of downloading the at least one selected image portion.