XR Content Synthesis via Spatial Quality Adaptation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current content processing and streaming schemes for extended reality applications, especially those with 6DoF capabilities, require a high density of pre-rendered view areas (PRVAs) to maintain an immersive experience, leading to significant bandwidth and storage costs due to the exponential increase in resource usage for multi-directional content.
Innovation Solution
The method involves synthesizing views for a user's position in a modelled 3D space using pre-rendered content from multiple viewing areas (VAs) of different qualities, where the quality of content is determined based on distance and orientation, allowing for reduced bandwidth usage without degrading the quality of synthesized views by using alpha-blending or similar interpolation algorithms.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If a high density of pre-rendered view areas (PRVAs) is used to maintain immersive experience quality, then the quality of the extended reality application is improved, but bandwidth consumption and storage requirements increase significantly
Solution Approach 1:
The patent applies local quality by assigning different quality levels to different PRVAs based on their spatial relationship to the user. Specifically, PRVAs closer to the user's current position and viewing direction are rendered at higher quality, while those farther away are rendered at lower quality. This spatially-adaptive quality distribution maintains the immersive experience where it matters most while reducing overall bandwidth consumption and storage requirements.
Solution Approach 2:
The patent changes the quality parameter of PRVAs dynamically based on user position and viewing direction. The system adjusts resolution, bitrate, and detail levels of pre-rendered content according to the user's location in the 3D space, transforming a static quality approach into a dynamic one that adapts to real-time user interaction patterns.
2Manufacturing precision
If a high density of pre-rendered view areas (PRVAs) is used to maintain immersive experience quality, then the quality of the extended reality application is improved, but storage requirements increase significantly
Solution Approach 1:
The patent reduces storage requirements by storing pre-rendered content at varying quality levels across different spatial locations. Instead of uniformly high-quality storage across all PRVAs, the system stores only the necessary quality level for each location based on its importance to the user experience, thereby reducing total storage volume while maintaining local quality where needed.
Solution Approach 2:
The patent implements partial action by selectively rendering and storing only the most critical PRVAs at high quality, while using lower quality or synthesized versions for less critical areas. This partial approach to quality distribution avoids the excessive storage requirements of uniform high-quality rendering across all possible viewpoints.
3Quantity of substance
If content of different quality versions is used for synthesis, then bandwidth consumption is reduced, but the complexity of selecting and processing content increases
Solution Approach 1:
The patent applies preliminary action by pre-calculating and storing quality selection criteria and synthesis parameters for different user positions and viewing directions. This pre-computation of selection logic reduces the real-time processing complexity, as the system only needs to look up and apply pre-determined quality levels rather than making complex decisions during runtime.
Solution Approach 2:
The patent uses copying by creating simplified representations or proxies of the complex quality selection process. Instead of implementing full-blown real-time quality optimization algorithms, the system uses pre-computed lookup tables, template-based selection, or simplified heuristics that replicate the essential behavior of complex quality management with reduced processing overhead.
Data Source
Figure 1A~1B
Figure 2
Figure 3
AI summary
Methods and systems of processing content by a client are described wherein the method includes: receiving scene information defining two or more viewpoints in a scene, each of the two or more viewpoints being associated with an area in the scene and with a plurality of quality versions of video content, the video content defining one or more views in the scene associated with a viewpoint; receiving position information associated with a user of a playback device, the position information defining at least a user position and, optionally, a viewing direction of the user in the scene; if the user position is outside the areas associated with the two or more viewpoints, then selecting first content of a first quality associated with a first viewpoint and second content of a second quality associated with a second viewpoint in the scene based on the position information and the scene information; sending a request to a server system, the request comprising information about the first content of the first quality and the second content of the second quality and, optionally, the position and/or viewing direction of the user in the scene; and, receiving, in response to the request, video frames comprising the first and second content for generating synthesized content on the basis of the first and second content, wherein the synthesized content is associated with the position of the user in the scene; or, receiving video frames comprising synthesized content associated with the position of the user in the scene, wherein the synthesized content is synthesized based on the first and second content.