Foveated Mesh Rendering for Central Clarity in Immersive Video
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Rendering immersive video content while optimizing video quality is a difficult and resource-intensive task, particularly in immersive environments such as virtual and augmented reality, where existing methods fail to efficiently manage pixel density and resolution.
Innovation Solution
The use of foveated meshes, which increase pixel density in central portions of video content by trading off pixel density in peripheral areas, is implemented through mechanisms that generate non-uniform UV or three-dimensional position maps using polynomial fitting functions to deform meshes like cropped equirectangular meshes.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If uniform pixel density is used across the entire video content, then the rendering is simple and resource-efficient, but the perceived video quality in central regions is insufficient
Solution Approach 1:
The patent applies different pixel density characteristics to different regions of the video content. The central region uses higher pixel density to improve perceived video quality, while peripheral regions use lower pixel density to reduce computational complexity and resource consumption. This local differentiation resolves the contradiction by making pixel density adaptive rather than uniform.
Solution Approach 2:
The patent dynamically adjusts pixel density based on the content characteristics and viewing conditions. The system can switch between foveated rendering modes and uniform rendering modes depending on the situation, making the pixel density distribution flexible and adaptive rather than fixed, thus resolving the contradiction between quality and complexity.
2Measurement precision
If high pixel density is applied across the entire video, then the perceived video quality is high, but the computational resources and processing time increase significantly
Solution Approach 1:
Instead of applying high pixel density uniformly across the entire video, the patent concentrates computational resources on the central region where viewers naturally focus attention. The peripheral regions are rendered at lower pixel density, significantly reducing the overall computational load while maintaining high perceived quality in the critical central area.
Solution Approach 2:
The patent applies high pixel density partially only to the extent necessary for perceived quality (central region) rather than excessively across the entire video. This partial application of high resolution achieves the minimum necessary quality threshold without the excessive computational cost of uniform high-resolution rendering.
3Measurement precision
If foveated mesh is used to increase central pixel density, then the perceived video quality in central region is improved, but the pixel density in peripheral areas is reduced
Solution Approach 1:
The patent intentionally creates local quality differences where the central region has high pixel density for clarity and the peripheral regions have lower pixel density. This local differentiation is acceptable because human vision is less sensitive to quality in peripheral areas, thus resolving the contradiction between central clarity and peripheral pixel density.
Solution Approach 2:
The patent uses a foveated mesh that copies and redistributes pixel density from peripheral regions to the central region. By warping the mesh, it effectively copies the visual information from lower-density peripheral areas and concentrates it in the central region, maintaining overall information content while improving central clarity.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
Methods, systems, and media for rendering immersive video content with foveated meshes are provided. In some embodiments, the method comprises: receiving a video content item; determining, using a hardware processor, whether the video content item meets at least one criterion; in response to determining that the video content item meets the at least one criterion, generating, using the hardware processor, a foveated mesh in accordance with a foveation ratio parameter on which frames of the video content item are to be projected, wherein the foveated mesh has a non-uniform position map that increases pixel density in a central portion of each frame of the video content item in comparison with peripheral portions of each frame of the video content item; and storing the video content item in a file format that includes the generated foveated mesh, wherein the immersive video content is rendered by applying the video content item as a texture to the generated foveated mesh.