Optical Flow Mipmap Adjustment for Cloud Gaming Bandwidth
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current graphics processing systems face challenges in efficiently managing memory bandwidth when applying high-quality textures, especially in scenarios with high relative velocities or motion blur, leading to unnecessary resource consumption and quality degradation.
Innovation Solution
Implementing optical flow mipmap level of detail adjustment techniques to dynamically adjust texture quality based on motion, rendering mipmap levels in a separate buffer and using optical flow data to preserve quality in high-detail areas while reducing it in low-detail areas, thereby optimizing memory usage and maintaining visual fidelity.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If high quality textures are applied to objects in a scene, then rendering quality is improved, but memory bandwidth consumption increases
Solution Approach 1:
The patent applies different texture quality levels to different regions of the scene based on their visual importance. High-quality textures are applied only to objects in regions where they will be visible and contribute to the final image, while low-quality or no textures are applied to objects in regions that will be occluded or blurred by motion. This selective application of quality levels reduces overall memory bandwidth consumption while maintaining rendering quality in visible areas.
Solution Approach 2:
The patent dynamically adjusts texture quality levels based on real-time scene conditions, including object velocity, camera motion, and occlusion relationships. As objects move through different regions of the scene or change velocity, their texture quality is adjusted accordingly. This dynamic adaptation allows the system to optimize memory bandwidth usage while maintaining high rendering quality when needed and reducing quality (and thus bandwidth consumption) when high quality would be imperceptible.
2Manufacturing precision
If high quality textures are applied to moving objects, then rendering quality is improved, but memory bandwidth is wasted when motion blur obscures detail
Solution Approach 1:
The patent performs preliminary analysis of scene geometry, object velocities, and camera motion to predict which objects will be subject to motion blur or occlusion before rendering. Based on this preliminary analysis, texture quality levels are pre-adjusted for objects that will be blurred or hidden, preventing the waste of memory bandwidth on textures that would not be visible in the final image. This predictive approach allows the system to allocate memory bandwidth efficiently before the actual rendering process begins.
3Quantity of substance
If mipmap levels are reduced for memory optimization, then memory bandwidth is conserved, but visual fidelity deteriorates in high-detail areas
Solution Approach 1:
The patent selectively applies different mipmap levels to different objects and regions of the scene based on their visual importance and the likelihood of being visible. High mipmap levels (high detail) are applied only to objects in regions where they will be visible and contribute significantly to the final image, while low mipmap levels (low detail) are applied to objects that will be occluded or blurred by motion. This selective quality adjustment conserves memory bandwidth while maintaining visual fidelity in areas where high detail is actually needed.
Solution Approach 2:
The patent uses feedback from scene analysis, including occlusion relationships, object velocities, and camera motion, to dynamically adjust mipmap level selection. The system continuously monitors which objects are visible and which are subject to motion blur or occlusion, and adjusts texture quality levels accordingly. This feedback mechanism ensures that memory bandwidth is conserved in areas where high detail would be imperceptible while maintaining visual fidelity in areas where high detail contributes to the final image quality.
Data Source
AI summary
Described herein are techniques to perform optical flow mipmap level of detail adjustment for rendering and performing cloud gaming encoding adjustment based on mipmap level of detail. One embodiment provides a graphics processor comprising first circuitry to process input data via a processing resource, the first circuitry to render a first frame of a scene of a three dimensional environment, determine motion data between the first frame and a second frame, determine speeds for pixels and objects in the scene based on the determined motion data, adjust a mipmap level of detail (LOD) associated with the objects based on their motion relative to a motion threshold, and render the objects in a third frame of the scene with an adjusted mipmap LOD.