Tiled Layer Composition to Reduce Remote Rendering Latency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing ER systems face latency and computational inefficiencies due to the need for per-pixel processing of multiple image layers, leading to user discomfort and resource-intensive operations that fail to scale with increasing layer counts.
Innovation Solution
Implementing a tiled composition approach that uses guidance composition meshes to restrict the consideration of layers during image composition, performing late stage reprojection and occlusion culling to optimize shader selection and reduce unnecessary processing.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If per-pixel processing of multiple image layers is performed, then image composition quality is improved, but processing time and computational resources increase significantly
Solution Approach 1:
The patent divides the image composition process into two distinct stages: a coarse culling stage that processes image layers at a lower resolution to identify potentially visible layers, and a fine composition stage that processes only those identified layers at full resolution. This segmentation reduces the total computational burden while maintaining final image quality.
Solution Approach 2:
The patent performs processing at two different levels of detail: an initial partial processing at coarse resolution to eliminate obviously hidden layers, followed by complete processing only for layers that pass the culling stage. This avoids the excessive action of processing all layers at full resolution while ensuring no visible layers are missed.
2Measurement precision
If all image layers are processed during composition, then visibility accuracy is improved, but computational resources and complexity increase
Solution Approach 1:
The patent segments the layer processing into a culling phase using simplified depth comparisons at coarse resolution, and a composition phase using full-resolution data only for layers that pass culling. This reduces computational complexity while preserving visibility accuracy for actually visible layers.
Solution Approach 2:
The patent extracts and processes only the subset of layers that are potentially visible, separating them from the complete set of all layers. By taking out only the relevant layers for final composition, the system reduces computational complexity without sacrificing visibility accuracy.
3Manufacturing precision
If high resolution processing is performed on all layers, then image quality is improved, but energy consumption and processing overhead increase
Solution Approach 1:
The patent segments processing resolution by stage: coarse resolution for culling and full resolution only for final composition of visible layers. This reduces total energy consumption while maintaining final image quality, as high-resolution processing is applied only where necessary.
Solution Approach 2:
The patent applies full high-resolution processing only partially to the subset of layers that are actually visible, rather than excessively applying it to all layers. This reduces energy consumption while preserving image quality for the visible portions of the scene.
Data Source
Figure 1
Figure 2
Figure 3A
AI summary
Techniques for performing tiled composition to intelligently restrict which layers are considered when performing an image composition process are disclosed. A service accesses, for each of multiple image layers, a corresponding color image and a corresponding depth image. The service performs LSR on the image layers to produce corresponding reprojected color images and corresponding reprojected depth images. The service uses the LSR's correction matrix to generate a set of guidance composition meshes. The service uses the guidance composition meshes to guide performance of the image composition process.