Tiled Layer Composition with Guidance Meshes for Remote Rendering
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing extended reality (ER) systems suffer from latency and user discomfort due to inefficient late stage reprojection (LSR) processes, which fail to scale well with increasing layer counts, leading to significant computational resource consumption and reduced performance.
Innovation Solution
Implement a tiled composition approach that uses guidance composition meshes to intelligently restrict which layers are considered during image composition, performing late stage reprojection (LSR) on multiple layers and generating a set of guidance composition meshes to optimize shader selection and reduce unnecessary processing, thereby improving computational efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If traditional image composition processes consider all image layers, then image quality is maintained, but computational resource consumption increases and rendering speed decreases
Solution Approach 1:
The patent divides the image composition process into tiled regions and segments the consideration of image layers based on depth information. Instead of processing all layers uniformly across the entire image, the system segments both the spatial domain (tiles) and the layer domain (based on depth), allowing selective processing of only relevant layers in each tile region. This segmentation dramatically reduces computational resources while maintaining visual quality.
Solution Approach 2:
The patent applies local quality by determining which image layers are relevant based on local depth information in each tile region. The system evaluates depth values locally to identify occlusion relationships, allowing different processing strategies for different regions of the image. This ensures high processing efficiency in regions where layers can be culled while maintaining full quality in regions requiring detailed composition.
2Manufacturing precision
If late stage reprojection is performed on all image layers, then accurate image correction is achieved, but processing time increases
Solution Approach 1:
The patent performs preliminary depth evaluation and layer selection before executing the full late stage reprojection process. By pre-processing depth information and identifying which layers will be visible in each tile region, the system prepares a reduced set of layers that need full reprojection processing. This preliminary action maintains correction accuracy for visible layers while avoiding unnecessary processing of hidden layers.
Solution Approach 2:
The patent applies partial action by performing complete late stage reprojection only on the subset of image layers that are determined to be visible, rather than processing all layers. The depth-based culling identifies the minimum necessary set of layers that must undergo full reprojection, achieving accurate correction where needed while reducing overall processing time by excluding layers that would not contribute to the final image.
3Reliability
If the system processes a large number of image layers, then image quality and completeness are maintained, but the system fails to scale efficiently
Solution Approach 1:
The patent segments the image processing into independent tile regions and applies depth-based layer culling within each segment. This allows the system to handle large numbers of layers by processing them in manageable, parallelizable units. Each tile can be processed independently based on its local depth information, enabling the system to scale efficiently as the number of layers increases without compromising overall image quality.
Solution Approach 2:
The patent changes the processing parameters dynamically based on depth information. Instead of using fixed processing parameters for all layers, the system adjusts which layers are processed by evaluating depth parameters in each tile region. This parameter change approach allows the system to maintain high image quality by processing necessary layers while achieving scalability by reducing the number of layers processed in regions where depth culling applies.
Data Source
AI summary
Techniques for performing tiled composition to intelligently restrict which layers are considered when performing an image composition process are disclosed. A service accesses, for each of multiple image layers, a corresponding color image and a corresponding depth image. The service performs LSR on the image layers to produce corresponding reprojected color images and corresponding reprojected depth images. The service uses the LSR's correction matrix to generate a set of guidance composition meshes. The service uses the guidance composition meshes to guide performance of the image composition process.