Sparse Frame Composition With Adaptive Reprojection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current techniques for augmented reality (AR) and extended reality (XR) frame processing do not differentiate between sparse and non-sparse frames, leading to inefficient computation, memory usage, bandwidth utilization, and increased power consumption.
Innovation Solution
A method and apparatus that process sparse frames by predicting boundary coordinates, adjusting display order, and selecting adaptive reprojection modes based on sparsity to optimize computation, memory, and power consumption.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If current techniques process sparse frames and non-sparse frames in the same manner, then processing is simplified, but computation efficiency deteriorates
Solution Approach 1:
The patent segments frame processing into two distinct paths: sparse frame processing and non-sparse frame processing. The system identifies whether a frame is sparse by checking if the number of modified pixels is below a threshold, then routes it to the appropriate processing pipeline. This segmentation allows optimized processing for each frame type, improving computation efficiency while maintaining manageable complexity through clear separation of processing logic.
Solution Approach 2:
The patent implements dynamic processing by adjusting the reprojection mode based on frame sparsity. The system dynamically selects between full reprojection and sparse reprojection techniques depending on the characteristics of each frame. This dynamic adaptation allows the system to optimize computation efficiency for sparse frames while maintaining quality for non-sparse frames, resolving the contradiction between processing simplicity and computational efficiency.
2Device complexity
If current techniques use the same reprojection method for all frames, then implementation is simpler, but memory usage increases
Solution Approach 1:
The patent applies local quality by using different reprojection methods for different regions of the display buffer. For sparse frames, only the regions containing modified pixels are reprojected and updated in memory, while the rest of the buffer remains unchanged. This localized processing significantly reduces memory usage compared to full-frame reprojection, while the implementation complexity is managed through region-based processing logic.
3Reliability
If full reprojection is used for all frames, then image quality is maintained, but power consumption increases
Solution Approach 1:
The patent applies partial action by performing reprojection only on the necessary portions of sparse frames rather than the entire frame. The system identifies modified pixel regions and applies reprojection processing only to these areas, maintaining image quality in the regions that require it while avoiding unnecessary processing in unchanged areas. This significantly reduces power consumption while preserving image quality where needed.
Data Source
AI summary
This disclosure provides systems, devices, apparatus, and methods, including computer programs encoded on storage media, for sparse composition. A processor obtains at least one frame including at least one component, where the at least one component includes (1) data of the at least one component and (2) metadata of the at least one component, and where the metadata includes enclosing boundary information of the at least one component. The processor predicts, based on the enclosing boundary information and pose information, a set of boundary coordinates of the at least one component. The processor adjusts, based on a display order, the predicted set of boundary coordinates of the at least one component. The processor determines whether the predicted set of boundary coordinates overlaps with at least one component boundary associated with a composition engine. The processor processes, based on the determination, the data.