GPU Shader Interleaving With Tiled On-Chip Buffering
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing graphics processing unit (GPU) operations require frequent access to external memory for intermediate image data, leading to increased memory traffic and inefficiencies due to the inability to store large image sizes in cache.
Innovation Solution
Interleaving graphics shader operations and compute shader operations by maintaining intermediate results in an on-chip memory, reducing the need for external memory access through tile-based buffering and scheduling optimizations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If intermediate images are stored in external memory, then cache capacity limitations are overcome, but memory traffic increases and performance decreases
Solution Approach 1:
The rendering process is divided into independent tile units that can be processed separately. Each tile is rendered, stored in on-chip cache, and processed by compute shaders before moving to the next tile. This segmentation allows the system to handle large intermediate images without requiring proportional external memory capacity, as only one tile needs to be stored at a time in the cache.
Solution Approach 2:
The patent introduces a temporal dimension to the rendering process by interleving graphics shader operations with compute shader operations across multiple time steps. Instead of completing all graphics rendering before compute processing (sequential in time), the system alternates between rendering tiles and processing tiles in an interleaved manner, effectively using time to manage memory constraints.
2Ease of operation
If graphics shader and compute shader operations are executed sequentially, then processing simplicity is maintained, but overall processing time increases
Solution Approach 1:
The system performs preliminary actions by rendering and storing tile data in on-chip cache before compute shaders need it. This allows compute shaders to immediately process the data without waiting for external memory operations, overlapping the rendering of subsequent tiles with the computation of previous tiles.
Solution Approach 2:
The interleaved execution model ensures continuous utilization of both graphics and compute shader resources. While graphics shaders are rendering new tiles, compute shaders are simultaneously processing previously rendered tiles, eliminating idle time and maintaining continuous productive action across the entire rendering pipeline.
3Loss of energy
If intermediate results are kept in on-chip memory, then external memory access is reduced, but on-chip memory capacity is limited
Solution Approach 1:
By segmenting the intermediate image into smaller tile units, the patent enables storage of multiple tiles in on-chip cache simultaneously. Each tile is processed independently, allowing the limited on-chip memory to hold sufficient data for continuous processing without requiring large external memory bandwidth.
4Reliability
If all render target operations are completed before reading results, then processing completeness is ensured, but memory access efficiency decreases
Solution Approach 1:
The system performs preliminary rendering of tiles and stores them in on-chip cache before compute shaders require the data. This preliminary action ensures data availability without requiring complete rendering of all render targets before any computation begins, enabling early processing of available data.
Solution Approach 2:
The interleaved execution model maintains continuous productive action by ensuring that compute shaders always have data available from previously rendered tiles, while graphics shaders continuously generate new data. This eliminates idle waiting time and ensures processing completeness through systematic progression through all tiles.
Data Source
Figure 1~2
Figure 3
Figure 4A~4C
AI summary
A graphics system interleaves a combination of graphics shader operations and compute shader operations. A set of application programming interface (API) calls is analyzed so as to determine dependencies and to identify candidates for interleaving. A compute shader is adapted to have a tiled access pattern. The interleaving is scheduled so as to reduce requests to access an external memory so as to perform reads and writes of intermediate data.