GPU Multi-Tag Cache Architecture for Tile Fragmentation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing cache architectures in SoCs are inefficient for GPU workloads due to cache line sizes optimized for CPU access, leading to cache fragmentation, reduced capacity, and performance issues in graphics-intensive applications.
Innovation Solution
A multi-tag cache architecture that allows each cache line to store multiple compressed GPU tiles, with larger cache line sizes and sector-based organization, using metadata to track tile locations, reducing fragmentation and improving efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If small cache line sizes (64 bytes) are used to optimize for CPU workloads, then CPU access efficiency is improved, but GPU performance deteriorates due to cache fragmentation and reduced effective capacity
Solution Approach 1:
The cache architecture dynamically adapts cache line size based on the accessing processor type. For GPU workloads, it uses larger cache line sizes (e.g., 4 KB) to accommodate tile-based memory access patterns, while maintaining compatibility with CPU workloads. This dynamic adaptation resolves the contradiction by allowing the system to optimize for the current workload type rather than being fixed to a single optimal size.
Solution Approach 2:
The patent changes the cache line size parameter from the conventional 64 bytes to larger sizes (e.g., 4 KB) specifically for GPU workloads. This parameter change enables better alignment with GPU's tile-based memory architecture, reducing cache fragmentation and increasing effective cache capacity for GPU-accessed data while maintaining system-wide functionality.
2Device complexity
If traditional single-tag cache architecture is used, then device complexity is kept low, but cache fragmentation increases and effective cache capacity is reduced for GPU workloads
Solution Approach 1:
The cache line is segmented into multiple sectors (e.g., 32-byte sectors within a 4 KB line), and multiple tags are associated with each cache line to track different compressed tiles. This segmentation allows the cache to store multiple GPU tiles per cache line, reducing fragmentation and increasing effective capacity without requiring a complete architectural overhaul.
Solution Approach 2:
The multi-tag cache architecture makes each cache line multi-functional by allowing it to store multiple compressed GPU tiles with different tags. This universal design enables a single cache line to serve multiple purposes (storing different tiles) thereby increasing effective cache capacity without proportionally increasing the number of physical cache lines needed.
3Productivity
If larger cache line sizes are used to store more GPU data, then cache hit rates improve, but memory traffic increases due to more cache lines required
Solution Approach 1:
Multiple compressed GPU tiles are merged into a single cache line using the multi-tag architecture. Instead of requiring separate cache lines for each tile, the system combines multiple tiles (e.g., two or more compressed tiles per 4 KB line) into unified cache entries, thereby reducing the total number of cache lines needed and decreasing memory traffic while maintaining high cache hit rates.
Data Source
AI summary
A system and a method are disclosed. The method includes the steps of storing a first portion of a first compressed tile in a first cache line of a cache storage device, and storing a second portion of the first compressed tile in a second cache line of a cache storage device.