Shared Local Memory Tiling for GPU Bank Utilization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current graphics processing units (GPUs) suffer from underutilization of off-chip memory due to data layout, data locality, and inter-kernel redundancy, which affects performance and efficiency in parallel processing.
Innovation Solution
Implement a memory tiling mechanism that applies a tiling pattern to memory addresses for data stored in shared local memory (SLM) to optimize data access and reduce redundancy, enhancing memory utilization and performance.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of manufacture
If linear mapping of address space to SLM banks is used, then implementation is simple, but memory utilization is low due to underutilization of off-chip memory
Solution Approach 1:
The patent applies a tiling pattern to the linear address space, transforming it into a two-dimensional grid structure. This dimensional transformation allows data to be organized in tiles that can be efficiently mapped to SLM banks, improving memory utilization without significantly complicating the implementation. The tiling logic divides the address space into manageable tiles that can be processed in parallel.
Solution Approach 2:
The patent changes the mapping parameters by introducing a tiling pattern with specific tile dimensions and bank mapping rules. Instead of direct linear mapping, the address translation now involves calculating tile indices and bank assignments based on the tiling configuration. This parameter transformation enables better utilization of SLM banks while maintaining a systematic approach to address mapping.
2Speed
If highly banked memory architecture is used, then parallel access is improved, but data locality is reduced due to linear mapping
Solution Approach 1:
By transforming the linear address space into a two-dimensional tiled structure, the patent preserves spatial locality within each tile while enabling parallel access across multiple banks. Data elements that are close in the original address space remain close within a tile, maintaining locality, while the tiling structure allows efficient distribution across banks for parallel access.
Solution Approach 2:
The patent segments the address space into discrete tiles that can be independently mapped to SLM banks. This segmentation allows data within each tile to maintain its locality while different tiles can be accessed in parallel from different banks, thus preserving both data locality and parallel access capability.
3Device complexity
If conventional SLM design is used, then hardware complexity is low, but performance is limited due to inter-kernel redundancy
Solution Approach 1:
The tiling pattern introduces a hierarchical structure to memory organization, creating tiles that can be reused across different kernels. This dimensional organization allows the same tile data to serve multiple computational kernels, reducing inter-kernel redundancy and improving performance without requiring complex hardware changes.
Solution Approach 2:
The patent pre-organizes data into tiles within SLM before kernel execution, preparing data in a format that can be efficiently reused by multiple kernels. This preliminary tiling action reduces the need for redundant data loading and processing across different kernels, improving overall performance while maintaining simple hardware architecture.
Data Source
Figure 1
Figure 2A
Figure 2B
AI summary
An apparatus to facilitate memory tiling is disclosed. The apparatus includes a memory, one or more execution units (EUs) to execute a plurality of processing threads via access to the memory and tiling logic to apply a tiling pattern to memory addresses for data stored in the memory.