Super Shared Local Memory for Heterogeneous Computing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In heterogeneous computing frameworks like OpenCL, the performance of memory access between Shared Local Memory (SLM) and System Global Memory (SGM) is hindered by the large number of operations required for writing data from SLM to SGM, particularly in scenarios with numerous work groups, leading to inefficiencies and performance penalties.
Innovation Solution
The introduction of a Super-Workgroup concept, which groups multiple work groups together to share a Super-SLM, reduces the number of memory writing operations by allowing data to be written from Super-SLM to SGM instead of traditional SLM, thereby improving memory access performance.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If data is written from each work group's SLM to SGM individually, then each work group can complete its operations independently, but the number of memory writing operations increases significantly, leading to performance degradation
Solution Approach 1:
Multiple work groups are merged into a super-work group that shares a common SLM (super-SLM). Instead of each work group writing to SGM separately, the super-work group performs a single collective write operation from the shared super-SLM to SGM, reducing the number of memory transactions from N (number of work groups) to 1.
Solution Approach 2:
The super-SLM serves multiple work groups simultaneously, acting as a universal shared memory resource. This multi-functional memory structure allows any work group within the super-work group to access and write data to the same memory space, eliminating redundant memory operations.
2Productivity
If multiple work groups operate with separate SLMs, then work group isolation is maintained, but the total number of memory writing operations to SGM increases, reducing overall system efficiency
Solution Approach 1:
Multiple SLMs are merged into a single super-SLM that is shared by all work groups in the super-work group. This consolidation reduces the total number of memory write operations to SGM from N separate operations to a single operation, thereby reducing power consumption associated with memory access.
Solution Approach 2:
The memory architecture parameter is changed from individual SLM per work group to a shared super-SLM for multiple work groups. This parameter change transforms the memory access pattern from multiple small transactions to a single larger transaction, improving both performance and power efficiency.
3Productivity
If traditional SLM architecture is used with individual work groups, then implementation simplicity is maintained, but the burden on memory access increases with many work groups, leading to performance penalties
Solution Approach 1:
Multiple work groups are combined into a super-work group structure with a shared super-SLM. This merging approach manages memory access more efficiently by reducing the number of independent memory management operations required, thereby improving overall system performance despite the increased architectural concept.
Data Source
AI summary
An apparatus, method and machine-readable storage medium to improve memory access performance between shared local memory and system global memory are described. The method comprises grouping two or more work groups to form a super workgroup, and positioning a portion of a memory space into one or more super shared local memories (Super SLMs), wherein the memory space which is shared within the super workgroup forms at least one Super SLM of the one or more Super SLMs. The apparatus comprises: a plurality of execution units; a cache memory having a portion which operates as a shared local memory (SLM), which is shared with the plurality of execution units, at least one of which operates on a work group of a sub-slice, wherein the SLM is shared within the work group; and at least one Super-SLM for providing shared memory accessible by different work groups in the sub-slice, wherein the at least one of the execution units operates on the different work groups. A last work group may write data back to a system global memory (SGM) from a Super-SLM.