GPU Sub-Core L1 Cache Atomic Operations
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current GPU architectures face inefficiencies in performing atomic operations due to high latency and lower bandwidth when using global memory atomic operations, which are performed in the last-level cache or memory controller, and the limitations of local memory atomic operations in managing shared local memory.
Innovation Solution
Implementing global memory atomic operations directly in the private L1 cache of a sub-core GPU, allowing for a new 'scope' parameter to distinguish between L3 and L1 cache operations, enabling local-scope memory atomics to be performed closer to execution units, reducing latency and increasing bandwidth.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If global memory atomic operations are performed in the last-level cache or memory controller, then the scope of atomic operations covers the full GPU, but the latency is high and bandwidth is low
Solution Approach 1:
The patent introduces a new scope parameter that allows atomic operations to be performed in different locations: L1 cache for local sub-core operations and L3 cache/memory controller for global GPU operations. This enables threads to choose the appropriate scope based on their needs, achieving low latency for local operations while maintaining global scope capability when required.
2Adaptability or versatility
If global memory atomic operations are performed in the last-level cache or memory controller, then the scope of atomic operations covers the full GPU, but the bandwidth is low
Solution Approach 1:
The patent enables atomic operations to be performed in the L1 cache of individual sub-cores, providing high bandwidth for local operations. The new scope parameter allows applications to use L1 cache atomics when they need high bandwidth and only involve threads within the same sub-core, while reserving L3 cache atomics for cases requiring broader scope.
3Device complexity
If local memory atomic operations are performed in shared local memory, then management complexity is reduced, but the memory capacity is limited
Solution Approach 1:
The patent makes the L1 cache serve dual purposes: it functions as both the private cache for execution units and as the memory space for atomic operations. This eliminates the need for separate shared local memory management while providing sufficient capacity, as the L1 cache is already allocated to each sub-core and can be used for atomics without additional management overhead.
Data Source
AI summary
Embodiments are directed to systems and methods for performing global memory atomics in a private cache of a sub-core of a GPU. An embodiment of a GPU includes multiple sub-cores each including a load/store pipeline. The load/store pipeline is operable to receive information specifying an atomic operation to be performed within a primary data cache of the load/store pipeline. The load/store pipeline is also operable to read data to be modified by the atomic operation into the primary data cache from a memory hierarchy shared by the multiple sub-cores. The load/store pipeline is further operable to produce an atomic result of the atomic operation by modifying the data within the primary data cache based on the atomic operation.