GPU Asynchronous DMA for Reducing Duplicate Memory Loads
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Legacy graphics processors experience duplicative memory loads from multiple thread groups accessing the same data, leading to unnecessary bandwidth pressure on the L2 cache and interconnect.
Innovation Solution
Implementing asynchronous Direct Memory Access (DMA) operations across graphics cores to optimize data transfer and reduce redundant memory loads, enabling efficient parallel processing.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If multiple thread groups load the same tile from memory using separate L2 cache loads, then each thread group can access the data independently, but duplicative loads create unnecessary bandwidth pressure on the L2 cache and interconnect
Solution Approach 1:
The system performs preliminary actions by having thread groups load data into shared L1 cache memory before individual thread execution. The L1 cache is configured to perform duplicate load operations automatically, so that when multiple thread groups need the same data, it is already loaded into the shared cache space, eliminating the need for separate L2 cache loads for each thread group.
Solution Approach 2:
Multiple separate L2 cache load operations are merged into a single shared L1 cache load operation. The L1 cache acts as a common data space that can be accessed by multiple thread groups, consolidating what would otherwise be duplicate memory transactions into one unified data transfer operation, thereby reducing bandwidth pressure on the L2 cache and interconnect.
2Reliability
If separate L2 cache loads are performed for each thread group, then data can be loaded into individual caches, but this creates resource contention and bandwidth pressure on the L2 cache interconnect
Solution Approach 1:
The shared L1 cache memory acts as an intermediary between the L2 cache and the thread groups. Instead of thread groups directly accessing L2 cache individually (creating contention and complexity), the L1 cache serves as a mediator that pre-loads data into a shared space, allowing multiple thread groups to access data without direct L2 cache contention, thereby simplifying the memory access architecture.
3Productivity
If asynchronous DMA operations are implemented across graphics cores, then data transfer efficiency is optimized and redundant memory loads are reduced, but this requires coordination mechanisms to manage parallel access
Solution Approach 1:
The L1 cache is configured to automatically perform duplicate load operations without requiring explicit coordination instructions from the CPU or memory controller. When multiple thread groups access the same data, the L1 cache self-manages the loading process, using its own bandwidth to pre-load data into shared space, thereby achieving efficient asynchronous data transfer without adding complex coordination mechanisms.
Data Source
AI summary
One embodiment provides a graphics processor comprising a base die including a plurality of chiplet sockets and a plurality of chiplets coupled with the plurality of chiplet sockets. A chiplet of the plurality of chiplets including a graphics core cluster including a plurality of graphics cores, a distributed shared local memory including shared local memory within each of the plurality of graphics cores, and a direct memory access engine within each of the plurality of graphics cores, the direct memory access engine configured to asynchronously copy data from a memory device to the distributed shared local memory.