GPU Memory Access Sorting to Reduce Cache and DRAM Latency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
As graphics processing units increase in size and complexity, the latency to the last level cache and graphics processor memory can lead to increased latency, resulting in thread stalls due to cache and graphics processor memory, which lowers the utilization of the graphics processor, especially for wider compute width resources like systolic arrays or wide vector engines, and traditional methods to hide this latency by adding more hardware threads are costly in terms of silicon area and power consumption.
Innovation Solution
Implementing a graphics processing unit with memory access architecture that sorts memory access requests by address and divides them into reads and writes in address-sorted batches to minimize DRAM bus turnaround and page open penalties, using typed 2D block messages for loading and storing pixel/texel data, and prefetching data into GPU cache memory to reduce latency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If the graphics processor size and complexity increase, then the overall memory throughput increases, but the latency to the last level cache and memory increases
Solution Approach 1:
The patent implements address sorting logic that sorts memory access requests by address before they are sent to the memory interface. This preliminary organization of requests ensures that memory accesses are coalesced and delivered in an optimized sequence, reducing the time spent waiting for memory operations to complete while maintaining high throughput capability.
2Productivity
If more hardware threads are added to hide cache/memory latency, then thread stalls are reduced, but silicon area and power consumption increase
Solution Approach 1:
The patent introduces an address sorting logic unit as an intermediary component between the memory access request generation stage and the memory interface. This sorting logic reorganizes memory requests to improve access patterns, effectively reducing latency without requiring additional hardware threads or increasing silicon area and power consumption.
3Loss of time
If memory access requests are not sorted by address, then DRAM bus turnaround and page open penalties increase, but implementing sorting logic adds device complexity
Solution Approach 1:
The patent implements address sorting logic that sorts memory access requests by address before they are sent to the memory interface. This preliminary organization of requests ensures that memory accesses are coalesced and delivered in an optimized sequence, reducing the time spent waiting for memory operations to complete while maintaining high throughput capability.
Data Source
AI summary
One embodiment provides a graphics processor including a processing resource including a register file, memory, a cache, and load/store/cache circuitry to process load, store, and prefetch messages from the processing resource. The circuitry will sort received memory access messages into address sorted lists of reads and writes. The circuitry schedules a first set of address sorted requests from a first request buffer for a first period of time, then schedules a second set of address sorted requests from a second request buffer for a second period of time.