Load Store Cache Microarchitecture for GPU Memory Bottlenecks
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current graphics processors face inefficiencies in cache and memory access, particularly in handling loads, stores, and cache operations, which can lead to bottlenecks in processing graphics and compute workloads.
Innovation Solution
The introduction of a Graphics Processing Unit (GPU) load store cache (LSC) microarchitecture that includes a dynamic cache with partitioned memory banks, out-of-order scheduling, and a parameterized bus to enhance cache and memory functionality, allowing for efficient processing of loads, stores, and cache operations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If traditional cache and memory access circuitry is used in graphics processors, then device complexity is reduced, but memory access efficiency and processing productivity deteriorate due to bottlenecks in handling loads, stores, and cache operations
Solution Approach 1:
The cache memory is divided into multiple partitioned memory banks (e.g., bank 0, bank 1, bank 2, bank 3) that can be accessed in parallel. Each bank handle independently manages a specific bank, enabling simultaneous access to multiple banks without conflict, thereby increasing memory access throughput and efficiency.
Solution Approach 2:
The system implements dynamic scheduling through a scheduler that assigns memory access requests to appropriate banks based on current bank availability and request characteristics. Bank handles are dynamically allocated and reassigned to optimize parallel access patterns, allowing the system to adapt to varying workload demands and maintain high productivity.
2Speed
If parallel processing of memory operations is implemented, then processing speed improves, but device complexity increases due to need for multiple memory banks and scheduling circuitry
Solution Approach 1:
The cache memory is segmented into multiple independent banks that can be accessed simultaneously. This segmentation enables parallel processing of memory operations, where multiple load/store requests can be serviced at the same time by different banks, directly increasing operation speed without requiring complex inter-bank coordination.
Solution Approach 2:
Each memory bank is equipped with its own bank handle that independently manages access to that bank. The bank handles autonomously handle memory access requests for their respective banks, reducing the need for complex centralized control logic and minimizing scheduling overhead, thereby achieving high speed with controlled complexity.
3Loss of time
If out-of-order scheduling is implemented in the load store cache, then latency is reduced, but device complexity increases due to scheduling mechanisms
Solution Approach 1:
The scheduler implements out-of-order scheduling by dynamically evaluating pending memory requests and executing them in an optimal sequence rather than strict program order. When a memory bank becomes available, the scheduler assigns the next appropriate request regardless of its original position in the instruction stream, reducing idle time and latency. The bank handles work with the scheduler to dynamically adjust assignment based on real-time bank availability.
Data Source
AI summary
One embodiment provides a graphics processor comprising memory access circuitry configured to receive a message from an instruction execution resource and determine a destination for the message, the destination one of shared function circuitry of a graphics core or a set of memory banks within the graphics core. The memory access circuitry then routes the message to the shared function circuitry in response to a determination that the message is directed to the shared function circuitry or routes the message to a message sequencer associated with the instruction execution resource in response to a determination that the message is directed to the set of memory banks.