Broadcast Load Mechanism for GPU Shared Memory Contention
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current graphics processing units (GPUs) face limitations in data access rates when multiple cores access shared cache memory, leading to reduced performance in parallel processing operations.
Innovation Solution
Implementing hardware enhancements for load/store instructions that allow a single graphics core to perform asynchronous broadcasts of data to other cores in a cluster, along with reusable barriers for synchronization, and techniques to broadcast data within cache or shared local memory without redundant reads.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If multiple graphics processor cores access data stored in a single cache memory bank, then data access rate becomes limited due to contention, but using multiple cache memory banks increases device complexity
Solution Approach 1:
The patent introduces a broadcast mechanism as an intermediary layer between the cache memory bank and multiple graphics processor cores. Instead of allowing direct simultaneous access from multiple cores to the same cache bank (which causes contention), a single core acts as an intermediary that reads data from the cache and broadcasts it to other cores through a dedicated broadcast bus, thereby eliminating access conflicts while maintaining a simple single-bank cache structure
Solution Approach 2:
The broadcast mechanism creates copies of data in the register files of multiple consumer cores from a single read operation. When one core reads data from the cache memory bank, the same data is copied and distributed to the register files of all cores that have requested it, allowing multiple cores to access the same data without additional cache read operations, thus improving data access rate without increasing cache memory complexity
2Productivity
If each graphics core performs independent address calculations and memory accesses, then each core must read data individually from cache, but this increases the total number of memory accesses and reduces processing efficiency
Solution Approach 1:
The patent merges multiple independent memory access operations into a single unified read operation. Instead of having each graphics core independently read data from the cache memory bank (which would generate multiple sequential access requests), the system combines all read requests into one operation where a single producer core performs the read and then broadcasts the data to all consuming cores simultaneously, thereby reducing the total number of memory access cycles and improving parallel processing efficiency
Solution Approach 2:
The system performs preliminary address calculations and data gathering in advance using a single producer core before the actual computation phase. The producer core pre-loads data from the cache memory bank into its register file, and other cores receive this pre-prepared data through broadcasting, eliminating the need for each core to perform redundant address calculations and memory access operations during the critical computation path
Data Source
AI summary
Embodiments described herein provide a technique enable a broadcast load from an L1 cache or shared local memory to register files associated with hardware threads of a graphics core. One embodiment provides a graphics processor comprising a cache memory and a graphics core coupled with the cache memory. The graphics core includes a plurality of hardware threads and memory access circuitry to facilitate access to memory by the plurality of hardware threads. The graphics core is configurable to process a plurality of load request from the plurality of hardware threads, detect duplicate load requests within the plurality of load requests, perform a single read from the cache memory in response to the duplicate load requests, and transmit data associated with the duplicate load requests to requesting hardware threads.