Graphics Core Multicast Loads for Shared Cache Bottlenecks
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
When multiple graphics processor cores access data stored in a single cache memory bank, the data access rate becomes limited, leading to inefficiencies in parallel processing operations.
Innovation Solution
Implementing hardware enhancements for load/store instructions that allow a single graphics core to perform a broadcast or multicast asynchronous load, distributing data to other cores in a graphics core cluster without redundant cache access, and using re-usable barriers for synchronization of producer and consumer threads during asynchronous data exchanges.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If multiple graphics processor cores access data stored in a single cache memory bank, then parallel processing operations can be performed, but the data access rate becomes limited
Solution Approach 1:
The patent divides the cache memory access into multiple independent cache memory banks. Each graphics processor core can access different cache banks simultaneously, eliminating the bottleneck of a single cache bank. This segmentation allows multiple cores to perform parallel access operations without contending for the same memory resource, thereby maintaining high data access rates while enabling parallel processing.
Solution Approach 2:
The patent introduces a new dimension of parallelism by implementing a cache memory structure where multiple banks can be accessed concurrently. Instead of having cores compete for a single cache bank (one-dimensional access), the system provides multiple independent access paths through multiple banks (multi-dimensional access). This dimensional expansion allows the data access rate to scale with the number of banks, resolving the contradiction between parallel processing capability and data access speed.
2Loss of time
If data is stored in cache memory to enable higher data rate access, then latency is reduced, but multiple cores accessing the same cache bank creates access conflicts
Solution Approach 1:
By segmenting the cache memory into multiple independent banks, the patent allows different graphics processor cores to access different banks simultaneously without conflicts. Each bank operates independently, so core A accessing bank 1 does not interfere with core B accessing bank 2. This eliminates access conflicts while maintaining the low-latency benefits of cache memory, thereby preserving both fast access times and high parallel processing efficiency.
Solution Approach 2:
The patent creates multiple copies of the cache memory structure in the form of separate banks. Instead of having all cores share a single cache bank, each bank serves as a copy that can be independently accessed. This copying approach distributes the access load across multiple instances, eliminating contention while maintaining the speed advantages of cached data.
3Adaptability or versatility
If a single cache memory bank is shared by multiple graphics cores, then memory resource utilization is improved, but the data access rate becomes a bottleneck
Solution Approach 1:
The patent segments the shared cache memory into multiple independent banks, allowing multiple graphics processor cores to access different banks simultaneously. This segmentation maintains the resource sharing model (improving adaptability) while eliminating the speed bottleneck by providing multiple parallel access paths. Each bank can be independently utilized, so the system achieves both high resource utilization and high data access rates.
Solution Approach 2:
The patent adds a new dimension to memory resource sharing by introducing multiple cache banks that can be accessed in parallel. Instead of single-dimensional sharing where cores compete for one bank, the system provides multi-dimensional sharing where cores can simultaneously access different banks. This dimensional expansion resolves the bottleneck while maintaining versatile resource sharing capabilities.
Data Source
AI summary
Embodiments described herein provide a technique to facilitate the broadcast or multicast of asynchronous loads to shared local memory of a plurality of graphics cores within a graphics core cluster. One embodiment provides a graphics processor including a cache memory a graphics core cluster coupled with the cache memory. The graphics core cluster includes a plurality of graphics cores. The plurality of graphics cores includes a graphics core configured to receive a designation as a producer graphics core for a multicast load, read data from the cache memory; and transmit the data read from the cache memory to a consumer graphics core of the plurality of graphics cores.