Shared Local Registers for GPU Thread Team Latency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The increasing number of processing resources in graphics processing units (GPUs) leads to increased latency in memory access due to the growing distance between resources and data storage, as well as competition for memory bandwidth, particularly in systems using shared local memory accessible by multiple threads.
Innovation Solution
Implementing a thread team construct with shared local registers that are accessible to each thread, allowing data to be processed in close proximity and reducing memory access latency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If the number of processing resources in GPUs is increased to improve processing capacity, then productivity is improved, but the distance between resources and data storage increases causing memory access latency to worsen
Solution Approach 1:
The patent segments the memory hierarchy into multiple levels: shared local memory (SLM) accessible by thread teams, and global memory. By dividing the memory access paths and creating localized memory regions close to processing resources, the system reduces the effective distance for frequent accesses while maintaining the scaled-up processing capacity.
Solution Approach 2:
The patent introduces a new dimension in the memory hierarchy by placing shared local memory physically closer to processing resources in the architectural layout. This spatial reorganization creates a near-far memory access dimension, where frequently accessed data resides in SLM close to processors, reducing latency without limiting the number of processing resources.
2Productivity
If shared local memory is made accessible to multiple threads to improve parallel processing, then productivity is improved, but competition for memory bandwidth increases causing memory access latency to worsen
Solution Approach 1:
The patent applies local quality by providing each thread team with its own dedicated shared local memory region. This localized memory allocation ensures that threads within a team have preferential, low-latency access to their specific data without competing with threads from other teams, thereby maintaining high parallel processing capability while reducing access latency.
Solution Approach 2:
The shared local memory acts as an intermediary buffer between global memory and processing resources. By caching frequently accessed data in SLM, the system mediates the bandwidth competition - threads access fast local memory rather than contending for global memory bandwidth, reducing latency while supporting multi-thread parallelism.
Data Source
AI summary
Shared local registers for thread team processing is described. An example of an apparatus includes one or more processors including a graphic processor having multiple processing resources; and memory for storage of data, the graphics processor to allocate a first thread team to a first processing resource, the first thread team including hardware threads to be executed solely by the first processing resource; allocate a shared local register (SLR) space that may be directly reference in the ISA instructions to the first processing resource, the SLR space being accessible to the threads of the thread team and being inaccessible to threads outside of the thread team; and allocate individual register spaces to the thread team, each of the individual register spaces being accessible to a respective thread of the thread team.