Thread Team Dispatch for GPU Memory Latency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The increasing number of processing resources in graphics processing units (GPUs) leads to increased latency in memory access due to the growing distance between resources and data storage, as well as competition for memory bandwidth, necessitating improvements in architectural efficiency for thread processing.
Innovation Solution
The implementation of a thread team construct, where a set of hardware threads is executed by a processing resource, with shared local registers accessible to each thread, allowing for ordered dispatch to a single processing resource, thereby keeping data processing in close proximity and reducing memory access latency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Power
If the number of processing resources in GPUs is increased to improve processing capability, then processing power is improved, but memory access latency increases due to greater distance between resources and data storage
Solution Approach 1:
The patent divides the monolithic GPU architecture into multiple computing tiles, each with its own local memory and processing resources. This segmentation allows each tile to operate independently with reduced memory access latency, while collectively providing high processing power through parallel operation of multiple tiles.
Solution Approach 2:
The patent implements local memory within each computing tile that is specifically optimized for fast access by the tile's processing resources. This local memory provides high-speed data storage close to the processing units, reducing memory access latency for frequently accessed data while maintaining overall system processing capacity.
2Productivity
If more processing resources are added to GPUs, then computational capacity is improved, but competition for memory bandwidth increases
Solution Approach 1:
The patent segments the memory hierarchy into local memory within each tile and global memory shared across all tiles. This segmentation allows multiple tiles to access their respective local memories simultaneously without competing for the same memory bandwidth, thereby maintaining high computational capacity while reducing memory bandwidth competition.
Solution Approach 2:
The patent introduces a two-level memory hierarchy dimension: local memory at the tile level and global memory at the GPU level. This dimensional organization allows parallel memory access operations to occur at the local level without saturating the global memory bandwidth, effectively increasing computational capacity while managing memory bandwidth competition.
3Productivity
If processing resources are increased, then processing capability is improved, but architectural efficiency decreases due to increased distance between resources and data storage
Solution Approach 1:
The patent organizes the GPU into multiple independent computing tiles, each with integrated processing resources and local memory. This segmentation creates modular units that maintain efficient data access patterns within each tile, preserving architectural efficiency while scaling processing capability through parallel tile execution.
Solution Approach 2:
The patent implements a nested memory hierarchy where local memory is nested within each computing tile, and global memory encompasses all tiles. This nested structure allows data to be stored at the appropriate level of the hierarchy, with frequently accessed data in local memory and less frequently accessed data in global memory, maintaining architectural efficiency while supporting expanded processing capability.
Data Source
AI summary
An apparatus to facilitate ordered thread dispatch for thread teams is disclosed. The apparatus includes one or more processors including a graphic processor, the graphics processor including a plurality of processing resources, and wherein the graphics processor is to: allocate a thread team local identifier (ID) for respective threads of a thread team comprising a plurality of hardware threads that are to be executed solely by a processing resource of the plurality of processing resources; and dispatch the respective threads together into the processing resource, the respective threads having the thread team local ID allocated.