Thread Group Dispatch in Clustered Graphics Architecture
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current thread dispatch policies in clustered graphics architectures, such as round-robin dispatch, fail to fully utilize the benefits of clustered caches and break memory access patterns, leading to suboptimal performance due to skipping of CFEs and disruption of intended memory access sequences.
Innovation Solution
Implementing a batched dispatch policy that dispatches consecutive thread groups in batches across cycles in a strict round-robin manner or separates streams of thread groups to clusters, preserving programmer-intended memory access patterns and enhancing cache utilization.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If round-robin dispatch is used to balance load across CFEs, then load balancing is improved, but memory access patterns are broken and cache utilization deteriorates
Solution Approach 1:
The patent segments the dispatch process into two independent dimensions: (1) round-robin selection of CFE instances for load balancing, and (2) sequential assignment of thread groups within each CFE's queue to preserve memory access patterns. This segmentation allows both load balancing and cache utilization to be optimized simultaneously without conflict.
Solution Approach 2:
The patent introduces dynamic queue management where each CFE maintains a separate queue of thread groups. The dispatcher dynamically assigns thread groups to appropriate CFE queues based on round-robin selection, while each CFE processes its queue sequentially. This dynamic structure enables flexible load distribution while preserving the sequential execution order needed for memory access patterns.
2Speed
If consecutive thread groups are dispatched to consecutive CFEs, then parallel processing is maximized, but memory access sequences are disrupted
Solution Approach 1:
The patent adds a temporal dimension to the dispatch process by introducing per-CFE queues with sequential processing. While spatial parallelism is maintained across multiple CFEs, the sequential queue processing within each CFE restores the temporal order of memory accesses, allowing both parallel processing and memory access sequences to coexist.
3Productivity
If CFEs are skipped when unable to accept thread groups, then load balancing is maintained, but dispatch efficiency deteriorates
Solution Approach 1:
The patent performs preliminary actions by pre-allocating and maintaining ready queues at each CFE. Thread groups are added to the appropriate CFE's queue in advance based on round-robin selection, so that when a CFE becomes available, it can immediately process the next thread group from its queue without skipping or waiting, thereby eliminating dispatch delays.
Data Source
AI summary
Thread group dispatch in a clustered graphics architecture is described. An example of an apparatus includes of compute front end (CFE) clusters to receive dispatched thread groups, the CFE clusters including at least a first CFE cluster and a second CFE cluster; processing resources coupled with the CFE clusters to execute threads within thread groups; and cache clusters to cache data including thread groups, wherein the apparatus is to receive thread groups for dispatch, and to dispatch the thread groups to the CFE clusters according to a dispatch operation, the dispatch operation including dispatching multiple thread groups to each of multiple CFEs in the first CFE cluster and multiple thread groups to each of multiple CFEs in the second CFE cluster.