Streaming Wave Coalescer Spill-After Lane Swap
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The throughput of SIMD micro-architectures is hindered by divergent points in execution, which cause parallel lanes to become inactive, leading to reduced performance.
Innovation Solution
The implementation of a streaming wave coalescer (SWC) circuit that performs a spill-after operation when swapping lanes between waves, reducing the need for temporary data storage and minimizing latency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If traditional lane swapping is used, then thread migration between parallel lanes is achieved, but temporary data storage is required and latency increases
Solution Approach 1:
The patent applies preliminary action by performing the spill operation after the lane swap is identified but before the actual data transfer begins. The system pre-determines which lanes need to be swapped and prepares the spill buffer allocation in advance, allowing the actual data migration to occur with minimal latency once the swap decision is made.
Solution Approach 2:
The patent introduces an intermediary mechanism where the spill buffer acts as a temporary holding area that enables lane swapping without requiring permanent temporary storage structures. The spill buffer is allocated dynamically and used only during the lane swap operation, eliminating the need for dedicated temporary storage resources and reducing overall latency.
2Productivity
If more temporary data storage is used, then lane swapping can be performed, but hardware resource efficiency decreases
Solution Approach 1:
The patent applies discarding and recovering by treating the spill buffer as a temporary resource that is allocated only when needed for lane swapping and then discarded after use. Instead of maintaining permanent temporary storage structures, the system recycles the spill buffer allocation after the lane swap operation completes, significantly reducing hardware resource usage and improving efficiency.
Solution Approach 2:
The patent introduces dynamics by making the spill buffer allocation dynamic rather than static. The spill buffer is allocated and deallocated based on the actual needs of lane swap operations, allowing the hardware resources to be adapted dynamically to workload requirements rather than being reserved permanently, thus improving hardware resource efficiency.
3Productivity
If spill-before operation is used, then lane swapping is enabled, but more temporary data storage is required
Solution Approach 1:
The patent applies the other way round by inverting the traditional spill-before approach to a spill-after operation. Instead of spilling data before the lane swap and then reading it back, the system spills data after the lane swap has been identified, using the spill buffer only for the duration of the actual data transfer. This inversion significantly reduces the quantity of temporary data storage required.
Data Source
AI summary
An apparatus and method for efficiently migrating the execution of threads between multiple parallel lanes of execution. In various implementations, a computing system includes multiple vector processing circuits of a compute circuit that executes multiple lanes of multiple waves. Each lane includes a key indicating a path of execution. When a lane of the multiple lanes of execution executes a stream wave coalescing (SWC) reorder instruction, a control circuit compares keys of waves that have previously executed the SWC reorder instruction. When the number of lanes with a matching key exceeds a threshold and after identifying at least this number of lanes to swap, the control circuit swaps continuation state information (live active state information) between lanes of an emitting wave that do not have a matching key and lanes of contributing waves that do have a matching key. The resulting (reordered) emitting wave executes more efficiently, which increases performance.