Persistent Thread Rescheduling for SIMD Wavefront Progress
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing parallel data processing systems using SIMD micro-architectures face issues such as deadlock due to serialized execution of separate branches and long latency stalls, which reduce memory-level parallelism and performance, limiting the types of applications that can be executed, especially when wavefronts exceed processor capacity.
Innovation Solution
Implementing per-wavefront context save and restore operations, allowing parallel data processing circuits to execute instructions at a finer granularity, enabling independent progress of wavefronts and avoiding deadlock by storing and loading context state information efficiently.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Volume of stationary object
If context state is saved at the granularity of an entire compute circuit, then data storage space is reduced, but performance is limited and application types are restricted
Solution Approach 1:
The patent segments the context save/restore granularity from the compute circuit level down to the wavefront level. Instead of saving context for entire compute circuits, the system now saves context for individual wavefronts (groups of threads), enabling finer-grained parallelism while reducing the storage burden compared to per-thread context saving.
Solution Approach 2:
The patent introduces a new dimensional approach by implementing context saving at the wavefront level, which is a intermediate granularity between compute circuit and individual thread. This dimensional change in context management granularity enables both reduced storage requirements and improved performance by allowing more flexible wavefront scheduling and reducing serialization overhead.
2Adaptability or versatility
If more wavefronts are launched than can simultaneously fit on the processor, then application flexibility is improved, but deadlock occurs
Solution Approach 1:
The patent implements preliminary context saving for wavefronts before they are executed. By saving wavefront context state in advance at the wavefront level, the system enables efficient context switching when wavefronts need to be rescheduled or when more wavefronts are launched than can fit simultaneously on the processor, preventing deadlock while maintaining flexibility.
Solution Approach 2:
The patent introduces wavefront-level context state as an intermediary mechanism between the compute circuits and individual threads. This intermediary layer enables the system to manage more wavefronts than can simultaneously execute by saving and restoring context at the wavefront level, allowing flexible launching of excess wavefronts without causing deadlock.
3Productivity
If separate branches are executed in parallel, then instruction throughput is improved, but serialization occurs reducing memory-level parallelism
Solution Approach 1:
The patent implements dynamic wavefront scheduling that can adaptively manage the execution of multiple wavefronts with different branch patterns. By saving and restoring context at the wavefront level, the system can dynamically switch between different wavefronts to hide memory latency and reduce serialization, maintaining high instruction throughput while minimizing the loss from branch serialization.
Data Source
AI summary
An apparatus and method for efficiently processing instructions in hardware parallel execution lanes within a processing circuit. In various implementations, a computing system includes a host processing circuit and a parallel data processing circuit that uses multiple single instruction multiple data (SIMD) circuits, each with multiple parallel lanes of execution. The host processing circuit generates an indication specifying a host trap event has occurred, which includes an asynchronous interruption. The host processing circuit stores the indication and information of a host trap handler in a predetermined memory location specifying subsequent tasks to execute. The parallel data processing circuit accesses this predetermined memory location to check for the indication of a trap event. The instructions of the trap handler directs the parallel data processing circuit to store context state information and initiate processing of other tasks.