GPU Exception Handling via Two-Pass Execution Masking
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Accelerated processing devices (APDs) typically stall when encountering exception-related instructions, unlike central processing units (CPUs), which can execute speculatively and out-of-order, leading to inefficiencies in parallel processing.
Innovation Solution
Implementing a two-pass execution technique, where instructions are executed in an exception-wait mode and then replayed, with a mask generated to control which instructions execute in each mode based on analysis of read and write sets and data dependencies, allowing some execution to proceed past exception-related instructions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If APDs stall when encountering exception-related instructions, then execution correctness is maintained, but processing productivity decreases
Solution Approach 1:
The patent segments instruction execution into two distinct passes: an exception-wait pass that executes instructions up to and including exception-related instructions, and a replay pass that executes instructions after the exception point. This segmentation allows the system to maintain correctness in the first pass while enabling productive work in the second pass, resolving the contradiction between reliability and productivity.
Solution Approach 2:
The patent performs preliminary analysis of instruction dependencies (read sets, write sets, and data dependencies) before execution to generate a mask that predicts which instructions can safely execute past exception points. This preliminary action enables the replay pass to execute instructions that are guaranteed to be correct, improving productivity without sacrificing reliability.
2Device complexity
If APDs execute instructions in-order and stall on exceptions, then execution simplicity is maintained, but time consumption increases
Solution Approach 1:
The patent implements periodic action by executing instructions in repeated passes: the exception-wait pass executes instructions up to the exception point, then the replay pass executes subsequent instructions. This periodic execution pattern allows the system to maintain relatively simple in-order execution logic while reducing time loss by continuously progressing through instruction streams in manageable cycles rather than stalling indefinitely.
3Productivity
If APDs allow speculative execution past exceptions, then productivity improves, but execution reliability decreases
Solution Approach 1:
The patent implements feedback mechanisms by analyzing instruction dependencies (read sets, write sets, and data dependencies) to generate execution masks that guide which instructions can safely execute in the replay pass. This feedback loop ensures that only instructions guaranteed to be correct are executed speculatively, maintaining reliability while enabling productivity improvements through parallel and out-of-order execution of safe instructions.
Data Source
AI summary
Techniques for improving performance of accelerated processing devices (“APDs”) when exceptions occur are provided. In APDs, the very large number of parallel processing execution units, and the complexity of the hardware used to execute a large number of work-items in parallel, means that APDs typically stall when an exception occurs (unlike in central processing units (“CPUs”), which are able to execute speculatively and out-of-order). However, the techniques provided herein allow at least some execution to occur past exceptions. Execution past an exception generating instruction occurs by executing instructions that would not lead to a corruption while skipping those that would lead to a corruption. After the exception has been satisfied, execution occurs in a replay mode in which the potentially exception-generating instruction is executed and in which instructions that did not execute in the exception-wait mode are executed. A mask and counter are used to control execution in replay mode.