Resumable Graph Execution on SIMT Hardware via Circular Buffer Iteration
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current technologies face challenges in handling buffer overflows during graph execution on single-instruction multiple-thread (SIMT) architecture, leading to errors and partial data processing, which limits the number of operators that can be offloaded to SIMT hardware and increases query runtime.
Innovation Solution
The approach involves using circular buffers and maintaining state across iterations of graph execution, allowing for seamless resumability and avoiding redundant operations by tracking the start and end indexes of input and output buffers with counters.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If buffer size is conservatively allocated to fit all operators on SIMT hardware, then device buffer memory usage is reduced, but output buffer size may be insufficient causing overflow errors and execution failure
Solution Approach 1:
The patent divides the graph execution into multiple iterations or stages, processing operators in batches rather than all at once. This segmentation allows the system to handle large output volumes by processing them in manageable chunks that fit within allocated device buffers, preventing overflow errors while maintaining reliable execution
Solution Approach 2:
The patent implements dynamic buffer allocation and iteration mechanisms where buffer usage is adjusted based on actual execution needs. The system dynamically manages buffer allocation across multiple execution iterations, allowing flexible adaptation to varying output sizes without requiring conservative static allocation that would waste memory or cause overflows
2Reliability
If buffer size is increased to accommodate all output data, then overflow errors are avoided, but device buffer memory consumption increases
Solution Approach 1:
The patent segments the execution of graph operators into multiple iterations, where each iteration processes a subset of operators or a portion of the data flow. This allows the system to use smaller device buffers for each iteration while still handling large overall output volumes by accumulating results across iterations, thus avoiding overflow without requiring large memory allocation
Solution Approach 2:
The patent implements partial processing in each iteration, where not all operators need to complete fully in a single pass. The system performs partial actions on subsets of data or operators, then resumes in subsequent iterations, allowing reliable execution with limited buffer space by processing only what fits in memory at each step
3Reliability
If graph execution falls back to CPU processing on overflow, then execution continues without error, but query runtime increases and throughput decreases
Solution Approach 1:
The patent performs preliminary actions by allocating sufficient host buffers and planning the execution strategy before graph launch. The system prepares buffer allocation schemes and iteration plans in advance, enabling the graph to execute fully on SIMT hardware across multiple iterations without needing to fall back to CPU processing, thus maintaining both reliability and productivity
Solution Approach 2:
The patent ensures continuous useful action by implementing resumable graph execution that maintains state across iterations. The system continues processing on SIMT hardware through multiple coordinated iterations rather than interrupting to use CPU, preserving the high-speed parallel processing capability throughout the entire execution flow and avoiding runtime penalties
4Quantity of substance
If graph execution is broken into multiple subgraphs to avoid overflow, then buffer size requirements are reduced, but end-to-end query runtime increases
Solution Approach 1:
The patent segments the graph execution into iterations rather than creating separate subgraphs. This segmentation approach maintains the graph as a unified execution unit that can be resumed across iterations, avoiding the overhead of breaking into independent subgraphs while still reducing per-iteration buffer requirements through controlled分批 processing
Solution Approach 2:
The patent implements a universal iteration framework that handles all graph operators and overflow scenarios through a single coordinated mechanism. This multi-functional approach allows the same graph execution engine to handle both small and large workloads by adjusting iteration parameters, eliminating the need for separate subgraph handling logic and reducing overall runtime
Data Source
AI summary
An approach is provided for a thorough and clean way of handling graph overflows in graph execution on single instruction, multiple threads (SIMT) hardware with resumable graph support. The solution does not assume that the input and output fit in the buffers allocated in the SIMT hardware. The approach maintains state of the execution for each kernel and uses multiple iterations of graph execution, making progress in each iteration until all data items are processed through the graph on SIMT hardware. This iterative processing of the graph is transparent to the end user. For resumability, the approach treats buffers as circular buffers instead of serial buffers. With the help of counters, the approach keeps track of the start and end indexes of input and output buffers, thus achieving seamless graph resumability when re-execution is required for only a subset of kernels.