Graph Streaming Processor Stage Reduction via Virtual Scheduling
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Graph streaming processors face limitations in processing depth due to a finite amount of scheduling hardware, requiring graphs to be split into sub-graphs, which hampers performance, especially in computation-intensive applications like neural networks.
Innovation Solution
The system reduces the number of stages in graph streaming processing by merging nodes into stages based on write command thresholds, using a scheduler with input and alternate command buffers to manage thread generation and dispatching, allowing for virtual staging and efficient data movement.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If graphs are split into sub-graphs due to finite scheduling hardware, then the system can process graphs with limited depth, but processing performance deteriorates in computation-intensive applications
Solution Approach 1:
The patent introduces a new dimension to the scheduling architecture by implementing virtual stages that operate alongside physical stages. This allows the system to process graphs with depth exceeding the number of physical stages by utilizing command buffers to store and retrieve commands across multiple virtual stage iterations, effectively adding a temporal dimension to the spatial stage structure.
Solution Approach 2:
The patent creates virtual copies of staging functionality through command buffers that can be reused across multiple stages. Instead of requiring a unique physical stage for each graph depth level, the same physical staging hardware is copied virtually through buffer reuse, allowing unlimited graph depth processing with finite hardware resources.
2Productivity
If multiple stages are used to process threads, then processing capability increases, but circuitry requirements and system complexity increase
Solution Approach 1:
The patent makes the staging hardware universal by designing stages that can handle multiple functions: processing current stage commands, storing commands for future stages in output command buffers, and retrieving commands from alternate command buffers. This multi-functionality allows a single physical stage to replace multiple dedicated stages, reducing overall circuitry requirements while maintaining processing capability.
Solution Approach 2:
The patent merges the functionality of multiple sequential stages into a single physical stage by combining command parsing, thread generation, and command buffer management within one stage unit. This consolidation reduces the number of separate circuitry blocks needed while achieving the same through-put through virtual staging mechanisms.
3Length of stationary object
If command buffers are increased to support more stages, then graph processing depth increases, but cache size and memory requirements increase
Solution Approach 1:
The patent prepares command buffers in advance by storing commands for future stages in output command buffers during current stage execution. This preliminary action allows commands to be ready for retrieval without requiring large cache structures, as the buffering is done as part of the normal stage operation rather than requiring dedicated cache memory for each stage.
Solution Approach 2:
The patent implements a command buffer recycling mechanism where alternate command buffers are discarded and recovered for reuse. Instead of allocating permanent cache space for each stage's command buffer, the system discards buffers that have served their purpose and recovers them for reuse in subsequent stages, dramatically reducing total memory requirements while supporting unlimited graph depth.
Data Source
AI summary
Methods, systems, and apparatuses for graph streaming processing system are disclosed. One system includes a plurality of graph streaming processors operative to process a plurality of threads, wherein the plurality of threads is organized as nodes. The system further includes a scheduler that includes a plurality of stages. Each stage includes a command parser operative to interpret commands within a corresponding input command buffer, an alternate command buffer, and a thread generator coupled to the command parser. The thread generator is operative to generate the plurality of threads, and dispatch the plurality of threads, where the processing of the plurality of thread for each stage includes storing write commands in the corresponding output command buffer or in the alternate command buffer.