Graph Stream Processing Scheduler with Prefetch Invalidate Threads
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Centralized computing systems face limitations in executing computation-intensive applications due to serial processing, which restricts throughput, and managing data dependencies across multiple processors in parallel computing is challenging, especially when processors require data from previous executions.
Innovation Solution
A graph stream processing system with a scheduler that segments code into blocks, uses prefetch and invalidate threads to manage data buffers efficiently, and schedules threads based on resource availability, leveraging hardware implementation for performance advantages.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If code is executed in serial fashion on one processor, then system complexity is low, but throughput is limited to processor speed
Solution Approach 1:
The code is divided into smaller code blocks that can be executed independently in parallel. The scheduler segments the computation graph into stages, with each stage containing multiple code blocks that can be dispatched to different processors simultaneously, thereby increasing throughput while managing complexity through structured division.
Solution Approach 2:
The system transitions from single-processor sequential execution to multi-processor parallel execution by adding the dimension of concurrent processing. The scheduler manages multiple processors operating simultaneously on different code blocks, transforming the execution model from one-dimensional sequential to multi-dimensional parallel processing.
2Productivity
If multiple processors execute code in parallel, then throughput increases, but managing data dependencies becomes challenging
Solution Approach 1:
The system performs preliminary actions by pre-fetching data into buffers before code blocks need them and pre-invalidating buffers after data is consumed. The scheduler dispatches code blocks only when their required data is available in buffers, eliminating wait states and ensuring parallel execution can proceed without data dependency bottlenecks.
Solution Approach 2:
Buffers serve as intermediaries between processors and data storage. The scheduler manages these buffers to decouple data production from data consumption, allowing parallel code blocks to execute independently while maintaining correct data flow. The buffer management system mediates between multiple processors and the underlying data storage, abstracting away the complexity of direct data dependency management.
3Reliability
If processors wait for data from previous executions, then data accuracy is maintained, but processor idle time increases
Solution Approach 1:
Data is pre-fetched into buffers before the code blocks that need it are executed. The scheduler identifies data dependencies and initiates data loading operations in advance, so that when a code block is ready to execute, its required data is already available in the buffer, eliminating processor idle time while maintaining data accuracy.
Solution Approach 2:
The system maintains continuous useful action by overlapping data prefetching with code execution. While one set of code blocks is executing, the scheduler simultaneously prefetches data for subsequent code blocks into buffers. This continuous pipeline ensures processors remain busy executing code without idle wait states, while data accuracy is preserved through proper buffer management and invalidation protocols.
Data Source
AI summary
Systems, apparatuses, and methods are disclosed for scheduling threads comprising of code blocks in a graph streaming processor (GSP) system. One system includes a scheduler for scheduling plurality of prefetch threads, main threads, invalidate threads. The plurality of prefetch threads includes prefetching data from main memory required for execution of the main threads of the next stage. The plurality of main threads includes a set of instructions operating on the graph streaming processors of GSP system. The plurality of the invalidate threads includes invalidating data location/s consumed by the plurality of the main threads of the previous stage. A portion of the scheduler is implemented in hardware.