Multi-Level Static Scheduling for Neural Network Pipelines
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing FPGA systems face challenges in efficiently scheduling large-scale neural networks due to the complexity of manual process scheduling and the exponential growth of scheduling complexity, which can lead to non-convergent scheduling issues.
Innovation Solution
The proposed solution involves statically scheduling a neural network using a multi-level approach, comprising an upper level that establishes a sequential order of functions, an intermediate level that forms a systolic array with balanced data flow channels, and a lower level where the HLS compiler assigns operations to hardware elements, allowing for concurrent execution.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If manual scheduling with complex state machines is used for HDL approach, then scheduling control is achieved, but time consumption increases and scheduling may not converge for large scale systems
Solution Approach 1:
The patent segments the scheduling process into three distinct hierarchical levels: upper level (layer-level scheduling), intermediate level (process-level scheduling within layers), and lower level (instruction-level scheduling within processes). This segmentation divides the complex monolithic scheduling problem into manageable sub-problems, enabling convergence for large-scale FPGA systems while reducing overall scheduling time through parallel processing at different levels.
2Device complexity
If single thread software function scheduling is used, then implementation simplicity is maintained, but scheduling complexity grows exponentially with system scale
Solution Approach 1:
The patent introduces a multi-dimensional hierarchical scheduling architecture that adds temporal and organizational dimensions to the scheduling process. By organizing scheduling across three levels (upper, intermediate, lower) and enabling concurrent execution of multiple processes within the same layer, the system transforms exponential complexity growth into polynomial scaling, significantly improving scheduling throughput for massively parallel FPGA systems.
3Productivity
If static scheduling is used to enable concurrent execution, then throughput is improved, but scheduling complexity increases
Solution Approach 1:
The patent performs preliminary static scheduling at compile time, establishing the execution order and timing of all processes before runtime. The multi-level scheduling approach pre-determines which processes execute concurrently and their relative timing, allowing hardware elements to operate in parallel during runtime without requiring complex dynamic scheduling mechanisms, thus achieving high throughput with manageable complexity.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
Embodiments herein describe techniques for static scheduling a neural network (100) implemented in a massively parallel hardware system (205). The neural network (100) may be scheduled using three different scheduling levels referred to herein as an upper level, an intermediate level, and a lower level. In one embodiment, the upper level includes a hardware or software model (400) of the layers in the neural network (100) that establishes a sequential order of functions that operate concurrently in the hardware system (205). In the intermediate level, identical processes in the functions defined in the upper level are connected to form a systolic array (280) or mesh and balanced data flow channels are used to minimize latency. In the lower level, a compiler (265) can assign the operations performed by the processing elements in the systolic array to different portions of the hardware system (205) to provide a static schedule for the neural network (100).