Compiler Scheduler for Hardware Accelerator Data Movement
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Hardware accelerators often lack memory management logic, leading to inefficient data movement and idle periods due to reliance on software drivers and lack of control flow features, which increases development time and cost.
Innovation Solution
A compiler-based scheduler determines optimal operation schedules for hardware accelerators by analyzing data flow graphs, estimating execution times, and managing memory usage to keep execution engines busy, even in the absence of memory management logic.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If hardware accelerators lack memory management logic to reduce device complexity and cost, then device complexity and manufacturing cost are reduced, but data movement efficiency deteriorates and idle periods increase
Solution Approach 1:
The patent introduces a compiler-based scheduler as an intermediary that manages data movement and scheduling for hardware accelerators. The scheduler analyzes data flow graphs, estimates execution times, and generates optimized schedules that coordinate data transfer between memory and execution engines, compensating for the absence of hardware memory management logic.
Solution Approach 2:
The compiler performs preliminary scheduling and optimization before code execution. It pre-calculates execution times, determines optimal data movement schedules, and prepares execution plans in advance, allowing the hardware accelerator to execute efficiently without runtime memory management overhead.
2Device complexity
If hardware accelerators lack control flow features to simplify hardware design, then device complexity is reduced, but software development time increases
Solution Approach 1:
The patent replaces hardware control flow mechanisms with software-based scheduling. The compiler scheduler generates detailed execution schedules and data movement instructions that substitute for hardware control flow logic, allowing simple hardware to achieve complex control functionality through carefully crafted software schedules.
3Device complexity
If software drivers manage data movement for hardware accelerators, then device complexity is reduced, but execution efficiency deteriorates due to idle periods
Solution Approach 1:
The system achieves self-service optimization through the compiler scheduler, which automatically analyzes the specific computational workload, generates optimized execution schedules, and coordinates data movement without requiring external software drivers. This self-contained approach eliminates the idle periods associated with driver-mediated data transfer.
Solution Approach 2:
The scheduler incorporates feedback mechanisms by analyzing execution time estimates and workload characteristics to continuously optimize scheduling decisions. It adjusts data movement timing and execution scheduling based on predicted performance, ensuring minimal idle time while maintaining hardware simplicity.
Data Source
AI summary
Scheduling of the operations of an integrated circuit device such as a hardware accelerator, including scheduling of movement of data into and out of the accelerator, can be performed by a compiler that produces program code for the accelerator. The compiler can produce a graph that represents operations to be performed by the accelerator. Using the graph, the compiler can determine estimated execution times for the operations represented by each node in the graph. The compiler can schedule operations by determining an estimated execution time for set of dependent operations that depend from an operation. The compiler can then select an operation that has a shortest estimated execution time from among a set of operations and which has a set of dependent operations that has a longest estimated execution time as compared to other sets of dependent operations.