Accelerator Task Scheduling via Data Flow Graphs
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current systems for programming GPUs face challenges due to programming difficulty, lack of modularity, and unpredictable performance artifacts, as they require explicit control over data movement, leading to trade-offs between performance and modularity.
Innovation Solution
An application programming interface (API) that encapsulates executable code snippets into accelerator tasks, generates a graph for data flow and dependencies, and abstracts memory using datablocks to schedule task execution across multiple accelerators, minimizing data copying.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If explicit control of data movement is implemented, then performance can be optimized, but programming complexity and difficulty increase
Solution Approach 1:
The patent introduces a compiler as an intermediary between the programmer and the GPU hardware. The compiler automatically generates data movement code based on algorithm code and execution schedules, eliminating the need for programmers to manually control data movement while still achieving optimized performance through automated code generation and scheduling
2Productivity
If data movement code is coupled with algorithm code, then performance optimization is possible, but modularity and reusability are sacrificed
Solution Approach 1:
The patent segments the programming model into distinct components: algorithm code that describes computation logic, and data movement code that handles data transfer. The compiler separates these concerns by automatically generating data movement operations based on the algorithm code and execution schedule, allowing algorithm code to remain modular and reusable while performance is optimized through automated code generation
3Productivity
If manual data movement control is required, then performance can be tuned, but the programming environment becomes less user-friendly
Solution Approach 1:
The patent implements a self-service programming model where the compiler automatically performs data movement optimization and code generation based on the provided algorithm code. The system serves itself by analyzing the computational graph, determining optimal execution schedules, and generating appropriate data movement operations without requiring manual intervention from the programmer
Data Source
AI summary
An application programming interface is provided that allows programmers to encapsulate snippets of executable code of a program into accelerator tasks. A graph is generated with a node corresponding to each of the accelerator tasks with edges that represent the data flow and data dependencies between the accelerator tasks. The generated graph is used by a scheduler to schedule the execution of the accelerator tasks across multiple accelerators. The application programming interface further provides an abstraction of the various memories of the accelerators called a datablock. The programmer can store and use data stored on the datablocks without knowing where on the accelerators the data is stored. The application programming interface can further schedule the execution of accelerator tasks to minimize the amount of data that is copied to and from the accelerators based on the datablocks and the generated graph.