Neural Network Scheduling for Computation Graphs Without Retraining
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing neural network computation tasks require significant computational resources and time due to the large number and size of operations, and existing scheduling techniques often necessitate retraining for each new computation graph, consuming additional resources.
Innovation Solution
A system that combines a neural network policy with an optimization algorithm, such as a genetic algorithm, to generate instance-specific proposal distributions for scheduling computation graphs across multiple devices, reducing resource consumption and execution time.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If existing scheduling techniques are used for computation graphs, then scheduling can be performed, but retraining is required for each new computation graph, consuming additional computational resources and time
Solution Approach 1:
The system performs preliminary training on a diverse set of computation graph structures during an offline phase, enabling the neural network to generalize to new computation graphs without requiring retraining. This preliminary action stores learned scheduling patterns that can be directly applied to unseen computation graphs, eliminating the need for repeated retraining while maintaining adaptability.
Solution Approach 2:
The neural network scheduler is designed with a universal architecture that can handle multiple types of computation graphs and scheduling scenarios through a single model. By training on diverse computation graph structures and operation types, the system achieves multi-functionality where one trained model serves all scheduling needs across different computation graphs, eliminating the need for separate training for each new graph type.
2Measurement precision
If neural networks are used to process computation tasks, then accurate predictions can be made, but significant computational resources and time are consumed due to the large number and size of operations
Solution Approach 1:
The scheduling problem is segmented into multiple independent components: operation-level scheduling decisions, device assignment decisions, and graph-level coordination. The neural network makes localized scheduling decisions for individual operations and nodes independently, rather than optimizing the entire computation graph as a single unit. This segmentation enables parallel processing of scheduling decisions across different computation graph nodes, significantly improving execution speed while maintaining scheduling quality through local optimization.
3Reliability
If more computational resources are allocated to scheduling, then better schedules can be generated, but the time and resources consumed in generating the schedule increase
Solution Approach 1:
The neural network scheduler is designed to be self-sufficient after initial training, requiring no additional retraining or resource-intensive optimization processes when faced with new computation graphs. The model directly generates high-quality schedules by applying learned patterns from training data to new inputs, making autonomous scheduling decisions without requiring external retraining resources or time-consuming iterative optimization, thus achieving both high reliability and fast schedule generation.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
Methods, systems, and apparatus, including computer programs encoded on computer storage media, for generating a schedule for a computation graph. One of the methods includes obtaining data representing an input computation graph; processing the data representing the input computation graph using a graph neural network to generate one or more instance-specific proposal distributions; and generating a schedule for the input computation graph by performing an optimization algorithm in accordance with the one or more instance-specific proposal distributions.