Generative Flow Network Neural Network Scheduling
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional scheduling systems for artificial neural networks face challenges in efficiently assigning operations to resources, leading to variations in computation time due to the use of proxies that may not accurately represent hardware characteristics, resulting in unsatisfactory performance and high computational costs.
Innovation Solution
A processor-implemented method using a generative flow network (GFlowNet) to generate and test multiple schedules based on a computation graph, selecting a schedule that satisfies a selection condition, thereby reducing makespan and improving resource utilization.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of energy
If conventional scheduling systems use proxies to evaluate schedules, then computational costs are reduced, but scheduling accuracy and performance become unsatisfactory due to inaccurate representation of hardware characteristics
Solution Approach 1:
The patent creates a simplified proxy model that copies essential hardware characteristics (processor speeds, memory capacities, communication latencies) to enable efficient schedule evaluation. This proxy serves as a lightweight representation that maintains sufficient accuracy for scheduling decisions without requiring full hardware complexity, thus reducing computational costs while preserving scheduling precision.
2Measurement precision
If multiple schedules are generated and tested on actual hardware, then scheduling accuracy improves, but computational costs and time consumption increase significantly
Solution Approach 1:
The patent performs preliminary schedule evaluation using the proxy model before committing to actual hardware testing. By pre-filtering and ranking schedules based on proxy evaluations, the system identifies promising candidates that are then tested on actual hardware. This preliminary action reduces the number of expensive hardware tests needed while maintaining high scheduling accuracy.
Solution Approach 2:
The patent evaluates multiple schedules through proxy modeling (excessive action in the cheap domain) to ensure sufficient coverage of the schedule space, then selectively tests only the most promising candidates on actual hardware. This partial hardware testing approach maintains productivity while achieving high scheduling precision through the combination of extensive proxy evaluation and targeted hardware validation.
3Productivity
If a single schedule is selected without extensive testing, then productivity is maintained, but reliability of task execution decreases due to variations in computation time
Solution Approach 1:
The patent performs preliminary evaluation of multiple schedules using proxy models to predict computation times and identify potential reliability issues before execution. By pre-assessing schedule robustness through proxy-based simulation, the system selects schedules that are more likely to execute reliably without requiring extensive actual hardware testing, thus maintaining productivity while improving reliability.
Data Source
AI summary
A processor-implemented method includes generating, by a scheduling model, a group of schedules from a computation graph associated with a task, each node on the computation graph being associated with an operation of an artificial neural network, each schedule of the group of schedules associating each node of the computation graph with a processor of a group of processors of a hardware device. The processor-implemented method also includes testing one or more schedules of the group of schedules on the hardware device or a model of the hardware device. The processor-implemented method further includes selecting a schedule of the one or more schedules based on testing the one or more schedules, the selected schedule satisfying a selection condition.