Supercomputer Task Scheduler Using Offline Reinforcement Learning
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current scheduling methods for supercomputers require manual intervention and are inefficient, as they rely on expert-defined policies and online reinforcement learning, which is slow to adapt and often results in suboptimal task distribution, and the use of simulators is impractical due to the complexity of supercomputers.
Innovation Solution
Implementing offline reinforcement learning using a trained scheduler based on execution histories from other supercomputers, allowing for automated scheduling without the need for a supercomputer simulator, by utilizing a training database with state, action, and reward data to optimize task allocation and resource utilization.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Extent of automation
If expert-defined scheduling policies are used, then the scheduling can be established manually, but it requires continuous expert intervention when supercomputer characteristics or usage patterns change
Solution Approach 1:
The scheduling system performs self-service through automated policy establishment using reinforcement learning. The scheduler autonomously learns optimal scheduling policies by interacting with the supercomputer environment and receiving feedback through reward signals, eliminating the need for continuous expert intervention while adapting to changing supercomputer characteristics and usage patterns
Solution Approach 2:
The system implements feedback mechanisms where the scheduler receives reward signals based on scheduling performance metrics. This feedback loop enables the scheduler to continuously learn and adjust its policies automatically, improving adaptability to changes in supercomputer characteristics without requiring expert reconfiguration
2Extent of automation
If online reinforcement learning is used to automate scheduling, then expert intervention is reduced, but the scheduler interacts slowly with the supercomputer resulting in suboptimal task distribution for long periods
Solution Approach 1:
The system performs preliminary action by pre-training the scheduler offline using historical execution data from the supercomputer. This pre-training phase allows the scheduler to learn optimal policies in advance without interfering with live supercomputer operations, so that when deployed, it can immediately provide high-quality task distribution from the first moment rather than requiring lengthy online learning periods
Solution Approach 2:
The system creates a copy of the supercomputer's operational data through execution histories stored in a database. The scheduler is trained on these copied historical data rather than interacting directly with the live supercomputer during training, enabling offline learning that doesn't impact real system productivity while still learning from actual supercomputer usage patterns
3Extent of automation
If a supercomputer simulator is used for training, then online interaction is avoided, but creating a high-fidelity simulator is impractical due to supercomputer complexity
Solution Approach 1:
Instead of creating a complex simulator, the system copies actual execution data from the real supercomputer into a database. This approach avoids the need to model supercomputer complexity while still providing realistic training data, as the scheduler learns from actual historical execution patterns rather than simulated approximations
Solution Approach 2:
The system inverts the traditional approach by not trying to simulate the supercomputer forward, but instead using historical execution data that already captures the supercomputer's behavior. This inversion allows training without simulation while maintaining fidelity to actual supercomputer operations
Data Source
AI summary
A method for scheduling computing tasks on a supercomputer including offline reinforcement learning (OFRL) of a scheduler on a database (LDB). The database includes at least one execution history (HIST) that includes the state (LHPCS) of a learning supercomputer at several moments (T, T−1); the actions (LACT) related to the scheduling of learning tasks on the learning supercomputer at those moments (T); and a reward (REW) related to each task. The method also includes the use of the scheduler trained on the computing tasks to be scheduled.