Batch Scheduler Virtual Machine Migration
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current batch scheduling systems in high-performance computing often overutilize certain resources while underutilizing others, leading to inefficient resource allocation and potential loss of work when compute nodes become resource-constrained.
Innovation Solution
A batch scheduler generates and deploys virtual machines (VMs) to optimize resource allocation by choreographing task distribution across compute nodes, allowing dynamic rebalancing and migration of VMs to ensure high-priority tasks are processed efficiently, without discarding active processes.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If tasks are submitted directly to the base operating system or unmanaged virtual machines, then the batch scheduling system is simple to operate, but resource utilization becomes unbalanced with overutilization of certain resources and underutilization of others
Solution Approach 1:
The patent introduces a batch scheduler as an intermediary layer between the operating system and compute nodes. This scheduler manages virtual machine lifecycles and task distribution, balancing resource allocation across the computing cluster while maintaining ease of operation through centralized control.
Solution Approach 2:
The system segments compute nodes into managed virtual machines, each handling specific tasks. This segmentation allows independent resource allocation and management for each VM, enabling balanced resource utilization across the entire system while maintaining operational simplicity.
2Productivity
If compute nodes are resource constrained, then resource allocation efficiency improves, but tasks must be discarded and rerun on other compute nodes causing loss of time
Solution Approach 1:
The batch scheduler dynamically monitors resource availability and VM performance, continuously adjusting task allocation in real-time. When resource constraints are detected, the scheduler dynamically migrates tasks to alternative VMs or compute nodes without discarding task state, thereby maintaining efficiency while avoiding time loss.
Solution Approach 2:
The system performs preliminary resource allocation and VM provisioning before tasks are submitted. By pre-configuring multiple VM instances and monitoring their resource availability, the scheduler can quickly redirect tasks to ready VMs when constraints occur, preventing time loss from task discarding and rerunning.
3Productivity
If the batch scheduling system manages virtual machines directly, then resource utilization is optimized, but the device complexity increases
Solution Approach 1:
The batch scheduler is designed as a universal system that handles multiple functions: VM lifecycle management, task scheduling, resource allocation, and dynamic migration. By consolidating these diverse functions into a single multi-functional scheduler, the system achieves optimized resource utilization without proportionally increasing complexity.
Solution Approach 2:
The scheduler manages VM parameters such as resource allocation, priority levels, and migration thresholds. By systematically controlling these parameters through automated policies, the system achieves complex resource optimization goals without requiring complex manual intervention or system architecture.
4Productivity
If tasks are discarded to free up resources, then resource constraints are relieved, but work is lost and must be rerun
Solution Approach 1:
Instead of discarding tasks when resources are constrained, the system discards only the task-to-VM binding and recovers the task for reallocation to another VM. This approach relieves resource constraints while preserving task integrity and work progress, maintaining both resource availability and work completion reliability.
Data Source
AI summary
A batch scheduler receives a task. Further, a processor, at the batch scheduler, generates a virtual machine corresponding to the task. In addition, the virtual machine is deployed from the batch scheduler to a compute node. The task is also dispatched from the batch scheduler to the virtual machine at the compute node.