Grid Computing Job Migration for Resource Fragmentation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Grid computing environments face inefficiencies due to resource fragmentation when scheduling jobs across multiple machines, as existing methods restrict jobs to single machines, preventing optimal utilization of available resources across clusters.
Innovation Solution
A method for managing resources in a grid computing environment involves determining if a new job can be run on a cluster with sufficient resources, and if not, migrating a currently running job to another cluster with available resources, using a scale across module that includes a resource monitor, checkpoint module, and job migrator to efficiently allocate resources.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If jobs are restricted to single machines in grid computing, then job scheduling is simplified, but resource utilization efficiency deteriorates due to fragmentation
Solution Approach 1:
The system segments the grid computing environment into multiple computing clusters, each capable of independently executing jobs. The job scheduling system is divided into cluster-level schedulers and a grid-level coordinator, allowing localized decision-making while maintaining global resource optimization. This segmentation enables jobs to be confined to single clusters for simplicity while still allowing resource sharing across the broader grid.
Solution Approach 2:
The invention introduces a new dimension to job scheduling by enabling jobs to scale across multiple computing clusters rather than being confined to a single machine. This dimensional expansion from single-machine to multi-cluster execution allows the system to overcome resource fragmentation while maintaining scheduling manageability through hierarchical control structures.
2Productivity
If jobs are allowed to scale across multiple machines, then resource utilization efficiency improves, but job scheduling complexity increases
Solution Approach 1:
The scheduling system is segmented into hierarchical levels: individual cluster schedulers handle local job placement and resource management, while a grid-level coordinator manages cross-cluster job migration and resource allocation. This segmentation distributes scheduling complexity across multiple manageable components rather than requiring a single complex centralized scheduler.
Solution Approach 2:
The system implements dynamic job migration capabilities that allow jobs to move between computing clusters based on real-time resource availability and workload conditions. This dynamic behavior enables the system to adapt to changing conditions and optimize resource utilization without requiring static, pre-configured job assignments across multiple machines.
3Ease of operation
If earliest-start criteria is used for job scheduling, then job scheduling simplicity is maintained, but resource fragmentation increases reducing grid efficiency
Solution Approach 1:
The system performs preliminary resource assessment and job pre-positioning actions before actual job execution. By evaluating cluster resource availability in advance and pre-positioning jobs in optimal clusters, the system avoids the resource fragmentation problems associated with simple earliest-start criteria while maintaining operational simplicity through automated decision-making rules.
Solution Approach 2:
The scheduling system incorporates feedback mechanisms that monitor resource utilization across computing clusters and adjust job allocation decisions accordingly. This feedback loop allows the system to respond to resource fragmentation conditions by migrating jobs to underutilized clusters, thereby maintaining grid efficiency while preserving the simplicity of automated scheduling operations.
Data Source
AI summary
A method, grid computing environment, and computer readable medium for managing available resources in a grid computing environment are disclosed. The method includes determining, when a new job is submitted, if a first of the computing clusters has sufficient available resources to run the new job. If the first computing cluster does not have sufficient available resources to run the new job, the method further includes determining if a second of the computing clusters has sufficient available resources to run a first job that is currently running on the first computing cluster. If the second computing cluster has sufficient available resources to run the first job, the first job is migrated to the second computing cluster.