Distributed Resource Allocation via Negotiated Return Deadlines
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In distributed systems, the immediate recycling of resources from low-priority jobs to high-priority jobs leads to inefficient resource utilization, as it results in the termination of low-priority job execution and requires restarting from scratch, causing resource waste and reduced efficiency.
Innovation Solution
Implement a method where the resource scheduling server and job management server negotiate resource return times based on job execution information and progress, determining a resource return deadline to ensure efficient resource allocation and utilization without immediate recycling.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If the resource scheduling server immediately recycles resources from low-priority jobs to high-priority jobs, then high-priority jobs can quickly obtain resources, but low-priority jobs are terminated and must restart from scratch, reducing overall resource utilization efficiency
Solution Approach 1:
The system creates backup copies of low-priority job data before preemption occurs. This preliminary action ensures that when resources are reallocated to high-priority jobs, the low-priority jobs can resume from the backup position rather than restarting completely, thus maintaining resource utilization efficiency while still enabling quick resource allocation to high-priority jobs
Solution Approach 2:
The system changes the state parameter of low-priority jobs from 'running' to 'suspended' instead of terminating them. This parameter change allows the jobs to be paused during preemption and later resumed from their backup position, resolving the contradiction between fast resource allocation and maintaining resource utilization efficiency
2Reliability
If low-priority jobs are preempted to allocate resources to high-priority jobs, then high-priority resource applications can be satisfied, but the low-priority jobs lose their execution progress and system resource utilization decreases
Solution Approach 1:
The system performs preliminary backup of low-priority job data before preemption. This ensures that when jobs are suspended to make way for high-priority jobs, their execution progress is preserved in the backup, allowing them to resume later without losing progress, thus maintaining both job execution continuity and system resource utilization
Solution Approach 2:
The system creates a copy (backup) of the low-priority job data and stores it in a backup position. This copying mechanism ensures that the original job progress is preserved even when the job is suspended during preemption, maintaining execution continuity while enabling resource reallocation to high-priority jobs
Data Source
Figure 1~2
Figure 3~4
Figure 5
AI summary
A distributed system resource allocation method, apparatus, and system are provided in embodiments of this application. The method comprises: receiving a resource preemption request sent by a resource scheduling server, the resource preemption request comprising job execution information corresponding to a first job management server; determining, according to the job execution information corresponding to the first job management server and comprised in the resource preemption request, resources to be returned by a second job management server and a resource return deadline; and returning, according to the determined resources to be returned and resource return deadline and a current job execution progress of the second job management server, the resources to be returned to the resource scheduling server before expiration of the resource return deadline. With the method provided in the embodiments of this application, the utilization of system resources can be effectively improved, and resource waste can be reduced.