Dynamic Resource Quota Allocation for Data Center Job Prioritization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Data centers face inefficiencies in resource utilization during system failures or maintenance, leading to impaired user operations and resource underutilization.
Innovation Solution
A method and system that identify high-availability and low-availability jobs within a distributed system, determining quotas for resource usage based on normal operations and tolerated events, allowing for resource migration, re-execution, or data reconstruction to maintain accessibility and efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If resources are allocated to high-availability jobs during normal operations, then service continuity is improved, but resource utilization efficiency deteriorates due to over-provisioning
Solution Approach 1:
The system dynamically adjusts resource quotas for high-availability jobs based on system state. During normal operations, HA jobs are constrained to a first quota to allow resource utilization. When failures occur, the system transitions to a second quota that ensures sufficient resources for service continuity. This dynamic adjustment resolves the contradiction by making resource allocation flexible rather than static.
Solution Approach 2:
The invention changes the parameter of resource quota allocation based on system conditions. The resource manager modifies the quota parameters for HA jobs from a conservative first quota during normal operations to a more generous second quota when failures are detected, thereby adapting resource allocation to actual needs and resolving the efficiency-reliability tradeoff.
2Productivity
If resources are constrained during normal operations to allow utilization, then resource efficiency is improved, but system resilience to failures deteriorates
Solution Approach 1:
The system performs preliminary actions by pre-configuring multiple quota levels (first quota for normal operations, second quota for failure scenarios) before failures occur. The resource manager is prepared with knowledge of alternative quota allocations, enabling rapid response to failures without compromising normal resource efficiency. This preliminary preparation resolves the contradiction by having resilience strategies ready in advance.
Solution Approach 2:
The invention implements beforehand cushioning by maintaining the capability to switch to alternative quota allocations that provide a buffer against failures. The system cushions against potential failures by having pre-planned resource allocation strategies (second quota) that ensure service continuity, while not permanently allocating those resources during normal operations, thus maintaining both efficiency and resilience.
3Reliability
If resource quotas are increased for high-availability jobs, then service continuity during failures is improved, but resource underutilization during normal operations worsens
Solution Approach 1:
The system employs dynamic resource quota adjustment, switching between a first quota (lower, for normal operations) and a second quota (higher, for failure scenarios). During normal operations, the lower quota prevents resource underutilization. When failures occur, the system transitions to the higher quota to ensure service continuity. This dynamic behavior resolves the contradiction by matching resource allocation to actual system needs in real-time.
Data Source
Figure 1A
Figure 1B
Figure 2
AI summary
A method (600) includes identifying high-availability jobs (122, 122a) and low- availability jobs (122, 122b) that demand usage of resources (110, 112, 114, 116, 422, 424, 426, 432, 434, 436) of a distributed system (100). The method includes determining a first quota (Q1) of the resources available to low-availability jobs as a quantity of the resources available during normal operations, and determining a second quota (Q2) of the resources available to high-availability jobs as a quantity of the resources available during normal operations minus a quantity of the resources lost due to a tolerated event. The method includes executing the jobs on the distributed system and constraining a total usage of the resources by both the high-availability jobs and the low-availability jobs to the quantify of the resources available during normal operations.