A resource scheduling method and system

Abstract

The embodiment of the invention discloses a resource scheduling method and system. The method comprises the following steps: judging whether resource queues are priority state resource queues according to the priority corresponding to each job in the resource queues; when all resource queues are non-priority state resource queues, performing resource scheduling on all resource queues according toa fair scheduling policy; and at least one priority state resource queue exist in all resource queues, performing resource scheduling on all resource queues according to a priority scheduling policy.

Description

technical field [0001] The present invention relates to the field of resource scheduling, in particular to a resource scheduling method and system. Background technique [0002] With the continuous development of cloud computing big data technology, more and more enterprises meet the needs of data applications by applying large-scale distributed computing clusters. The use of large-scale distributed computing clusters can effectively overcome the scalability problem of single-node computing resources, and can effectively improve the efficiency of resource use by sharing clusters. At present, in a multi-tenant and multi-task scenario, the following existing technologies are often used to effectively allocate cluster physical resources to each computing task: [0003] (1) First-in-first-out scheduling strategy. All jobs are strictly in accordance with the submitted time sequence, and priority is given to satisfying the resource request of the job submitted first. After the jo...

AI Technical Summary

Problems solved by technology

[0007] (1) The first-in-first-out scheduling strategy is not suitable for multi-tenant scenarios, nor can high-priority jobs obtain more resources

[0008] (2) Although the fair scheduling strategy can ensure the fair allocation of resources in the multi-group household scenario, it cannot obtain more resources for tasks with high priority

[0009] (3) Although the priority scheduling strategy can enable high-priority tasks to obtain resources as soon as possible, it cannot guarantee the fairness among multi-tenant resource queues, and it is difficult to directly apply in multi-tenant scenarios

[0010] To sum up, although the existing large-scale cluster resource scheduling methods can achieve relative fairness in the use of multi-tenant resources, or achieve high-priority tasks to obtain resources as soon as possible, they cannot meet the needs of each tenant in a multi-tenant scenario. Under the premise of fair allocation of resources in the resource queue, at the same time, it can realize the demand for more resources for high-priority tasks

Images