Cross-data center task scheduling and bandwidth allocation method based on hypergraph segmentation

Abstract

The invention relates to the technical field of cloud computing, and aiming to solve the task scheduling problem of a cross-regional data center and optimize the scheduling result of a task in the data center so as to minimize the overall completion time, the invention relates to a cross-data center task scheduling and bandwidth allocation method based on hypergraph segmentation. The method is composed of a task scheduling control part and a network bandwidth distribution part, the task scheduling control part is responsible for realizing scheduling and deploying the tasks among the data centers based on network interconnection, periodically detects the arrival of a job and starts a scheduling algorithm, and generates a scheduling scheme by combining the information, such as a data centerresource capacity and a task completion state which are acquired by current monitoring, etc.; and the network flow control part is divided into two types of a real-time online application interactivenetwork flow and a data transmission non-interactive network flow in a task scheduling process. The method is mainly applied to the communication control occasions.

Description

technical field [0001] The invention relates to the technical field of cloud computing, in particular to the field of task scheduling and optimization of data centers. Specifically, it relates to a cross-data center task scheduling and bandwidth allocation method based on hypergraph partitioning. Background technique [0002] The latest trends show that cloud computing data centers are developing towards super-large scale, and the deployment form is also shifting from a single centralized deployment to a global distributed deployment. At the same time, in the face of cross-regional data centers, it is urgent to optimize the scheduling algorithm to better integrate the computing, storage and network resources of the data center, so as to achieve efficient resource scheduling and task deployment, and improve the production efficiency of the data center. But cross-regional data center management must also address a series of issues such as wide-area distributed data, data shar...

AI Technical Summary

Problems solved by technology

[0003] At present, there have been many studies on resource management and task scheduling in data centers, such as Greedy, CDS[1], HPS[2], Flutter[3] and other scheduling methods, but these methods have more or less Few defects, difficult to adapt to complex scheduling needs

Greedy uses a greedy algorithm to schedule tasks based on the idea of ​​maximum data localization; CDS models task scheduling as a community discovery problem, and achieves the optimal goal of data localization through an iterative scheduling algorithm. However, both Greedy and CDS Ignoring the potential impact of task scheduling order on WAN data transmission volume, resulting in excessive data volume transmission; HPS adopts the idea of ​​hypergraph segmentation to achieve the balance of the overall time of each data center at the cost of increasing the total amount of file transmission. At the same time, it focuses on the situation where the resource needs are much greater than the resource capacity, and its scheduling performance is not good in scenarios with sufficient computing resources; Flutter formalizes the scheduling of cross-center centers as a lexicographic minimum-maximum integer linear programming problem, but its The solution process requires a series of restrictive conditions, resulting in a poor scope of application, and the ability to solve scheduling scenarios with insufficient bandwidth is poor

Images