Cross-data center task scheduling and bandwidth allocation method based on hypergraph partition

Abstract

The invention relates to the technical field of cloud computing. In order to solve the task scheduling problem of cross-regional data centers, the scheduling results of tasks in the data center are optimized, thereby minimizing the overall completion time. For this reason, in the present invention, the cross-data center task scheduling and bandwidth allocation method based on hypergraph segmentation is composed of a task scheduling control part and a network bandwidth allocation part, and the task scheduling control part is responsible for realizing tasks between data centers based on network interconnection Scheduling deployment, the task scheduling part periodically detects the arrival of jobs and starts the scheduling algorithm, and generates a scheduling plan by combining the data center resource capacity and task completion status collected by current monitoring; the network traffic control part is divided into two categories: real-time online application Data transfer during interactive network streams and task scheduling non-interactive network streams. The invention is mainly applied to communication control occasions.

Description

technical field [0001] The invention relates to the technical field of cloud computing, in particular to the field of task scheduling and optimization of data centers. Specifically, it relates to a cross-data center task scheduling and bandwidth allocation method based on hypergraph partitioning. Background technique [0002] The latest trends show that cloud computing data centers are developing towards super-large scale, and the deployment form is also shifting from a single centralized deployment to a global distributed deployment. At the same time, in the face of cross-regional data centers, it is urgent to optimize the scheduling algorithm to better integrate the computing, storage and network resources of the data center, so as to achieve efficient resource scheduling and task deployment, and improve the production efficiency of the data center. But cross-regional data center management must also address a series of issues such as wide-area distributed data, data shar...

AI Technical Summary

Problems solved by technology

[0003] At present, there have been many studies on resource management and task scheduling in data centers, such as Greedy, CDS[1], HPS[2], Flutter[3] and other scheduling methods, but these methods have more or less Few defects, difficult to adapt to complex scheduling needs

Greedy uses a greedy algorithm to schedule tasks based on the idea of ​​maximum data localization; CDS models task scheduling as a community discovery problem, and achieves the optimal goal of data localization through an iterative scheduling algorithm. However, both Greedy and CDS Ignoring the potential impact of task scheduling order on WAN data transmission volume, resulting in excessive data volume transmission; HPS adopts the idea of ​​hypergraph segmentation to achieve the balance of the overall time of each data center at the cost of increasing the total amount of file transmission. At the same time, it focuses on the situation where the resource needs are much greater than the resource capacity, and its scheduling performance is not good in scenarios with sufficient computing resources; Flutter formalizes the scheduling of cross-center centers as a lexicographic minimum-maximum integer linear programming problem, but its The solution process requires a series of restrictive conditions, resulting in a poor scope of application, and the ability to solve scheduling scenarios with insufficient bandwidth is poor

Images