Minimum-cost maximum-flow based large-scale resource scheduling system and minimum-cost maximum-flow based large-scale resource scheduling method

A resource scheduling and maximum flow technology, applied in the field of big data resource management, can solve problems such as low resource utilization and job performance degradation, and achieve the effect of reducing the solution time

CN106708625AInactive Publication Date: 2017-05-24INST OF SOFTWARE - CHINESE ACAD OF SCI
3 Cites 15 Cited by

Patent Information

Authority / Receiving Office
CN · China
Current Assignee / Owner
Publication Date
2017-05-24
Estimated Expiration
Not applicable · inactive patent

Smart Images

  • Figure 1
    Figure 1
  • Figure 2
    Figure 2
  • Figure 3
    Figure 3
Patent Text Reader

Abstract

The invention relates to a minimum-cost maximum-flow based large-scale resource scheduling system and a minimum-cost maximum-flow based large-scale resource scheduling method. The system comprises a task state table, a cluster state table, a scheduling target table, a minimum-cost maximum-flow constructor, a minimum-cost maximum-flow solver and a task executor. The task state table is used for receiving and storing task states submitted by users, and the task states include task CPU (central processing unit) utilization rate, memory utilization rate, network I / O, magnetic disk I / O and priority. The cluster state table is used for storing cluster state information including cluster CPU utilization rate, memory utilization rate and network and magnetic disk I / O and updating cluster states when the cluster state change. The scheduling target table is used for storing scheduling targets configured by the users, and the scheduling targets include priority, placement constraint and fairness currently. The minimum-cost maximum-flow constructor is used for selecting the scheduling targets from the scheduling target table according to information of the task state table and the cluster state table to construct a minimum-cost maximum-flow graph. The minimum-cost maximum-flow solver is used for solving the minimum-cost maximum-flow graph constructed by the minimum-cost maximum-flow constructor according to an incremental algorithm. The task executor is responsible for specific execution of tasks. The minimum-cost maximum-flow based large-scale resource scheduling system and the minimum-cost maximum-flow based large-scale resource scheduling method meet the requirement on flexibility of practical business scenarios.
Need to check novelty before this filing date? Find Prior Art

Description

technical field

[0001] The invention relates to a large-scale resource scheduling system and method based on minimum cost and maximum flow, which belongs to the field of big data resource management, especially resource scheduling problems in large-scale environments. Background technique

[0002] With the rapid development of technologies such as the Internet (Internet) and the Internet of Things (IoT), data (Data) has begun to transform from simple processing objects to basic services, and multiple jobs (Job) are submitted at the same time and decomposed into parallel execution tasks (Task). , running processing on physical servers with a scale of at least 10,000 has become a mainstream application mode, which is called the "concurrent job" problem. For example, Github needs to process more than 20 million jobs every year, and Facebook responds to nearly 10,000 job requests every day.

[0003] Large-scale resource scheduling refers to the decision-making process of optima...

Examples

Embodiment Construction

[0035] In order to make the present invention easier to understand, the present invention will be further described in conjunction with an example, but this example does not constitute any limitation to the present invention.

[0036] as attached figure 1 , The technical solution of the present invention: a large-scale resource scheduling system based on minimum cost and maximum flow, including a task state table, a cluster state table, a scheduling target table, a minimum cost maximum flow constructor, a minimum cost maximum flow solver, and a task executor. in:

[0037] The task status table receives and saves the task status submitted by the user, including the CPU usage, memory usage, network I / O, disk I / O, and priority of the task;

[0038] The cluster status table stores cluster status information, including cluster CPU usage, memory usage, network and disk I / O, and updates the cluster status table when the cluster status changes;

[0039] Scheduling target table: stor...