Multi-tenant resource optimization scheduling method facing different types of loads

A resource optimization and scheduling method technology, applied in resource allocation, multi-program device, program control design, etc., can solve the problems of long response time, low parallel efficiency, and not fully considering different load types of jobs, etc., to improve throughput , high resource utilization, and reduced response time

Active Publication Date: 2017-03-15
SOUTH CHINA UNIV OF TECH
View PDF6 Cites 37 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

At the same time, most of the existing delay scheduling algorithms are based on fixed waiting time, and do not fully consider the different load types of jobs
Assuming that most of the data of the job is concentrated on a certain node or a small number of data nodes in the cluster, computing tasks may also be concentrated on a certain node or some nodes in the cluster, resulting in low parallel efficiency and long response time of the job time

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Multi-tenant resource optimization scheduling method facing different types of loads
  • Multi-tenant resource optimization scheduling method facing different types of loads
  • Multi-tenant resource optimization scheduling method facing different types of loads

Examples

Experimental program
Comparison scheme
Effect test

example

[0035] In order to verify the effectiveness of the multi-tenant resource optimization scheduling method for different types of loads, we simulated and used the proposed scheduling method to analyze and compare the optimized job response time and cluster throughput.

[0036] Now select four ordinary PCs on the same rack to build a Hadoop cluster, one machine as the master node, and the other three as slave nodes (computing nodes), assuming that each computing node has only one resource slot, at any time a certain There is only one task running on each node, and the tasks on the nodes are in a completely serial relationship, so the running time of tasks on a certain node can be directly accumulated.

[0037] First, establish a mathematical model to analyze the effectiveness of judging whether to perform delayed scheduling according to different load types. Assuming that a task takes D seconds longer to run on a non-local node than on a local node, the arrival of the job request ...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention relates to a multi-tenant resource optimization scheduling method facing different types of loads. The multi-tenant resource optimization scheduling method comprises the following steps: 1, submitting jobs by system tenants and adding the jobs into a job queue; 2, collecting job load information and sending the job load information to a resource manager; 3, judging different load types of the jobs according to the job load information by the resource manager, and sending type information to a job scheduler; 4, carrying out job scheduling according to the different load types by the job scheduler; if the job is a calculation intensive type job, scheduling at the node; if the job is an I / O intensive type job, delaying and waiting; and 5, collecting job scheduling decision-making information to a scheduling reconstruction decision-making device, reconstructing a target calculation node, and carrying out the job scheduling according to a final decision-making result. According to the method provided by the invention, a multi-tenant shared cluster is realized, so that the cost of establishing an independent cluster is reduced, and meanwhile, a plurality of tenants can share more big data set resources. The better data locality is realized facing optimization of the different types of loads, and the balance between the equity and efficiency in a job scheduling process can be realized very well; and calculation performances of the whole cluster, such as throughput rate and job responding time, are improved.

Description

technical field [0001] The invention relates to a multi-tenant resource management technology, in particular to a multi-tenant resource optimization scheduling method for different types of loads. Background technique [0002] In recent years, network information technology has advanced by leaps and bounds, and the contradiction between massive digital information and people's ability to obtain the information they need has become increasingly prominent. On the other hand, the application of distributed computing frameworks such as MapReduce, Dryad, and Spark makes it possible for users to find correct information from massive databases within a tolerable time. Traditional web multi-tenant systems need to process complex business logic, and therefore need to ensure a high level of data consistency and support complex data queries. In contrast, a multi-tenant system based on a big data platform often has the characteristics of a very large amount of data, does not require st...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G06F9/50
CPCG06F9/5083
Inventor 林伟伟温昂展张子龙张国强李进
Owner SOUTH CHINA UNIV OF TECH
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products