Load-monitoring-based hybrid scheduling method under Hadoop cloud platform

A technology of load monitoring and hybrid scheduling, applied in resource allocation, multi-programming devices, etc., can solve the problems of resource occupation, failure to execute, and long average job completion time, so as to improve fairness, reduce scheduling consumption, and The effect of reducing the average completion time

Active Publication Date: 2013-12-18
金数信息科技(苏州)有限公司
View PDF4 Cites 10 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

The FIFO method is simple to implement and consumes less scheduling. However, when faced with massive data processing, some jobs that require a lot of calculations will occupy resources for a long time, and subsequent jobs

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Load-monitoring-based hybrid scheduling method under Hadoop cloud platform
  • Load-monitoring-based hybrid scheduling method under Hadoop cloud platform
  • Load-monitoring-based hybrid scheduling method under Hadoop cloud platform

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0039] The present invention will be further described below in conjunction with the accompanying drawings.

[0040] Assuming that there are m resources in the system, the number of idle resources is k, and the number of tasks to be executed is numT. The load degree of the system is defined as:

[0041] HL = numT m

[0042] If 0≤HL<1, the system is under low load;

[0043] If 1≤HL≤2, the system is in load balance;

[0044] If HL>2, the system is overloaded.

[0045] The hybrid scheduling strategy based on load monitoring includes a load monitor and a scheduling selector. The load monitor is responsible for monitoring and calculating the load of the system, and the scheduling selector schedules tasks according to the load conditions provided by the load monitor.

[0046] When there are idle resources requesting task execution, the scheduling selector will obtain the real-time system load status through the load...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a load-monitoring-based hybrid scheduling method under a Hadoop cloud platform. A load-monitoring-based hybrid scheduling scheme is provided by analyzing scheduling effects and respective application scenarios of a Max-D algorithm, an FIFO algorithm and a fair scheduling algorithm. The one the most proper for the current load condition is selected from the three algorithms by monitoring a system load in real time. Compared with the single application of one scheduling algorithm, the scheme has significant advantages and is adaptive to changes in the load in a Hadoop system, so that the system can retain good performance.

Description

technical field [0001] The invention relates to a cloud platform job scheduling method, in particular to a load monitoring-based hybrid job scheduling method under a Hadoop cloud platform. technical background [0002] Wireless sensory network technology is a technology that has gradually emerged in recent years. It integrates sensor technology, embedded computing technology, distributed information processing technology and communication technology, and is a new information acquisition and processing technology. It is different from traditional networks in terms of theoretical design, application realization, and development direction, so the theoretical basis and system design of sensor networks are very different from traditional network systems. Hadoop is a distributed system with high reliability and good scalability. It can run applications on clusters composed of a large number of cheap hardware devices, and provides a set of stable and reliable interfaces for applica...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
IPC IPC(8): G06F9/50
Inventor 李千目陆路侯君
Owner 金数信息科技(苏州)有限公司
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap