Job scheduling method based on cluster node load state prediction

A technology of job scheduling and load status, which is applied in the field of information technology processing, can solve problems such as energy waste, and achieve the effects of reducing energy consumption, improving resource utilization, and improving energy efficiency

Inactive Publication Date: 2019-08-06
SHANDONG UNIV OF SCI & TECH
View PDF9 Cites 10 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0002] The load level of Hadoop cluster nodes is relatively low during non-peak hours. In most cases, the computing power of the cluster will far exceed the amou

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Job scheduling method based on cluster node load state prediction
  • Job scheduling method based on cluster node load state prediction
  • Job scheduling method based on cluster node load state prediction

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0053] Below in conjunction with accompanying drawing and specific embodiment the present invention is described in further detail:

[0054] 1. Load analysis and node task processing characteristics

[0055] In order to complete the energy-saving scheduling of tasks, it is necessary to perceive the resource consumption of computing nodes, and improve the resource allocation strategy on the premise of obtaining the resource consumption characteristics of pending tasks.

[0056] 1.1. Node load analysis

[0057] Measures the load impact of different types of processing tasks, denoted by the parameter θ. use theta c Indicates the impact of the CPU component, with θ m Denote the impact of the memory component, with θ o Indicates the influence of other components of the system.

[0058] (1) Single node load

[0059] Assuming that the load of a node i in the Hadoop cluster is expressed as CPU load with Indicates that the load of memory with Indicates that other system loa...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a job scheduling method based on cluster node load state prediction, belonging to the field of information technology processing, provides a job scheduling method based on cluster node load state prediction comprising the following steps: firstly, introducing the design idea of the algorithm, and dividing nodes of a cluster into queues of active nodes and queues of dormantnodes according to the load state of the prediction algorithm; then, dynamically activating dormant nodes according to the resource quantity required by calculation so as to achieve the purpose of dynamically adjusting the cluster scale; describing the flow of an algorithm, the steps of the algorithm and the implementation of pseudo codes of the algorithm, and then analyzing the energy-saving effect of the algorithm; and introducing the experimental verification process, including experimental design and data set selection, and finally carrying out quantitative analysis on experimental results, so that the method provided by the invention has an energy-saving effect with practical significance.

Description

technical field [0001] The invention belongs to the field of information technology processing, and in particular relates to a job scheduling method based on cluster node load state prediction. Background technique [0002] The load level of Hadoop cluster nodes is relatively low during non-peak hours. In most cases, the computing power of the cluster will far exceed the amount of resources required for input computing tasks, which leads to the fact that many computing nodes may always run at a low The load is running, resulting in a waste of energy. Contents of the invention [0003] Aiming at the above-mentioned technical problems existing in the prior art, the present invention proposes a job scheduling method based on the prediction of cluster node load status, which is reasonably designed, overcomes the deficiencies of the prior art, and has good effects. [0004] In order to achieve the above object, the present invention adopts the following technical solutions: ...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
IPC IPC(8): G06F9/48G06F9/50G06N3/04G06N3/08
CPCG06F9/4881G06F9/5016G06F9/5027G06N3/08G06F2209/5019G06N3/044G06N3/045Y02D10/00
Inventor 张金泉杨振宇禹继国韩庆亮倪丽娜张福新
Owner SHANDONG UNIV OF SCI & TECH
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap