A Job Scheduling Method Based on Cluster Node Load Status Prediction

A technology for job scheduling and load status, applied in the field of information technology processing, can solve problems such as energy waste, and achieve the effects of reducing energy consumption, improving cluster access performance, and improving energy efficiency

Inactive Publication Date: 2020-03-06
SHANDONG UNIV OF SCI & TECH
View PDF9 Cites 2 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0002] The load level of Hadoop cluster nodes is relatively low during non-peak hours. In most cases, the computing power of the cluster will far exceed the amount of resources required for input computing tasks, which leads to many computing nodes may always be at a low of the load is running, resulting in wasted energy

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • A Job Scheduling Method Based on Cluster Node Load Status Prediction
  • A Job Scheduling Method Based on Cluster Node Load Status Prediction
  • A Job Scheduling Method Based on Cluster Node Load Status Prediction

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0054] Below in conjunction with accompanying drawing and specific embodiment the present invention is described in further detail:

[0055] 1. Load analysis and node task processing characteristics

[0056] In order to complete the energy-saving scheduling of tasks, it is necessary to perceive the resource consumption of computing nodes, and improve the resource allocation strategy on the premise of obtaining the resource consumption characteristics of pending tasks.

[0057] 1.1. Node load analysis

[0058] Measures the load impact of different types of processing tasks, denoted by the parameter θ. use theta c Indicates the impact of the CPU component, with θ m Denote the impact of the memory component, with θ o Indicates the influence of other components of the system.

[0059] (1) Single node load

[0060] Assuming that the load of a node i in the Hadoop cluster is expressed as CPU load with Indicates that the load of memory with Indicates that other system loa...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a job scheduling method based on cluster node load status prediction, which belongs to the field of information technology processing. The invention proposes a job scheduling algorithm based on cluster node load status prediction. The load state of the prediction algorithm is divided into active node queues and dormant node queues; then the dormant nodes are dynamically activated according to the amount of resources required by the calculation to achieve the purpose of dynamically adjusting the cluster size; then the algorithm flow and algorithm steps are described. and the pseudo-code implementation of the algorithm, and then analyzed the energy-saving effect of the algorithm; and introduced the process of experimental verification, including the design of the experiment, the selection of data sets, and finally quantitatively analyzed the experimental results, proving the application's The method has a practical energy-saving effect.

Description

technical field [0001] The invention belongs to the field of information technology processing, and in particular relates to a job scheduling method based on cluster node load state prediction. Background technique [0002] The load level of Hadoop cluster nodes is relatively low during non-peak hours. In most cases, the computing power of the cluster will far exceed the amount of resources required for input computing tasks, which leads to the fact that many computing nodes may always run at a low The load is running, resulting in a waste of energy. Contents of the invention [0003] Aiming at the above-mentioned technical problems existing in the prior art, the present invention proposes a job scheduling method based on the prediction of cluster node load status, which is reasonably designed, overcomes the deficiencies of the prior art, and has good effects. [0004] In order to achieve the above object, the present invention adopts the following technical solutions: ...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Patents(China)
IPC IPC(8): G06F9/48G06F9/50G06N3/04G06N3/08
CPCG06F9/4881G06F9/5016G06F9/5027G06N3/08G06F2209/5019G06N3/044G06N3/045Y02D10/00
Inventor 张金泉杨振宇禹继国韩庆亮倪丽娜张福新
Owner SHANDONG UNIV OF SCI & TECH
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap