Energy-saving storage method for cold and hot area division based on cluster node load state prediction

A technology of area division and load status, applied in the field of information technology processing, can solve the problems of occupying computing node storage resources and energy waste, and achieve the effect of reducing operating costs, energy consumption, and total energy consumption.

Inactive Publication Date: 2020-05-05
SHANDONG UNIV OF SCI & TECH
View PDF1 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

This causes some computing nodes to run at a lower load, resulting in wasted energy
At the same time, the data blocks stored by the data nodes will become cold data for a considerable proportion of the time, and these data will occupy the storage resources of the computing nodes

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Energy-saving storage method for cold and hot area division based on cluster node load state prediction
  • Energy-saving storage method for cold and hot area division based on cluster node load state prediction
  • Energy-saving storage method for cold and hot area division based on cluster node load state prediction

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0032] Below in conjunction with accompanying drawing and specific embodiment the present invention is described in further detail:

[0033] There are general rules for data block access in Hadoop clusters. The application obtains the general explanation of the law by consulting the literature, and the application designs the storage strategy of the Hadoop cluster data block according to the data access law.

[0034] 1. Data block storage design idea

[0035] Access to a data block generally occurs within a short period of time after it is uploaded. As time goes by, the data block will become colder and the probability of being accessed will decrease. Verifying the rationality of this law is of great help to the design of storage strategies for data blocks.

[0036] 1.1, Hadoop cluster data block storage analysis

[0037] (1) Analysis of copy storage mechanism

[0038] For a data block Data, the default copy storage mechanism of the Hadoop cluster will store its copies in ...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

Disclosed is a cold and hot area dividing energy-saving storage method based on cluster node load state prediction, belonging to the field of technical processing of information. The concept of the method is introduced first in the present application, then the execution steps of the method are described, and a pseudocode of the method is realized, next the energy-saving effect of an energy-saving policy is theoretically analyzed, then the procedure of an experiment is introduced, wherein the procedure comprises an environment of the experiment, selection of a data set and an access format of a data block, and finally a result of the experiment is analyzed. By means of quantitative analysis and comparison with energy consumption, in the case where cold and hot areas are not divided, in a Hadoop cluster, it is verified that an energy-saving storage policy of the present application has a certain actual energy-saving effect.

Description

technical field [0001] The invention belongs to the field of information technology processing, and in particular relates to an energy-saving storage method for dividing hot and cold areas based on cluster node load state prediction. Background technique [0002] The development of cloud computing technology has continuously expanded the scale and number of data centers, and at the same time, it has also brought about the problem of higher and higher energy consumption costs of data center clusters. In order to achieve data reliability in the Hadoop cluster, each data block has multiple backups. As time goes by, the access frequency of data blocks gradually decreases, and the computing load of nodes also gradually decreases. This causes some computing nodes to run at a lower load, resulting in wasted energy. At the same time, the data blocks stored by the data nodes will become cold data for a considerable proportion of the time, and these data will occupy the storage reso...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Patents(China)
IPC IPC(8): G06F9/48G06F9/50
CPCG06F9/4893G06F9/5088G06F2209/5019Y02D10/00
Inventor 倪丽娜韩庆亮禹继国张金泉
Owner SHANDONG UNIV OF SCI & TECH
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap