Method and device for predicting hardware resource use ratio of equipment cluster

Abstract

The invention discloses a method and a device for predicting the hardware resource use ratio of an equipment cluster, and is used for improving accuracy for predicting the hardware resource use ratioof the equipment cluster. The method comprises the following steps that: prediction equipment obtains the performance feature library of a first equipment cluster, wherein the performance feature library comprises execution duration and resource expenditures generated when equipment is under an idle state and any one task is operated, the execution duration is required when different rounds of each stage contain different task amounts in different stages of one piece of equipment processing data contained in the first equipment cluster; the prediction equipment predicts practical execution duration required when each piece of equipment in a second equipment cluster executes tasks independently distributed in different stages in a data processing process according to the execution duration;and the prediction equipment predicts the hardware resource use ratio of the second equipment cluster on the basis of the practical execution duration, the resource expenditures and the configurationparameter of the second equipment cluster, wherein the second equipment cluster is a virtual simulation cluster obtained on the basis of the expansion of the first equipment cluster.

Description

technical field [0001] The present application relates to the field of computers, and in particular to a method and device for predicting the utilization rate of hardware resources of a device cluster. Background technique [0002] With the development of science and technology and the Internet, the amount of information in modern society is increasing rapidly, which accumulates large-scale data, and some of these data will be stored in the cloud platform or processed with the help of the cloud platform. With the help of Hadoop, users can efficiently store, manage and analyze these storages by writing distributed parallel programs and running them on a device cluster composed of multiple devices without knowing the underlying details of the distribution. Data in the cloud platform. [0003] Hadoop is a software framework capable of distributed processing of a large amount of data. At the bottom is the distributed file system (Hadoop Distributed File System, HDFS), which use...

AI Technical Summary

Problems solved by technology

[0004] At present, the prediction of the hardware resource utilization rate of the equipment cluster after expansion is generally based on the algorithm and architecture prototype verification, which can only realize functional simulation or qualitative prediction, and cannot achieve quantitative prediction.

Therefore, in the prior art, the prediction accuracy of the hardware resource utilization rate of the device cluster after expansion is low

Images