Equipment cluster scaling performance prediction method and device

Abstract

The invention provides an equipment cluster scaling performance prediction method and device which are used for improving the equipment cluster scaling performance prediction precision. The method comprises the steps that prediction equipment acquires a prediction reference information set, wherein the prediction reference information set is obtained by testing the process that at least one equipment in a first equipment cluster process data in advance and comprises operating resource parameters correspondingly needed when different rounds of each stage contain different task numbers in the process that one equipment processes the data in different stages; the prediction equipment predicts actual resource parameters needing to be consumed when each equipment in a second equipment cluster executes allocated tasks in different stages in the data processing process on the basis of the prediction reference information set; and the prediction equipment predicts the resource consuming condition when the second equipment cluster processes the data on the basis of the actual resource parameters of each equipment, wherein the second equipment cluster is a virtual simulation cluster obtained by scaling the first equipment cluster.

Description

technical field [0001] The present application relates to the field of computers, and in particular to a method and device for predicting performance of equipment cluster expansion. Background technique [0002] With the development of science and technology and the Internet, the amount of information in modern society is increasing rapidly, which accumulates large-scale data, and some of these data will be stored in the cloud platform or processed with the help of the cloud platform. With the help of Hadoop, users can efficiently store, manage and analyze these storages by writing distributed parallel programs and running them on a device cluster composed of multiple devices without knowing the underlying details of the distribution. Data in the cloud platform. [0003] Hadoop is a software framework capable of distributed processing of a large amount of data. At the bottom is the distributed file system (Hadoop Distributed File System, HDFS), which uses distributed storag...

AI Technical Summary

Problems solved by technology

[0004] At present, the expansion performance of equipment clusters is generally predicted based on algorithm and architecture prototype verification, which can only realize functional simulation or qualitative prediction of expansion performance trends, and quantitative prediction cannot be achieved

Therefore, the prediction accuracy of the expansion performance of the equipment cluster in the prior art is low

Images