Storage performance prediction method, device, equipment and medium

Abstract

The present application provides a storage performance prediction method, apparatus, device, and medium, relating to the field of storage system technology. The method can train a random forest classifier to be trained comprising multiple decision trees using a sample data set including real indicator parameters of multiple storage systems and corresponding real input and output volumes and real response times to obtain a pre-trained storage performance prediction model. The pre-trained storage performance prediction model predicts the storage performance of the storage system to be tested based on the performance data of the storage system to be tested, and obtains a predicted value and a predicted category of the storage performance in a first prediction result of the storage performance of the storage system to be tested, thereby enabling the input and output volumes and response time of the storage system to be quickly and efficiently predicted.

Description

Technical Field

[0001] The present application relates to the technical field of storage systems, and in particular to a storage performance prediction method, apparatus, device, and medium. Background Art

[0002] With the rapid development of the big data era, data storage has also received significant attention. As businesses grow, data volumes will continue to grow, necessitating sufficient storage capacity to meet business needs. Monitoring the storage capacity of enterprise cloud disks can help companies plan expansion plans in a timely manner and avoid storage space shortages. Implementing measures such as data cleansing, compression, and archiving can also improve storage utilization and extend the lifespan of storage space. Centralized storage performance metrics include read and write speeds, throughput, and latency. Regularly monitoring storage device performance can help companies identify potential issues and implement timely optimization and adjustments. Companies can perform performance testing on storage devices to identify bottlenecks and optimize storage architectures, upgrade hardware, or adjust storage policies to improve performance and ensure a better user experience. Due to the numerous parameters that influence storage performance, traditional storage performance prediction methods are time-consuming and labor-intensive, hindering accurate prediction of storage performance parameters and, consequently, hindering the accuracy of storage performance prediction results. Summary of the Invention

[0003] In view of this, embodiments of the present application provide a storage performance prediction method, device, and medium to overcome the above-mentioned problems or at least partially solve the above-mentioned problems.

[0004] A first aspect of an embodiment of the present application provides a storage performance prediction method, the method comprising:

[0005] Obtain the index parameters of the storage system to be tested;

[0006] The index parameters of the storage system to be tested are input into a pre-trained storage performance prediction model to obtain a first prediction result of the storage performance of the storage system to be tested, wherein the pre-trained storage performance prediction model is obtained by training a random forest classifier to be trained containing multiple decision trees using a sample data set, and the sample data set includes real index parameters of multiple storage systems and corresponding real input and output quantities and real response times, and the first prediction result includes a predicted category of storage performance and a predicted value of storage performance.

[0007] Optionally, the method further includes:

[0008] Before training the random forest classifier to be trained, an original data set comprising a plurality of original samples is obtained, each original sample comprising input data and output data; the input data is the real indicator parameter, and the output data is the real input and output quantity and the real response time;

[0009] Performing data cleaning on multiple original samples of the original data set, removing missing values ​​and outliers in the multiple original samples, and performing mean-variance normalization processing on the multiple cleaned samples to obtain a sample data set, wherein the sample data set includes multiple samples;

[0010] Construct a random forest classifier to be trained containing multiple decision trees;

[0011] The sample data set is divided to obtain multiple sample data subsets, each of which includes a training sample set and a validation sample set.

[0012] Optionally, dividing the sample data set to obtain multiple sample data subsets includes:

[0013] For each decision tree, extracting a plurality of samples from the sample data set with replacement as the training sample set;

[0014] taking a plurality of samples not extracted from the sample data set as a validation sample set;

[0015] The multiple samples included in the training sample set are all randomly selected from the sample data set.

[0016] Optionally, constructing a random forest classifier to be trained comprising multiple decision trees includes:

[0017] For each decision tree in the random forest classifier to be trained, arbitrarily select a first indicator parameter from multiple samples in the training sample set corresponding to the decision tree as a first feature, and use the first feature as the root node of the decision tree;

[0018] Dividing a plurality of samples in a training sample set corresponding to the decision tree according to the first feature to obtain a plurality of first subsets, wherein the first subsets include a plurality of samples having the same first feature value, the first feature corresponds to a plurality of first feature values, and each first subset corresponds to a first feature value;

[0019] For each child node in the decision tree, arbitrarily select a second indicator parameter other than the first indicator parameter;

[0020] generating a second feature based on the second indicator parameter, and using the second feature as a child node of the decision tree;

[0021] According to the second feature, each first subset in the plurality of first subsets is divided into a plurality of second subsets, wherein the second subset is smaller than the first subset;

[0022] When the number of samples corresponding to the child nodes of the decision tree reaches a minimum number of samples or the decision tree reaches a maximum depth, it is determined that the creation of the random forest classifier to be trained is completed.

[0023] Optionally, training the random forest classifier to be trained comprises the following steps:

[0024] For each decision tree, input data of the training sample set corresponding to the decision tree into the decision tree, so that the input data passes through the root node of the decision tree and reaches the leaf node of the decision tree, and record the second prediction result corresponding to the leaf node where the input data reaches;

[0025] Comparing the second prediction result with the output data corresponding to the input data, and calculating a first absolute error value, where each sample corresponds to a first absolute error value;

[0026] Adjusting parameters of the random forest classifier to be trained based on the first absolute error value;

[0027] Testing the random forest classifier to be trained after the parameter adjustment using the verification sample set to obtain a third prediction result corresponding to a leaf node reached by the input data of the sample in the verification sample set;

[0028] Comparing the third prediction result with the output data corresponding to the input data of the sample in the validation sample set, and calculating a second absolute error value, where each sample corresponds to a second absolute error value;

[0029] For the multiple decision trees of the random forest classifier to be trained, the mean absolute error value is calculated for the multiple second absolute error values. When the mean absolute error value is less than a preset threshold, it is determined that the training of the pre-trained storage performance prediction model is completed.

[0030] Optionally, inputting the index parameters of the storage system to be tested into a pre-trained storage performance prediction model to obtain a first prediction result of the storage performance of the storage system to be tested includes:

[0031] Inputting the index parameters of the storage system to be tested into multiple decision trees of a pre-trained storage performance prediction model, and recording multiple third prediction results corresponding to multiple leaf nodes reached by the index parameters of the storage system to be tested, wherein each decision tree corresponds to one third prediction result;

[0032] An average result or a majority result of the plurality of third prediction results is used as a first prediction result of the storage performance of the storage system to be tested.

[0033] Optionally, the method further includes:

[0034] Before training the random forest classifier to be trained, searching for a combination of the number of the decision trees and the number of leaf nodes of the decision trees;

[0035] In the process of training the random forest classifier to be trained, traversing multiple combinations in an ascending order of the number of decision trees, and determining the model performance of the random forest classifier to be trained corresponding to each combination;

[0036] Based on the model performance, the optimal number of decision trees and the number of leaf nodes of the decision trees are determined.

[0037] A second aspect of an embodiment of the present application provides a storage performance prediction device, the device comprising:

[0038] An acquisition module is used to obtain the index parameters of the storage system to be tested;

[0039] A prediction module is used to input the indicator parameters of the storage system to be tested into a pre-trained storage performance prediction model to obtain a first prediction result of the storage performance of the storage system to be tested, wherein the pre-trained storage performance prediction model is obtained by training a random forest classifier to be trained containing multiple decision trees through a sample data set, and the sample data set includes the real indicator parameters of multiple storage systems and the corresponding real input and output quantities and real response time, and the first prediction result includes the predicted category of storage performance and the predicted value of storage performance.

[0040] A third aspect of an embodiment of the present application provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the storage performance prediction method as described in the first aspect.

[0041] A fourth aspect of an embodiment of the present application provides a computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the storage performance prediction method as described in the first aspect is implemented.

[0042] Beneficial effects of this application:

[0043] An embodiment of the present application provides a storage performance prediction method, apparatus, device, and medium, comprising: obtaining indicator parameters of a storage system to be tested; inputting the indicator parameters of the storage system to be tested into a pre-trained storage performance prediction model to obtain a first prediction result of the storage performance of the storage system to be tested, wherein the pre-trained storage performance prediction model is obtained by training a random forest classifier to be trained that includes multiple decision trees using a sample data set, the sample data set including real indicator parameters of multiple storage systems and corresponding real input and output quantities and real response times, and the first prediction result includes a predicted category of storage performance and a predicted value of storage performance.

[0044] A pre-trained storage performance prediction model can be obtained by training a random forest classifier to be trained that includes multiple decision trees using the real indicator parameters of multiple storage systems and the corresponding real input and output volumes and real response times in a sample data set. The pre-trained storage performance prediction model can be used to predict the storage performance of the storage system to be tested based on the performance data of the storage system to be tested, and the predicted value and the predicted category of the storage performance in the first prediction result of the storage performance of the storage system to be tested can be obtained, thereby enabling the input and output volumes and response time of the storage system to be predicted quickly and efficiently. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] The drawings that constitute a part of this application are used to provide further understanding of this application. The illustrative embodiments of this application and their descriptions are used to explain this application and do not constitute improper limitations on this application.

[0046] In order to more clearly illustrate the technical solution of the present application, the following is a brief introduction to the drawings required for the description of the present application. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0047] Figure 1 This is a flow chart of a storage performance prediction method provided by an embodiment of the present application;

[0048] Figure 2 This is a schematic diagram of various nodes for training a random forest classifier in a storage performance prediction method provided in one embodiment of the present application;

[0049] Figure 3 This is a schematic diagram of predicting a first prediction result in a storage performance prediction method provided by an embodiment of the present application;

[0050] Figure 4 This is a structural diagram of a storage performance prediction device provided in one embodiment of the present application. DETAILED DESCRIPTION

[0051] It should be noted that, unless there is any conflict, the embodiments and features in the embodiments of this application can be combined with each other.

[0052] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0053] Figure 1 FIG. 1 is a flow chart of a storage performance prediction method provided by an embodiment of the present application. Figure 1 As shown, an embodiment of the present application provides a storage performance prediction method, the method comprising:

[0054] Step S101, obtaining index parameters of the storage system to be tested;

[0055] Step S102: Input the index parameters of the storage system to be tested into a pre-trained storage performance prediction model to obtain a first prediction result of the storage performance of the storage system to be tested, wherein the pre-trained storage performance prediction model is obtained by training a random forest classifier to be trained that includes multiple decision trees using a sample data set, and the sample data set includes real index parameters of multiple storage systems and corresponding real input and output quantities and real response times, and the first prediction result includes a predicted category of storage performance and a predicted value of storage performance.

[0056] First, in step S101, the index parameters of the storage system to be tested are obtained. The index parameters are parameters that can affect the performance of the storage system, which are determined after pre-collecting and analyzing relevant data on the storage performance of multiple different types of storage systems. The index parameters of the storage system may include but are not limited to: number of CPU cores, CPU main frequency, memory main frequency, memory size, RAID type, number of disks, disk read and write speed, external card transmission speed, cache partitioning and automatic tiering.

[0057] Furthermore, in step S102, the acquired index parameters of the storage system are input into a pre-trained storage performance prediction model. Through the storage system prediction model, a first prediction result of the storage performance of the storage system to be tested can be obtained. The first prediction result can represent the storage performance of the storage system, such as the predicted value of the input and output volume and the predicted value of the response time, as well as the predicted category of the storage performance of the storage system, for example, excellent storage performance, good storage performance, average storage performance, poor storage performance, etc.

[0058] Among them, the pre-trained storage performance prediction model has pre-learned how to predict the storage performance of the storage system based on its index parameters. The training process of the storage performance prediction model is obtained by training a random forest classifier to be trained containing multiple decision trees through a sample data set. The sample data set includes the real index parameters of multiple storage systems and the corresponding real input and output volumes (IOPS, Input / Output Operations Per Second) and real response time. The first prediction result includes the predicted category of storage performance and the predicted value of storage performance. The storage performance prediction model is obtained by training a random forest classifier to be trained containing multiple decision trees. The sample data set used for training includes the real index parameters of multiple storage systems and the corresponding real input and output volumes and real response time.

[0059] The actual input and output as well as the actual response data can be obtained in the following ways:

[0060] 1. Collect data through actual testing: Use specialized testing tools or equipment to simulate actual workloads and record the corresponding input, output, and response times.

[0061] 2. Simulation: In the absence of actual hardware, simulation can be used to create sample data sets. This typically involves simulating a performance model of the storage system to predict input and output volumes and response times under different workloads.

[0062] 3. Historical data analysis: Extract samples from the historical performance data contained in the existing storage system. During this process, the historical performance data needs to be analyzed and processed in detail to ensure the accuracy and representativeness of the data.

[0063] 4. Open source datasets: Open source datasets usually contain a large amount of real-time input, output, and response time data, which can be used as reference data for training or testing models.

[0064] Through the above embodiments, a random forest classifier to be trained including multiple decision trees can be trained using the real indicator parameters of multiple storage systems and the corresponding real input and output volumes and real response times in a sample data set to obtain a pre-trained storage performance prediction model. The storage performance of the storage system to be tested is predicted based on the performance data of the storage system to be tested through the pre-trained storage performance prediction model, and the predicted value and predicted category of the storage performance in the first prediction result of the storage performance of the storage system to be tested are obtained, so that the input and output volumes and response time of the storage system can be predicted quickly and efficiently.

[0065] Optionally, the method further includes:

[0066] Before training the random forest classifier to be trained, an original data set comprising a plurality of original samples is obtained, each original sample comprising input data and output data; the input data is the real indicator parameter, and the output data is the real input and output quantity and the real response time;

[0067] Performing data cleaning on multiple original samples of the original data set, removing missing values ​​and outliers in the multiple original samples, and performing mean-variance normalization processing on the multiple cleaned samples to obtain a sample data set, wherein the sample data set includes multiple samples;

[0068] Construct a random forest classifier to be trained containing multiple decision trees;

[0069] The sample data set is divided to obtain multiple sample data subsets, each of which includes a training sample set and a validation sample set.

[0070] Figure 2 Schematic diagram of various nodes for training a random forest classifier in a storage performance prediction method provided by an embodiment of the present application, such as Figure 2 shown.

[0071] The technical solution of this application uses ensemble learning to predict storage performance data. The entire process is briefly described as follows:

[0072] 1. Data processing: creating a sample data set from the original sample set and preprocessing the data;

[0073] 2. Data partitioning: split the processed sample data set into a training sample set and a validation sample set;

[0074] 3. Model creation, creating a random forest classifier, building multiple decision trees, and conducting modeling research on sample data sets;

[0075] 4. Model training: Select appropriate hyperparameters to train the most appropriate random forest classifier model as the trained storage performance prediction model;

[0076] 5. Model evaluation: Use evaluation indicators to evaluate the prediction accuracy of the model.

[0077] Specifically, in one embodiment, before training the random forest classifier to be trained, it is necessary to first construct a sample data set for training. First, it is necessary to obtain an original data set containing multiple original samples, each original sample includes input data and output data, and each original sample is obtained by testing the actual storage system using a special testing tool or equipment. The same storage system can obtain multiple original samples. The input data is the real indicator parameter of the storage system, and the output data is the real input and output volume and the real response time.

[0078] Furthermore, after obtaining the original data set, data cleaning is required for multiple original samples. The purpose of data cleaning is to remove missing values ​​and outliers from multiple original samples. Missing values ​​refer to the lack of values ​​in a certain field in the original sample, while outliers refer to values ​​that are significantly different from other data in the original sample. Missing values ​​and outliers can be identified by drawing box plots, scatter plots, or using statistical methods (such as IQR, Z-score, etc.). Once missing values ​​and outliers are identified, they can be deleted, filled, or corrected according to the specific situation.

[0079] Furthermore, after data cleaning, the cleaned multiple samples are subjected to mean-variance normalization processing to obtain a sample data set, which includes multiple samples. The normalization processing can eliminate the differences between features of different scales, which helps to improve the convergence speed and generalization ability of the model.

[0080] Next, a random forest classifier containing multiple decision trees is constructed for training. Random forest is an ensemble learning method that improves model accuracy and stability by building multiple decision trees and combining their predictions. When constructing the random forest classifier, parameters such as the decision tree depth and splitting criteria can be initialized to control the complexity of the storage performance prediction model.

[0081] Finally, the sample data set is divided into multiple sample data subsets, each of which contains a training sample set and a validation sample set. The number of sample data subsets can be set according to the parameters for constructing the random forest classifier, and the proportion of the training sample set and the validation sample set in each sample data subset can be set according to the actual situation. For example, 80% of the data in the sample data subset is used as the training sample set for model training, and 20% of the data is used as the validation sample set for model verification.

[0082] Through the above embodiment, before training the random forest classifier, a sample data set can be obtained through data processing, and a random forest classifier to be trained can be constructed. Finally, the sample data set is divided to obtain multiple sample data subsets, thereby completing the preparation process before training the model.

[0083] Optionally, dividing the sample data set to obtain multiple sample data subsets includes:

[0084] For each decision tree, extracting a plurality of samples from the sample data set with replacement as the training sample set;

[0085] taking a plurality of samples not extracted from the sample data set as a validation sample set;

[0086] The multiple samples included in the training sample set are all randomly selected from the sample data set.

[0087] Specifically, in one embodiment, the obtained sample data set can be divided according to the number of decision trees, each decision tree corresponds to a sample data subset, and the sample data subset construction process of each decision tree (for example, it can be extracted by Bootstrap sampling) is carried out by randomly extracting multiple samples from the sample data set with replacement each time and adding them to the training sample set, so that each decision tree is trained on the entire sample data set, which helps to improve the generalization ability of the storage performance prediction model. Moreover, since multiple extractions are performed with replacement, the training sample sets are slightly different, and the training data of each decision tree will also be different. This increases the diversity of each decision tree, because different decision trees are trained based on different training sample sets, which helps to reduce overfitting and improve the generalization ability of the storage performance prediction model. Moreover, after extracting multiple samples with replacement multiple times, the multiple samples that were not extracted in the sample training set can be used as the verification sample set of the decision tree, and the verification sample set can be used to evaluate the performance of the random forest classifier.

[0088] Through the above embodiment, it is possible to ensure that each decision tree is trained on the entire sample data set, and the performance of the storage performance prediction model is evaluated by using the validation sample set. This design helps to improve the generalization ability and prediction accuracy of the storage performance prediction model.

[0089] Optionally, constructing a random forest classifier to be trained comprising multiple decision trees includes:

[0090] For each decision tree in the random forest classifier to be trained, arbitrarily select a first indicator parameter from multiple samples in the training sample set corresponding to the decision tree as a first feature, and use the first feature as the root node of the decision tree;

[0091] Dividing a plurality of samples in a training sample set corresponding to the decision tree according to the first feature to obtain a plurality of first subsets, wherein the first subsets include a plurality of samples having the same first feature value, the first feature corresponds to a plurality of first feature values, and each first subset corresponds to a first feature value;

[0092] For each child node in the decision tree, arbitrarily select a second indicator parameter other than the first indicator parameter;

[0093] generating a second feature based on the second indicator parameter, and using the second feature as a child node of the decision tree;

[0094] According to the second feature, each first subset in the plurality of first subsets is divided into a plurality of second subsets, wherein the second subset is smaller than the first subset;

[0095] When the number of samples corresponding to the child nodes of the decision tree reaches a minimum number of samples or the decision tree reaches a maximum depth, it is determined that the creation of the random forest classifier to be trained is completed.

[0096] Specifically, in one embodiment, for each decision tree in the constructed random forest classifier, a first indicator parameter is first randomly selected from the training sample set corresponding to the decision tree as a first feature, and the first feature is used as the root node of the decision tree. The first feature is used to divide the training sample set corresponding to the decision tree into multiple first subsets, where the samples contained in each first subset have the same first feature value. The first feature corresponds to multiple first feature values, and each first subset corresponds to one first feature value. The features of the root nodes of different decision trees are not necessarily the same.

[0097] For example, for one of the decision trees, the number of CPU cores in the indicator parameter is used as the first indicator parameter. At this time, the number of CPU cores is the first feature. Therefore, the training sample set is divided according to the number of CPU cores to obtain multiple first subsets. The number of CPU cores of the samples contained in different first subsets is different. The first feature corresponds to multiple first eigenvalues, that is, the number of CPU cores can be divided into single-core, dual-core, quad-core, eight-core, and sixteen-core. Therefore, single-core, dual-core, quad-core, eight-core, and sixteen-core are all first eigenvalues. The first subset with the first eigenvalue of dual-core indicates that the indicator parameters of multiple samples therein are all dual-core.

[0098] Furthermore, for each child node in the decision tree, a second indicator parameter other than the first indicator parameter is arbitrarily selected, a second feature is generated based on the second indicator parameter, and the second feature is used as the child node of the decision tree. For example, since the number of CPU cores is already the root node of this decision tree, when selecting the feature of the child node directly connected to the root node, it is selected from the indicator parameters other than the number of CPU cores, such as memory size. Therefore, the second feature is the memory size.

[0099] Furthermore, each of the multiple first subsets is further divided based on the second feature, and each first subset is divided into multiple second subsets, and the second subsets are smaller than the first subsets. For example, for a sub-node whose second feature is memory size, the samples in the first subset that have been classified by the number of CPU cores can be further classified according to memory size, and the memory sizes of different second subsets are different. The features of other sub-nodes are selected and the subsets are divided in this way until the sample corresponding to the sub-node of the decision tree is the minimum number of samples or the decision tree reaches the maximum depth, for example, the number of samples in the subset is 1 and further division cannot be performed, or the sub-node of the decision tree with the most recently assigned feature is a leaf node and further division cannot be performed. At this time, it is determined that the creation of the random forest classifier to be trained is complete.

[0100] Through the above embodiment, a random forest classifier consisting of multiple decision trees can be constructed. Each decision tree selects a feature from the sample dataset and divides the dataset into smaller subsets based on this feature. The next feature is then selected, and this process is repeated until a predetermined tree depth is reached or other stopping criteria are met.

[0101] Optionally, training the random forest classifier to be trained comprises the following steps:

[0102] For each decision tree, input data of the training sample set corresponding to the decision tree into the decision tree, so that the input data passes through the root node of the decision tree and reaches the leaf node of the decision tree, and record the second prediction result corresponding to the leaf node where the input data reaches;

[0103] Comparing the second prediction result with the output data corresponding to the input data, and calculating a first absolute error value, where each sample corresponds to a first absolute error value;

[0104] Adjusting parameters of the random forest classifier to be trained based on the first absolute error value;

[0105] Testing the random forest classifier to be trained after the parameter adjustment using the verification sample set to obtain a third prediction result corresponding to a leaf node reached by the input data of the sample in the verification sample set;

[0106] Comparing the third prediction result with the output data corresponding to the input data of the sample in the validation sample set, and calculating a second absolute error value, where each sample corresponds to a second absolute error value;

[0107] For the multiple decision trees of the random forest classifier to be trained, the mean absolute error value is calculated for the multiple second absolute error values. When the mean absolute error value is less than a preset threshold, it is determined that the training of the pre-trained storage performance prediction model is completed.

[0108] Specifically, in one embodiment, for each decision tree in the random forest, the corresponding training sample set is first input into the decision tree. The input data (i.e., indicator parameters) of the input sample starts from the root node and propagates along the path of the decision tree until it reaches the leaf node. In this process, it will propagate according to the different features set, and finally record the second prediction result predicted by the leaf node corresponding to the input sample. The second prediction result of each leaf node includes the prediction category and prediction value of the sample in the training sample set (i.e., the predicted value of the input and output volume and the predicted value of the response time and the predicted category of the storage system performance corresponding to the sample).

[0109] Furthermore, the second prediction result is compared with the output data corresponding to the input data of the sample (i.e., the actual input and output volume and the actual response time, as well as the actual storage performance category), and the first absolute error value is calculated, and each sample corresponds to a first absolute error value.

[0110] Furthermore, the parameters of the random forest classifier to be trained are adjusted based on the calculated first absolute error value, such as changing the structure of the tree, increasing or decreasing the number of trees, adjusting the depth of the tree, etc.

[0111] Furthermore, the random forest classifier to be trained after parameter adjustment is tested through the verification sample set to obtain a third prediction result corresponding to the leaf node where the input data of the sample in the verification sample set arrives. The third prediction result includes the prediction category and prediction value of the sample in the verification sample set (that is, the predicted value of the input and output volume and the predicted value of the response time and the prediction category of the storage system performance corresponding to the sample).

[0112] Furthermore, the third prediction result is compared with the output data corresponding to the input data of the sample in the verification sample set (the actual input and output volume, the actual response time of the sample in the verification sample set, and the actual category of the storage system corresponding to the sample), and the second absolute error value is calculated, and each sample corresponds to a second absolute error value.

[0113] Finally, for multiple decision trees of the random forest classifier to be trained, the average absolute error value is calculated using multiple second absolute error values. When the calculated average absolute error value is less than a preset threshold, it can be determined that the pre-trained storage performance prediction model has been trained.

[0114] The mean absolute error value can be selected from the mean absolute percentage error EMAPE and the root mean square error ERMSE. The mean absolute percentage error EMAPE represents the percentage of the average deviation between the predicted value and the actual value. The root mean square error is used to measure the deviation between the observed value and the true value. The two are calculated as follows, where N represents the number of samples and y n Represents the actual output data, represents the prediction result, and ∑ represents the summation operation.

[0115]

[0116] Because EMAPE and ERMSE use percentage and square root to measure errors, respectively, the direct correlation between them is not obvious. However, from a broader perspective, they are both effective tools for measuring forecasting performance, but with different focuses and sensitivities.

[0117] Optionally, the step S102 includes:

[0118] Inputting the index parameters of the storage system to be tested into multiple decision trees of a pre-trained storage performance prediction model, and recording multiple third prediction results corresponding to multiple leaf nodes reached by the index parameters of the storage system to be tested, wherein each decision tree corresponds to one third prediction result;

[0119] An average result or a majority result of the plurality of third prediction results is used as a first prediction result of the storage performance of the storage system to be tested.

[0120] Figure 3 This is a schematic diagram of predicting a first prediction result in a storage performance prediction method provided by an embodiment of the present application.

[0121] Specifically, such as Figure 3As shown, when it is necessary to predict the storage performance of a storage system to be tested, the index parameters of the storage system to be tested are input into multiple decision trees of a pre-trained storage performance prediction model. Each decision tree will predict the storage performance based on the index parameters of the storage system to be tested and output a third prediction result. Different decision trees predict the storage performance for the same index parameters of the storage system to be tested, and the output third prediction results are not exactly the same. Therefore, multiple third prediction results can be processed and integrated to obtain a first prediction result, such as using the average result or majority result of the calculated multiple third prediction results as the first prediction result of the storage performance of the storage system to be tested. In addition, other statistical methods, such as weighted average, median, etc., can also be used to calculate the statistics of multiple third prediction results to obtain the first prediction result.

[0122] Through the above embodiments, the first prediction result of storage performance can be obtained by inputting the index parameters of the storage system into a pre-trained decision tree model and calculating the statistics of the prediction results of multiple decision trees, thereby improving the accuracy of the prediction results of the storage performance of the storage system.

[0123] Optionally, the method further includes:

[0124] Before training the random forest classifier to be trained, searching for a combination of the number of the decision trees and the number of leaf nodes of the decision trees;

[0125] In the process of training the random forest classifier to be trained, traversing multiple combinations in an ascending order of the number of decision trees, and determining the model performance of the random forest classifier to be trained corresponding to each combination;

[0126] Based on the model performance, the optimal number of decision trees and the number of leaf nodes of the decision trees are determined.

[0127] Specifically, before training the random forest classifier, it is first necessary to search for multiple combinations of the number of decision trees and the number of leaf nodes. This can be achieved by traversing all possible combinations. For example, you can start with only 3 decision trees and gradually increase the number of decision trees until a certain upper limit is reached. During the training process, multiple combinations are traversed from small to large in the number of decision trees, and the model performance of the random forest classifier to be trained corresponding to each combination is determined. This can be achieved by calculating a loss function (such as cross entropy loss or mean square error loss) to evaluate the predictive performance of the classifier. Based on the model performance, the optimal number of decision trees and the number of leaf nodes are selected from all possible combinations. This can be achieved by comparing the model performance indicators (such as accuracy, recall rate, F1 score, etc.) under different combinations. Selecting the parameter combination with the best performance can make the random forest classifier have better predictive performance.

[0128] Through the above embodiment, it is possible to determine the optimal model parameter settings by searching for multiple combinations of the number of decision trees and the number of leaf nodes and evaluating the model performance corresponding to each combination. This helps to improve the prediction performance of the random forest classifier and make it more suitable for practical application scenarios.

[0129] Optionally, in one embodiment, the algorithm of the random forest classifier can be optimized based on the Grey Wolf Optimization algorithm (GWO). When the random forest classifier algorithm is used to predict storage performance, the prediction result will be affected by the number of decision trees and the number of leaf nodes of the random forest. If the number of decision trees is too large, it may cause overfitting, and if the number of decision trees is too small, the performance of the storage performance prediction model may be unstable. The leaf nodes are the number of features randomly selected from all features and used as the number of segmentation variables. In order to improve the prediction performance of the random forest classifier algorithm, the Grey Wolf Optimization algorithm GWO (Grey Wolf Optimization) can be used to search for the best number of trees and leaf nodes.

[0130] The Gray Wolf Optimization (GWO) algorithm, proposed by Seyedali Mirjalili et al. in 2014, is a swarm intelligence optimization algorithm that mimics the hunting behavior of gray wolves. This algorithm primarily simulates the social hierarchy and hunting strategies of gray wolves and is used to solve various optimization problems. The GWO algorithm solves problems by simulating the social hierarchy and hunting process of gray wolves, including tracking and approaching prey, harassing, pursuing, and surrounding prey, and finally attacking them. In GWO, the optimal solution is defined as alpha (α), which simulates the position of the leader wolf. The suboptimal solution and candidate solutions are named beta (β) and delta (δ), respectively, which simulate the positions of second- and third-order wolves. The remaining candidate solutions are assumed to be omega (ω), which simulate the positions of subordinate wolves. The hunting process is guided by alpha, beta, and delta wolves, with omega wolves following these three wolves in their search, aiming to find better solutions and then update the optimal solution.

[0131] Specifically, the process of obtaining the optimal number of trees and leaf nodes by the gray wolf optimization algorithm includes the following steps:

[0132] First, the optimization problem is defined and the parameters of the random forest classifier (such as the number of decision trees and the number of leaf nodes) are used as optimization variables of the gray wolf optimization algorithm.

[0133] Furthermore, the optimization objectives are defined, such as the mean absolute percentage error (EMAPE) and the root mean square error (ERMSE).

[0134] Furthermore, a population of gray wolves is initialized, and a certain number of gray wolves are randomly generated in the solution space. The position of each gray wolf represents a set of random forest parameters (i.e., a combination of the number of decision trees and the number of leaf nodes mentioned above). Based on the social hierarchy of gray wolves, the three gray wolves with the best fitness are selected as α (the wolf king), β (the wolf king's left and right guards), and δ wolves. The remaining gray wolves are designated as ω wolves.

[0135] Furthermore, at each iteration, the position of each wolf is updated based on its social behavior. Specifically, the position updates of wolves α, β, and δ guide the other wolves toward a more optimal solution space. The wolf's position update strategy includes searching for prey (global search) and surrounding prey (local search), which are controlled by parameters in the algorithm. The performance of the random forest classifier corresponding to each wolf is evaluated, and the social hierarchy of the wolf is updated.

[0136] Furthermore, the iterative process is repeated until a preset number of iterations is reached or other convergence conditions are met (such as the fitness value no longer increases significantly), and the optimal number of decision trees and the number of leaf nodes of the decision trees are determined.

[0137] Finally, the position of the α wolf represents the optimal random forest parameter combination. The hyperparameter combination represented by Alpha (α) is considered the best hyperparameter combination. As an example, the optimal hyperparameter combination is found while setting the model parameters to: 30 gray wolves in the GWO algorithm, 2 variables to be optimized (i.e., the number of decision trees and leaf nodes), and a maximum number of iterations of 100.

[0138] Through the above embodiment, the Grey Wolf Optimization Algorithm is combined with the Random Forest Algorithm to find the optimal hyperparameters, which can further improve the accuracy of storage performance prediction.

[0139] Figure 4 FIG. 1 is a schematic diagram of a storage performance prediction device provided by an embodiment of the present application. Figure 4 shown.

[0140] Based on the same inventive concept, another embodiment of the present application further provides a storage performance prediction device, the device comprising:

[0141] An acquisition module 11 is used to obtain index parameters of the storage system to be tested;

[0142] The prediction module 12 is used to input the indicator parameters of the storage system to be tested into a pre-trained storage performance prediction model to obtain a first prediction result of the storage performance of the storage system to be tested, wherein the pre-trained storage performance prediction model is obtained by training a random forest classifier to be trained containing multiple decision trees through a sample data set, and the sample data set includes the real indicator parameters of multiple storage systems and the corresponding real input and output quantities and real response time, and the first prediction result includes the predicted category of storage performance and the predicted value of storage performance.

[0143] Optionally, the device further comprises:

[0144] an original data set acquisition module, configured to acquire an original data set comprising a plurality of original samples before training the random forest classifier to be trained, wherein each original sample comprises input data and output data; the input data is the real indicator parameter, and the output data is the real input and output quantity and the real response time;

[0145] a data cleaning module, configured to perform data cleaning on a plurality of original samples of the original data set, remove missing values ​​and outliers in the plurality of original samples, and perform mean-variance normalization processing on the plurality of cleaned samples to obtain a sample data set, wherein the sample data set includes a plurality of samples;

[0146] A building module for building a random forest classifier to be trained containing multiple decision trees;

[0147] The sample data set partitioning module is used to partition the sample data set to obtain multiple sample data subsets, each of which includes a training sample set and a validation sample set.

[0148] Optionally, the sample data set division module includes:

[0149] an extraction unit, configured to extract, with replacement, a plurality of samples from the sample data set for each decision tree as the training sample set;

[0150] a verification sample set acquisition unit, configured to use a plurality of samples not extracted from the sample data set as a verification sample set;

[0151] The multiple samples included in the training sample set are all randomly selected from the sample data set.

[0152] Optionally, the building block includes:

[0153] a first indicator parameter selection unit, configured to select, for each decision tree in the random forest classifier to be trained, any first indicator parameter from a plurality of samples in a training sample set corresponding to the decision tree as a first feature, and use the first feature as a root node of the decision tree;

[0154] a partitioning unit, configured to partition a plurality of samples in a training sample set corresponding to the decision tree according to the first feature to obtain a plurality of first subsets, wherein the first subsets include a plurality of samples having the same first feature value, the first feature corresponds to the plurality of first feature values, and each first subset corresponds to a first feature value;

[0155] A second indicator parameter selection unit, configured to arbitrarily select a second indicator parameter other than the first indicator parameter for each child node in the decision tree;

[0156] a second feature generating unit, configured to generate a second feature based on the second indicator parameter, and use the second feature as a child node of the decision tree;

[0157] According to the second feature, each first subset in the plurality of first subsets is divided into a plurality of second subsets, wherein the second subset is smaller than the first subset;

[0158] A determining unit is used to determine that the creation of the random forest classifier to be trained is completed when the number of samples corresponding to the child nodes of the decision tree is a minimum number of samples or the decision tree reaches a maximum depth.

[0159] Optionally, the device further comprises:

[0160] A training module is configured to input, for each decision tree, input data of a training sample set corresponding to the decision tree into the decision tree, such that the input data passes through a root node of the decision tree and reaches a leaf node of the decision tree, and record a second prediction result corresponding to the leaf node reached by the input data;

[0161] A first comparison module is configured to compare the second prediction result with the output data corresponding to the input data, and calculate a first absolute error value, where each sample corresponds to a first absolute error value;

[0162] an adjustment module, configured to adjust parameters of the random forest classifier to be trained based on the first absolute error value;

[0163] A testing module, configured to test the random forest classifier to be trained after the parameter adjustment using the verification sample set, and obtain a third prediction result corresponding to a leaf node reached by the input data of the sample in the verification sample set;

[0164] A second comparison module is configured to compare the third prediction result with the output data corresponding to the input data of the samples in the validation sample set, and calculate a second absolute error value, where each sample corresponds to a second absolute error value;

[0165] A determination module is used to calculate the average absolute error value for multiple decision trees of the random forest classifier to be trained and multiple second absolute error values, and when the average absolute error value is less than a preset threshold, it is determined that the pre-trained storage performance prediction model has been trained.

[0166] Optionally, the prediction module 12 includes:

[0167] a prediction unit, configured to input the index parameters of the storage system to be tested into multiple decision trees of a pre-trained storage performance prediction model, and record multiple third prediction results corresponding to multiple leaf nodes reached by the index parameters of the storage system to be tested, wherein each decision tree corresponds to one third prediction result;

[0168] The processing unit is configured to use an average result or a majority result of the plurality of third prediction results as a first prediction result of the storage performance of the storage system to be tested.

[0169] Optionally, the device further comprises:

[0170] A search module, configured to search for a plurality of combinations of the number of decision trees and the number of leaf nodes of the decision trees before training the random forest classifier to be trained;

[0171] A traversal module is used to traverse a plurality of combinations in an ascending order of the number of decision trees during the training of the random forest classifier to be trained, and determine the model performance of the random forest classifier to be trained corresponding to each combination;

[0172] The optimal combination determination module is used to determine the optimal number of decision trees and the number of leaf nodes of the decision trees based on the model performance.

[0173] Based on the same inventive concept, another embodiment of the present application further provides an electronic device, comprising a memory, a processor, and a computer program stored on the memory, wherein the processor executes the computer program to implement the storage performance prediction method as described in any of the above embodiments.

[0174] Based on the same inventive concept, another embodiment of the present application further provides a computer program product, including a computer program, which is executed by a processor to implement the storage performance prediction method described in any of the above embodiments.

[0175] Based on the same inventive concept, another embodiment of the present application further provides a computer-readable medium having a computer program stored thereon, wherein when the program is executed by a processor, the storage performance prediction method as described in any of the above embodiments is implemented.

[0176] As for the device, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0177] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0178] Those skilled in the art will appreciate that the embodiments of the present application can be provided as methods, devices, or computer program products. Therefore, the present application can adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present application can adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0179] The embodiments of the present application are described with reference to the flowcharts and / or block diagrams of the methods, terminal devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing terminal device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal device generate instructions for implementing the steps in the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0180] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing terminal device to operate in a specific manner, so that the instructions stored in the computer readable memory produce a manufactured product including an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0181] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal device so that a series of operating steps are executed on the computer or other programmable terminal device to produce a computer-implemented process, thereby providing instructions for executing on the computer or other programmable terminal device to implement the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0182] Although preferred embodiments of the present invention have been described, those skilled in the art may make additional changes and modifications to these embodiments once they become aware of the basic inventive concepts. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the embodiments of the present invention.

[0183] Finally, it should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or terminal device that includes a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or terminal device. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of additional identical elements in the process, method, article, or terminal device that includes the element.

[0184] The above is a detailed introduction to the storage performance prediction method, device, equipment and medium provided by the present application. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea; at the same time, for general technical personnel in this field, based on the ideas of the present application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. A storage performance prediction method, characterized in that: The method comprises: Obtain the index parameters of the storage system to be tested; Inputting the index parameters of the storage system to be tested into a pre-trained storage performance prediction model to obtain a first prediction result of the storage performance of the storage system to be tested, wherein the pre-trained storage performance prediction model is obtained by training a random forest classifier to be trained comprising multiple decision trees using a sample data set, the sample data set comprising real index parameters of multiple storage systems and corresponding real input and output volumes and real response times, and the first prediction result comprising a predicted category of storage performance and a predicted value of storage performance; The construction process of a random forest classifier to be trained containing multiple decision trees is: For each decision tree in the random forest classifier to be trained, arbitrarily select a first indicator parameter from multiple samples in the training sample set corresponding to the decision tree as a first feature, and use the first feature as the root node of the decision tree; Dividing a plurality of samples in a training sample set corresponding to the decision tree according to the first feature to obtain a plurality of first subsets, wherein the first subsets include a plurality of samples having the same first feature value, the first feature corresponds to a plurality of first feature values, and each first subset corresponds to a first feature value; For each child node in the decision tree, arbitrarily select a second indicator parameter other than the first indicator parameter; generating a second feature based on the second indicator parameter, and using the second feature as a child node of the decision tree; According to the second feature, each first subset in the plurality of first subsets is divided into a plurality of second subsets, wherein the second subset is smaller than the first subset; When the number of samples corresponding to the child nodes of the decision tree reaches a minimum number of samples or the decision tree reaches a maximum depth, it is determined that the creation of the random forest classifier to be trained is completed.

2. The storage performance prediction method according to claim 1, wherein: The method further comprises: Before training the random forest classifier to be trained, an original data set comprising a plurality of original samples is obtained, each original sample comprising input data and output data; the input data is the real indicator parameter, and the output data is the real input and output quantity and the real response time; Performing data cleaning on multiple original samples of the original data set, removing missing values ​​and outliers in the multiple original samples, and performing mean-variance normalization processing on the multiple cleaned samples to obtain a sample data set, wherein the sample data set includes multiple samples; Construct a random forest classifier to be trained containing multiple decision trees; The sample data set is divided to obtain multiple sample data subsets, each of which includes a training sample set and a validation sample set.

3. The storage performance prediction method according to claim 2, wherein: The sample data set is divided to obtain multiple sample data subsets, including: For each decision tree, extracting a plurality of samples from the sample data set with replacement as the training sample set; taking a plurality of samples not extracted from the sample data set as a validation sample set; The multiple samples included in the training sample set are all randomly selected from the sample data set.

4. The storage performance prediction method according to claim 3, characterized in that: Training the random forest classifier to be trained comprises the following steps: For each decision tree, input data of the training sample set corresponding to the decision tree into the decision tree, so that the input data passes through the root node of the decision tree and reaches the leaf node of the decision tree, and record the second prediction result corresponding to the leaf node where the input data reaches; Comparing the second prediction result with the output data corresponding to the input data, and calculating a first absolute error value, where each sample corresponds to a first absolute error value; Adjusting parameters of the random forest classifier to be trained based on the first absolute error value; Testing the random forest classifier to be trained after the parameter adjustment using the verification sample set to obtain a third prediction result corresponding to a leaf node reached by the input data of the sample in the verification sample set; Comparing the third prediction result with the output data corresponding to the input data of the sample in the validation sample set, and calculating a second absolute error value, where each sample corresponds to a second absolute error value; For the multiple decision trees of the random forest classifier to be trained, the mean absolute error value is calculated for the multiple second absolute error values. When the mean absolute error value is less than a preset threshold, it is determined that the training of the pre-trained storage performance prediction model is completed.

5. The storage performance prediction method according to claim 3, wherein: Inputting the index parameters of the storage system to be tested into a pre-trained storage performance prediction model to obtain a first prediction result of the storage performance of the storage system to be tested includes: Inputting the index parameters of the storage system to be tested into multiple decision trees of a pre-trained storage performance prediction model, and recording multiple third prediction results corresponding to multiple leaf nodes reached by the index parameters of the storage system to be tested, wherein each decision tree corresponds to one third prediction result; An average result or a majority result of the plurality of third prediction results is used as a first prediction result of the storage performance of the storage system to be tested.

6. The storage performance prediction method according to claim 1, wherein: The method further comprises: Before training the random forest classifier to be trained, searching for a combination of the number of the decision trees and the number of leaf nodes of the decision trees; In the process of training the random forest classifier to be trained, traversing multiple combinations in an ascending order of the number of decision trees, and determining the model performance of the random forest classifier to be trained corresponding to each combination; Based on the model performance, the optimal number of decision trees and the number of leaf nodes of the decision trees are determined.

7. A storage performance prediction device, characterized in that: The device comprises: An acquisition module is used to obtain the index parameters of the storage system to be tested; a prediction module, configured to input the index parameters of the storage system to be tested into a pre-trained storage performance prediction model to obtain a first prediction result of the storage performance of the storage system to be tested, wherein the pre-trained storage performance prediction model is obtained by training a random forest classifier to be trained comprising multiple decision trees using a sample data set, the sample data set comprising real index parameters of multiple storage systems and corresponding real input and output volumes and real response times, and the first prediction result comprising a predicted category of storage performance and a predicted value of storage performance; The device further comprises: a first indicator parameter selection unit, configured to select, for each decision tree in the random forest classifier to be trained, any first indicator parameter from a plurality of samples in a training sample set corresponding to the decision tree as a first feature, and use the first feature as a root node of the decision tree; a partitioning unit, configured to partition a plurality of samples in a training sample set corresponding to the decision tree according to the first feature to obtain a plurality of first subsets, wherein the first subsets include a plurality of samples having the same first feature value, the first feature corresponds to the plurality of first feature values, and each first subset corresponds to a first feature value; A second indicator parameter selection unit, configured to arbitrarily select a second indicator parameter other than the first indicator parameter for each child node in the decision tree; a second feature generating unit, configured to generate a second feature based on the second indicator parameter, and use the second feature as a child node of the decision tree; According to the second feature, each first subset in the plurality of first subsets is divided into a plurality of second subsets, wherein the second subset is smaller than the first subset; A determining unit is used to determine that the creation of the random forest classifier to be trained is completed when the number of samples corresponding to the child nodes of the decision tree is a minimum number of samples or the decision tree reaches a maximum depth.

8. An electronic device, characterized in that: The method comprises a memory, a processor and a computer program stored in the memory, wherein the processor executes the computer program to implement the storage performance prediction method according to any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that A computer program is stored thereon, wherein when the computer program is executed by a processor, the storage performance prediction method according to any one of claims 1 to 6 is implemented.

Images