[0026] Example 1
[0027] figure 1 A flowchart of a method for predicting resource occupancy data according to Embodiment 1 of the present invention is shown, as figure 1 As shown, the resource occupancy data prediction method specifically includes the following steps:
[0028] Step S101: Obtain indicator data generated by the current operation of at least one service.
[0029] When the service is running, the performance of the service will be measured according to various indicator data generated by its operation. Based on the health of the service, the resulting metric data will also vary. The acquisition of indicator data can be obtained by monitoring the operation of the service. For example, when the service is running, in order to ensure the normal operation of the service, daily monitoring of the service operation is performed, so that the indicator data generated by the service operation can be easily obtained.
[0030] Metric data includes query rate per second, throughput, concurrency, response time, etc. Query Per Second (qps) is a measure of the amount of traffic processed by a service within a specified time, reflecting the processing efficiency of the service; throughput reflects the processing speed and capability of the service, such as the service capacity per unit time. The ability to handle how many transactions, requests, data, etc. Corresponding to different services, such as database services, throughput is the number of executions of different SQL statements, the amount of returned data, etc. in unit time; network services, throughput is the data traffic transmitted by the network, etc.; concurrency is used to measure the concurrent capacity and synchronization of services. Coordination and other capabilities reflect the service's ability to deal with concurrent situations; the response time is the time it takes for the service to respond to user operations after receiving user operations, such as from the time the user clicks on a page, and ends when the page is fully displayed time spent. The response time can also be subdivided into: server-side response time, network response time, etc.
[0031] Step S102, predicting the current index data to obtain the predicted index data corresponding to a specified time point in the future.
[0032] By counting the historical indicator data generated by the service in the historical time period, it can be found that the indicator data generated by the service operation has periodic characteristics, and the indicator data will change periodically over time. For example, if the book reading service is provided to the user group, from 7:00 to 9:00 every morning, the user group will read more books on the way to work, and the indicator data will have more qps, higher throughput, more concurrency, and response. The time will be slower, etc.; from 9:00 to 12:00 every day, fewer user groups will read books, the qps in the indicator data will be less, the throughput will be less, the number of concurrency will be less, the response time will be faster, etc.; The indicator data of each time period and working day will also change periodically with different time periods.
[0033] When predicting the current index data, based on the periodic characteristics of the index data, a preset prediction algorithm may be used to predict the current index data. For example, the cubic smoothing index algorithm is used to reasonably predict the specified time point in the future, and obtain the prediction index data corresponding to the specified time point in the future. Here, in order to ensure the accuracy of the prediction, and to facilitate real-time monitoring of the subsequently obtained predicted resource occupancy data, and timely and reasonable resource arrangement, the specified time point in the future is preferably a shorter time, such as 1 hour in the future.
[0034] Step S103 , input the prediction index data into the pre-established resource occupation training model, and obtain the prediction resource occupation data required for the operation of at least one service at a corresponding specified time point in the future.
[0035] Service operation needs to occupy resources. Resource occupation data includes memory resource occupation data, CPU resource occupation data, disk occupation data, traffic occupation data, etc. Input the predictive indicator data into the pre-established resource occupancy training model, and the resource occupancy training model can obtain the corresponding predictive resource occupancy data required for service operation at a specified future time point according to the input predictive indicator data of the service at a specified time point in the future.
[0036] When there are multiple services, the prediction indicator data of each service at a specified time point in the future needs to be input into the pre-established resource occupation training model to obtain the corresponding predicted resource occupation data required for each service to run at a specified time point in the future. The predicted resource occupancy data required for each service to run is accumulated to obtain the predicted resource occupancy data required for all services to run at a specified time in the future.
[0037] Further, the training process of the pre-established resource occupation training model includes the following steps:
[0038] Step S1031, collect sample data and sample labeling data.
[0039] Before the resources are occupied by the training model for training, it is necessary to collect enough sample data and sample annotation data required for training. The sample data includes historical indicator data, and the sample labeling data includes historical resource occupation data occupied by at least one service corresponding to the historical indicator data. The above sample data and sample annotation data can be obtained through daily monitoring and recording of services to obtain historical indicator data, and historical resource occupancy data can be obtained by recording the resources occupied by each service during operation. For example, historical indicator data generated by at least one service operation in the past three months and historical resource occupation data occupied by corresponding at least one service operation are collected.
[0040]When collecting sample data and sample labeling data, it is necessary to collect indicator data generated by at least one service operation at the same historical time point and resource occupation data occupied by corresponding at least one service operation, so as to be used for training the resource occupation training model.
[0041] Step S1032, input the sample data into the model to be trained for training, and obtain the output result of the model to be trained.
[0042] The sample data collected above are input into the model to be trained as input data for training, and the output result of the model to be trained is obtained, that is, the resource occupation data occupied by the service operation.
[0043] The model to be trained may use a machine algorithm model such as a linear regression model, which is specifically set according to the implementation, and is not limited here.
[0044] Step S1033 , according to the loss between the output result and the sample labeling data, adjust the weight parameters of the model to be trained until the preset conditions are met, and the resource occupation training model is obtained.
[0045] The preset condition may include, for example, calculating the accuracy of the output result and the sample labeled data, when the accuracy meets a certain threshold, such as 99%; or the preset condition is that the deviation between the output result and the sample labeled data is less than a certain threshold, such as the deviation is less than 5% Wait. Taking the model to be trained as a linear regression model as an example, it is assumed that there is a linear relationship between the sample data and the output result, and training is performed according to the sample data. There will be an error between the output result and the sample labeling data, that is, there is a loss. According to the existing loss, the loss function is optimized to minimize the loss function, thereby obtaining the resource occupancy training model.
[0046] This step can be performed repeatedly, and continuously adjust the weight parameters of the model to be trained according to the loss between the output result and the sample labeling data until the preset conditions are met, thereby obtaining the resource-occupying training model.
[0047] Further, since each service is different, when the training model is occupied by training resources, a training model of resource occupancy targeted for each service can be obtained for different services. That is, during training, according to the historical indicator data generated by the operation of each service and the corresponding historical resource occupation data occupied by the operation of the service, train separately according to the service, and obtain the resource occupation training model of each service. When inputting the prediction index data of each service at a specified time in the future, the corresponding input is entered into the resource occupation training model pre-established for the service, and the predicted resource occupation data required for the operation of the service at the corresponding specified time in the future is obtained. .
[0048] According to the resource occupancy data prediction method provided by the present invention, the indicator data generated by the current operation of at least one service is obtained; the current indicator data is predicted to obtain the prediction indicator data corresponding to a specified time point in the future; the prediction indicator data is input into the pre-established resource Occupy the training model to obtain the predicted resource occupancy data required for at least one service to run at a corresponding specified time point in the future. Using the present invention to obtain the predicted resource occupancy data required for the operation of at least one service at a specified time point in the future, it is convenient to know the possible resource occupancy situation at the specified time point in the future in advance, make a good response to the resource occupancy situation, and timely analyze the existing resources. Make adjustments to avoid problems such as inability to run services due to insufficient resources, and avoid resource idleness caused by unreasonable expansion of resources, and provide resource utilization. Further, training is performed in advance according to historical indicator data and historical resource occupancy data to obtain a resource occupancy training model, so as to ensure the accuracy of the predicted resource occupancy data.
