Resource occupancy data prediction method, electronic device, and storage medium

A technology for resource occupancy and data prediction, applied in the software field, can solve problems such as poor experience effects, and achieve the effects of avoiding resource idleness, providing resource utilization, and ensuring accuracy

Inactive Publication Date: 2018-12-18
ZHANGYUE TECH CO LTD
5 Cites 12 Cited by

AI-Extracted Technical Summary

Problems solved by technology

However, in this way, the server is often expanded and adjusted after the problem occurs, that is, the re...
View more

Method used

According to the resource occupancy data prediction method provided by the present invention, obtain the index data that current at least one service operation produces; Predict the current index data, obtain the predictive index data corresponding to the specified time point in the future; Input the predictive index data in advance The established resource occupancy training model obtains the predicted resource occupancy data required for the operation of at least one service at a corresponding specified time point in the future. Using the present invention to obtain the predicted resource occupation data required for the operation of at least one service at a specified time point in the future, it is convenient to know the possible resource occupation situation at a specified time point in the future in advance, to make a good response to the resource occupation situation, and to timely respond to existing resources. Make adjustments to avoid problems such as service inoperability caused by insufficient resources, and avoid idle resources caused by unreasonable expansion of resources, and improve resource utilization. Further, training is performed in advance based on historical index data and historical resource occupancy data to obtain a resource occupancy training model to ensure the accuracy of the predicted ...
View more

Abstract

The invention discloses a method for predicting resource occupancy data, an electronic device and a storage medium. The method comprises the following steps: obtaining index data generated by at leastone service operation at present; forecasting the current index data to obtain the corresponding forecast index data at the specified time point in the future; the forecasting index data being inputinto a pre-established resource occupancy training model to obtain the forecasting resource occupancy data needed for at least one service operation at a corresponding future specified time point. Atleast one predict resource occupancy data required for that operation of a service at a future designate time point is obtained by using the invention, It is convenient to know in advance the possibleresource occupancy situation at the appointed time point in the future, to make a good response to the resource occupancy situation, to adjust the existing resources in time, to avoid the problems such as the service can not run due to the insufficient resources, and to avoid the resource idleness caused by the unreasonable expansion of resources, and to provide the resource utilization ratio.

Application Domain

Resource allocationHardware monitoring

Technology Topic

Data predictionData needs +5

Image

  • Resource occupancy data prediction method, electronic device, and storage medium
  • Resource occupancy data prediction method, electronic device, and storage medium
  • Resource occupancy data prediction method, electronic device, and storage medium

Examples

  • Experimental program(4)

Example Embodiment

[0026] Example 1
[0027] figure 1 A flowchart of a method for predicting resource occupancy data according to Embodiment 1 of the present invention is shown, as figure 1 As shown, the resource occupancy data prediction method specifically includes the following steps:
[0028] Step S101: Obtain indicator data generated by the current operation of at least one service.
[0029] When the service is running, the performance of the service will be measured according to various indicator data generated by its operation. Based on the health of the service, the resulting metric data will also vary. The acquisition of indicator data can be obtained by monitoring the operation of the service. For example, when the service is running, in order to ensure the normal operation of the service, daily monitoring of the service operation is performed, so that the indicator data generated by the service operation can be easily obtained.
[0030] Metric data includes query rate per second, throughput, concurrency, response time, etc. Query Per Second (qps) is a measure of the amount of traffic processed by a service within a specified time, reflecting the processing efficiency of the service; throughput reflects the processing speed and capability of the service, such as the service capacity per unit time. The ability to handle how many transactions, requests, data, etc. Corresponding to different services, such as database services, throughput is the number of executions of different SQL statements, the amount of returned data, etc. in unit time; network services, throughput is the data traffic transmitted by the network, etc.; concurrency is used to measure the concurrent capacity and synchronization of services. Coordination and other capabilities reflect the service's ability to deal with concurrent situations; the response time is the time it takes for the service to respond to user operations after receiving user operations, such as from the time the user clicks on a page, and ends when the page is fully displayed time spent. The response time can also be subdivided into: server-side response time, network response time, etc.
[0031] Step S102, predicting the current index data to obtain the predicted index data corresponding to a specified time point in the future.
[0032] By counting the historical indicator data generated by the service in the historical time period, it can be found that the indicator data generated by the service operation has periodic characteristics, and the indicator data will change periodically over time. For example, if the book reading service is provided to the user group, from 7:00 to 9:00 every morning, the user group will read more books on the way to work, and the indicator data will have more qps, higher throughput, more concurrency, and response. The time will be slower, etc.; from 9:00 to 12:00 every day, fewer user groups will read books, the qps in the indicator data will be less, the throughput will be less, the number of concurrency will be less, the response time will be faster, etc.; The indicator data of each time period and working day will also change periodically with different time periods.
[0033] When predicting the current index data, based on the periodic characteristics of the index data, a preset prediction algorithm may be used to predict the current index data. For example, the cubic smoothing index algorithm is used to reasonably predict the specified time point in the future, and obtain the prediction index data corresponding to the specified time point in the future. Here, in order to ensure the accuracy of the prediction, and to facilitate real-time monitoring of the subsequently obtained predicted resource occupancy data, and timely and reasonable resource arrangement, the specified time point in the future is preferably a shorter time, such as 1 hour in the future.
[0034] Step S103 , input the prediction index data into the pre-established resource occupation training model, and obtain the prediction resource occupation data required for the operation of at least one service at a corresponding specified time point in the future.
[0035] Service operation needs to occupy resources. Resource occupation data includes memory resource occupation data, CPU resource occupation data, disk occupation data, traffic occupation data, etc. Input the predictive indicator data into the pre-established resource occupancy training model, and the resource occupancy training model can obtain the corresponding predictive resource occupancy data required for service operation at a specified future time point according to the input predictive indicator data of the service at a specified time point in the future.
[0036] When there are multiple services, the prediction indicator data of each service at a specified time point in the future needs to be input into the pre-established resource occupation training model to obtain the corresponding predicted resource occupation data required for each service to run at a specified time point in the future. The predicted resource occupancy data required for each service to run is accumulated to obtain the predicted resource occupancy data required for all services to run at a specified time in the future.
[0037] Further, the training process of the pre-established resource occupation training model includes the following steps:
[0038] Step S1031, collect sample data and sample labeling data.
[0039] Before the resources are occupied by the training model for training, it is necessary to collect enough sample data and sample annotation data required for training. The sample data includes historical indicator data, and the sample labeling data includes historical resource occupation data occupied by at least one service corresponding to the historical indicator data. The above sample data and sample annotation data can be obtained through daily monitoring and recording of services to obtain historical indicator data, and historical resource occupancy data can be obtained by recording the resources occupied by each service during operation. For example, historical indicator data generated by at least one service operation in the past three months and historical resource occupation data occupied by corresponding at least one service operation are collected.
[0040]When collecting sample data and sample labeling data, it is necessary to collect indicator data generated by at least one service operation at the same historical time point and resource occupation data occupied by corresponding at least one service operation, so as to be used for training the resource occupation training model.
[0041] Step S1032, input the sample data into the model to be trained for training, and obtain the output result of the model to be trained.
[0042] The sample data collected above are input into the model to be trained as input data for training, and the output result of the model to be trained is obtained, that is, the resource occupation data occupied by the service operation.
[0043] The model to be trained may use a machine algorithm model such as a linear regression model, which is specifically set according to the implementation, and is not limited here.
[0044] Step S1033 , according to the loss between the output result and the sample labeling data, adjust the weight parameters of the model to be trained until the preset conditions are met, and the resource occupation training model is obtained.
[0045] The preset condition may include, for example, calculating the accuracy of the output result and the sample labeled data, when the accuracy meets a certain threshold, such as 99%; or the preset condition is that the deviation between the output result and the sample labeled data is less than a certain threshold, such as the deviation is less than 5% Wait. Taking the model to be trained as a linear regression model as an example, it is assumed that there is a linear relationship between the sample data and the output result, and training is performed according to the sample data. There will be an error between the output result and the sample labeling data, that is, there is a loss. According to the existing loss, the loss function is optimized to minimize the loss function, thereby obtaining the resource occupancy training model.
[0046] This step can be performed repeatedly, and continuously adjust the weight parameters of the model to be trained according to the loss between the output result and the sample labeling data until the preset conditions are met, thereby obtaining the resource-occupying training model.
[0047] Further, since each service is different, when the training model is occupied by training resources, a training model of resource occupancy targeted for each service can be obtained for different services. That is, during training, according to the historical indicator data generated by the operation of each service and the corresponding historical resource occupation data occupied by the operation of the service, train separately according to the service, and obtain the resource occupation training model of each service. When inputting the prediction index data of each service at a specified time in the future, the corresponding input is entered into the resource occupation training model pre-established for the service, and the predicted resource occupation data required for the operation of the service at the corresponding specified time in the future is obtained. .
[0048] According to the resource occupancy data prediction method provided by the present invention, the indicator data generated by the current operation of at least one service is obtained; the current indicator data is predicted to obtain the prediction indicator data corresponding to a specified time point in the future; the prediction indicator data is input into the pre-established resource Occupy the training model to obtain the predicted resource occupancy data required for at least one service to run at a corresponding specified time point in the future. Using the present invention to obtain the predicted resource occupancy data required for the operation of at least one service at a specified time point in the future, it is convenient to know the possible resource occupancy situation at the specified time point in the future in advance, make a good response to the resource occupancy situation, and timely analyze the existing resources. Make adjustments to avoid problems such as inability to run services due to insufficient resources, and avoid resource idleness caused by unreasonable expansion of resources, and provide resource utilization. Further, training is performed in advance according to historical indicator data and historical resource occupancy data to obtain a resource occupancy training model, so as to ensure the accuracy of the predicted resource occupancy data.

Example Embodiment

[0049] Embodiment 2
[0050] image 3 A flowchart of a method for predicting resource occupancy data according to Embodiment 2 of the present invention is shown, such as image 3 As shown, the resource occupancy data prediction method includes the following steps:
[0051] Step S301 , acquiring index data generated by the current operation of at least one service.
[0052] Step S302: Predict the current index data to obtain the predicted index data corresponding to a specified time point in the future.
[0053] Step S303 , input the prediction index data into the pre-established resource occupation training model, and obtain the prediction resource occupation data required for the operation of at least one service at a corresponding specified time point in the future.
[0054] For the above steps, reference is made to the description of steps S101-S103 in the first embodiment, and details are not repeated here.
[0055] Step S304, obtaining resource availability data provided by the server.
[0056] The resource availability data provided by the server itself can be obtained according to the pre-configured data of the server. Here, in order to make each service run well and deal with possible emergencies, the resource availability data may not be all the resource availability data that the server itself can provide, but the resource data within the preset threshold range of the server is obtained as the resource availability data. data. If the threshold is 90%, the resource data provided by the server accounting for 90% of all resource data is obtained as the resource available data, and 10% of the resource data is used as the resource data for responding to emergencies.
[0057] Step S305, it is judged whether the resource available data can support the service running on the server.
[0058] Obtain each service currently running on the server, and calculate the difference between the predicted resource occupancy data required for all services running on the server to run at a specified time point in the future based on the obtained predicted resource occupancy data required for each service to run at a specified time point in the future. and. For example, if the server currently runs services A, B, and C, the predicted resource occupancy data required to run the services A, B, and C at a specified time point in the future is accumulated to obtain the sum of the required predicted resource occupancy data.
[0059] Determine whether the resource availability data is greater than or equal to the sum of the predicted resource occupancy data required for the operation of all services at a specified time in the future. If it is greater than or equal to, it means that the server can support the normal operation of all services at a specified time in the future; The server cannot support the normal operation of all services at the specified time point in the future, and step S306 is executed.
[0060] Step S306, performing server resource alarm processing.
[0061] When it is judged that the server cannot support the normal operation of all services at a specified time point in the future, it is necessary to further judge the specific shortage of resources, so that the server resource alarm processing can alert the shortage of resources, such as insufficient memory capacity and excessive CPU usage. It is convenient to take corresponding measures for specific resources, and avoid problems such as idle resources caused by supplementing all resources. Further, the server resource alarm processing can also issue an alarm based on the specific value of the resource shortage. When it is convenient to supplement, the resource can be supplemented according to the specific data, so as to avoid too little supplement, which cannot guarantee the normal operation of the service, and avoid too much supplement which leads to insufficient resource utilization. higher issues.
[0062] The specific alarm mode may adopt various existing alarm modes such as email alarm, display screen alarm, etc., which is not limited here.
[0063] According to the resource occupancy data prediction method provided by the present invention, after the predicted resource occupancy data required for each service to run at a specified time point in the future is predicted, it is compared with the resource availability data provided by the server. Support the services running on the server, and process the server resource alarms in time, so that the problems that may occur in the future can be quickly solved in advance and the normal operation of the services can be guaranteed.

Example Embodiment

[0064] Embodiment 3
[0065] The third embodiment of the present application provides a non-volatile computer storage medium, where the computer storage medium stores at least one executable instruction, and the computer executable instruction can execute the resource occupancy data prediction method in any of the foregoing method embodiments.
[0066] Executable instructions can specifically be used to cause the processor to perform the following operations:
[0067] Obtain the indicator data generated by the current operation of at least one service; make predictions on the current indicator data to obtain the prediction indicator data corresponding to the specified time point in the future; input the prediction indicator data into the pre-established resource occupation training model to obtain the corresponding future specified time point Predicted resource usage data required for at least one service to run.
[0068] In an optional embodiment, the executable instructions further cause the processor to perform the following operations: use a preset prediction algorithm to predict the current indicator data according to the cycle characteristics of the indicator data, and obtain the prediction indicator corresponding to a specified time point in the future data.
[0069] In an optional embodiment, the executable instructions further cause the processor to perform the following operations: collect sample data and sample labeling data; input the sample data into the model to be trained for training, and obtain the output result of the model to be trained; The loss between the output result and the sample labeling data, adjust the weight parameters of the model to be trained until the preset conditions are met, and obtain the resource occupancy training model.
[0070] In an optional implementation manner, the sample data includes historical indicator data generated by the operation of at least one service; the sample labeling data includes historical resource occupation data occupied by at least one service corresponding to the historical indicator data.
[0071] In an optional implementation manner, the executable instructions further cause the processor to perform the following operations: obtain resource availability data provided by the server; determine whether the resource availability data can support the service running on the server; if not, perform server resource alarm processing .
[0072] In an optional implementation manner, the executable instructions further cause the processor to perform the following operations: according to at least one service running on the server, calculate the sum of the predicted resource occupancy data required for the running of all services at a specified time point in the future; determine the resource Whether the available data is greater than or equal to the sum of the predicted resource occupancy data required for all services to run at a specified time in the future; if not, perform server resource alarm processing.
[0073] In an optional implementation manner, the indicator data includes query rate, throughput, concurrency and/or response time per second; resource usage data includes memory resource usage data, CPU resource usage data, disk usage data and/or traffic Occupy data.

PUM

no PUM

Description & Claims & Application Information

We can also present the details of the Description, Claims and Application information to help users get a comprehensive understanding of the technical details of the patent, such as background art, summary of invention, brief description of drawings, description of embodiments, and other original content. On the other hand, users can also determine the specific scope of protection of the technology through the list of claims; as well as understand the changes in the life cycle of the technology with the presentation of the patent timeline. Login to view more.

Similar technology patents

Solar azimuth angle tracking controller and tracking method thereof

InactiveCN102156482AGuaranteed accuracyavoid mistracking
Owner:COMBA TELECOM SYST CHINA LTD
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products