A computing power resource scheduling method and system

Abstract

The present invention relates to the technical field of data processing. More specifically, the present invention relates to a computing power resource scheduling method and system. The method includes: collecting the computing power demand data of a target node for consecutive multiple days, and taking the data volume of each divided data segment as a data point to construct a historical demand sequence; performing ARIMA prediction based on the historical demand sequence. During the prediction process, using the weights of each reference data, weighting and normalizing the moving average coefficients of the corresponding reference data obtained by maximum likelihood estimation to obtain the actual moving average coefficients of the corresponding reference data, and based on the actual moving average coefficients of each reference data, obtaining the moving average values of all reference data for ARIMA prediction, and performing computing power scheduling based on the prediction results. The present invention can accurately predict the data volume processed by the target node within a specific time period, so as to achieve precise scheduling of computing power resources.

Description

Technical Field

[0001] The present invention relates to the technical field of data processing. More specifically, the present invention relates to a computing power resource scheduling method and system. Background Art

[0002] With the rapid development of technologies such as cloud computing, big data, and artificial intelligence, computing power resources have become one of the core competitiveness of modern enterprises and research institutions. However, the efficient scheduling and management of computing power resources face many challenges, mainly involving the problem of resource allocation under dynamic load, which increases the difficulty of computing power resource management. Therefore, it is urgent to find a precise computing power resource scheduling method to achieve the efficient allocation and utilization of computing power resources.

[0003] In related technologies, time series prediction algorithms are usually used to predict the computing power resource requirements of each node, and dynamic allocation of computing power resources is performed according to the prediction results of computing power requirements. The ARIMA model consists of three parts: autoregressive (AR), integrated (I), and moving average (MA). By modeling the historical data of the time series, the future trend can be predicted, and the computing power requirements of the nodes can be predicted.

[0004] However, in the process of using the ARIMA model to predict the computing power requirements of nodes, when the MA model in this model captures the dependence relationship between error terms, the moving average coefficients of each error term are usually directly calculated according to the maximum likelihood function. However, since the maximum likelihood estimation is determined based on the probability distribution, it is impossible to specifically process the abnormal data existing in the error terms, resulting in inaccurate calculation results of the traditional MA model, affecting the accuracy of the prediction effect, and unable to accurately schedule the computing power resources. Summary of the Invention

[0005] In order to solve the problem that the accuracy of the prediction result is poor when predicting the computing power requirements of nodes based on the ARIMA model, resulting in the inability to accurately schedule the computing power, the present invention provides a computing power resource scheduling method and system.

[0006] According to a first aspect of the present invention, there is provided a computing power resource scheduling method, including:

[0007] Collect the computing power requirement data of the target node for consecutive multiple days, divide it into several data segments with the same length, take the data volume of each data segment as a data point, and construct a historical requirement sequence;

[0008] Based on the historical demand sequence for ARIMA prediction, during the prediction process, several previous error values of the current error value are used as reference data. Using the weights of each reference data, the moving average coefficients of the corresponding reference data obtained by maximum likelihood estimation are weighted and normalized to obtain the actual moving average coefficients of the corresponding reference data. Based on the actual moving average coefficients of each reference data, the moving average values of all reference data are obtained for ARIMA prediction, and computing power scheduling is performed based on the prediction results;

[0009] The method for obtaining weights includes: calculating the credibility of any reference data, where the credibility is negatively correlated with the difference between this any reference data and the average value of all reference data, and the difference between this any reference data and the average error value of all data points in the historical demand sequence;

[0010] Fitting all reference data, and based on the fitting error of this any reference data and the fitting errors of all reference data, obtaining the trend reference of this any reference data to correct the credibility, where the correction value is positively correlated with the trend reference to obtain the weight of this any reference data.

[0011] When using the ARIMA model to predict the computing power demand of nodes, the present invention can specifically focus on the interference of abnormal data in the reference data of the current error value, and can weight the moving average coefficients of the corresponding reference data obtained by maximum likelihood estimation through the weights determined by evaluating the possibility of each reference data being abnormal data, so that the determined moving average value can reduce the influence of abnormal data existing in the reference data, and further improve the accuracy of the computing power demand prediction result, realizing the precise scheduling of computing power resources.

[0012] Preferably, the number of reference data of the current error value is the same as the moving average order of the pre-constructed ARIMA model.

[0013] Preferably, the credibility of any reference data satisfies the following relational expression:

[0014] ;

[0015] In the formula, is the credibility of the th reference data of the current error value; is the value of the th reference data of the current error value; is the average value of the error values of all data points in the historical demand sequence; is the average value of all reference data; is the absolute value symbol; is the standard deviation of the error values of all data points in the historical demand sequence; The standard deviation of all reference data for the current error value.

[0016] By comparing and analyzing the values of each reference data with all error data, the present invention can accurately measure the possibility of each reference data being abnormal data. Taking the standard deviation as the reference standard can eliminate the influence of different dimension differences and ensure the accuracy of the determined credibility.

[0017] Preferably, the method for obtaining the error value of any data point in the historical demand sequence includes:

[0018] For any data point in the historical demand sequence, calculate the difference between the value of this data point and the predicted value corresponding to this data point to obtain the error value of this data point.

[0019] Preferably, based on the fitting error of any reference data and the fitting errors of all reference data, obtain the trend reference of any reference data, which satisfies the following relational expression:

[0020] ;

[0021] In the formula, is the trend reference of the th reference data of the current error value; is the value of the th reference data of the current error value; is the fitted value of the th reference data of the current error value; is the absolute value symbol; is the number of reference data of the current error value; is the natural exponential function.

[0022] The present invention can evaluate the possibility of each reference data being abnormal data from the dimension of the change trend, so as to adjust the credibility of each reference data, and thus can more accurately evaluate the abnormal degree of each reference data.

[0023] Preferably, the process of correcting the credibility of any reference data by using the trend reference of any reference data includes:

[0024] Perform a multiplication operation on the trend reference of any reference data and the credibility of this reference data to obtain the corrected value of the credibility of this reference data, and use the corrected value as the weight of this reference data.

[0025] The present invention can set a larger weight for the reference data with a lower abnormal degree, thereby reducing the influence of abnormal data existing in the reference data.

[0026] Preferably, the method for obtaining the actual moving average coefficient of any reference data includes:

[0027] Obtain the weighted values of the moving average coefficients of each reference data, sum them up, calculate the ratio of the weighted value of the moving average coefficient of any reference data to the obtained cumulative sum, obtain the normalized value of the weighted value, and use the normalized value as the actual moving average coefficient of the any reference data.

[0028] Preferably, the process of obtaining the moving average value of all reference data based on the actual moving average coefficients of each reference data includes:

[0029] Use the actual moving average coefficients of each reference data to perform weighted summation on all reference data to obtain the moving average value of all reference data.

[0030] The present invention can ensure the accuracy of the determined moving average value.

[0031] Preferably, the target node is a computer device.

[0032] According to the second aspect of the present invention, there is provided a computing power resource scheduling system, which includes a memory and a processor. A computer program is stored on the memory, and the processor executes the computer program to implement the steps of the first aspect of the present invention.

[0033] The present invention has the following effects:

[0034] The method for determining the actual moving average coefficients of each reference data in the present invention can reduce the influence of abnormal data existing in the reference data on the moving average value of all reference data, ensure the accuracy of the moving average value, and thus can accurately predict the computing power demand of the target node based on the relatively accurate moving average value, realizing the precise scheduling of computing power resources. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] By reading the following detailed description with reference to the accompanying drawings, the above and other objects, features, and advantages of the exemplary embodiments of the present invention will become easily understandable. In the drawings, several embodiments of the present invention are shown in an exemplary rather than restrictive manner, and the same or corresponding reference numerals represent the same or corresponding parts, wherein:

[0036] Figure 1 It is a schematic flowchart of the steps of a computing power resource scheduling method according to an embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0037] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative efforts shall fall within the protection scope of the present invention.

[0038] The following will describe in detail the specific implementation manners of the present invention in conjunction with the accompanying drawings.

[0039] Refer to Figure 1 , a computing power resource scheduling method, including steps S1 - S3, specifically as follows:

[0040] S1: Collect the computing power demand data of the target node for consecutive multiple days, divide it into several data segments of the same length, take the data volume of each data segment as a data point, and construct a historical demand sequence.

[0041] In an exemplary embodiment of the present invention, the target node refers to a randomly selected node, and a node is a computing unit or device in a network, responsible for executing computing tasks, storing data, or participating in network communication. A node can be a physical device (such as a server, a computer), or a virtual instance (such as a virtual machine). The type of the node is not particularly limited in this embodiment.

[0042] The computing power demand data refers to a quantitative index of the actual computing power required by the node to complete its computing tasks (such as data processing, algorithm execution, model training, transaction verification, etc.) within a specific time period, reflecting the degree of demand of the node for computing resources (such as CPU, GPU, memory, etc.) during operation.

[0043] In an exemplary embodiment of the present invention, the target node is a computer device.

[0044] Further, after the target node is selected, the computing power demand data of the target node for consecutive multiple days, such as within consecutive 7 days, can be obtained, and the computing power demand data every ten minutes is taken as a data segment, so as to divide the computing power demand data within these 7 days into several data segments; then, the data volume of each data segment is taken as a data point, so as to obtain a historical demand sequence composed of all data points. The length of the data segment is not particularly limited in this embodiment.

[0045] S2: Perform ARIMA prediction based on the historical demand sequence. During the prediction process, take the previous several error values of the current error value as reference data, use the weights of each reference data to weight the moving average coefficients of the corresponding reference data obtained by maximum likelihood estimation, and normalize them to obtain the actual moving average coefficients of the corresponding reference data.

[0046] It should be noted that in view of the problem of inaccurate calculation results obtained by the traditional MA model, the present invention improves the traditional MA model. The specific improvement content is as follows: by evaluating several error values before the current error value, that is, the performance of the reference data, the weight corresponding to the reference data is determined, and the weight is used to correct the moving average coefficient of the corresponding reference data obtained by maximum likelihood estimation, so as to obtain the actual moving average coefficient of each reference data, and subsequent operations are performed based on the actual moving average coefficient of each reference data to improve the accuracy of the calculation results of the traditional MA model.

[0047] It should be further noted that the data expression of the MA model is: , where is the current observation value; is the current error value; is the moving average coefficient; is the order of the MA model. Among them, the present invention only improves the determination of the moving average coefficient in the traditional MA model, that is, only improves , ..., the determination method of the value, and does not improve other contents of the MA model and the trend prediction using the ARIMA model.

[0048] In an exemplary embodiment of the present invention, the number of reference data of the current error value is the same as the moving average order of the pre-constructed ARIMA model.

[0049] Among them, the moving average order of the ARIMA model refers to the order of the MA model. The order of the MA model can be determined by the ACF graph (autocorrelation graph) or by the PACF graph (partial autocorrelation graph). The present embodiment does not make a special limitation on the determination method of the order of the MA model. It should be noted that the process of determining the order of the MA model using the ACF graph and the PACF graph is a prior art, and the present embodiment will not elaborate on this here.

[0050] Furthermore, after determining the reference data of the current error value, the weight of each reference data can be determined through the following steps:

[0051] Step 1: Calculate the credibility of any reference data. The credibility is negatively correlated with the difference between the any reference data and the average value of all reference data, and the difference between the any reference data and the average error value of all data points in the historical demand sequence.

[0052] Among them, the credibility refers to the possibility that any reference data is normal data. For example, when any reference data is likely to be abnormal data, the credibility of the reference data is relatively low; otherwise, the credibility of the reference data is relatively high.

[0053] In an exemplary embodiment of the present invention, the determination of the error value of any data point in the historical demand sequence can be achieved through the following steps:

[0054] For any data point in the historical demand sequence, calculate the difference between the value of the data point and the predicted value corresponding to the data point to obtain the error value of the data point.

[0055] It should be noted that since each data point in the historical demand sequence is the data volume of a data segment, therefore, the predicted value corresponding to each data point is the data volume of the corresponding data segment predicted by the ARIMA model. Thus, based on the actual value and the predicted value of each data point, the error value of the corresponding data point can be determined.

[0056] Furthermore, after determining the error values of each data point in the historical demand sequence, the credibility of each reference data can be calculated. Specifically, the credibility of any reference data satisfies the following relational expression:

[0057] ;

[0058] In the formula, is the credibility of the th reference data of the current error value; is the value of the th reference data of the current error value; is the average value of the error values of all data points in the historical demand sequence; is the average value of all reference data; is the absolute value symbol; is the standard deviation of the error values of all data points in the historical demand sequence, which is used as a comparison standard to eliminate the influence of different dimension differences; is the standard deviation of all reference data of the current error value, which is used as a comparison standard to eliminate the influence of different dimension differences.

[0059] Among them, reflects the difference between the th reference data and the average value of the error values of all data points in the historical demand sequence. The smaller this value is, the greater the deviation of the reference data from all error values, and further indicates that the value of the reference data is more likely to be abnormal, and the credibility of the corresponding reference data is lower.

[0060] reflects the difference between the th reference data and the average value of all reference data. The smaller this value is, the greater the deviation of the reference data from all reference data, and further indicates that the reference data has local prominent characteristics, then the value of the reference data is more likely to be abnormal, and the credibility of the corresponding reference data is lower.

[0061] Step 2: Fit all reference data, and obtain the trend reference of any one of the reference data based on the fitting error of the any one of the reference data and the fitting errors of all reference data.

[0062] It should be noted that since the ARIMA model can analyze and predict the change trend of time series, so that the data error gradually becomes smaller or remains unchanged. Therefore, the present invention utilizes this feature to evaluate the change trend of each reference data, and corrects the credibility of each reference data by using the evaluation result, so as to further ensure the accuracy of the determined credibility.

[0063] Optionally, all reference data can be fitted by using the least squares method, polynomial fitting, etc. This embodiment does not make a special limitation on the selected fitting method.

[0064] Further, after fitting all reference data, the trend reference of any one of the reference data can be determined based on the fitting error of the any one of the reference data and the fitting errors of all reference data. Specifically, the trend reference of any one of the reference data satisfies the following relational expression:

[0065] ;

[0066] In the formula, is the trend reference of the th reference data of the current error value; is the value of the th reference data of the current error value; is the fitting value of the th reference data of the current error value; is the absolute value symbol; is the number of reference data of the current error value; is the natural exponential function.

[0067] Among them, reflects the fitting error of the th reference data of the current error value. The larger this value is, the worse the fitting effect of the reference data is, and further it indicates that the change trend of the reference data may be abnormal, and correspondingly the trend reference of the reference data is lower.

[0068] reflects the cumulative sum of the fitting errors of all reference data. The larger this value is, the worse the function fitting effect is, and it is impossible to accurately predict the subsequent error values. At this time, if the When the fitting error of a reference data is also large, it indicates that the change trend of this reference data is relatively consistent with the change trends of all reference data. Furthermore, it shows that the change trend of this reference data is a normal change trend, and the trend reference of this reference data is relatively high.

[0069] Step 3: Use the trend reference to correct the credibility. The correction value is positively correlated with the trend reference to obtain the weight of any one of the reference data.

[0070] In an exemplary embodiment of the present invention, the correction of the credibility of any one of the reference data can be achieved through the following steps:

[0071] Perform a multiplication operation on the trend reference of any one of the reference data and the credibility of this reference data to obtain the correction value of the credibility of this reference data, and use the correction value as the weight of this reference data.

[0072] In another embodiment, it is also possible to select linear correction, logarithmic correction, or other methods for correction according to specific circumstances. This embodiment does not particularly limit the selected method for correcting the credibility, as long as the relationship that the correction value is positively correlated with the trend reference of the corresponding reference data is ensured.

[0073] Furthermore, after determining the weights of each reference data, the weights of each reference data can be used to weight the moving average coefficients of the corresponding reference data obtained by maximum likelihood estimation, so as to obtain the weighted values of the moving average coefficients of each reference data. It should be noted that the process of determining the moving average coefficients of each error value by maximum likelihood estimation is a prior art, and this embodiment will not elaborate on it here.

[0074] In an exemplary embodiment of the present invention, the determination of the actual moving average coefficients of each reference data can be achieved through the following steps:

[0075] Obtain the weighted values of the moving average coefficients of each reference data and sum them. Calculate the ratio of the weighted value of the moving average coefficient of any one of the reference data to the obtained cumulative sum to obtain the normalized value of the weighted value, and use the normalized value as the actual moving average coefficient of any one of the reference data.

[0076] In another implementation, it is also possible to use a linear normalization function, such as functions, etc. to normalize the weighted values of the moving average coefficients of each reference data, so as to obtain the actual moving average coefficients of each reference data. This embodiment does not particularly limit the selected normalization method.

[0077] S3: Based on the actual moving average coefficients of each reference data, obtain the moving average value of the error for ARIMA prediction, and perform computing power scheduling based on the prediction result.

[0078] In an exemplary embodiment of the present invention, the determination of the moving average of all reference data can be achieved through the following steps:

[0079] Using the actual moving average coefficients of the respective reference data, perform a weighted summation of all reference data to obtain the moving average value.

[0080] It should be noted that obtaining the moving average of the error based on the moving average coefficients of the respective error terms is prior art in the MA model, and this embodiment will not elaborate on it here.

[0081] Furthermore, after obtaining the moving average of all reference data, an ARIMA prediction can be performed based on this moving average value to obtain a predicted value of the data volume processed by the target node in the next data segment, so that real-time computing power resource scheduling can be performed based on the prediction result, enabling the computing power resources to achieve the highest utilization rate. It should be noted that the process of performing ARIMA prediction based on the known moving average value is prior art, and this embodiment will not elaborate on it here.

[0082] The present invention also provides a computing power resource scheduling system, which includes a memory and a processor, and a computer program is stored on the memory. The computer program integrates the functions of a computing power resource scheduling method. When the computer program is executed, a computing power resource scheduling method can accurately predict the data volume processed by the target node within a specific period, thereby enabling precise scheduling of the computing power resources.

[0083] In the description of this specification, the meanings of "a plurality" and "several" are at least two, such as two, three, or more, unless otherwise specifically defined.

[0084] Although this specification has shown and described multiple embodiments of the present invention, it will be apparent to those skilled in the art that such embodiments are provided only by way of example. Those skilled in the art will think of many changes, alterations, and alternative ways without departing from the spirit and concept of the present invention. It should be understood that various alternative solutions to the embodiments of the present invention described herein can be adopted in the process of practicing the present invention.

Claims

1. A computing resource scheduling method, characterized in that: include: Collect the computing power demand data of the target node for multiple consecutive days and divide it into several data segments of the same length. Take the data volume of each data segment as a data point to build a historical demand sequence. ARIMA forecasting is performed based on the historical demand sequence. During the forecasting process, the previous several error values ​​of the current error value are used as reference data. The moving average coefficient of the corresponding reference data obtained by maximum likelihood estimation is weighted and normalized using the weight of each reference data to obtain the actual moving average coefficient of the corresponding reference data. Based on the actual moving average coefficient of each reference data, the moving average of all reference data is obtained for ARIMA forecasting, and computing power scheduling is performed based on the forecast results. The method for obtaining the weight includes: calculating the credibility of any reference data, wherein the credibility is negatively correlated with the difference between the any reference data and the average value of all reference data, and the difference between the any reference data and the average error value of all data points in the historical demand sequence; All reference data are fitted, and based on the fitting error of any reference data and the fitting errors of all reference data, the trend reference of any reference data is obtained, which satisfies the following relationship: , is the current error value The trend of reference data is referenceable. is the current error value The value of the reference data, is the current error value The fitted values ​​of the reference data, is the absolute value symbol, is the number of reference data for the current error value, is the natural exponential function; Using the trend reference of any reference data, the credibility of any reference data is corrected to obtain the weight of any reference data, including: multiplying the trend reference of any reference data and the credibility of the reference data to obtain a corrected value of the credibility of the reference data, and using the corrected value as the weight of the reference data.

2. A computing resource scheduling method according to claim 1, characterized in that: The number of reference data of the current error value is the same as the moving average order of the pre-built ARIMA model.

3. A computing resource scheduling method according to claim 1, characterized in that: The credibility of any reference data satisfies the following relationship: ； In the formula, is the current error value The credibility of the reference data; is the current error value The value of reference data; is the average value of the error values ​​of all data points in the historical demand sequence; is the average value of all reference data; is the absolute value symbol; is the standard deviation of the error values ​​of all data points in the historical demand sequence; is the standard deviation of all reference data for the current error value.

4. A computing resource scheduling method according to claim 3, characterized in that: The method for obtaining the error value of any data point in the historical demand sequence includes: For any data point in the historical demand sequence, the difference between the value of the data point and the predicted value corresponding to the data point is calculated to obtain the error value of the data point.

5. A computing resource scheduling method according to claim 1, characterized in that: A method for obtaining the actual moving average coefficient of any reference data includes: Obtain the weighted values ​​of the moving average coefficients of each reference data and sum them up, calculate the weighted value of the moving average coefficient of any reference data, and the ratio of the weighted value to the obtained cumulative sum to obtain the normalized value of the weighted value, and use the normalized value as the actual moving average coefficient of any reference data.

6. A computing resource scheduling method according to claim 1, characterized in that: The process of obtaining the moving average of all reference data based on the actual moving average coefficient of each reference data includes: The actual moving average coefficient of each reference data is used to perform weighted summation on all reference data to obtain the moving average of all reference data.

7. A computing resource scheduling method according to claim 1, characterized in that: The target node is a computer device.

8. A computing resource scheduling system, characterized in that: The computing power resource scheduling system includes a memory and a processor, the memory stores a computer program, and the processor executes the computer program to implement the steps of a computing power resource scheduling method as described in any one of claims 1-7.

Images