Resource planning method and device, electronic equipment and computer readable medium

Abstract

The invention discloses a resource planning method and device, electronic equipment and a computer readable medium, and relates to the technical field of computers, and the method comprises the steps: obtaining a target resource pool and a historical service node demand corresponding to a target task type according to a resource planning request; calling a time sequence prediction model to calculate a predicted service node demand quantity in a future preset time period based on a historical service node demand quantity, updating parameter values in a time sequence prediction index, and solving the target function to obtain an optimal service node demand quantity of the target resource pool; obtaining a statistical service node demand based on a heuristic algorithm; and according to the optimal service node demand quantity and the statistical service node demand quantity, determining the target service node demand quantity and comparing the target service node demand quantity with the current service node quantity in the target resource pool, and if the comparison result is inconsistent, adjusting the service node quantity of the target resource pool. The resource utilization efficiency is improved, and the queuing time is shortened.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a resource planning method, device, electronic device and computer-readable medium. Background Art

[0002] At present, with the growth of training needs and the increase in the complexity of training models, the demand for the underlying computing resources of the algorithm platform is also expanding, the cost of enterprise infrastructure is rising, the utilization efficiency of computing resources is low, and the waiting time for computing resource requests from submission to processing is long. Summary of the invention

[0003] In view of this, the embodiments of the present application provide a resource planning method, device, electronic device and computer-readable medium, which can solve the existing problems that as the training demand grows and the complexity of the training model increases, the demand for the underlying computing power resources of the algorithm platform is also expanding, the utilization efficiency of computing power resources is low, and the waiting time in queue from submission to processing of computing power resource requests is long.

[0004] To achieve the above object, according to one aspect of an embodiment of the present application, a resource planning method is provided, including:

[0005] Obtain the historical service node demand corresponding to the target resource pool and the target task type according to the received resource planning request;

[0006] Call the time series prediction model to calculate the predicted service node demand for a preset period of time in the future based on the historical service node demand, update the parameter value in the time series prediction index according to the predicted service node demand, and then solve the objective function containing the time series prediction index to obtain the optimal service node demand of the target resource pool;

[0007] Perform statistical analysis on historical service node demands based on a heuristic algorithm to obtain statistical service node demands;

[0008] Determine the target service node demand based on the optimal service node demand and the statistical service node demand;

[0009] The target service node demand is compared with the current number of service nodes in the target resource pool, and in response to the comparison result being inconsistent, the number of service nodes in the target resource pool is adjusted based on the target service node demand and the current number of service nodes.

[0010] Optionally, according to the predicted service node demand, the parameter values ​​in the time series prediction index are updated, including:

[0011] Calculate the number of service resource requests per unit time based on the predicted service node demand;

[0012] The parameter value of the field parameter corresponding to the number of service resource requests per unit time in the time series prediction indicator is updated to the calculated number of service resource requests per unit time.

[0013] Optionally, solving an objective function including a time series prediction indicator to obtain an optimal service node demand of a target resource pool includes:

[0014] Obtain the indicator calculation formula for each time series prediction indicator in the queuing system;

[0015] Based on the calculation formulas of various indicators, the corresponding relationship between the number of service resource requests per unit time of the target resource pool, the number of service nodes released per unit time, the service node utilization rate, the average waiting time of service resource requests and the demand for service nodes is established;

[0016] Based on the corresponding relationship, the timing prediction indicators in the objective function are converted into parameters corresponding to the number of service resource requests per unit time, the number of service nodes released per unit time, and the service node demand. Then, the objective function is optimized under the conditions of maximizing the service node utilization and minimizing the average waiting time of service resource requests, and the service node demand obtained by the solution is used as the optimal service node demand.

[0017] Optionally, adjusting the number of service nodes in the target resource pool based on the target service node demand and the current number of service nodes includes:

[0018] If the demand for the target service node is less than the current number of service nodes, the target node is determined based on the node priority when selecting nodes in the target resource pool corresponding to the type of the target resource pool, and the target node is selected from the target resource pool and added to the non-target resource pool.

[0019] Optionally, adjusting the number of service nodes in the target resource pool based on the target service node demand and the current number of service nodes includes:

[0020] If the demand for the target service node is greater than the current number of service nodes, the target node is determined based on the node priority when selecting nodes in the non-target resource pool corresponding to the type of the target resource pool, and the target node is selected from the non-target resource pool and added to the target resource pool.

[0021] Optionally, determining the target service node demand according to the optimal service node demand and the statistical service node demand includes:

[0022] Perform weighted average calculation on the optimal service node demand and the statistical service node demand to obtain a weighted average value;

[0023] The weighted average is determined as the target service node demand.

[0024] Optionally, determining the target service node demand according to the optimal service node demand and the statistical service node demand includes:

[0025] Calculate the absolute value of the difference between the optimal service node demand and the statistical service node demand;

[0026] If the absolute value of the difference is greater than a preset threshold, the statistical service node demand is determined as the target service node demand.

[0027] In addition, the present application also provides a resource planning device, including:

[0028] An acquisition unit is configured to acquire the historical service node demand corresponding to the target resource pool and the target task type according to the received resource planning request;

[0029] The objective function solving unit is configured to call the time series prediction model to calculate the predicted service node demand in the future preset time period based on the historical service node demand, update the parameter value in the time series prediction index according to the predicted service node demand, and then solve the objective function including the time series prediction index to obtain the optimal service node demand of the target resource pool;

[0030] A statistical analysis unit is configured to perform statistical analysis on historical service node demands based on a heuristic algorithm to obtain statistical service node demands;

[0031] A demand determination unit is configured to determine a target service node demand based on the optimal service node demand and the statistical service node demand;

[0032] The adjustment unit is configured to compare the target service node demand with the current number of service nodes in the target resource pool, and in response to an inconsistent comparison result, adjust the number of service nodes in the target resource pool based on the target service node demand and the current number of service nodes.

[0033] Optionally, the objective function solving unit is further configured to:

[0034] Calculate the number of service resource requests per unit time based on the predicted service node demand;

[0035] The parameter value of the field parameter corresponding to the number of service resource requests per unit time in the time series prediction indicator is updated to the calculated number of service resource requests per unit time.

[0036] Optionally, the objective function solving unit is further configured to:

[0037] Obtain the indicator calculation formula for each time series prediction indicator in the queuing system;

[0038] Based on the calculation formulas of various indicators, the corresponding relationship between the number of service resource requests per unit time of the target resource pool, the number of service nodes released per unit time, the service node utilization rate, the average waiting time of service resource requests and the demand for service nodes is established;

[0039] Based on the corresponding relationship, the timing prediction indicators in the objective function are converted into parameters corresponding to the number of service resource requests per unit time, the number of service nodes released per unit time, and the service node demand. Then, the objective function is optimized under the conditions of maximizing the service node utilization and minimizing the average waiting time of service resource requests, and the service node demand obtained by the solution is used as the optimal service node demand.

[0040] Optionally, the adjustment unit is further configured to:

[0041] If the demand for the target service node is less than the current number of service nodes, the target node is determined based on the node priority when selecting nodes in the target resource pool corresponding to the type of the target resource pool, and the target node is selected from the target resource pool and added to the non-target resource pool.

[0042] Optionally, the adjustment unit is further configured to:

[0043] If the demand for the target service node is greater than the current number of service nodes, the target node is determined based on the node priority when selecting nodes in the non-target resource pool corresponding to the type of the target resource pool, and the target node is selected from the non-target resource pool and added to the target resource pool.

[0044] Optionally, the demand quantity determination unit is further configured to:

[0045] Perform weighted average calculation on the optimal service node demand and the statistical service node demand to obtain a weighted average value;

[0046] The weighted average is determined as the target service node demand.

[0047] Optionally, the demand quantity determination unit is further configured to:

[0048] Calculate the absolute value of the difference between the optimal service node demand and the statistical service node demand;

[0049] If the absolute value of the difference is greater than a preset threshold, the statistical service node demand is determined as the target service node demand.

[0050] In addition, the present application also provides a resource planning electronic device, including: one or more processors; a storage device for storing one or more programs, when the one or more programs are executed by one or more processors, the one or more processors implement the resource planning method as described above.

[0051] In addition, the present application also provides a computer-readable medium on which a computer program is stored, and when the program is executed by a processor, the resource planning method as described above is implemented.

[0052] To achieve the above objective, according to another aspect of the embodiments of the present application, a computer program product is provided.

[0053] A computer program product according to an embodiment of the present application includes a computer program, and when the program is executed by a processor, the resource planning method provided by the embodiment of the present application is implemented.

[0054] An embodiment of the above invention has the following advantages or beneficial effects: the present application obtains the historical service node demand corresponding to the target resource pool and the target task type according to the received resource planning request; calls the time series prediction model to calculate the predicted service node demand for a preset time period in the future based on the historical service node demand, updates the parameter value in the timing prediction index according to the predicted service node demand, and then solves the objective function containing the timing prediction index to obtain the optimal service node demand of the target resource pool; performs statistical analysis on the historical service node demand based on the heuristic algorithm to obtain the statistical service node demand; determines the target service node demand according to the optimal service node demand and the statistical service node demand; compares the target service node demand with the current number of service nodes in the target resource pool, and in response to the inconsistency of the comparison result, adjusts the number of service nodes in the target resource pool based on the target service node demand and the current number of service nodes. Through objective functions and statistical analysis, resources are rationally planned with the goal of minimizing the waiting time of tasks in queue and maximizing the utilization of computing resources. A suitable recommended setting for the number of service nodes in the resource pool is given to improve resource utilization efficiency and shorten the waiting time for requests for computing resources from submission to processing.

[0055] The further effects of the above-mentioned non-conventional optional manner will be described below in conjunction with the specific implementation manner. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] The accompanying drawings are used to better understand the present application and do not constitute an improper limitation on the present application.

[0057] Figure 1 is a schematic diagram of the main process of a resource planning method according to an embodiment of the present application;

[0058] Figure 2 is a schematic diagram of the main process of a resource planning method according to an embodiment of the present application;

[0059] Figure 3It is a schematic diagram of the main process of an intelligent elastic resource pool solution of a resource planning method according to an embodiment of the present application;

[0060] Figure 4 It is a schematic diagram of a timing prediction module of an intelligent analysis service of a resource planning method according to an embodiment of the present application;

[0061] Figure 5 is a schematic diagram of a queuing system of a resource planning method according to an embodiment of the present application;

[0062] Figure 6 is a schematic diagram of an M / M / C queuing system of a resource planning method according to an embodiment of the present application;

[0063] Figure 7 is a schematic diagram of main units of a resource planning device according to an embodiment of the present application;

[0064] Figure 8 is an exemplary system architecture diagram to which the embodiments of the present application can be applied;

[0065] Fig. 9 It is a structural diagram of a computer system of a terminal device or server suitable for implementing an embodiment of the present application. DETAILED DESCRIPTION

[0066] The following is an explanation of the exemplary embodiments of the present application in conjunction with the accompanying drawings, including various details of the embodiments of the present application to facilitate understanding, which should be considered as merely exemplary. Therefore, those of ordinary skill in the art should recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the present application. Similarly, for the sake of clarity and conciseness, the description of known functions and structures is omitted in the following description. It should be noted that the acquisition, transmission, storage, use, and processing of data in the technical solution of the present application are in compliance with the relevant provisions of national laws and regulations. It should be noted that in the embodiments of the present application, certain software, components, models, and other existing solutions in the industry may be mentioned, which should be considered as exemplary, and their purpose is only to illustrate the feasibility of the implementation of the technical solution of the present application, but it does not mean that the applicant has or must have used the solution. In the technical solution of the present application, the collection, collection, update, analysis, processing, use, transmission, storage, etc. of the user personal information involved are in compliance with the provisions of relevant laws and regulations, are used for legal and reasonable purposes, and do not violate public order and good customs, are not shared, leaked or sold outside these legal uses, and are subject to supervision and management by regulatory authorities. Take necessary measures to prevent illegal access to such user personal information data, maintain the security of user personal information, network security and national security, ensure that persons who have access to personal information data comply with relevant laws and regulations, and ensure the security of user personal information. Once such user personal information data is no longer needed, the risk should be minimized by limiting or even prohibiting data collection and / or deleting data.

[0067] When used, including in certain related applications, user privacy is protected by de-identifying data, such as by removing specific identifiers when used, controlling the amount or specificity of data stored, controlling how data is stored, and / or other methods of de-identification.

[0068] Figure 1 is a schematic diagram of the main process of a resource planning method according to an embodiment of the present application, such as Figure 1 As shown, the resource planning method mainly includes the following steps S101 to S105.

[0069] Step S101: acquiring the historical service node demand corresponding to the target resource pool and the target task type according to the received resource planning request.

[0070] In this embodiment, the execution subject of the resource planning method (for example, it can be a server) can receive a resource planning request through a wired connection or a wireless connection. For example, the resource planning request can be a request for planning the underlying computing power resources of the algorithm platform, or it can be a request for planning storage resources. The embodiment of the present application is explained by taking the resource planning request as a request for planning the underlying computing power resources of the algorithm platform as an example. After receiving the resource planning request, the execution subject can obtain the target resource pool corresponding to the request according to the resource pool identifier carried in the request, and determine the target task type according to the target resource pool. When the target resource pool is a whole machine resource pool, the target task type can be, for example, a whole machine task; when the target resource pool is an ordinary resource pool, the target task type can be, for example, an ordinary task, and the corresponding historical service node demand is obtained according to the target task type. For example, all n GPU cards in the target resource pool are abstracted into n service desks, and the GPU resource demand of the platform task submitted to this target resource pool is abstracted into a service request for the GPU card. Each GPU card in the target resource pool serves one GPU request, and the service node can be the GPU card in the resource pool. Then, obtaining the corresponding historical service node demand according to the target task type can be obtaining the corresponding historical GPU card demand according to the target task type.

[0071] Step S102, calling the time series prediction model to calculate the predicted service node demand for a preset future period based on the historical service node demand, updating the parameter value in the time series prediction index according to the predicted service node demand, and then solving the objective function including the time series prediction index to obtain the optimal service node demand of the target resource pool.

[0072] For example, the time series prediction model is called, and the commonly used time series prediction method (that is, the time series prediction algorithm) is used to predict the time series data at a certain point in the future. For example, based on the historical service node demand of the whole machine task, for example, the historical GPU card resource demand time series data (recorded once every hour) can be used to predict the GPU card resource demand of the whole machine task of the platform in the next three hours. The time series prediction methods (that is, the time series prediction algorithms) adopted here include the triple exponential smoothing method holt-winter, the traditional machine learning method XGBoost based on decision trees, and the deep learning LSTM time series prediction method.

[0073] Specifically, according to the predicted service node demand, the parameter value in the timing prediction index is updated, including: calculating the number of service resource requests per unit time according to the predicted service node demand; updating the parameter value of the field parameter corresponding to the number of service resource requests per unit time in the timing prediction index to the calculated number of service resource requests per unit time.

[0074] For example, the number of service resource requests per unit time can be, for example, the number of GPU resource requests arriving at the target resource pool per unit time. Timing prediction indicators may include: the overall GPU utilization of the target resource pool, the probability of the system being idle, the average length of the waiting queue for GPU requests in the target resource pool, the average amount of GPU requests in the queuing system of the target resource pool, the average queuing waiting time and average stay time of GPU requests in the target resource pool, and other indicators. The number of service resource requests per unit time can be recorded as λ, so as to accurately update the parameter value of the field parameter corresponding to the number of service resource requests per unit time in the timing prediction indicator.

[0075] Step S103: Statistically analyze the historical service node demands based on a heuristic algorithm to obtain statistical service node demands.

[0076] For example, a heuristic algorithm is used to perform a statistical analysis of historical service node demands. For example, the average resource demand of related types of tasks (such as whole-machine tasks or ordinary tasks) in the past day is calculated as the statistical service node demand, thereby improving the scientific nature of the determination of the statistical service node demand.

[0077] Step S104: determining the target service node demand according to the optimal service node demand and the statistical service node demand.

[0078] Specifically, the target service node demand is determined according to the optimal service node demand and the statistical service node demand, including: performing weighted average calculation on the optimal service node demand and the statistical service node demand to obtain a weighted average value; and determining the weighted average value as the target service node demand.

[0079] According to the heuristic algorithm, the weighted average of the optimal service node demand obtained by prediction and the statistical service node demand obtained by statistics can be calculated, and then the weighted average is used as the target service node demand of the target resource pool. This improves the accuracy of determining the target service node demand in the target resource pool, improves resource utilization efficiency, and shortens the waiting time for computing resource requests from submission to processing.

[0080] Step S105, comparing the target service node demand with the current number of service nodes in the target resource pool, and in response to the comparison result being inconsistent, adjusting the number of service nodes in the target resource pool based on the target service node demand and the current number of service nodes.

[0081] Specifically, the number of service nodes in the target resource pool is adjusted based on the demand for target service nodes and the current number of service nodes, including: if the demand for target service nodes is less than the current number of service nodes, based on the node priority when selecting nodes in the target resource pool corresponding to the type of the target resource pool, the target node is determined, the target node is selected from the target resource pool and added to the non-target resource pool.

[0082] For example, the non-target resource pool may be a common resource pool, and the target resource pool may be a whole machine resource pool.

[0083] The target resource pool is determined to be expanded or reduced based on the result of the comparison between the target service node demand and the current number of service nodes in the target resource pool. When the target service node demand is less than the current number of service nodes, the target resource pool is reduced (for example, the entire machine resource pool is reduced), and then the target node is determined based on the node priority when selecting nodes in the target resource pool corresponding to the type of the target resource pool, and the target node is selected from the target resource pool and added to the non-target resource pool.

[0084] Specifically, when the whole machine resource pool is reduced in capacity, the target node is selected from the whole machine resource pool and added to the common resource pool. The target node is selected in the following order:

[0085] 1) Nodes running Jupyter notebook task Pods have higher priority than nodes without notebook task Pods;

[0086] 2) Nodes without whole-machine tasks running have higher priority than nodes with whole-machine tasks running;

[0087] 3) Nodes with fewer running tasks have higher priority than nodes with more running tasks. Nodes with the same number of running tasks and shorter average predicted remaining running time have higher priority. In this way, idle nodes with no resource usage or nodes that release machine resources faster can be added to the general resource pool first, thereby effectively responding to the increased general resource demand predicted in advance.

[0088] Among them, the comparison rules 1) and 2) are also intended to avoid fragmented allocation of GPU cards in the whole-machine resource pool nodes. For example, non-whole-machine allocation nodes that occupy 3 cards will be preferentially scaled down to the common resource pool. The rule 3) is also intended to effectively expand the available resources of the common resource pool to cope with the subsequent increase in resource demands of common resource pool tasks.

[0089] This embodiment obtains the historical service node demand corresponding to the target resource pool and the target task type according to the received resource planning request; calls the time series prediction model to calculate the predicted service node demand for the future preset period based on the historical service node demand, updates the parameter value in the time series prediction index according to the predicted service node demand, and then solves the objective function containing the time series prediction index to obtain the optimal service node demand of the target resource pool; statistically analyzes the historical service node demand based on the heuristic algorithm to obtain the statistical service node demand; determines the target service node demand according to the optimal service node demand and the statistical service node demand; compares the target service node demand with the current number of service nodes in the target resource pool, and in response to the inconsistent comparison result, adjusts the number of service nodes in the target resource pool based on the target service node demand and the current number of service nodes. Through the objective function and statistical analysis, the resources are reasonably planned, and the appropriate recommended setting of the number of service nodes in the resource pool is given with the goal of minimizing the queuing waiting time of the task and maximizing the utilization rate of the computing resources, so as to improve the resource utilization efficiency and shorten the queuing waiting time of the request for computing resources from submission to processing.

[0090] Figure 2 is a schematic diagram of the main flow of a resource planning method according to an embodiment of the present application, such as Figure 2 As shown, the resource planning method mainly includes the following steps S201 to S208.

[0091] Step S201: Obtain historical service node demands corresponding to a target resource pool and a target task type according to a received resource planning request.

[0092] The target resource pool may be a whole machine resource pool. The target task type may be a whole machine task. The historical service node demand corresponding to the target task type may be the historical service node demand corresponding to the whole machine task.

[0093] Step S202, calling the time series prediction model to calculate the predicted service node demand for a preset period of time in the future based on the historical service node demand, and updating the parameter value in the time series prediction index according to the predicted service node demand.

[0094] For example, the embodiments of the present application can use the machine learning model XGBoost as the time series prediction model of the intelligent analysis service timing prediction module, and use the triple exponential smoothing Holt-Winters as the time series prediction model of the intelligent analysis service statistical analysis module.

[0095] Step S203, obtaining the indicator calculation formula of each time series prediction indicator in the queuing system.

[0096] For example, the calculation formulas for the various time series prediction indicators in the queuing system are as shown in the following formulas (1) to (6):

[0097]

[0098]

[0099] In the above formula, λ refers to the average arrival rate of GPU requests in the resource pool, that is, the number of GPU resource requests arriving per unit time, and μ refers to the service rate of GPU requests, that is, the number of GPU requests that can be processed per unit time. Among them, λ is the number of GPU resource requests arriving per unit time calculated by the number of GPU resource demand cards in the target resource pool predicted by the timing prediction algorithm in the next few hours, that is, the average arrival rate of GPU requests in the resource pool, and μ is a statistical estimate that can be calculated by statistical analysis of the running time data of the resource pool tasks. Cμ in formula (1) represents the number of GPU requests that can be processed per unit time by a total of C GPU cards in the resource pool. Therefore, ρ calculated by formula (1) represents the overall GPU utilization of the resource pool, also known as the system load. Formula (2) represents the probability that there are 0 GPU resource requests in the system, that is, the probability that the system is in an idle state. k is an indicator variable of the summation symbol, k = 0, 1, 2, ..., C-1, which is an integer. Formula (3) calculates the average length of the waiting queue for GPU requests in the resource pool. Formula (4) represents the average number of GPU requests in the queuing system of the resource pool. Formulas (5) and (6) respectively calculate the average queuing waiting time and average stay time of GPU requests in the resource pool.

[0100] Step S204, based on various indicator calculation formulas, establish a corresponding relationship between the number of service resource requests per unit time in the target resource pool, the number of service nodes released per unit time, the service node utilization rate, the average waiting time of service resource requests and the service node demand.

[0101] Through the indicator calculation formulas (1) to (6) of the various timing prediction indicators in the above queuing system, we can establish the variables λ (the number of service resource requests per unit time in the target resource pool), μ (the number of service nodes released per unit time, which is a statistical estimate and can be calculated by statistically analyzing the running time data of the target resource pool tasks) and the resource pool GPU resource utilization ρ (service node utilization), the average waiting time W of the resource pool GPU request q Relationship with service node demand C.

[0102] Step S205, based on the corresponding relationship, converts the timing prediction indicators in the objective function into the form of parameters corresponding to the number of service resource requests per unit time, the number of service nodes released per unit time, and the service node demand, and then optimizes the objective function under the conditions of maximizing the service node utilization and minimizing the average waiting time of the service resource request, so that the service node demand obtained by the solution is used as the optimal service node demand.

[0103] Based on the calculation formulas (1) to (6) of various indicators, the target resource pool is established to calculate the number of service resource requests per unit time λ, the number of service nodes released per unit time μ, the service node utilization rate ρ, and the average waiting time W of service resource requests. q The corresponding relationship between the service node demand C and the objective function The time series prediction index in is converted into the form of parameters corresponding to the number of service resource requests per unit time λ, the number of service nodes released per unit time μ, and the service node demand C, as shown in the following formula corresponding to the optimization problem (i.e., the objective function): As shown in formula (7)-formula (11), the objective function is optimized under the conditions of maximizing the service node utilization and minimizing the average waiting time of service resource requests, and the service node demand C obtained by the solution is used as the optimal service node demand.

[0104] The core of the resource planning method of the embodiment of the present application is to dynamically adjust the number of resource pool nodes to maximize the utilization of GPU resources and minimize the average waiting time of resource pool tasks, that is, by adjusting the value of variable C to maximize the target variable ρ and minimize the target variable W q , thus converting the capacity planning problem into an optimization problem. The formula corresponding to the optimization problem is as follows:

[0105]

[0106] The formula corresponding to the above optimization problem is solved by formula (1)-formula (11):

[0107] stC1+C2=N,C1>0,C2>0 (7)

[0108] ρ1=f(λ1,μ1,C1) (8)

[0109] ρ2=f(λ2,μ2,C2) (9)

[0110] W q1 =g(λ1,μ1,C1) (10)

[0111] W q2 =g(λ2,μ2,C2) (11)

[0112] Where C1 and C2 represent the number of GPU cards in the whole resource pool and the common resource pool respectively, N represents the total number of cards in the two resource pools, and the functions f and g can be derived from the queuing theory model formula (1) to formula (6). i is the GPU request arrival rate per unit time calculated based on the GPU demand of the resource pool predicted by the timing prediction algorithm, μ i is the average service rate of the GPU cards in the resource pool task (which can be a statistical estimate), that is, the number of GPU cards released per unit time. This is an integer programming problem, and the optimal resource pool capacities C1 and C2 can be determined by program solution to maximize the system GPU resource utilization and minimize the task queue waiting time.

[0113] The embodiment of the present application performs intelligent resource pool planning based on time series prediction and queuing theory, adopts a predictive resource pool expansion and contraction method to respond to changes in resource demand for resource pool tasks in advance, and uses queuing theory methods to provide accurate resource pool capacity planning, thereby minimizing the average waiting time for task scheduling and maximizing the occupancy rate of cluster computing resources. Based on the time series prediction method, the resource demand of the resource pool in the future period is accurately predicted; based on queuing theory and optimization methods, the average queuing waiting time of cluster tasks and the maximization of cluster resource occupancy rate are used as optimization goals to obtain the predicted optimal resource pool capacity planning.

[0114] Step S206: Perform statistical analysis on historical service node demands based on a heuristic algorithm to obtain statistical service node demands.

[0115] In some embodiments, the statistical analysis of historical service node demands based on the heuristic algorithm may be to use the median of the historical service node demands as the statistical service node demands, so as to ensure the efficiency and accuracy of determining the statistical service node demands.

[0116] Step S207: determining the target service node demand according to the optimal service node demand and the statistical service node demand.

[0117] Specifically, the target service node demand is determined based on the optimal service node demand and the statistical service node demand, including: calculating the absolute value of the difference between the optimal service node demand and the statistical service node demand; if the absolute value of the difference is greater than a preset threshold, determining the statistical service node demand as the target service node demand.

[0118] When the gap between the predicted optimal service node demand and the statistical service node demand is too large, a more reasonable target service node demand can be accurately obtained by using the statistical service node demand as a backup.

[0119] Step S208, comparing the target service node demand with the current number of service nodes in the target resource pool, and in response to the comparison result being inconsistent, adjusting the number of service nodes in the target resource pool based on the target service node demand and the current number of service nodes.

[0120] Specifically, the number of service nodes in the target resource pool is adjusted based on the demand for the target service node and the current number of service nodes, including: if the demand for the target service node is greater than the current number of service nodes, based on the node priority when selecting nodes in the non-target resource pool corresponding to the type of the target resource pool, the target node is determined, the target node is selected from the non-target resource pool and added to the target resource pool.

[0121] The target resource pool is determined to be expanded or reduced based on the result of comparing the target service node demand with the current number of service nodes in the target resource pool. When the target service node demand is greater than the current number of service nodes, the target resource pool is expanded (for example, the entire machine resource pool is expanded), and then the target node is determined based on the node priority when selecting nodes in the non-target resource pool corresponding to the type of the target resource pool, and the target node is selected from the non-target resource pool and added to the target resource pool.

[0122] For example, the non-target resource pool may be a common resource pool, and the target resource pool may be a whole machine resource pool.

[0123] For example, when the whole machine resource pool (i.e., the target resource pool) is expanded, target nodes are selected from the common resource pool (i.e., the non-target resource pool) to be added to the whole machine resource pool. The target nodes are selected from the common resource pool (i.e., the non-target resource pool) in the following order:

[0124] 1. Nodes running Jupyter notebook task Pods have lower priority than nodes without notebook task Pods. This is because notebook tasks usually take longer to run than ordinary training tasks. If a node with 7 idle GPU cards running a notebook task that applies for 1 GPU card is added to the whole machine resource pool, the 1 GPU card on the node will not be released for a long time, resulting in the 7 idle GPU cards not being effectively utilized because they cannot meet the resource requirements of the task that applies for 8 GPU cards for the whole machine. Notebook tasks usually refer to data processing and analysis tasks performed in the Notebook environment.

[0125] 2. Nodes without whole-machine tasks have lower priority than nodes with whole-machine tasks. For example, if Node 1 occupies 5 GPU cards when running Task A and Node 2 occupies 8 GPU cards when running Task B, Node 1 has lower priority than Node 2 when expanding the whole-machine resource pool.

[0126] 3. Nodes with fewer running tasks have higher priority than nodes with more running tasks. Nodes with the same number of running tasks and shorter average predicted remaining running time have higher priority. In this way, idle nodes with no resource usage or nodes that release machine resources faster can be added to the machine resource pool first, thereby effectively responding to the increased machine resource demand predicted in advance.

[0127] Among them, the first and second comparison rules are to avoid the fragmentation of the whole machine resource pool and ensure that the GPU cards on the nodes of the whole machine resource pool are allocated to the whole machine as much as possible; the third rule is to effectively expand the available resources of the whole machine resource pool to meet the increased demand for the whole machine resource pool.

[0128] The relevant terms that may be involved in the embodiments of the present application are interpreted as follows: Kubernetes: container orchestration system; Pod: the smallest unit that can be created and deployed in Kubernetes; Job / Task: A task is the smallest unit for resource scheduling in the system, corresponding to the Pod resource object in the Kubernetes container orchestration system. The scheduling process of a task refers to binding a node that meets the resource requirements to the task and starting the task on the node; A job is the basic unit for a user to submit a computing task to a cluster. A job usually contains multiple tasks. For example, a training job under the distributed training PS-Worker architecture usually contains multiple PS tasks (responsible for storing and synchronizing model parameters) and Worker tasks (responsible for calculating model gradients). The scheduling process of a job refers to scheduling the tasks contained in the job to the cluster nodes and the job will only start running when all the tasks of the job (or the pre-specified minimum number of running tasks) are scheduled to the cluster nodes; Tenant: A cluster is usually shared by multiple organizations or users, and these organizations or users are called tenants; Node label: Kubernetes will add labels to the nodes in the cluster, and will select the appropriate node for the Pod based on the node label when scheduling tasks.

[0129] Figure 3It is a schematic diagram of the main process of the intelligent elastic resource pool solution of the resource planning method according to an embodiment of the present application. The embodiment of the present application focuses on the management of the underlying computing power resources of the algorithm platform and the scheduling of training tasks. The training task scheduler adopts a variety of scheduling strategies to make the use of node resources after scheduling more balanced. Intelligent resource pool planning based on time series prediction and queuing theory adopts a time series prediction method to predict the future GPU resource demand of the resource pool, and further adopts queuing theory to give the optimal resource pool capacity planning. Finally, according to the recommended number of nodes of the resource pool given by the capacity planning, the node scale of the resource pool is dynamically adjusted, thereby improving the utilization efficiency of cluster computing resources and reducing the overall queuing waiting time of training tasks. The underlying cluster computing power resources of the current algorithm platform are divided into several independent resource pools, which may include a collection resource pool and an ordinary resource pool. The whole machine resource pool runs whole machine tasks, and the ordinary resource pool runs ordinary tasks.

[0130] The embodiment of the present application can perform intelligent resource pool planning based on time series prediction and queuing theory, and can dynamically adjust the number of resource pool nodes according to changes in resource application amounts of different types of tasks.

[0131] In particular, the embodiment of the present application adopts a predictive resource pool expansion and contraction method. Compared with the responsive resource pool expansion and contraction, it can respond to the changes in the resource application amount of the resource pool task in advance. At the same time, the predictive resource pool expansion and contraction has higher accuracy. When it is predicted that the demand for a certain type of task will increase, the resource pool will be expanded in advance, thereby effectively responding to the increase in resource demand and effectively reducing the waiting time of this type of task; when it is predicted that the demand for a certain type of task will decrease, the resource pool will be reduced in advance, thereby avoiding idle computing resources in the resource pool and improving the utilization of computing resources. In addition, the embodiment of the present application adopts the queuing theory in operations research to plan the capacity of the resource pool, with the goal of minimizing the waiting time of the platform task in the queue and maximizing the occupancy rate of cluster computing resources, and giving a suitable recommended setting for the number of resource pool nodes.

[0132] The intelligent resource pool planning method based on time series prediction and queuing theory proposed in the embodiment of the present application is as follows: Figure 3 As shown, here we take the intelligent automatic adjustment of the whole machine resource pool and the common resource pool as an example. The overall technical solution includes engine (configurable, scheduled periodic execution), node agent (k8s operator), intelligent analysis service (intelligent analysis module), data source (Datasource) and grafana visualization, among which:

[0133] The engine is a periodic execution engine that provides a series of configurable parameters to control the behavior of the intelligent resource pool planning system, such as supporting the configuration of the periodic duration, that is, how often to adjust the node scale of the target resource pool, and periodically accessing the intelligent analysis module (IAM service) to obtain the appropriate resource pool node scale at the next moment.

[0134] Node agent (in k8s cluster) is a custom k8s operator. Node agent listens to custom resource objects based on the watch mechanism in k8sclient-go. When custom resource objects are updated, they read the spec.recommendations[0].recommendedNodes and spec.recommendations[1].recommendedNodes fields in the custom resource objects (representing the recommended number of nodes for the whole resource pool and the common resource pool, respectively), and check whether the current number of resource pool nodes is consistent with the recommended number of nodes. If they are inconsistent, dynamic expansion and contraction of the resource pool is performed. For example, it can be based on the common resource pool (easyalgo=train) and the whole resource pool (easyalgo=train. When the whole resource pool is expanded, nodes are selected from the common resource pool and marked as Figure 3 The label scheduling-type=slaNode is shown; when the whole machine resource pool is scaled down, a node is selected from the whole machine resource pool and the label scheduling-type=slaNode on the whole machine node is removed) to dynamically adjust the scale of the whole machine resource pool; the node agent is a management agent that monitors the application servers on the host system and routes management requests to the servers.

[0135] The intelligent analysis module (IAM) is the core component of the intelligent resource pool planning solution. It is used by the engine component to periodically access and obtain the optimal number of resource pool nodes for the next periodic moment. The intelligent analysis service mainly includes three main modules:

[0136] Time series prediction module: predicts the resource demand for a period of time in the future based on the historical resource demand of tasks of related types in the resource pool, and recommends the number of resource pool nodes based on the time series prediction indicators and capacity planning methods;

[0137] Statistical analysis module: Statistical analysis of historical resource demand indicators of tasks of related types in the resource pool, such as calculating the average resource demand of tasks of related types in the past day and giving a recommended setting for the number of resource pool nodes based on this;

[0138] Resource pool size decision module: Based on the results given by the timing prediction module and the statistical analysis module, the recommended setting of the number of resource pool nodes in the next cycle time period is given according to the heuristic rules and returned to the engine component. The heuristic rules can be, for example, a weighted average of the recommended settings of the number of resource pool nodes given by the timing prediction module and the statistical analysis module, and when the number of resource pool nodes given by the timing prediction module deviates greatly from the result of the statistical analysis module, the result of the statistical analysis module is used as a backup.

[0139] Data source and visualization: Data sources mainly include Prometheus, ClickHouse and MySQL databases, which store data such as historical resource demand of resource pool tasks for access by intelligent analysis services. At the same time, the intelligent analysis service periodically writes the predicted indicators and observed indicators of resource demand into the ClickHouse database and displays them in the grafana visualization panel, so as to compare the prediction effects of different time series prediction algorithms.

[0140] The overall execution process of the resource planning method of the embodiment of the present application is as follows: (1) the engine component periodically accesses the intelligent analysis service to request the recommended number of nodes for the whole machine resource pool and the ordinary resource pool within the next period; (2) the intelligent analysis service accesses the data source to obtain the historical GPU resource demand indicators of the whole machine resource pool and the ordinary resource pool related types of tasks (whole machine tasks and ordinary tasks), calculates the recommended number of nodes for the two resource pools and returns it to the engine component, and writes the observed value and time series prediction value of the task GPU resource demand to the ClickHouse database for visual display; (3) the engine component writes the recommended number of nodes in the resource pool returned by the intelligent analysis service into a specific field in the custom resource object. For example, the engine component rewrites the CRD Spec according to the analysis result of the intelligent analysis service. For example, the rewritten custom resource (CRD) xxx spec: slaNodeNum: 4xxx; (4) the node agent component listens (watches) to the update event of the custom resource object and lists the resource pool nodes to detect whether the actual number of nodes in the whole machine resource pool and the ordinary resource pool is consistent with the recommended number of nodes set for the next period. If not, the resource pool is expanded or reduced.

[0141] Detailed design of intelligent resource pool planning component: The engine component periodically accesses the intelligent analysis service to obtain the recommended node number setting of the target resource pool in the next cycle, and writes the setting into the custom resource object. The engine module provides a series of configurable parameters to control the behavior of the intelligent resource pool planning system. The configurable parameters are shown in the following table:

[0142]

[0143] The intelligent analysis module (IAM) is periodically accessed by the engine component, and provides the recommended node number setting for the resource pool in the future based on the historical resource demand indicator data of the target resource pool-related tasks. The intelligent analysis service consists of three main modules, including the time series prediction module, the statistical analysis module, and the resource pool size decision module.

[0144] Figure 4 Schematic diagram of a timing prediction module of an intelligent analysis service of a resource planning method according to an embodiment of the present application. Figure 4 As shown in the figure, the execution of this module is divided into two stages: the first stage predicts the timing indicators based on the timing data (timing data: timing indicators such as the number of GPU card applications and the number of task submissions within a time interval (i.e., the average arrival rate per unit time)), and adopts the commonly used time series prediction method to predict the timing data at a certain point in the future. For example, the historical GPU card resource demand timing data of the whole machine task is used to predict the GPU card resource demand of the whole machine task of the platform in the next three hours. The time series prediction methods adopted here include the three-time exponential smoothing method holt-winter, the traditional machine learning method XGBoost based on the decision tree, and the deep learning LSTM timing prediction method; the second stage solves the optimization problem according to the timing prediction indicators (the solution goal of the optimization problem is to minimize the average waiting time of the task and maximize the occupancy rate of the GPU card (computing resource) of the resource pool) to give the optimal node number setting of the target resource pool. Here, the M / M / C model in the queuing theory is used as the theoretical basis.

[0145] The statistical analysis module serves as a backup. It uses the heuristic strategy to solve the recommended node number setting of the resource pool from a statistical perspective based on the task GPU resource demand data over a period of time. It performs statistical analysis on the historical resource demand indicators of tasks of related types in the resource pool, such as calculating the average resource demand of tasks of related types in the past day and giving the recommended setting of the number of resource pool nodes based on this.

[0146] The resource pool scale decision module uses a certain strategy to combine the results of the timing prediction module and the statistical analysis module to obtain the final target resource pool recommended node number setting and returns it to the engine component. Based on the results given by the timing prediction module and the statistical analysis module, the recommended setting of the number of resource pool nodes in the next cycle time period is given according to the heuristic rules and returned to the engine component. The heuristic rules include, for example, taking a weighted average of the recommended setting of the number of resource pool nodes given by the timing prediction module and the statistical analysis module, or taking the result of the statistical analysis module as a backup when the number of resource pool nodes given by the timing prediction module deviates greatly from the result of the statistical analysis module.

[0147] The node agent component is implemented as a k8s operator, which includes a custom resource CRD and a controller control module. When executing elastic expansion and contraction of the resource pool, the engine component writes the recommended node number setting of the target resource pool returned by the intelligent analysis service into the custom resource object spec.recommendations[0].recommendedNodes and spec.recommendations[1].recommendedNodes fields (indicating the recommended node number settings for the entire resource pool and the common resource pool). After the node agent component monitors the changes in the custom resource object, it detects whether the actual number of nodes in the resource pool is consistent with the recommended number of nodes given in the custom resource object. If they are inconsistent, the resource pool is expanded or contracted. When executing the expansion of the entire resource pool, select nodes from the common resource pool and mark the nodes with the following: Figure 3 The label of scheduling-type=slaNode is shown in the figure; when the whole machine resource pool is scaled down, a node is selected from the whole machine resource pool and the label of scheduling-type=slaNode on the whole machine node is removed. The cluster resource pool division adopts the soft isolation method based on the node special label. The custom scheduler distinguishes different resource pools according to the node special label and only schedules specific types of tasks to run on the corresponding resource pool node.

[0148] In particular, when performing capacity expansion and contraction of the entire machine resource pool, how to select the target node needs to follow certain rules to avoid the fragmentation of GPU card resources of the entire machine resource pool node due to capacity expansion and contraction of the node.

[0149] Specifically, when the whole machine resource pool is expanded, nodes are selected from the common resource pool and added to the whole machine resource pool. The nodes are selected in the following order:

[0150] 1. Nodes running Jupyter notebook task Pods have lower priority than nodes without notebook task Pods. This is because notebook tasks usually take longer to run than ordinary training tasks. If a node with 7 idle GPU cards running a notebook task that applies for 1 GPU card is added to the whole machine resource pool, the 1 GPU card on the node will not be released for a long time, resulting in the 7 idle GPU cards not being effectively utilized because they cannot meet the resource requirements of the task that applies for 8 GPU cards for the whole machine. Notebook tasks usually refer to data processing and analysis tasks performed in the Notebook environment.

[0151] 2. Nodes without whole-machine tasks have lower priority than nodes with whole-machine tasks. For example, if Node 1 occupies 5 GPU cards when running Task A and Node 2 occupies 8 GPU cards when running Task B, Node 1 has lower priority than Node 2 when expanding the whole-machine resource pool.

[0152] 3. Nodes with fewer running tasks have higher priority than nodes with more running tasks. Nodes with the same number of running tasks and shorter average predicted remaining running time have higher priority. In this way, idle nodes with no resource usage or nodes that release machine resources faster can be added to the machine resource pool first, thereby effectively responding to the increased machine resource demand predicted in advance.

[0153] Among them, the first and second comparison rules are to avoid the fragmentation of the whole machine resource pool and ensure that the GPU cards on the nodes of the whole machine resource pool are allocated to the whole machine as much as possible; the third rule is to effectively expand the available resources of the whole machine resource pool to meet the increased demand for the whole machine resource pool.

[0154] Specifically, when the whole machine resource pool is reduced in capacity, nodes are selected from the whole machine resource pool and added to the common resource pool. The nodes are selected in the following order:

[0155] 1) Nodes running Jupyter notebook task Pods have higher priority than nodes without notebook task Pods;

[0156] 2) Nodes without whole-machine tasks running have higher priority than nodes with whole-machine tasks running;

[0157] 3) Nodes with fewer running tasks have higher priority than nodes with more running tasks. Nodes with the same number of running tasks and shorter average predicted remaining running time have higher priority. In this way, idle nodes with no resource usage or nodes that release machine resources faster can be added to the general resource pool first, thereby effectively responding to the increased general resource demand predicted in advance.

[0158] Among them, the comparison rules 1) and 2) are also intended to avoid fragmented allocation of GPU cards in the whole-machine resource pool nodes. For example, non-whole-machine allocation nodes that occupy 3 cards will be preferentially scaled down to the common resource pool. The rule 3) is also intended to effectively expand the available resources of the common resource pool to cope with the subsequent increase in resource demands of common resource pool tasks.

[0159] Figure 5 It is a schematic diagram of a queuing system according to a resource planning method according to an embodiment of the present application. Specifically, the timing prediction algorithm (also referred to as the time series prediction method in the embodiment of the present application) and the queuing theory: the intelligent elastic resource pool planning of the resource planning method in the embodiment of the present application adopts a timing prediction algorithm to predict the GPU resource requirements of the resource pool tasks in the future period, and uses queuing theory as a theoretical basis to solve the optimization problem and give the optimal number of resource pool nodes. Recommendations, so as to respond in advance to changes in resource demand for the resource pool in the future period, - expand the resource pool in advance to cope with the sudden increase in GPU card resource demand or shrink the resource pool in advance to avoid idle GPU resources in the resource pool. The relevant timing prediction algorithm and queuing theory will be explained as follows:

[0160] The goal of the time series prediction algorithm is to predict the GPU resource demand in the next few hours based on the historical GPU resource demand time series data of the resource pool for offline tasks. It is a typical regression problem. Taking the whole machine resource pool as an example, the input data is the historical GPU resource demand of the whole machine task, and the output data is the GPU resource demand of the whole machine task in the next few hours. Common time series prediction methods can be divided into the following categories: Statistical strategy: Use the results of recent moments to perform mean, median, time decay and other calculation methods to directly obtain future results. This method is relatively simple; Time series model: Commonly used time series models include exponential smoothing method, gray prediction model, ARIMA prediction model, seasonal SARIMA model, etc. This type of time series prediction model can only characterize sequence information and cannot add other information for training; Machine learning model: Adopt the method of feature engineering + machine learning regression. Common time series prediction models based on machine learning methods include decision tree models such as XGBoost and LightGBM. This type of model relies on manual feature extraction and requires manual construction of a large number of time series related features; Deep learning model: The deep learning model based on deep neural network is an end-to-end sequence prediction that does not rely on feature engineering and can automatically extract time series features. Common deep learning models include recurrent neural network model LSTM, DeepAR and Informer model based on Transformer introducing attention mechanism.

[0161] For example, the embodiment of the present application uses the machine learning model XGBoost as the time series prediction model of the intelligent analysis service timing prediction module, and uses the triple exponential smoothing Holt-Winters as the time series prediction model of the intelligent analysis service statistical analysis module.

[0162] Compared with deep learning models, the XGBoost model is more lightweight, has a smaller model size and lower training cost, and also has higher prediction accuracy; the Holt-Winters triple exponential smoothing model uses statistical analysis methods to extract the trend and seasonal characteristics of time series, and introduces attenuation characteristics to give recent time series indicators a higher weight. At the same time, the Holt-Winters model also has the characteristics of lower training cost and faster training speed.

[0163] The embodiment of the present application uses queuing theory modeling to plan the resource pool capacity. Queuing theory is also known as random service system theory, and its research content covers the following three aspects: (1) Performance problems: mainly studying the probabilistic regularity of various queuing systems, including queue length distribution, customer waiting time distribution, and system busy period distribution; (2) Optimization problems: including static optimality and dynamic optimality. Static optimality refers to the optimal system design, that is, how to design the queuing system to maximize efficiency or minimize cost; dynamic optimality involves the optimal operation strategy, that is, how to adjust the service mechanism in actual operation to optimize system performance; (3) Statistical inference problems: mainly used to determine the type of queuing system for further analysis and optimization.

[0164] like Figure 5 As shown in the figure, in a queuing system, requests arrive (service objects) and queue up to wait for service from the service agency, and leave after the service is completed. For example, customers start from the customer source, queue up in front of the service agency and wait for service, and leave the system after the service is completed. The input process indicates how user access arrives, such as the probability distribution of user access time; the queuing rules indicate how user requests are queued in the system to wait for processing, whether the queue is a single queue or multiple queues, parallel or serial, and whether the queue capacity is limited or infinite; the service rules include the number of service nodes, the order of processing requests, etc. A queuing system can be represented by a six-tuple A / S / C / K / N / D: A represents the distribution of arriving requests, such as M represents exponential distribution, G represents the distribution of general service time, etc.; S represents the distribution of service time of the request receiving service desk; C represents the number of service desks; K represents the system capacity limit, that is, the maximum number of queued requests that can be accommodated; N represents the total number of requests, and ∞ represents an infinite number of requests; D represents the service scheduling policy of the request, and the default scheduling policy is first-come, first-served.

[0165] Figure 6It is a schematic diagram of an M / M / C queuing system according to a resource planning method according to an embodiment of the present application.

[0166] The technical solution of the present invention models the resource pool capacity planning problem as an M / M / C queuing system, such as Figure 6 As shown, in the M / M / C queuing system, a request arrives (service object), and the arrived request enters the GPU request waiting queue, and each GPU card (GPU1, GPU2, ..., GPUn) in the resource pool serves a GPU request in the GPU request waiting queue.

[0167] For example, all n GPU cards in the resource pool are abstracted as n service desks, and the GPU resource requirements of the platform tasks submitted to this resource pool are abstracted as service requests for GPU cards. Each GPU card in the resource pool serves one GPU request. When all GPUs in the resource pool are occupied, the remaining GPU requests will wait in the waiting queue. In addition, for the M / M / C queuing system, the first two M's indicate that the arrival time interval of GPU card requests and the time interval for the GPU service desk to end service and release the GPU card both obey the negative exponential distribution, or in other words, the arrival of GPU resource requests and the release of task-occupied GPU cards satisfy the Poisson distribution. C indicates that the number of service desks is C, that is, there are C GPU cards in the resource pool. The M / M / C queuing system also assumes that the length of the request waiting queue is infinite, the number of requests is also infinite, and the GPU requests in the waiting queue are served according to the first-come, first-served FIFO scheduling policy.

[0168] Furthermore, for the M / M / C queuing system, several important indicators in the queuing system can be calculated by the following formulas, including: the GPU utilization rate of the resource pool ρ, the average number of GPU resource requests waiting in the resource pool L q , the average number of GPU resource requests in the resource pool (including GPU resource requests waiting in queue and being served) L s , the average waiting time W of resource pool GPU request q and the average stay time of GPU requests in the resource pool (including waiting time in queue and receiving service time) W s The calculation of these indicators is shown in the following formula (1) to formula (6):

[0169]

[0170] In the above formula, λ refers to the average arrival rate of GPU requests in the resource pool, that is, the number of GPU resource requests arriving per unit time, and μ refers to the service rate of GPU requests, that is, the number of GPU requests that can be processed per unit time. Among them, λ is the number of GPU resource requests arriving per unit time calculated by the number of GPU resource demand cards in the target resource pool predicted by the timing prediction algorithm in the next few hours, that is, the average arrival rate of GPU requests in the resource pool, and μ is a statistical estimate that can be calculated by statistical analysis of the running time data of the resource pool tasks. Cμ in formula (1) represents the number of GPU requests that can be processed per unit time by a total of C GPU cards in the resource pool. Therefore, ρ calculated by formula (1) represents the overall GPU utilization of the resource pool, also known as the system load. Formula (2) represents the probability that there are 0 GPU resource requests in the system, that is, the probability that the system is in an idle state. k is an indicator variable of the summation symbol, k = 0, 1, 2, ..., C-1, which is an integer. Formula (3) calculates the average length of the waiting queue for GPU requests in the resource pool. Formula (4) represents the average number of GPU requests in the queuing system of the resource pool. Formulas (5) and (6) respectively calculate the average queuing waiting time and average stay time of GPU requests in the resource pool.

[0171] Through the above formulas (1) to (6), we can establish the relationship between variables λ, μ, resource pool GPU resource utilization ρ, and the average waiting time W of resource pool GPU requests. q The core of the resource planning method of the embodiment of the present application is to dynamically adjust the number of resource pool nodes to maximize the utilization of GPU resources and minimize the average waiting time of resource pool tasks, that is, by adjusting the value of variable C to maximize the target variable ρ and minimize the target variable W q , thus converting the capacity planning problem into an optimization problem. The formula corresponding to the optimization problem is as follows:

[0172]

[0173] Solve the formula corresponding to the above optimization problem through formula (1)-formula (11);

[0174] stC1+C2=N,C1>0,C2>0 (7)

[0175] ρ1=f(λ1,μ1,C1) (8)

[0176] ρ2=f(λ2,μ2,C2) (9)

[0177] W q1 =g(λ1,μ1,C1) (10)

[0178] W q2=g(λ2,μ2,C2) (11)

[0179] Where C1 and C2 represent the number of GPU cards in the whole resource pool and the common resource pool respectively, N represents the total number of cards in the two resource pools, and the functions f and g can be derived from the queuing theory model formula (1) to formula (6). i is the GPU request arrival rate per unit time calculated based on the GPU demand of the resource pool predicted by the timing prediction algorithm, μ i is the average service rate of GPU cards in resource pool tasks (a statistical estimate that can be calculated by statistical analysis of the running time data of resource pool tasks), that is, the number of GPU cards released per unit time. This is an integer programming problem, and the optimal resource pool capacities C1 and C2 can be determined by program solution to maximize the utilization of system GPU resources and minimize the waiting time of task queues.

[0180] The embodiment of the present application performs intelligent resource pool planning based on time series prediction and queuing theory, adopts a predictive resource pool expansion and contraction method to respond to changes in resource demand for resource pool tasks in advance, and uses queuing theory methods to provide accurate resource pool capacity planning, thereby minimizing the average waiting time for task scheduling and maximizing the occupancy rate of cluster computing resources. Based on the time series prediction method, the resource demand of the resource pool in the future period is accurately predicted; based on queuing theory and optimization methods, the optimal resource pool capacity planning is obtained with the optimization goals of minimizing the average queuing waiting time of cluster tasks and maximizing the cluster resource occupancy rate; based on the K8Scontroller tuning control logic, a set of automated resource pool scale expansion and contraction mechanisms is implemented.

[0181] Figure 7 Schematic diagram of the main units of the resource planning device according to an embodiment of the present application. Figure 7 As shown, the resource planning device 700 includes an acquisition unit 701 , an objective function solving unit 702 , a statistical analysis unit 703 , a demand quantity determination unit 704 and an adjustment unit 705 .

[0182] The acquisition unit 701 is configured to acquire the historical service node demand corresponding to the target resource pool and the target task type according to the received resource planning request.

[0183] The objective function solving unit 702 is configured to call the time series prediction model to calculate the predicted service node demand for a preset time period in the future based on the historical service node demand, update the parameter value in the time series prediction index according to the predicted service node demand, and then solve the objective function including the time series prediction index to obtain the optimal service node demand of the target resource pool.

[0184] The statistical analysis unit 703 is configured to perform statistical analysis on the historical service node demands based on a heuristic algorithm to obtain statistical service node demands.

[0185] The demand determination unit 704 is configured to determine the target service node demand according to the optimal service node demand and the statistical service node demand.

[0186] The adjustment unit 705 is configured to compare the target service node demand with the current number of service nodes in the target resource pool, and in response to inconsistent comparison results, adjust the number of service nodes in the target resource pool based on the target service node demand and the current number of service nodes.

[0187] In some embodiments, the objective function solving unit 702 is further configured to: calculate the number of service resource requests per unit time based on the predicted service node demand; and update the parameter value of the field parameter corresponding to the number of service resource requests per unit time in the timing prediction indicator to the calculated number of service resource requests per unit time.

[0188] In some embodiments, the objective function solving unit 702 is further configured to: obtain the indicator calculation formula for each timing prediction indicator in the queuing system; establish the correspondence between the number of service resource requests per unit time of the target resource pool, the number of service nodes released per unit time, the service node utilization rate, the average waiting time of service resource requests and the service node demand based on the various indicator calculation formulas; based on the corresponding relationship, convert the timing prediction indicators in the objective function into a form containing parameters corresponding to the number of service resource requests per unit time, the number of service nodes released per unit time and the service node demand, and then optimize the objective function under the conditions of maximizing the service node utilization rate and minimizing the average waiting time of the service resource request, so as to use the service node demand obtained by the solution as the optimal service node demand.

[0189] In some embodiments, the adjustment unit 705 is further configured to: if the demand for the target service node is less than the current number of service nodes, determine the target node based on the node priority when selecting a node in the target resource pool corresponding to the type of the target resource pool, select the target node from the target resource pool and add it to the non-target resource pool.

[0190] In some embodiments, the adjustment unit 705 is further configured to: if the demand for the target service node is greater than the current number of service nodes, determine the target node based on the node priority when selecting nodes in the non-target resource pool corresponding to the type of the target resource pool, select the target node from the non-target resource pool and add it to the target resource pool.

[0191] In some embodiments, the demand determination unit 704 is further configured to: perform weighted average calculation on the optimal service node demand and the statistical service node demand to obtain a weighted average value; and determine the weighted average value as the target service node demand.

[0192] In some embodiments, the demand determination unit 704 is further configured to: calculate the absolute value of the difference between the optimal service node demand and the statistical service node demand; if the absolute value of the difference is greater than a preset threshold, determine the statistical service node demand as the target service node demand.

[0193] It should be noted that the resource planning method and resource planning device of the present application have a corresponding relationship in terms of specific implementation contents, so the repeated contents will not be described again.

[0194] Figure 8 An exemplary system architecture 800 is shown to which the resource planning method or resource planning apparatus according to the embodiments of the present application can be applied.

[0195] like Figure 8 As shown, system architecture 800 may include terminal devices 801, 802, 803, a network 804, and a server 805. Network 804 is used to provide a medium for communication links between terminal devices 801, 802, 803 and server 805. Network 804 may include various connection types, such as wired, wireless communication links, or optical fiber cables, etc.

[0196] Users can use terminal devices 801, 802, and 803 to interact with server 805 through network 804 to receive or send messages, etc. Various communication client applications can be installed on terminal devices 801, 802, and 803, such as shopping applications, web browser applications, search applications, instant messaging tools, email clients, social platform software, etc. (only examples).

[0197] The terminal devices 801 , 802 , and 803 may be various electronic devices having a resource planning processing screen and supporting web browsing, including but not limited to smart phones, tablet computers, laptop computers, desktop computers, and the like.

[0198] Server 805 may be a server that provides various services, such as a background management server that supports resource planning requests submitted by users using terminal devices 801, 802, and 803 (for example only). The background management server may obtain the historical service node demand corresponding to the target resource pool and the target task type according to the received resource planning request; call the time series prediction model to calculate the predicted service node demand for a preset period of time in the future based on the historical service node demand, update the parameter value in the time series prediction index according to the predicted service node demand, and then solve the objective function containing the time series prediction index to obtain the optimal service node demand of the target resource pool; perform statistical analysis on the historical service node demand based on the heuristic algorithm to obtain the statistical service node demand; determine the target service node demand according to the optimal service node demand and the statistical service node demand; compare the target service node demand with the current number of service nodes in the target resource pool, and in response to the inconsistent comparison result, adjust the number of service nodes in the target resource pool based on the target service node demand and the current number of service nodes. Through objective functions and statistical analysis, resources are rationally planned with the goal of minimizing the waiting time of tasks in queue and maximizing the utilization of computing resources. A suitable recommended setting for the number of service nodes in the resource pool is given to improve resource utilization efficiency and shorten the waiting time for requests for computing resources from submission to processing.

[0199] It should be noted that the resource planning method provided in the embodiment of the present application is generally executed by the server 805 , and accordingly, the resource planning device is generally arranged in the server 805 .

[0200] It should be understood that Figure 8 The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided according to the implementation requirements.

[0201] Reference below Fig. 9 , which shows a schematic diagram of the structure of a computer system 900 of a terminal device suitable for implementing an embodiment of the present application. Fig. 9 The terminal device shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0202] like Fig. 9As shown, the computer system 900 includes a central processing unit (CPU) 901, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 902 or a program loaded from a storage part 908 into a random access memory (RAM) 903. In the RAM 903, various programs and data required for the operation of the computer system 900 are also stored. The CPU 901, the ROM 902, and the RAM 903 are connected to each other via a bus 904. An input / output (I / O) interface 905 is also connected to the bus 904.

[0203] The following components are connected to the I / O interface 905: an input section 906 including a keyboard, a mouse, etc.; an output section 907 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 908 including a hard disk, etc.; and a communication section 909 including a network interface card such as a LAN card, a modem, etc. The communication section 909 performs communication processing via a network such as the Internet. A drive 910 is also connected to the I / O interface 905 as needed. A removable medium 911, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 910 as needed, so that a computer program read therefrom is installed into the storage section 908 as needed.

[0204] In particular, according to the embodiments disclosed in the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present application include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 909, and / or installed from the removable medium 911. When the computer program is executed by the central processing unit (CPU) 901, the above-mentioned functions defined in the system of the present application are executed.

[0205] It should be noted that the computer-readable medium shown in the present application may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. Computer-readable storage media may include, for example, but are not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or devices, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present application, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device. In the present application, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, in which a computer-readable program code is carried. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer-readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, optical cable, RF, etc., or any suitable combination of the above.

[0206] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present application. In this regard, each box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the above-mentioned module, program segment or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flow chart, and the combination of the boxes in the block diagram or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0207] The units involved in the embodiments described in the present application may be implemented by software or hardware. The units described may also be set in a processor, for example, it may be described as: a processor includes an acquisition unit, an objective function solving unit, a statistical analysis unit, a demand determination unit, and an adjustment unit. The names of these units do not, in some cases, constitute limitations on the units themselves.

[0208] As another aspect, the present application also provides a computer-readable medium, which may be included in the device described in the above embodiment; or it may exist separately and not be assembled into the device. The above computer-readable medium carries one or more programs, and when the above one or more programs are executed by a device, the device obtains the historical service node demand corresponding to the target resource pool and the target task type according to the received resource planning request; calls the time series prediction model to calculate the predicted service node demand for the future preset period based on the historical service node demand, updates the parameter value in the time series prediction index according to the predicted service node demand, and then solves the objective function containing the time series prediction index to obtain the optimal service node demand of the target resource pool; performs statistical analysis on the historical service node demand based on the heuristic algorithm to obtain the statistical service node demand; determines the target service node demand according to the optimal service node demand and the statistical service node demand; compares the target service node demand with the current number of service nodes in the target resource pool, and in response to the inconsistent comparison result, performs adjustment on the number of service nodes in the target resource pool based on the target service node demand and the current number of service nodes.

[0209] The computer program product of the present application includes a computer program, and when the computer program is executed by a processor, the resource planning method in the embodiment of the present application is implemented.

[0210] According to the technical solution of the embodiment of the present application, resources are rationally planned through objective functions and statistical analysis, with the goal of minimizing the queuing waiting time of tasks and maximizing the utilization of computing resources, and a suitable recommended setting for the number of resource pool service nodes is given, thereby improving resource utilization efficiency and shortening the queuing waiting time for requests for computing resources from submission to processing.

[0211] The above specific implementations do not constitute a limitation on the protection scope of this application. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions may occur depending on design requirements and other factors. Any modifications, equivalent substitutions and improvements made within the spirit and principles of this application should be included in the protection scope of this application.

Claims

1. A resource planning method, characterized in that: include: Obtain the historical service node demand corresponding to the target resource pool and the target task type according to the received resource planning request; Calling a time series prediction model to calculate the predicted service node demand for a preset future period based on the historical service node demand, updating the parameter value in the time series prediction index according to the predicted service node demand, and then solving the objective function including the time series prediction index to obtain the optimal service node demand of the target resource pool; Performing statistical analysis on the historical service node demands based on a heuristic algorithm to obtain statistical service node demands; Determining a target service node demand according to the optimal service node demand and the statistical service node demand; The target service node demand is compared with the current number of service nodes in the target resource pool, and in response to an inconsistent comparison result, the number of service nodes in the target resource pool is adjusted based on the target service node demand and the current number of service nodes.

2. The method according to claim 1, characterized in that The updating of the parameter value in the time series prediction index according to the predicted service node demand includes: Calculate the number of service resource requests per unit time according to the predicted service node demand; The parameter value of the field parameter corresponding to the number of service resource requests per unit time in the time series prediction index is updated to the calculated number of service resource requests per unit time.

3. The method according to claim 1, characterized in that The solving of the objective function including the timing prediction index to obtain the optimal service node demand of the target resource pool includes: Obtain the indicator calculation formula for each time series prediction indicator in the queuing system; Based on the calculation formulas of each of the indicators, a corresponding relationship is established between the number of service resource requests per unit time of the target resource pool, the number of service nodes released per unit time, the service node utilization rate, the average waiting time of service resource requests and the demand for service nodes; Based on the corresponding relationship, the timing prediction index in the objective function is converted into a form including parameters corresponding to the number of service resource requests per unit time, the number of service nodes released per unit time and the service node demand, and then the objective function is optimized under the conditions of maximizing the service node utilization and minimizing the average waiting time of the service resource request, so that the service node demand obtained by the solution is used as the optimal service node demand.

4. The method according to claim 1, characterized in that: The adjusting the number of service nodes in the target resource pool based on the target service node demand and the current number of service nodes includes: If the demand for the target service node is less than the current number of service nodes, the target node is determined based on the node priority when selecting nodes in the target resource pool corresponding to the type of the target resource pool, and the target node is selected from the target resource pool and added to the non-target resource pool.

5. The method according to claim 1, characterized in that The adjusting the number of service nodes in the target resource pool based on the target service node demand and the current number of service nodes includes: If the demand for the target service node is greater than the current number of service nodes, the target node is determined based on the node priority when selecting nodes in the non-target resource pool corresponding to the type of the target resource pool, and the target node is selected from the non-target resource pool and added to the target resource pool.

6. The method according to claim 1, characterized in that: The step of determining the target service node demand according to the optimal service node demand and the statistical service node demand includes: Performing weighted average calculation on the optimal service node demand and the statistical service node demand to obtain a weighted average value; The weighted average value is determined as the target service node demand.

7. The method according to claim 1, characterized in that: The step of determining the target service node demand according to the optimal service node demand and the statistical service node demand includes: Calculating the absolute value of the difference between the optimal service node demand and the statistical service node demand; If the absolute value of the difference is greater than a preset threshold, the statistical service node demand is determined as the target service node demand.

8. A resource planning device, characterized in that: include: An acquisition unit is configured to acquire the historical service node demand corresponding to the target resource pool and the target task type according to the received resource planning request; The objective function solving unit is configured to call the time series prediction model to calculate the predicted service node demand for a preset period of time in the future based on the historical service node demand, update the parameter value in the time series prediction index according to the predicted service node demand, and then solve the objective function including the time series prediction index to obtain the optimal service node demand of the target resource pool; A statistical analysis unit, configured to perform statistical analysis on the historical service node demand based on a heuristic algorithm to obtain a statistical service node demand; A demand determination unit, configured to determine a target service node demand based on the optimal service node demand and the statistical service node demand; The adjustment unit is configured to compare the target service node demand with the current number of service nodes in the target resource pool, and in response to an inconsistent comparison result, adjust the number of service nodes in the target resource pool based on the target service node demand and the current number of service nodes.

9. A resource planning electronic device, characterized in that: include: one or more processors; a storage device for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 7.

10. A computer readable medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

11. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images