Resource scheduling method and device, equipment, storage medium and program product

Abstract

The invention discloses a resource scheduling method and device, equipment, a storage medium and a program product, and the specific technical scheme comprises the steps: obtaining task data of a to-be-executed task, and the node utilization popularity and the node resource proportion of each node, the node resource proportion is a ratio of the number of used resources of each node to the total number of resources of each node; based on the task data, a preset algorithm is adopted for calculation to obtain a task popularity value, and the task popularity value represents the importance degree of the to-be-executed task; inputting the node utilization popularity, the node resource proportion and the task popularity value into a priority prediction model to obtain a priority parameter of a to-be-executed task output by the priority prediction model and a priority parameter of each node; and determining a target node corresponding to the to-be-executed task according to the priority parameter of the to-be-executed task and the priority parameter of each node. Therefore, accurate scheduling of computing power resources can be realized.

Description

Technical Field

[0001] The present application belongs to the field of cloud computing technology, and in particular, relates to a resource scheduling method, device, equipment, storage medium and program product. Background Art

[0002] With the development of cloud computing and the fifth-generation mobile communication technology (5G), computing resources are distributed in various nodes of the entire network. In different application scenarios, computing resources need to be scheduled to meet the computing power requirements in different application scenarios.

[0003] Currently, the resource utilization of each node can be determined through the traffic and memory of each node in the network, and then computing resources can be scheduled according to the resource utilization of each node. However, this method is prone to delays or errors and cannot achieve accurate scheduling of computing resources. Summary of the invention

[0004] The embodiments of the present application provide a resource scheduling method, apparatus, device, storage medium and program product, which can accurately implement the scheduling of computing resources.

[0005] In a first aspect, an embodiment of the present application provides a resource scheduling method, including:

[0006] Obtaining task data of tasks to be executed and node utilization heat and node resource ratio of each node, wherein the node utilization heat represents the number trend of tasks executed by the node, and the node resource ratio is based on the ratio of the number of used resources of each node to the total number of resources of each node;

[0007] Based on the task data, a task heat value is calculated using a preset algorithm, where the task heat value indicates the importance of the task to be performed;

[0008] Input the node utilization heat, the node resource ratio and the task heat value into a priority prediction model to obtain the priority parameters of the task to be executed and the priority parameters of each node output by the priority prediction model;

[0009] The target node corresponding to the task to be executed is determined according to the priority parameter of the task to be executed and the priority parameters of each node.

[0010] In a possible implementation, before acquiring the task data of the task to be executed and the node utilization heat and node resource proportion of each node, the method further includes:

[0011] Obtaining node data of each node within a first preset time period;

[0012] For each indicator data in the node data of each node, when the load corresponding to the indicator data is greater than or equal to a preset threshold, calculate the change trend of the indicator data within the first preset time period to obtain a heat factor;

[0013] Determine a heat value corresponding to the indicator data based on the heat factor, wherein the heat value represents the usage of the node within the first preset time period;

[0014] For the node data of each node, calculate the average value of the heat value corresponding to each indicator data to obtain the node utilization heat;

[0015] For each node in the node data, the ratio of the used resources of the node to the total resources of the node is calculated to obtain the resource ratio of the node.

[0016] In a possible implementation, calculating the change trend of the indicator data within a first preset time period to obtain a heat factor includes:

[0017] Calculate the mean of the indicator data at each time point within the first preset time period;

[0018] When the mean value is less than a preset threshold, determining that the heat value corresponding to the indicator data is a preset value;

[0019] When the mean is greater than or equal to the preset threshold, constructing a value pair based on the indicator data at each time point within the first preset time period;

[0020] Calculate the difference of the indicator data in each value pair to get the number of negative differences and the number of positive differences;

[0021] The heat factor is calculated based on the number of negative differences and the number of positive differences.

[0022] In a possible implementation, the calculating the heat factor based on the number of negative difference values ​​and the number of positive difference values ​​includes:

[0023] The heat factor is calculated according to the following formula:

[0024] S=2×f(min(X1,X2),n,0.5)

[0025] Among them, S represents the heat factor, X1 represents the number of negative differences, X2 represents the number of positive differences, n represents the number of indicator data within the first preset time length, 0.5 is the preset parameter value, and f represents the probability distribution function.

[0026] In a possible implementation, the task data includes predicted running time, predicted execution cost, running status, and task length; the task heat value is calculated based on the task data using a preset algorithm, including:

[0027] Determining the task state value of the task to be executed corresponding to the data according to the preset corresponding relationship between the running state and the state value;

[0028] The task heat value is calculated according to the following formula:

[0029]

[0030] Wherein, W3 represents the task heat value, α represents the task length, t represents the predicted running time, N is the task state assignment, and F is the predicted execution cost.

[0031] In a possible implementation, determining the target node corresponding to the task to be executed according to the priority parameter of the task to be executed and the priority parameter of each node includes:

[0032] Sorting the tasks to be executed according to their priority parameters from high to low to obtain a queue of tasks to be executed;

[0033] Sort the nodes in descending order according to their priority parameters to get a node queue;

[0034] The nodes whose priority order in the node queue is consistent with the priority order in the to-be-executed task queue are used as the target nodes of the to-be-executed tasks.

[0035] In a possible implementation, before inputting the node utilization heat, the node resource proportion and the task heat value into a priority prediction model to obtain the priority parameters of the to-be-executed task and the priority parameters of each node output by the priority prediction model, the method further includes:

[0036] Obtain historical task data corresponding to the historical tasks within the second preset time period and the historical node utilization heat and historical node resource ratio of each historical node;

[0037] Based on the historical task data, the preset algorithm is used to calculate the historical task heat value, where the historical task heat value indicates the importance of the historical task;

[0038] The historical node utilization heat, the historical node resource ratio and the historical task heat value are used as training samples, and the priority parameters of the historical tasks and the priority parameters of the nodes corresponding to the historical tasks are used as label data to train the neural network model to obtain the priority prediction model.

[0039] In a possible implementation, before taking the historical node utilization heat, the historical node resource ratio, and the historical task heat value as training samples, and taking the priority parameters of the historical tasks and the priority parameters of the nodes corresponding to the historical tasks as label data to train the neural network model to obtain the priority prediction model, the method further includes:

[0040] Using the preset priority parameter of the historical task as the priority parameter of the historical task;

[0041] Determine whether the running time of the historical task in the corresponding historical node is greater than a third preset time;

[0042] When the running time is less than the third preset time, setting the priority parameter of the historical task at the corresponding historical node to a maximum value;

[0043] When the running time is greater than or equal to the third preset time, an adjusted historical node is obtained, and the priority parameter of the adjusted historical node is set as the priority parameter of the historical task.

[0044] In a second aspect, an embodiment of the present application provides a resource scheduling device, including:

[0045] Obtaining task data of tasks to be executed and node utilization heat and node resource ratio of each node, wherein the node utilization heat represents the number trend of tasks executed by the node, and the node resource ratio is based on the ratio of the number of used resources of each node to the total number of resources of each node;

[0046] Based on the task data, a task heat value is calculated using a preset algorithm, where the task heat value indicates the importance of the task to be performed;

[0047] Input the node utilization heat, the node resource ratio and the task heat value into a priority prediction model to obtain the priority parameters of the task to be executed and the priority parameters of each node output by the priority prediction model;

[0048] The target node corresponding to the task to be executed is determined according to the priority parameter of the task to be executed and the priority parameters of each node.

[0049] In a third aspect, an embodiment of the present application provides an electronic device, the device comprising: a processor and a memory storing computer program instructions;

[0050] When the processor executes the computer program instructions, the method according to any one of the first aspects is implemented.

[0051] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, on which computer program instructions are stored. When the computer program instructions are executed by a processor, the method described in any one of the first aspects is implemented.

[0052] In a fifth aspect, an embodiment of the present application provides a computer program product, the computer product comprising a computer program, and when the computer program is executed by a processor, the method as described in any one of the first aspects is implemented.

[0053] The resource scheduling method, device, equipment, storage medium and program product provided by the embodiment of the present application obtain the task data of the task to be executed and the node utilization heat and node resource proportion of each node, wherein the node utilization heat represents the number trend of the tasks executed by the node, and the node utilization heat can represent the load change trend of the node. The node resource proportion can reflect the resource usage in each node. The task heat value of each task is calculated by a preset algorithm, wherein the task heat value can represent the importance of the task to be executed. Then the calculated node utilization heat, node resource proportion and task heat value are input into the priority prediction model to obtain the priority parameters of each task to be executed and the priority parameters of each node, and the target node corresponding to each task to be executed is determined according to the priority parameters of the task to be executed and the priority parameters of each node. Among them, the node resource proportion represents the resource utilization rate of the node, and the task heat value represents the importance of the task. In this way, according to the importance of each task to be executed, the resource utilization rate of each node and the change trend of the node load, the priority prediction model can accurately predict the priority parameters of the task to be executed and the priority parameters of each node. Among them, the higher the priority parameter of the task to be executed, the more important the task to be executed is. The priority parameter of each node indicates that the node has more computing power resources, thereby allocating more computing power to important tasks to be executed, realizing accurate scheduling of computing power resources. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] In order to more clearly illustrate the technical solution of the embodiments of the present application, the following is a brief introduction to the drawings required for use in the embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0055] Figure 1 It is a flowchart of a resource scheduling method provided by an embodiment of the present application;

[0056] Figure 2 It is a flowchart of a method for calculating node utilization heat and node resource ratio provided by an embodiment of the present application;

[0057] Figure 3It is a flowchart of a priority prediction model training method provided by an embodiment of the present application;

[0058] Figure 4 is an exemplary schematic diagram of a resource scheduling method provided by an embodiment of the present application;

[0059] Figure 5 It is a structural schematic diagram of a resource scheduling device provided by an embodiment of the present application;

[0060] Figure 6 It is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0061] The features and exemplary embodiments of various aspects of the present application will be described in detail below. In order to make the purpose, technical solutions and advantages of the present application clearer, the present application will be further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only intended to explain the present application, rather than to limit the present application. For those skilled in the art, the present application can be implemented without the need for some of these specific details. The following description of the embodiments is only to provide a better understanding of the present application by illustrating the examples of the present application.

[0062] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the statement "include..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0063] In order to solve the problems of the prior art, the embodiments of the present application provide a resource scheduling method, apparatus, device, storage medium and program product. The following first describes the resource scheduling method provided in the embodiments of the present application, which can be applied to any node in a software-defined network (SDN). Figure 1 As shown, the method includes:

[0064] S101, obtaining task data of tasks to be executed and node utilization heat and node resource ratio of each node.

[0065] Among them, the node utilization heat represents the number trend of tasks executed by the node, and the node resource ratio is based on the ratio of the number of used resources of each node to the total number of resources of each node.

[0066] In one example, task data refers to the task status, task length, predicted operation market, and predicted operation cost of the task to be executed. The task status can be running, suspended, and waiting. The resources of each node may include the central processing unit (CPU), disk, and memory of the node, and the predicted operation cost is the CPU, disk, and memory resources required for the node to run the task.

[0067] Specifically, the method for calculating the node utilization heat and the node resource ratio is described in detail in the subsequent embodiments.

[0068] S102: Based on the task data, a preset algorithm is used to calculate the task heat value.

[0069] Among them, the task heat value indicates the importance of the task to be executed.

[0070] S103, input the node utilization heat, node resource ratio and task heat value into the priority prediction model, and obtain the priority parameters of the to-be-executed tasks and the priority parameters of each node output by the priority prediction model.

[0071] Among them, the priority prediction model uses the historical task data of historical tasks, the utilization heat of historical nodes and the resource proportion of historical nodes as the input data of the neural network model, and trains the priority parameters of historical tasks and the priority parameters of historical nodes corresponding to historical tasks as label data.

[0072] It can be understood that the higher the priority parameter of the task to be executed, the more important the task to be executed is, and computing resources are scheduled for the task to be executed first; the higher the priority parameter of the node, the more sufficient the computing resources of the node are, and the node is used first to execute the task to be executed.

[0073] S104: Determine a target node corresponding to the task to be executed according to the priority parameter of the task to be executed and the priority parameters of each node.

[0074] By adopting the above method, the task data of the task to be executed and the node utilization heat and node resource proportion of each node are obtained, wherein the node utilization heat represents the number trend of the tasks executed by the node, and the node utilization heat can represent the load change trend of the node. The node resource proportion can reflect the resource usage in each node. The task heat value of each task is calculated by a preset algorithm, wherein the task heat value can represent the importance of the task to be executed. Then the calculated node utilization heat, node resource proportion and task heat value are input into the priority prediction model to obtain the priority parameters of each task to be executed and the priority parameters of each node, and the target node corresponding to each task to be executed is determined according to the priority parameters of the task to be executed and the priority parameters of each node. Among them, the node resource proportion represents the resource utilization rate of the node, and the task heat value represents the importance of the task. In this way, according to the importance of each task to be executed, the resource utilization rate of each node and the change trend of the node load, the priority prediction model can accurately predict the priority parameters of the task to be executed and the priority parameters of each node. Among them, the higher the priority parameter of the task to be executed, the more important the task to be executed is. The priority parameter of each node indicates that the node has more computing power resources, thereby allocating more computing power to important tasks to be executed, realizing accurate scheduling of computing power resources.

[0075] The following describes how to obtain node utilization and node resource ratio. Figure 2 As shown, before the above S101, obtaining the task data of the task to be executed and the node utilization heat and node resource proportion of each node, the method further includes S201 to S205:

[0076] S201: Obtain node data of each node within a first preset time period.

[0077] The first preset duration is set based on experience. In one example, the node data includes resource usage of indicators such as CPU, memory, and disk of the node.

[0078] S202. For each indicator data in the node data of each node, when the load corresponding to the indicator data is greater than or equal to a preset threshold, calculate the change trend of the indicator data within a first preset time period to obtain a heat factor.

[0079] Among them, the indicator data is the data corresponding to each indicator in the above-mentioned node data, for example, the indicator includes CPU usage and disk remaining space, etc. In the embodiment of the present application, the indicator data included in the node data is not limited. In actual implementation, the indicator data included in the node data can be set in advance according to the actual application scenario.

[0080] It should be noted that, when the load corresponding to the indicator data is greater than or equal to the preset threshold, it means that the indicator data of the node occupies more resources. Conversely, when the load corresponding to the indicator data is less than the preset threshold, it means that the indicator data of the node occupies fewer resources. When the indicator data of the node occupies fewer resources, the indicator data does not have a high reference value. Therefore, the heat value corresponding to the indicator data can be set to the preset value.

[0081] Specifically, the above S202, for each indicator data in the node data of each node, when the load corresponding to the indicator data is greater than or equal to the preset threshold, calculate the change trend of the indicator data within the first preset time length to obtain the heat factor, which can be implemented as follows:

[0082] Step 1: Calculate the mean of the indicator data at each time point within the first preset time period.

[0083] The node data is obtained by taking the current time as the starting time and acquiring the node data within the first preset time period from the starting time. The average value of the indicator data at each time point within the first preset time period is the load of the indicator data.

[0084] Step 2: When the mean value is less than a preset threshold, determine that the heat value corresponding to the indicator data is a preset value.

[0085] The heat value corresponding to the indicator data is used to indicate the usage of the indicator data within a first preset time period.

[0086] Step 3: When the mean is greater than or equal to the preset threshold, construct a value pair based on the indicator data at each time point within the first preset time period.

[0087] Specifically, the method for constructing the value pair is related to the amount of indicator data within the first preset time period. There are two methods for determining the value pair according to the amount of indicator data:

[0088] Case 1: When the amount of indicator data is an odd number, assuming the amount of data is n, the number of value pairs that can be constructed is (n+1) / 2.

[0089] Case 2: When the amount of indicator data is an even number, assuming the amount of data is n, the number of value pairs that can be constructed is n / 2.

[0090] It can be understood that when n is an even number, k logarithmic value pairs can be constructed; when n is an odd number, k-1 logarithmic value pairs can be constructed.

[0091] Based on the above determination of the number of constructed value pairs, in the data set constructed using the indicator data within the first preset time length, starting from the first data, the calculated k value is used as the preset step length, and the k+1th data is selected at an interval of k data. The k+1th data and the first data are used as a value pair, the k+2th data and the second data are used as a value pair, and so on, until k value pairs are obtained.

[0092] Step 4: Calculate the difference of the indicator data in each value pair to obtain the number of negative differences and the number of positive differences.

[0093] Among them, the number of negative differences is equal to the number of value pairs in which the difference of indicator data in the value pair is negative; the number of positive differences is equal to the number of value pairs in which the difference of indicator data in the value pair is positive.

[0094] Step 5: Calculate the heat factor based on the number of negative differences and the number of positive differences.

[0095] The heat factor can be calculated according to the following formula:

[0096] S=2×f(min(X1,X2),n,0.5)

[0097] Among them, S represents the heat factor, X1 represents the number of negative differences, X2 represents the number of positive differences, n represents the number of indicator data within the first preset time length, 0.5 is the preset parameter value, and f represents the probability distribution function.

[0098] It can be understood that since the numerical pair includes data from two different time points in the data set constructed by the indicator data within the first preset time length, the difference between the two indicator data in the numerical pair represents the difference between the two different time points. If the difference is greater than 0, it indicates that the indicator data is in an upward trend. If the difference is less than 0, it indicates that the indicator data is in a downward trend. Therefore, the changing trend of the indicator data within the first preset time length can be accurately judged by the number of negative differences and the number of positive differences.

[0099] S203. Determine a heat value corresponding to the indicator data based on the heat factor.

[0100] The heat value indicates the usage of the node within the first preset time period.

[0101] In an example, as shown in the following formula, the heat value of the indicator data is determined according to the number of negative difference values, the number of positive difference values, and the heat factor:

[0102]

[0103] Among them, when X1 is greater than X2 and X1 is greater than S, the heat value of the indicator data is the first preset value; when X1 is less than X2 and X1 is less than S, the heat value of the indicator data is the second preset value.

[0104] The first preset value may be 2, and the second preset value may be 1. The first preset value and the second preset value may be set according to actual business requirements, and the embodiment of the present application does not limit the values ​​of the first preset value and the second preset value.

[0105] S204. For the node data of each node, calculate the average value of the heat value corresponding to each indicator data to obtain the node utilization heat.

[0106] Among them, after calculating the average value of the heat value corresponding to each indicator data, the average value of the heat value corresponding to each indicator data is summed up to obtain the node utilization heat.

[0107] S205 . For each node data, calculate the ratio of the used resources of the node to the total resources of the node to obtain the node resource ratio.

[0108] Among them, for each indicator data, the ratio of the amount of used resources corresponding to the indicator data to the total number of resources of the node is calculated to obtain the indicator data resource ratio, and then the resource ratio of each indicator data is summed to obtain the node resource ratio.

[0109] Specifically, the node resource ratio is calculated according to the following formula:

[0110]

[0111] Among them, P2 represents the node resource ratio, J is equal to the number of indicator data, and w j is the preset resource weight coefficient corresponding to each indicator data, n j is the number of used resources corresponding to the indicator data, N j The total number of resources corresponding to the indicator data.

[0112] By using the method provided in the embodiment of the present application, the heat value corresponding to each indicator data calculated based on the node data of each node within the first preset time period can reflect the usage of the node within the first preset time period, and the change trend of the indicator data within the first preset time period can be obtained based on the heat factor. The accuracy of the node priority predicted by the subsequent node utilization heat and node resource ratio is improved.

[0113] In some embodiments of the present application, the task data includes predicted running time, predicted execution cost, running status, and task length; for the above S102, based on the task data, a preset algorithm is used to calculate the task heat value, which indicates the importance of the task to be executed, and can be implemented as follows:

[0114] The task state assignment of the task to be executed is determined according to the preset correspondence between the running state and the state assignment.

[0115] In an example, when the running state is in the suspended state, the state value may be 6, when the running state is in the running state, the state value may be 10, and when the running state is in the waiting state, the state value may be 3.

[0116] The task heat value is calculated according to the following formula:

[0117]

[0118] Among them, W3 represents the task heat value, α represents the task length, t represents the predicted running time, N represents the task state assignment, F represents the predicted execution cost, and i represents the number of indicators used to run the task. The above formula for calculating the task heat value is the preset algorithm.

[0119] Based on the node utilization heat, node resource ratio and task heat value obtained by the above calculation, the priority parameter of the node and the priority parameter of the task to be executed can be obtained by using the priority prediction model, and then the execution node of each task to be executed is matched based on the priority parameter of the node and the priority parameter of the task to be executed. Based on this, the above S104, determining the target node corresponding to the task to be executed according to the priority parameter of the task to be executed and the priority parameter of each node, can be specifically implemented as follows:

[0120] The priority parameters of the tasks to be executed are sorted from high to low to obtain a queue of tasks to be executed. The priority parameters of each node are sorted from high to low to obtain a node queue. The nodes whose priority order in the node queue is consistent with that in the queue of tasks to be executed are taken as the target nodes of the tasks to be executed.

[0121] It should be noted that the number of priority parameters of the tasks to be executed in the output result of the priority prediction model can be set to be consistent with the number of priority parameters of the nodes, so the number of nodes in the node queue is consistent with the number of tasks to be executed in the task queue. According to the priority parameter of each task to be executed, the target node consistent with the priority parameter of the task to be executed can be determined in the node queue, and a one-to-one correspondence between the tasks to be executed and the nodes can be established.

[0122] Using the method provided in the embodiment of the present application, the tasks to be executed and the nodes are sorted according to the size of the priority parameter, and then the target queue corresponding to each task to be executed is determined according to the sorting of the nodes in the node queue. Among them, the priority parameter of the task to be executed is used to indicate the importance of the task to be executed. The higher the priority parameter, the more important the task to be executed is, and the computing power resources need to be scheduled for the task to be executed first. The priority parameter of the node is used to indicate whether the computing power resources of the node are sufficient. The higher the priority, the more abundant the computing power resources of the node. Therefore, by determining the target node of the task to be executed according to the priority parameter, more computing power resources can be scheduled for important tasks to be executed, thereby realizing accurate scheduling of computing power resources.

[0123] In some embodiments of the present application, before the priority parameters of the task to be executed and the priority parameters of the node are predicted by the priority prediction model, the neural network model is trained using the historical task data and the historical node data of the historical nodes of the historical tasks to obtain the priority prediction model. Figure 3 As shown, before the above S103, inputting the node utilization heat, node resource proportion and task heat value into the priority prediction model to obtain the priority parameters of the to-be-executed tasks and the priority parameters of each node output by the priority prediction model, the method further includes:

[0124] S301, obtaining historical task data corresponding to historical tasks within a second preset time period and historical node utilization heat and historical node resource proportion of each historical node.

[0125] Among them, the second preset duration can be the same as the first preset duration, and can also be different from the first preset duration. The embodiment of the present application does not specifically limit the relationship between the second preset duration and the first preset duration.

[0126] S302: Based on the historical task data, a preset algorithm is used to calculate the historical task heat value.

[0127] The historical task heat value indicates the importance of the historical task. The preset algorithm refers to the relevant description in the above embodiment, which will not be repeated here.

[0128] S303: Use the historical node utilization heat, historical node resource ratio and historical task heat value as training samples, and use the priority parameters of the historical tasks and the priority parameters of the nodes corresponding to the historical tasks as label data to train the neural network model to obtain a priority prediction model.

[0129] By adopting the method provided in the embodiment of the present application, historical task data, historical node utilization heat of each historical node and historical node resource ratio are used as training samples, and the priority parameters of historical tasks and the priority parameters of nodes corresponding to historical tasks are used as label data to train the priority prediction model. The task priority parameters and the node priority parameters can be accurately predicted based on the task data, node utilization heat and node resource ratio, thereby ensuring the accuracy of the prediction results of the priority prediction model.

[0130] It should be noted that the execution node corresponding to each historical task may not be the optimal execution node for the historical task. In order to ensure the accuracy of the output results of the trained priority prediction model. Determine the optimal execution node corresponding to each historical task. Based on this, in the above S303, the utilization heat of the historical node, the resource proportion of the historical node and the heat value of the historical task are used as training samples, and the priority parameters of the historical task and the priority parameters of the node corresponding to the historical task are used as label data to train the neural network model. Before obtaining the priority prediction model, the method also includes:

[0131] Step A: Using the preset priority parameter of the historical task as the priority parameter of the historical task.

[0132] Among them, the preset priority parameters of historical tasks are obtained by manual annotation.

[0133] Step B: determine whether the running time of the historical task in the corresponding historical node is greater than a third preset time.

[0134] Among them, the third preset time length is set based on experience.

[0135] Step C: when the running time is less than the third preset time, the priority parameter of the historical task at the corresponding historical node is set to the maximum value.

[0136] It should be noted that when the running time is less than the third preset time, it means that the computing power resources of the historical node are sufficient to support the historical node to quickly execute the historical task, that is, the historical node is the optimal execution node for the historical task.

[0137] The embodiment of the present application does not limit the execution order of the above steps B and C.

[0138] Step D: when the running time is greater than or equal to the third preset time, obtain the adjusted historical node, and set the priority parameter of the adjusted historical node as the priority parameter of the historical task.

[0139] It should be noted that, when the running time is greater than the third preset time, it means that the running time of the historical task on the historical node is too long, and the computing power resources of the historical node are insufficient to support the historical node to quickly execute the historical task, that is, the historical node is not the optimal execution node for the historical task. The execution node corresponding to the historical task can be adjusted, and then the priority parameter of the adjusted historical node is set to be consistent with the priority parameter of the historical task.

[0140] By adopting the method provided in the embodiment of the present application, the preset priority parameter of the historical task is used as the priority parameter of the historical task, and then the historical node is judged to be the optimal execution node according to the running time of the historical task in the corresponding historical node. If the historical node is not the optimal execution node, the adjusted historical node is obtained, and then the priority parameter of the adjusted historical node is set as the priority parameter of the historical task. In this way, each historical node corresponding to each historical task can be obtained, and the priority parameter of each historical task is the same as the priority parameter of the corresponding historical node, which is convenient for the subsequent training of the neural network model and improves the accuracy of the output result of the priority prediction model.

[0141] The following combination Figure 4 The resource scheduling method provided in the embodiment of the present application is introduced, such as Figure 4 As shown, the method includes:

[0142] S401, collecting real-time operation data and current task data of each node.

[0143] Among them, the real-time operation data is the node data within the above-mentioned first preset time period, and the current task data is the task data of the above-mentioned task to be executed.

[0144] S402: Calculate the node utilization and node resource ratio based on the real-time operation data of the node.

[0145] Among them, the method of calculating the node utilization heat and the node resource ratio refers to the relevant description in the above embodiment, which will not be repeated here.

[0146] S403: Calculate the task heat value according to the current task data.

[0147] Among them, the method of calculating the task heat value refers to the relevant description in the above embodiment, which will not be repeated here.

[0148] S404: Input the node utilization heat, node resource ratio and task heat value into the trained model, and output the task priority parameter and node priority parameter.

[0149] Among them, the task priority parameter is the priority parameter of the above-mentioned task to be executed, the node priority parameter is the priority parameter of the above-mentioned node, and the model is the above-mentioned priority prediction model.

[0150] S405 . Allocate the task to the corresponding node based on the obtained task priority parameter and node priority parameter.

[0151] Among them, the method of allocating tasks to corresponding nodes is the above-mentioned method of determining the target node corresponding to the task to be executed. For specific content, please refer to the relevant description in the above-mentioned embodiment, which will not be repeated here.

[0152] In this way, based on the calculated node utilization heat, node resource ratio and task heat value, the node priority parameter and task priority parameter can be accurately predicted through the pre-trained priority prediction model, where the node priority parameter can reflect whether the node's computing power resources are sufficient, and the task priority parameter can reflect the importance of the task. Then, according to the node priority parameter and task priority parameter, nodes with sufficient computing power resources are allocated to important tasks, thereby realizing the precise scheduling of computing power resources.

[0153] Based on the same concept, the embodiment of the present application provides a resource scheduling device, such as Figure 5 As shown, the device comprises:

[0154] The acquisition module 501 is used to acquire the task data of the task to be executed and the node utilization heat and node resource ratio of each node. The node utilization heat represents the number trend of the tasks executed by the node, and the node resource ratio is based on the ratio of the number of used resources of each node to the total number of resources of each node;

[0155] A calculation module 502 is used to calculate a task heat value based on the task data using a preset algorithm, where the task heat value indicates the importance of the task to be executed;

[0156] The prediction module 503 is used to input the node utilization heat, node resource ratio and task heat value into the priority prediction model to obtain the priority parameters of the to-be-executed tasks and the priority parameters of each node output by the priority prediction model;

[0157] The determination module 504 is used to determine the target node corresponding to the task to be executed according to the priority parameter of the task to be executed and the priority parameters of each node.

[0158] In a possible implementation, the acquisition module 501 is further configured to acquire node data of each node within a first preset time period;

[0159] The calculation module 502 is further used to calculate the change trend of each indicator data in the node data of each node within a first preset time period when the load corresponding to the indicator data is greater than or equal to a preset threshold, and obtain a heat factor;

[0160] The determination module 504 is further configured to determine a heat value corresponding to the indicator data based on the heat factor, where the heat value represents the usage of the node within a first preset time period;

[0161] The calculation module 502 is further used to calculate the average value of the heat value corresponding to each indicator data for the node data of each node, and obtain the node utilization heat;

[0162] The calculation module 502 is further used to calculate the ratio of the used resource quantity of the node to the total resource quantity of the node in the node data of each node, so as to obtain the node resource ratio.

[0163] In a possible implementation, the calculation module 502 is specifically configured to:

[0164] Calculate the mean of the indicator data at each time point within the first preset time period;

[0165] When the mean value is less than the preset threshold, the heat value corresponding to the indicator data is determined to be the preset value;

[0166] When the mean is greater than or equal to the preset threshold, constructing a value pair based on the indicator data at each time point within the first preset time period;

[0167] Calculate the difference of the indicator data in each value pair to get the number of negative differences and the number of positive differences;

[0168] Based on the number of negative differences and the number of positive differences, a heat factor is calculated.

[0169] In a possible implementation, the calculation module 502 is specifically configured to:

[0170] The heat factor is calculated according to the following formula:

[0171] S=2×f(min(X1,X2),n,0.5)

[0172] Among them, S represents the heat factor, X1 represents the number of negative differences, X2 represents the number of positive differences, n represents the number of indicator data within the first preset time length, 0.5 is the preset parameter value, and f represents the probability distribution function.

[0173] In a possible implementation, the task data includes predicted running time, predicted execution cost, running status, and task length; the calculation module 502 is specifically used to:

[0174] Determine the task state value corresponding to the task to be executed according to the preset correspondence relationship between the running state and the state value;

[0175] The task heat value is calculated according to the following formula:

[0176]

[0177] Among them, W3 represents the task heat value, α represents the task length, t represents the predicted running time, N is the task state assignment, and F is the predicted execution cost.

[0178] In a possible implementation, the determination module 504 is specifically configured to:

[0179] Sort the tasks to be executed in descending order according to their priority parameters to obtain a queue of tasks to be executed;

[0180] Sort the nodes in descending order according to their priority parameters to get a node queue;

[0181] The nodes whose priority order in the node queue is consistent with that in the task queue to be executed are taken as the target nodes of the task to be executed.

[0182] In a possible implementation, the device further includes:

[0183] The acquisition module 501 is further used to acquire the historical task data corresponding to the historical task within the second preset time period and the historical node utilization heat and historical node resource proportion of each historical node;

[0184] The calculation module 502 is further used to calculate the historical task heat value based on the historical task data using a preset algorithm, where the historical task heat value indicates the importance of the historical task;

[0185] The training module is used to use the historical node utilization heat, historical node resource ratio and historical task heat value as training samples, and use the priority parameters of the historical tasks and the priority parameters of the nodes corresponding to the historical tasks as label data to train the neural network model to obtain a priority prediction model.

[0186] In a possible implementation, the device further includes:

[0187] A determination module 504, configured to use the preset priority parameter of the historical task as the priority parameter of the historical task;

[0188] A judgment module, used to judge whether the running time of the historical task in the corresponding historical node is greater than a third preset time;

[0189] A setting module, used for setting the priority parameter of the historical task at the corresponding historical node to a maximum value when the running time is less than a third preset time;

[0190] The setting module is further used to obtain the adjusted historical node through the acquisition module 501 when the running time is greater than or equal to the third preset time, and set the priority parameter of the adjusted historical node as the priority parameter of the historical task.

[0191] It should be noted that the resource scheduling device 500 is a device corresponding to the above-mentioned resource scheduling method, and all implementation methods in the above-mentioned method embodiment are applicable to the embodiment of the device and can achieve the same technical effect.

[0192] Figure 6 A schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present application is shown.

[0193] The terminal device may include a processor 601 and a memory 602 storing computer program instructions.

[0194] Specifically, the processor 601 may include a central processing unit (CPU), or an application specific integrated circuit (ASIC), or may be configured to implement one or more integrated circuits of the embodiments of the present application.

[0195] The memory 602 may include a large capacity memory for data or instructions. By way of example and not limitation, the memory 602 may include a hard disk drive (HDD), a floppy disk drive, a flash memory, an optical disk, a magneto-optical disk, a magnetic tape, or a universal serial bus (USB) drive or a combination of two or more of these. In appropriate cases, the memory 602 may include a removable or non-removable (or fixed) medium. In appropriate cases, the memory 602 may be inside or outside the integrated gateway disaster recovery device. In a specific embodiment, the memory 602 is a non-volatile solid-state memory.

[0196] In certain embodiments, the memory 602 may include a read-only memory (ROM), a random access memory (RAM), a magnetic disk storage medium device, an optical storage medium device, a flash memory device, an electrical, optical or other physical / tangible memory storage device. Thus, in general, the memory includes one or more tangible (non-transitory) computer-readable storage media (e.g., a memory device) encoded with software including computer-executable instructions, and when the software is executed (e.g., by one or more processors), it is operable to perform the operations described with reference to the method according to an aspect of the present disclosure.

[0197] The processor 601 implements any one of the resource scheduling methods in the above embodiments by reading and executing computer program instructions stored in the memory 602 .

[0198] In one example, the electronic device may further include a communication interface 603 and a bus 604. Figure 6 As shown, the processor 601, the memory 602, and the communication interface 603 are connected via a bus 604 and communicate with each other.

[0199] The communication interface 603 is mainly used to implement communication between various modules, devices, units and / or equipment in the embodiments of the present application.

[0200] Bus 604 includes hardware, software or both, and the parts of online data flow billing equipment are coupled to each other.For example, but not limitation, bus may include accelerated graphics port (AGP) or other graphics bus, enhanced industrial standard architecture (EISA) bus, front side bus (FSB), hypertransport (HT) interconnection, industrial standard architecture (ISA) bus, infinite bandwidth interconnection, low pin count (LPC) bus, memory bus, micro channel architecture (MCA) bus, peripheral component interconnection (PCI) bus, PCI-Express (PCI-X) bus, serial advanced technology attachment (SATA) bus, video electronics standard association local (VLB) bus or other suitable bus or two or more of these combinations. In appropriate cases, bus 710 may include one or more buses. Although the present application embodiment describes and shows a specific bus, the application considers any suitable bus or interconnection.

[0201] In addition, in combination with the resource scheduling method in the above embodiment, the embodiment of the present application can provide a computer storage medium for implementation. The computer storage medium stores computer program instructions; when the computer program instructions are executed by a processor, any one of the resource scheduling methods in the above embodiment is implemented.

[0202] In combination with the resource scheduling method in the above-mentioned embodiment, an embodiment of the present application may provide a computer program product, where the computer product includes a computer program, and when the computer program is executed by a processor, any one of the resource scheduling methods in the embodiment is implemented.

[0203] It should be clear that the present application is not limited to the specific configuration and processing described above and shown in the figures. For the sake of simplicity, a detailed description of the known method is omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of the present application is not limited to the specific steps described and shown, and those skilled in the art can make various changes, modifications and additions, or change the order between the steps after understanding the spirit of the present application.

[0204] The functional blocks shown in the above block diagram can be implemented as hardware, software, firmware or a combination thereof. When implemented in hardware, it can be, for example, an electronic circuit, an application specific integrated circuit (ASIC), appropriate firmware, a plug-in, a function card, etc. When implemented in software, the elements of the present application are programs or code segments that are used to perform the required tasks. The program or code segment can be stored in a machine-readable medium, or transmitted on a transmission medium or a communication link by a data signal carried in a carrier wave. "Machine-readable medium" can include any medium capable of storing or transmitting information. Examples of machine-readable media include electronic circuits, semiconductor memory devices, ROM, flash memory, erasable ROM (EROM), floppy disks, CD-ROMs, optical disks, hard disks, optical fiber media, radio frequency (RF) links, etc. The code segment can be downloaded via a computer network such as the Internet, an intranet, etc.

[0205] It should also be noted that the exemplary embodiments mentioned in this application describe some methods or systems based on a series of steps or devices. However, this application is not limited to the order of the above steps, that is, the steps can be performed in the order mentioned in the embodiment, or in a different order from the embodiment, or several steps can be performed simultaneously.

[0206] Aspects of the present disclosure are described above with reference to the flowchart and / or block diagram of the method, device (system) and computer program product according to the embodiment of the present disclosure. It should be understood that each box in the flowchart and / or block diagram and the combination of each box in the flowchart and / or block diagram can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device to produce a machine so that these instructions executed by the processor of the computer or other programmable data processing device enable the implementation of the function / action specified in one or more boxes of the flowchart and / or block diagram. Such a processor can be, but is not limited to, a general-purpose processor, a special-purpose processor, a special application processor, or a field programmable logic circuit. It can also be understood that each box in the block diagram and / or flowchart and the combination of boxes in the block diagram and / or flowchart can also be implemented by dedicated hardware that performs a specified function or action, or can be implemented by a combination of dedicated hardware and computer instructions.

[0207] The above are only specific implementation methods of the present application. Those skilled in the art can clearly understand that for the convenience and simplicity of description, the specific working processes of the systems, modules and units described above can refer to the corresponding processes in the aforementioned method embodiments, and will not be repeated here. It should be understood that the protection scope of the present application is not limited to this. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed in this application, and these modifications or replacements should be included in the protection scope of this application.

Claims

1. A resource scheduling method, characterized in that: include: Obtaining task data of tasks to be executed and node utilization heat and node resource ratio of each node, wherein the node utilization heat represents the number trend of tasks executed by the node, and the node resource ratio is based on the ratio of the number of used resources of each node to the total number of resources of each node; Based on the task data, a task heat value is calculated using a preset algorithm, where the task heat value indicates the importance of the task to be performed; Input the node utilization heat, the node resource ratio and the task heat value into a priority prediction model to obtain the priority parameters of the task to be executed and the priority parameters of each node output by the priority prediction model; The target node corresponding to the task to be executed is determined according to the priority parameter of the task to be executed and the priority parameters of each node.

2. The method according to claim 1, characterized in that Before obtaining the task data of the task to be executed and the node utilization heat and node resource proportion of each node, the method further includes: Obtaining node data of each node within a first preset time period; For each indicator data in the node data of each node, when the load corresponding to the indicator data is greater than or equal to a preset threshold, calculate the change trend of the indicator data within the first preset time period to obtain a heat factor; Determine a heat value corresponding to the indicator data based on the heat factor, wherein the heat value represents the usage of the node within the first preset time period; For the node data of each node, calculate the average value of the heat value corresponding to each indicator data to obtain the node utilization heat; For each node in the node data, the ratio of the used resources of the node to the total resources of the node is calculated to obtain the resource ratio of the node.

3. The method according to claim 2, characterized in that The calculating the change trend of the indicator data within the first preset time period to obtain the heat factor includes: Calculate the mean of the indicator data at each time point within the first preset time period; When the mean value is less than a preset threshold, determining that the heat value corresponding to the indicator data is a preset value; When the mean is greater than or equal to the preset threshold, constructing a value pair based on the indicator data at each time point within the first preset time period; Calculate the difference of the indicator data in each value pair to get the number of negative differences and the number of positive differences; The heat factor is calculated based on the number of negative differences and the number of positive differences.

4. The method according to claim 3, characterized in that: The step of calculating the heat factor based on the number of negative difference values ​​and the number of positive difference values ​​includes: The heat factor is calculated according to the following formula: S=2×f(min(X1,X2),n,0.5) Among them, S represents the heat factor, X1 represents the number of negative differences, X2 represents the number of positive differences, n represents the number of indicator data within the first preset time length, 0.5 is the preset parameter value, and f represents the probability distribution function.

5. The method according to claim 1, characterized in that The task data includes predicted running time, predicted execution cost, running status, and task length; the task heat value is calculated based on the task data using a preset algorithm, including: Determine the task state value corresponding to the task to be executed according to the preset corresponding relationship between the running state and the state value; The task heat value is calculated according to the following formula: Wherein, W3 represents the task heat value, α represents the task length, t represents the predicted running time, N is the task state assignment, and F is the predicted execution cost.

6. The method according to claim 1, characterized in that The determining the target node corresponding to the task to be executed according to the priority parameter of the task to be executed and the priority parameter of each node includes: Sorting the tasks to be executed according to their priority parameters from high to low to obtain a queue of tasks to be executed; Sort the nodes in descending order according to their priority parameters to get a node queue; The nodes whose priority order in the node queue is consistent with that in the to-be-executed task queue are used as the target nodes of the to-be-executed tasks.

7. The method according to claim 1, characterized in that Before inputting the node utilization heat, the node resource proportion and the task heat value into the priority prediction model to obtain the priority parameters of the to-be-executed task and the priority parameters of each node output by the priority prediction model, the method further includes: Obtain historical task data corresponding to the historical tasks within the second preset time period and the historical node utilization heat and historical node resource ratio of each historical node; Based on the historical task data, the preset algorithm is used to calculate the historical task heat value, where the historical task heat value indicates the importance of the historical task; The historical node utilization heat, the historical node resource ratio and the historical task heat value are used as training samples, and the priority parameters of the historical tasks and the priority parameters of the nodes corresponding to the historical tasks are used as label data to train the neural network model to obtain the priority prediction model.

8. The method according to claim 7, characterized in that Before using the historical node utilization heat, the historical node resource ratio and the historical task heat value as training samples, and using the priority parameters of the historical tasks and the priority parameters of the nodes corresponding to the historical tasks as label data to train the neural network model to obtain the priority prediction model, the method further includes: Using the preset priority parameter of the historical task as the priority parameter of the historical task; Determine whether the running time of the historical task in the corresponding historical node is greater than a third preset time; When the running time is less than the third preset time, setting the priority parameter of the historical task at the corresponding historical node to a maximum value; When the running time is greater than or equal to the third preset time, an adjusted historical node is obtained, and the priority parameter of the adjusted historical node is set as the priority parameter of the historical task.

9. A resource scheduling device, characterized in that: include: An acquisition module is used to acquire task data of tasks to be executed and node utilization heat and node resource ratio of each node, wherein the node utilization heat represents the number trend of tasks executed by the node, and the node resource ratio is based on the ratio of the number of used resources of each node to the total number of resources of each node; A calculation module, configured to calculate a task heat value based on the task data using a preset algorithm, wherein the task heat value indicates the importance of the task to be executed; A prediction module, used to input the node utilization heat, the node resource proportion and the task heat value into a priority prediction model, and obtain the priority parameters of the to-be-executed task and the priority parameters of each node output by the priority prediction model; The determination module is used to determine the target node corresponding to the task to be executed according to the priority parameter of the task to be executed and the priority parameter of each node.

10. A terminal device, characterized in that: The device comprises: a processor and a memory storing computer program instructions; When the processor executes the computer program instructions, the method according to any one of claims 1 to 8 is implemented.

11. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer program instructions, and when the computer program instructions are executed by a processor, the method according to any one of claims 1 to 8 is implemented.

12. A computer program product, characterized in that The computer product comprises a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 8 is implemented.

Images