Resource dynamic scheduling method, device and equipment and computer readable storage medium

Abstract

The application relates to the field of cloud computing, and discloses a resource dynamic scheduling method, device and equipment and a computer readable storage medium. When the number of resource scheduling tasks of a user is greater than the number of schedulable computing nodes, the resource scheduling tasks are batched according to the number of schedulable computing nodes, a simulated annealing algorithm is used to take the batched resource scheduling tasks into schedulable computing nodes for scheduling, and the global resource utilization of each schedulable computing node in the batched resource scheduling tasks is optimized to the lowest, so that the global optimization of the overall scheduling of the cluster is realized, the scheduling difficulty is reduced, the scheduling efficiency is improved, the problem of unbalanced load in current cloud computing is solved, and on this basis, the evaluation mode of the optimization target can be configured to solve the multi-objective optimization problem in scheduling, and the automatic management requirement of the current cloud computing cluster is met.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of cloud computing, and particularly relates to a resource dynamic scheduling method and device, equipment and a computer readable storage medium. BACKGROUND

[0002] With the expansion of the Internet, the massive data generated by servers promotes the rapid development of cloud computing. Cloud computing refers to the unified management and scheduling of a large number of computing resources connected by a network to form a computing resource pool to provide services to users. One of the goals of cloud computing is to achieve efficient use of resources, and resource scheduling in a cloud computing environment is a critical issue that needs to consider the workload, frequency and scale of resource allocation and task scheduling. The resource scheduling problem is to allocate a certain number of tasks to appropriate nodes for execution to achieve the highest total execution efficiency, while also considering the load balancing of cluster nodes.

[0003] Traditional scheduling algorithms often optimize single resource scheduling tasks, which is difficult to achieve optimal scheduling of clusters, cannot consider the load of cluster nodes from a global perspective, and cannot meet the multi-objective scheduling requirements of clusters. The scheduling effect is not good, and the optimal scheduling of the cluster cannot be achieved. Moreover, this way of scheduling resources one by one has low scheduling efficiency and cannot adapt to the current resource scheduling requirements of large-scale clusters.

[0004] It is a technical problem for those skilled in the art to provide a more optimized resource scheduling scheme. SUMMARY

[0005] The purpose of the present application is to provide a resource dynamic scheduling method, device, equipment and computer readable storage medium, which realizes global optimization of cluster overall scheduling and improves scheduling efficiency.

[0006] To solve the above technical problems, the present application provides a resource dynamic scheduling method, comprising:

[0007] When the number of resource scheduling tasks of a user is greater than the number of schedulable computing nodes, the resource scheduling tasks are batched so that the number of resource scheduling tasks in each batch is not greater than the number of schedulable computing nodes;

[0008] From the first batch of resource scheduling tasks, the global resource utilization of each schedulable computing node after the current batch of resource scheduling tasks is put into the schedulable computing node scheduling is taken as the optimization goal, and a simulated annealing algorithm is applied to obtain the scheduling strategy of the current batch, until the scheduling strategy of each batch is obtained;

[0009] According to the scheduling strategy of each batch, the scheduling of each resource scheduling task is performed;

[0010] In one of the scheduling strategies, each of the schedulable computing nodes is allocated at most one of the resource scheduling tasks.

[0011] Optionally, the resource scheduling tasks are batched such that the number of the resource scheduling tasks in each batch is not greater than the number of the schedulable computing nodes, specifically including:

[0012] According to the parameter information of each of the resource scheduling tasks, a dependency relationship between the resource scheduling tasks is analyzed and obtained, and each of the resource scheduling tasks is sorted according to a priority;

[0013] According to the sorting of each of the resource scheduling tasks, while ensuring that the resource scheduling tasks having a dependency relationship are allocated to different batches and the number of the resource scheduling tasks in each batch is not greater than the number of the schedulable computing nodes, each of the resource scheduling tasks is batched.

[0014] Optionally, the resource scheduling tasks are batched such that the number of the resource scheduling tasks in each batch is not greater than the number of the schedulable computing nodes, specifically including:

[0015] The remainder obtained by dividing the number of the resource scheduling tasks by the number of the schedulable computing nodes is the number of the resource scheduling tasks in the last batch, and the number of the resource scheduling tasks in each of the other batches is equal to the number of the schedulable computing nodes.

[0016] Optionally, the objective function of the simulated annealing algorithm is:

[0017] ;

[0018] wherein,

[0019] ;

[0020] ;

[0021] ;

[0022] ;

[0023] ;

[0024] ;

[0025] is the objective function, is an evaluation function of a scheduling sequence generated in the simulated annealing algorithm, a CPU load condition evaluation function corresponding to the scheduling sequence, a weight of the CPU load condition evaluation function, a memory load condition evaluation function corresponding to the scheduling sequence, a weight of the memory load condition evaluation function, an IO load evaluation function corresponding to the scheduling sequence, a weight of the IO load evaluation function, an affinity matching condition evaluation function between the resource scheduling task corresponding to the scheduling sequence and the schedulable computing node, a weight of the affinity matching condition evaluation function, a resource utilization rationality evaluation function corresponding to the scheduling sequence, a weight of the resource utilization rationality evaluation function;

[0026] a number of the schedulable computing nodes, a CPU resource occupied by the resource scheduling task allocated to the i-th schedulable computing node under the scheduling sequence, a CPU resource used by the i-th schedulable computing node, a total amount of CPU resource of the i-th schedulable computing node;

[0027] a memory resource occupied by the resource scheduling task allocated to the i-th schedulable computing node under the scheduling sequence, a memory resource used by the i-th schedulable computing node, a total amount of memory resource of the i-th schedulable computing node;

[0028] an IO resource occupied by the resource scheduling task allocated to the i-th schedulable computing node under the scheduling sequence, an IO resource used by the i-th schedulable computing node, a total amount of IO resource of the i-th schedulable computing node;

[0029] an affinity matching condition function of the resource scheduling task allocated to the i-th schedulable computing node under the scheduling sequence, an affinity of the resource scheduling task allocated to the i-th schedulable computing node under the scheduling sequence, an affinity of the i-th schedulable computing node, when satisfying affinity = 0, when is not satisfied is satisfied = 1.

[0030] is a resource utilization rationality evaluation function of an ith schedulable computing node under a scheduling sequence, is a resource occupation amount of a resource scheduling task allocated to the ith schedulable computing node under the scheduling sequence, is a total amount of resources of the ith schedulable computing node, when is not satisfied is satisfied is not satisfied = 1.

[0031] Optionally, the scheduling strategy of the batch is obtained by using the simulated annealing algorithm, and specifically includes the following steps:

[0032] Starting from an initial scheduling sequence, in each iteration calculation, a preset perturbation method is used to perturb an last scheduling sequence corresponding to a last iteration number to obtain a current scheduling sequence corresponding to a current iteration number;

[0033] If a value of the evaluation function corresponding to the current scheduling sequence is less than a value of the evaluation function corresponding to the last scheduling sequence, the current scheduling sequence is taken as a last scheduling sequence corresponding to a next iteration number;

[0034] If the value of the evaluation function corresponding to the current scheduling sequence is greater than or equal to the value of the evaluation function corresponding to the last scheduling sequence, a Metropolis judgment is performed;

[0035] If the current scheduling sequence passes the Metropolis judgment compared with the last scheduling sequence, the current scheduling sequence is taken as a last scheduling sequence corresponding to a next iteration number;

[0036] If the current scheduling sequence does not pass the Metropolis judgment compared with the last scheduling sequence, the last scheduling sequence is taken as a last scheduling sequence corresponding to a next iteration number;

[0037] If a current temperature corresponding to the current iteration number is greater than a termination temperature of the simulated annealing algorithm, the current temperature is multiplied by an annealing coefficient to obtain a temperature of a next iteration number, and then the step of using the preset perturbation method to perturb the last scheduling sequence corresponding to the last iteration number to obtain the current scheduling sequence corresponding to the current iteration number is returned;

[0038] If the current temperature corresponding to the current iteration number is less than or equal to the termination temperature, the iteration calculation is stopped.

[0039] Optionally, after the stop iteration calculation, further comprising:

[0040] If the resource occupation of each of the schedulable computing nodes in the result scheduling sequence corresponding to the last iteration does not exceed the total amount of resources of the schedulable computing nodes after each of the schedulable computing nodes is allocated the corresponding resource scheduling task, the result scheduling sequence is taken as the scheduling strategy of the resource scheduling task of the batch;

[0041] If there is at least one schedulable computing node in the result scheduling sequence whose resource occupation exceeds the total amount of resources of the schedulable computing nodes after the schedulable computing node is allocated the corresponding resource scheduling task, the number of schedulable computing nodes is reduced, and the step of dividing the resource scheduling task into batches is returned until a first result scheduling sequence of the resource scheduling task of the batch after re-allocation is obtained; at the same time, pre-emptive scheduling is enabled, the high-priority task in the resource scheduling task of the batch is exchanged with the low-priority task running in the schedulable computing node, and the step of obtaining the scheduling strategy of the batch by applying the simulated annealing algorithm for optimization is returned to obtain a second result scheduling sequence of the resource scheduling task of the batch;

[0042] The scheduling strategy of the resource scheduling task of the batch is selected as the scheduling strategy of the batch with the smaller value of the evaluation function in the first result scheduling sequence and the second result scheduling sequence.

[0043] Optionally, further comprising:

[0044] When the number of resource scheduling tasks is less than the number of schedulable computing nodes, the schedulable computing nodes are sorted according to node load, and the schedulable computing node with smaller node load is selected to allocate the resource scheduling task.

[0045] To solve the above technical problems, the application further provides a resource dynamic scheduling device, comprising:

[0046] A batch unit is configured to divide the resource scheduling task into batches when the number of resource scheduling tasks of a user is greater than the number of schedulable computing nodes, so that the number of resource scheduling tasks of each batch is not greater than the number of schedulable computing nodes.

[0047] An optimization unit is configured to take the global resource utilization rate of each schedulable computing node as an optimization objective after the resource scheduling task of each batch is put into the schedulable computing node for scheduling, apply the simulated annealing algorithm to obtain the scheduling strategy of each batch until the scheduling strategy of each batch is obtained.

[0048] The scheduling unit is configured to perform scheduling of each of the resource scheduling tasks according to the scheduling strategy of each batch.

[0049] In one of the scheduling strategies, each of the schedulable computing nodes is allocated at most one of the resource scheduling tasks.

[0050] To solve the above technical problem, the present application further provides a resource dynamic scheduling device, comprising:

[0051] The memory is configured to store a computer program.

[0052] The processor is configured to execute the computer program, and the computer program, when executed by the processor, implements the steps of the resource dynamic scheduling method according to any one of the above.

[0053] To solve the above technical problem, the present application further provides a computer readable storage medium having a computer program stored thereon, and the computer program, when executed by a processor, implements the steps of the resource dynamic scheduling method according to any one of the above.

[0054] The resource dynamic scheduling method provided by the present application reduces the complexity of the many-to-many scheduling in the traditional scheduling by batching the resource scheduling tasks according to the number of schedulable computing nodes when the number of resource scheduling tasks of a user is greater than the number of schedulable computing nodes, and then uses the simulated annealing algorithm to optimize the global resource utilization of each schedulable computing node after the resource scheduling tasks in the batch are scheduled by the schedulable computing nodes, so as to obtain the optimal solution of load balancing while ensuring the randomness and globality of optimization. The method not only realizes the global optimization of the overall scheduling of the cluster, but also reduces the scheduling difficulty and improves the scheduling efficiency, solves the problem of load imbalance in current cloud computing, and provides a general and effective method for the resource scheduling problem of the cloud computing environment, which is suitable for the automatic management requirements of the current cloud computing cluster.

[0055] The present application further provides a resource dynamic scheduling device, equipment and computer readable storage medium, which have the above beneficial effects, and will not be described here. BRIEF DESCRIPTION OF DRAWINGS

[0056] In order to more clearly illustrate the technical solutions of the embodiments of the present application or the prior art, the drawings needed in the embodiment or prior art description will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present application, and those skilled in the art can obtain other drawings according to these drawings without creative labor.

[0057] Figure 1 A flowchart of a resource dynamic scheduling method provided by an embodiment of the present application;

[0058] Figure 2 A simulated annealing algorithm optimization flowchart provided by an embodiment of the present application;

[0059] Figure 3 A structural schematic diagram of a resource dynamic scheduling device provided by an embodiment of the present application;

[0060] Figure 4 A structural schematic diagram of a resource dynamic scheduling device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0061] The core of the present application is to provide a resource dynamic scheduling method, device, equipment and computer readable storage medium, which realizes global optimization of cluster overall scheduling and improves scheduling efficiency.

[0062] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the scope of protection of the present application.

[0063] Embodiment one

[0064] Figure 1 A flowchart of a resource dynamic scheduling method provided by an embodiment of the present application.

[0065] As shown in Figure 1 , the resource dynamic scheduling method provided by the embodiment of the present application comprises:

[0066] S101: When the number of resource scheduling tasks of a user is greater than the number of schedulable computing nodes, the resource scheduling tasks are divided into batches, so that the number of resource scheduling tasks in each batch is not greater than the number of schedulable computing nodes.

[0067] S102: From the first batch of resource scheduling tasks, the optimization goal is to achieve the lowest global resource utilization of each schedulable computing node after the resource scheduling tasks in the batch are put into the schedulable computing nodes for scheduling, and a simulated annealing algorithm is applied to obtain the scheduling strategy of the batch until the scheduling strategies of each batch are obtained.

[0068] S103: The scheduling of each resource scheduling task is performed according to the scheduling strategy of each batch.

[0069] In one scheduling strategy, each schedulable computing node is allocated at most one resource scheduling task.

[0070] In a specific implementation, the resource dynamic scheduling method provided by the embodiments of the present application can be executed based on a management node server in a server cluster corresponding to a cloud computing platform.

[0071] Suppose that the number of resource scheduling tasks is M, and the number of schedulable computing nodes in the server cluster is n. When M > n, the resource scheduling tasks are batched so that the number of resource scheduling tasks in each batch is not greater than the number of schedulable computing nodes. Specifically, the number of resource scheduling tasks in each batch can be M / n, and the number of resource scheduling tasks in the last batch is the remainder obtained by dividing the number of resource scheduling tasks by the number of schedulable computing nodes, and the number of resource scheduling tasks in the remaining batches is equal to the number of schedulable computing nodes.

[0072] Further, to adapt to different priorities of different resource scheduling tasks and different resource demand situations of different resource scheduling tasks, and to promote obtaining a more reasonable global optimal solution, the resource scheduling tasks are batched in step S101 so that the number of resource scheduling tasks in each batch is not greater than the number of schedulable computing nodes. Specifically, this can include:

[0073] According to the parameter information of each resource scheduling task, the dependency relationship between the resource scheduling tasks is analyzed, and each resource scheduling task is sorted according to the priority;

[0074] According to the sorting of each resource scheduling task, while ensuring that the resource scheduling tasks with a dependency relationship are allocated to different batches and the number of resource scheduling tasks in each batch is not greater than the number of schedulable computing nodes, the resource scheduling tasks are batched.

[0075] In actual application, a scoring function built in a K8s cluster can be used to score each resource scheduling task, and the priority of the resource scheduling task is sorted according to the score, while the resource demand information of each resource scheduling task is obtained, which can include CPU, memory, IO, bandwidth and affinity requirements. In addition, other evaluation methods can also be used to sort the priority of the resource scheduling task, such as setting multiple evaluation conditions for weighted comprehensive evaluation.

[0076] According to the current task to be scheduled, the parameter information of each to-be-scheduled resource is collected, the dependency relationship between different resources is analyzed, and the resources with a dependency relationship are allocated to different batches for batch scheduling.

[0077] Further, the resource dynamic scheduling method provided by the embodiment of the present application can further include: when the number of resource scheduling tasks is less than the number of schedulable computing nodes, ranking the schedulable computing nodes according to node load conditions, and assigning resource scheduling tasks to schedulable computing nodes with smaller node loads. When M < n, the top M schedulable computing nodes can be selected for scheduling using an intelligent algorithm. Specifically, the schedulable computing nodes can be ranked after scoring according to node load conditions using a scoring function built in a K8s cluster. In addition, other evaluation methods can be used to rank the schedulable computing nodes, for example, a weighted comprehensive evaluation can be performed from the perspectives of multiple types of resources such as CPU, memory, IO, bandwidth, and affinity. When setting the weights of each type of resource, the weights of each type of resource of the schedulable computing nodes can be further set according to the quality of the specific type of resource of the schedulable computing nodes. When the quality of a certain type of resource of a schedulable computing node is poor, the weight of the type of resource corresponding to the schedulable computing node is increased to make the resource scheduling tasks inclined to other schedulable computing nodes.

[0078] Further, when the number of resource scheduling tasks is equal to the number of schedulable computing nodes, the resource dynamic scheduling method provided by the embodiment of the present application can be directly used to skip the batch step and directly optimize and schedule all resource scheduling tasks.

[0079] The resource dynamic scheduling method provided by the embodiment of the present application can reduce the complexity of the traditional scheduling by batching the resource scheduling tasks according to the number of schedulable computing nodes when the number of resource scheduling tasks of a user is greater than the number of schedulable computing nodes, reducing the complexity O(M) of the traditional scheduling to O(N), where M is the number of resource scheduling tasks and N is the number of batches in the scheduling process. Then, the simulated annealing algorithm is used to optimize the global resource utilization of each schedulable computing node after the resource scheduling tasks in the batch are scheduled in the schedulable computing nodes, so as to obtain a load-balanced optimal solution while ensuring the randomness and globality of the optimization. The method not only realizes the global optimization of the whole cluster scheduling, but also reduces the scheduling difficulty and improves the scheduling efficiency, solves the problem of load imbalance in current cloud computing, and provides a general and effective method for resource scheduling problems in cloud computing environment, which meets the requirements of automatic management of current cloud computing clusters.

[0080] Embodiment Two

[0081] Figure 2 An optimization flowchart of the simulated annealing algorithm provided by the embodiment of the present application.

[0082] On the basis of the above-mentioned embodiments, the global resource utilization rate of each schedulable computing node in the batch is optimized after the batch of resource scheduling tasks is put into schedulable computing node scheduling, and the global resource utilization rate can be obtained by weighted average of multiple types of resources, and the resource types can include but are not limited to CPU, memory, IO, bandwidth, affinity and the like, and other indicators can also be included, and different evaluation function items and different weights can be added according to the actual situation of the cloud computing platform, and different performance concerns, so as to flexibly adapt to the multi-objective optimization problem in the scheduling process.

[0083] Let the schedulable computing node set N = {N1, N2, …, N i , …, N n}, the resource scheduling task set T = {T1, T2, …, T j , …, T m} of the batch, where m ≤ n, to speed up optimization, the batch in the front of the sorting is selected as m = n, and the last batch is less than n resource scheduling tasks.

[0084] In a specific implementation, the embodiments of the present application take CPU, memory, IO, affinity and resource utilization rationality as multi-objectives for example, and the corresponding evaluation function is as follows:

[0085] .

[0086] Wherein, is the CPU load condition evaluation function corresponding to the scheduling sequence, is the weight of the CPU load condition evaluation function, The expression of is as follows:

[0087] ;

[0088] Wherein, is the number of schedulable computing nodes, is the CPU resource occupied by the resource scheduling task allocated to the i-th schedulable computing node under the scheduling sequence, is the CPU resource used by the i-th schedulable computing node, is the total amount of CPU resources of the i-th schedulable computing node.

[0089] is the memory load condition evaluation function corresponding to the scheduling sequence, is the weight of the memory load condition evaluation function, The expression of is as follows:

[0090] ;​

[0091] wherein, is the number of schedulable computing nodes, is the memory resource occupied by the resource scheduling task allocated to the i-th schedulable computing node under the scheduling sequence, is the memory resource used by the i-th schedulable computing node, is the total amount of memory resource of the i-th schedulable computing node.

[0092] is the IO load evaluation function corresponding to the scheduling sequence, is the weight of the IO load evaluation function, is expressed as follows:

[0093] ;

[0094] wherein, is the number of schedulable computing nodes, is the IO resource occupied by the resource scheduling task allocated to the i-th schedulable computing node under the scheduling sequence, is the IO resource used by the i-th schedulable computing node, is the total amount of IO resource of the i-th schedulable computing node.

[0095] is the affinity matching condition evaluation function between the resource scheduling task and the schedulable computing node corresponding to the scheduling sequence, is the weight of the affinity matching condition evaluation function, is expressed as follows:

[0096] ;

[0097] wherein, is the number of schedulable computing nodes, is the affinity matching condition function of the resource scheduling task allocated to the i-th schedulable computing node under the scheduling sequence, is the affinity of the resource scheduling task allocated to the i-th schedulable computing node under the scheduling sequence, is the affinity of the i-th schedulable computing node, when satisfies affinity = 0, when does not satisfy affinity = 1.

[0098] is the resource utilization rationality evaluation function corresponding to the scheduling sequence, is the weight of the resource utilization rationality evaluation function, The expression is as follows:

[0099] ;

[0100] in, This represents the number of schedulable computing nodes. Let be the resource utilization rationality evaluation function for the i-th schedulable computing node under the scheduling sequence. This represents the resource consumption of the scheduling task allocated to the i-th schedulable computing node in the scheduling sequence. Let be the total resources of the i-th schedulable computing node, when hour =0, when hour =1.

[0101] The objective function for optimization using the simulated annealing algorithm in this embodiment is:

[0102] .

[0103] Based on this, let the initial temperature of the simulated annealing algorithm be T0, and the termination temperature be T. e The maximum number of iterations is L. An initial scheduling sequence is randomly generated. That is, the j-th resource scheduling task in this batch is assigned to the j-th resource scheduling task. On each schedulable computing node N = {N1, N2, …, N} i , …,N n r is the simulated annealing coefficient, a real number between 0 and 1, used to control the annealing rate. The smaller the value of r, the faster the simulated annealing rate.

[0104] In the calculation, the number of iterations can be controlled by only the simulated annealing coefficient and the termination temperature, or the maximum number of iterations can be used to control the number of iterations at the same time. That is, the iteration stops when the maximum number of iterations and / or the termination temperature is reached, and the corresponding scheduling sequence is the result scheduling sequence of this optimization.

[0105] Based on this, such as Figure 2 As shown, step S101 applies the simulated annealing algorithm to optimize and obtain the scheduling strategy for this batch, which may specifically include:

[0106] S201: Generate the initial scheduling sequence.

[0107] S202: In this iteration calculation, the previous scheduling sequence corresponding to the previous iteration number is perturbed using a preset perturbation method to obtain the current scheduling sequence corresponding to the current iteration number.

[0108] Specifically, the preset disturbance method can include the following methods: (1) exchanging the schedulable computing nodes corresponding to any two resource scheduling tasks; (2) randomly selecting to reverse the order of the schedulable computing nodes between the schedulable computing nodes corresponding to two resource scheduling tasks; (3) randomly selecting schedulable computing node a, schedulable computing node b and schedulable computing node c corresponding to three resource scheduling tasks, and moving the schedulable computing nodes between schedulable computing node a and schedulable computing node b to after schedulable computing node c. In each iteration calculation, one of the above methods can be randomly selected to generate a new scheduling sequence, or other methods can be used to generate a new scheduling sequence.

[0109] S203: determining whether the value of the evaluation function corresponding to the current scheduling sequence is less than the value of the evaluation function corresponding to the previous scheduling sequence; if yes, proceeding to step S204; if no, proceeding to step S205.

[0110] That is, if <0, then is updated to , is the previous scheduling sequence, is the value of the evaluation function corresponding to the previous scheduling sequence, is the current scheduling sequence, is the value of the evaluation function corresponding to the current scheduling sequence.

[0111] S204: taking the current scheduling sequence as the previous scheduling sequence corresponding to the next iteration number.

[0112] S205: performing Metropolis judgment.

[0113] Specifically, if ≥ 0, Metropolis judgment is performed, a random number r0= random(0, 1) between 0 and 1 is generated, and , wherein is the judgment result of Metropolis judgment, and T is the temperature corresponding to the current iteration number.

[0114] If the current scheduling sequence passes Metropolis judgment compared with the previous scheduling sequence, that is, >r0, proceeding to step S204; if the current scheduling sequence does not pass Metropolis judgment compared with the previous scheduling sequence, that is, ≤ r0, proceeding to step S206.

[0115] S206: taking the previous scheduling sequence as the previous scheduling sequence corresponding to the next iteration number.

[0116] That is, when the value of the evaluation function corresponding to the current scheduling sequence is not less than the value of the evaluation function corresponding to the last scheduling sequence, and the Metropolis criterion is not passed, the current scheduling sequence is not accepted, and the last scheduling sequence is used for iteration in the next iteration.

[0117] S207: determining whether the current temperature corresponding to the current iteration number is greater than the termination temperature of the simulated annealing algorithm; if yes, proceeding to step S208; if no, proceeding to step S209.

[0118] S208: multiplying the current temperature by the annealing coefficient to obtain the temperature of the next iteration number, and returning to step S202.

[0119] S209: stopping the iteration calculation and outputting the result scheduling sequence.

[0120] Specifically, it is determined whether the current temperature T is greater than the termination temperature T of the simulated annealing algorithm. e If yes, the temperature is updated, T is set to r x T, and the next iteration calculation is performed; otherwise, the simulated annealing algorithm is ended, and the result scheduling sequence is obtained.

[0121] Figure 2 The steps shown are a method for scheduling the current batch of resource scheduling tasks to obtain the optimization result (i.e., the result scheduling sequence) of the current batch of resource scheduling tasks. After obtaining the result scheduling sequence, the result scheduling sequence can be directly used as the scheduling strategy of the current batch of resource scheduling tasks for actual scheduling, and the resource states of all schedulable computing nodes are updated. After obtaining the scheduling strategies of all batches of resource scheduling tasks, actual scheduling can be performed. If there is an unscheduled resource scheduling task, the steps of applying the simulated annealing algorithm to obtain the scheduling strategy of the current batch are repeated until the scheduling strategies of all batches are obtained.

[0122] Embodiment Three

[0123] On the basis of the above embodiments, to ensure the rationality of resource scheduling and avoid scheduling failure, after step S209: stopping the iteration calculation, the resource dynamic scheduling method provided by the embodiments of the present application further comprises:

[0124] If the resource occupation of each schedulable computing node after being allocated the corresponding resource scheduling task in the result scheduling sequence corresponding to the last iteration does not exceed the total amount of resources of the schedulable computing node, the result scheduling sequence is used as the scheduling strategy of the current batch of resource scheduling tasks.

[0125] If there is at least one schedulable computing node in the first result scheduling sequence whose resource occupation exceeds the total amount of resources after being allocated with the corresponding resource scheduling task, the number of schedulable computing nodes is reduced, and the step of batching the resource scheduling tasks in step S101 is returned to until a first result scheduling sequence of the re-allocated resource scheduling tasks of the current batch is obtained; meanwhile, preemption scheduling is enabled, the high-priority tasks in the current batch of resource scheduling tasks are exchanged with the low-priority tasks running in the schedulable computing nodes, and the step of obtaining the scheduling strategy of the current batch of resource scheduling tasks by applying the simulated annealing algorithm in step S102 is returned to, so as to obtain a second result scheduling sequence of the current batch of resource scheduling tasks.

[0126] The scheduling strategy of the current batch of resource scheduling tasks is selected as the one with a smaller value of the evaluation function in the first result scheduling sequence and the second result scheduling sequence.

[0127] In a specific implementation, when the result scheduling sequence of the current batch is obtained, the matching of all schedulable computing nodes and resource scheduling tasks in the result scheduling sequence is further evaluated, and if it is reasonable, the actual scheduling is performed, and the scheduling of the next batch (if there is) is performed. Otherwise, it indicates that the scheduling of the current batch fails, and there may be unreasonable. The judgment method of reasonableness is that in the result scheduling sequence, it is ensured that the resource occupation of each schedulable computing node does not exceed the total amount of resources after being allocated with the corresponding resource scheduling task, otherwise the scheduling fails.

[0128] If the scheduling fails, the two operations of reducing the number of schedulable computing nodes and re-batching (such as reducing one schedulable computing node) and preemption scheduling are simultaneously performed, respectively obtaining a first result scheduling sequence (corresponding to an evaluation function value ) and a second result scheduling sequence (corresponding to an evaluation function value ), and the result scheduling sequence with a smaller evaluation function value is selected to be put into actual scheduling.

[0129] It should be noted that when preemption scheduling is performed, one or more high-priority tasks in the current batch of resource scheduling tasks can be randomly selected and exchanged with the low-priority tasks running in the schedulable computing nodes to perform simulation scheduling. When the above two operations are performed, the reasonableness of the first result scheduling sequence and the second result scheduling sequence should be ensured, otherwise the batching or optimization by the simulated annealing algorithm or preemption scheduling needs to be reconsidered until a globally optimal and reasonable solution is obtained.

[0130] The above describes various embodiments of the resource dynamic scheduling method, and on this basis, the application further discloses a resource dynamic scheduling device, equipment and computer readable storage medium corresponding to the above method.

[0131] Embodiment Four

[0132] Figure 3 A structural schematic diagram of a resource dynamic scheduling device provided by an embodiment of the present application.

[0133] As shown in Figure 3 , the resource dynamic scheduling device provided by the embodiment of the present application comprises:

[0134] The batch unit 301 is configured to batch the resource scheduling tasks when the number of the resource scheduling tasks of the user is greater than the number of the schedulable computing nodes, so that the number of the resource scheduling tasks of each batch is not greater than the number of the schedulable computing nodes.

[0135] The optimization unit 302 is configured to, from the first batch of resource scheduling tasks, take the global resource utilization of each schedulable computing node in the batch as the optimization target after the resource scheduling tasks in the batch are put into the schedulable computing nodes for scheduling, and apply the simulated annealing algorithm to obtain the scheduling strategy of the batch until the scheduling strategies of each batch are obtained.

[0136] The scheduling unit 303 is configured to perform the scheduling of each resource scheduling task according to the scheduling strategy of each batch.

[0137] In one scheduling strategy, each schedulable computing node is allocated at most one resource scheduling task.

[0138] Further, the resource dynamic scheduling device provided by the embodiment of the present application further comprises:

[0139] The sorting unit is configured to, when the number of the resource scheduling tasks is less than the number of the schedulable computing nodes, sort each schedulable computing node according to the node load, and select the schedulable computing node with smaller node load to allocate the resource scheduling task.

[0140] Since the embodiments of the device part correspond to the embodiments of the method part, the embodiments of the device part are described in the description of the embodiments of the method part, and will not be described here.

[0141] Embodiment five

[0142] Figure 4 A structural schematic diagram of a resource dynamic scheduling device provided by an embodiment of the present application.

[0143] As shown in Figure 4 , the resource dynamic scheduling device provided by the embodiment of the present application comprises:

[0144] The memory 410 is configured to store a computer program 411.

[0145] The processor 420 is configured to execute the computer program 411, and the computer program 411, when executed by the processor 420, implements the steps of the resource dynamic scheduling method according to any one of the above embodiments.

[0146] The processor 420 may include one or more processing cores, such as a 3-core processor or an 8-core processor. The processor 420 may be implemented using at least one hardware form selected from Digital Signal Processing (DSP), Field-Programmable Gate Array (FPGA), and Programmable Logic Array (PLA). The processor 420 may also include a main processor and a coprocessor. The main processor, also known as a Central Processing Unit (CPU), is used to process data in the wake-up state; the coprocessor is a low-power processor used to process data in the standby state. In some embodiments, the processor 420 may integrate a Graphics Processing Unit (GPU), which is responsible for rendering and drawing the content to be displayed on the screen. In some embodiments, the processor 420 may also include an Artificial Intelligence (AI) processor, which handles computational operations related to machine learning.

[0147] The memory 410 may include one or more computer-readable storage media, which may be non-transitory. The memory 410 may also include high-speed random access memory and non-volatile memory, such as one or more disk storage devices or flash memory devices. In this embodiment, the memory 410 is used to store at least the following computer program 411, wherein, after being loaded and executed by the processor 420, the computer program 411 is able to implement the relevant steps in the resource dynamic scheduling method disclosed in any of the foregoing embodiments. In addition, the resources stored in the memory 410 may also include an operating system 412 and data 413, and the storage method may be temporary storage or permanent storage. The operating system 412 may be Windows. The data 413 may include, but is not limited to, the data involved in the above methods.

[0148] In some embodiments, the resource dynamic scheduling device may further include a display screen 430, a power supply 440, a communication interface 450, an input / output interface 460, a sensor 470, and a communication bus 480.

[0149] Those skilled in the art will understand that Figure 4 The structure shown does not constitute a limitation on the resource dynamic scheduling device and may include more or fewer components than shown.

[0150] The resource dynamic scheduling device provided by the embodiments of the present application comprises a memory and a processor. When the processor executes the program stored in the memory, the resource dynamic scheduling method described above can be realized, and the effects are the same as above.

[0151] Embodiment six

[0152] It should be noted that the apparatus and device embodiments described above are merely illustrative, for example, the division of the modules is merely a logical function division, and actual implementation can have another division manner, for example, a plurality of modules or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the displayed or discussed modules can be indirect coupling or communication connection between the modules through some interfaces, devices or modules, and can be electrical, mechanical or other forms. The modules described as separate components can be or can not be physically separated, and the components displayed as modules can be or can not be physical modules, that is, they can be located in one place or distributed on a plurality of network modules. Part or all of the modules can be selected according to actual needs to achieve the purpose of the embodiments of the present application.

[0153] In addition, the function modules in each embodiment of the present application can be integrated in one processing module, or each module can be physically separated, or two or more modules can be integrated in one module. The integrated module can be realized in the form of hardware or in the form of a software function module.

[0154] If the integrated module is realized in the form of a software function module and sold or used as an independent product, it can be stored in a computer readable storage medium. Based on this understanding, the technical solutions of the present application essentially or the part that contributes to the prior art or the whole or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium, and executes all or part of the steps of the method described in each embodiment of the present application.

[0155] Therefore, the embodiments of the present application also provide a computer readable storage medium, and the computer readable storage medium stores a computer program. When the computer program is executed by a processor, the steps of the resource dynamic scheduling method are realized.

[0156] The computer readable storage medium can include a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and various storage program codes.

[0157] The computer program included in the computer readable storage medium provided in the embodiment can implement the steps of the resource dynamic scheduling method as described above when executed by the processor, and the effects are the same as above.

[0158] The resource dynamic scheduling method, device, equipment and computer readable storage medium provided by the present application are described in detail above. The embodiments in the specification are described in a progressive manner, and each embodiment mainly explains the difference from other embodiments. The same or similar parts of each embodiment can be referred to. For the device, equipment and computer readable storage medium disclosed by the embodiments, since they correspond to the method disclosed by the embodiments, the description is relatively simple, and the related parts can be referred to the method part. It should be pointed out that for ordinary skilled in the art, without departing from the principles of the present application, some improvements and modifications can be made to the present application, and these improvements and modifications also fall within the protection scope of the claims of the present application.

[0159] It should be further noted that in the present specification, the relationship terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between the entities or operations. Moreover, the terms "include", "contain" or any other variants thereof are intended to cover non-exclusive inclusion, so that the process, method, article or equipment including a series of elements not only includes those elements, but also includes other elements not explicitly listed or inherent to such process, method, article or equipment. Without more limitations, the element defined by the statement "including a" does not exclude the existence of other identical elements in the process, method, article or equipment including the element.

Claims

1. A method for dynamic resource scheduling, the method comprising: The method comprises the following steps: when the number of resource scheduling tasks of a user is greater than the number of schedulable computing nodes, the resource scheduling tasks are batched so that the number of resource scheduling tasks in each batch is not greater than the number of schedulable computing nodes; starting from the first batch of resource scheduling tasks, an optimization objective of achieving the lowest global resource utilization of each schedulable computing node after the resource scheduling tasks in the batch are scheduled to the schedulable computing nodes is used to obtain the scheduling strategy of the batch by using a simulated annealing algorithm until the scheduling strategies of each batch are obtained; the scheduling of each resource scheduling task is performed according to the scheduling strategy of each batch; in one scheduling strategy, each schedulable computing node is allocated at most one resource scheduling task; the simulated annealing algorithm is used to obtain the scheduling strategy of the batch, and the method comprises the following steps: starting from an initial scheduling sequence, in each iteration calculation, a preset perturbation method is used to perturb a previous scheduling sequence corresponding to a previous iteration time to obtain a current scheduling sequence corresponding to a current iteration time; if the value of an evaluation function corresponding to the current scheduling sequence is less than the value of the evaluation function corresponding to the previous scheduling sequence, the current scheduling sequence is used as the previous scheduling sequence corresponding to the next iteration time; if the value of the evaluation function corresponding to the current scheduling sequence is greater than or equal to the value of the evaluation function corresponding to the previous scheduling sequence, Metropolis judgment is performed; if the current scheduling sequence passes the Metropolis judgment compared with the previous scheduling sequence, the current scheduling sequence is used as the previous scheduling sequence corresponding to the next iteration time; if the current scheduling sequence does not pass the Metropolis judgment compared with the previous scheduling sequence, the previous scheduling sequence is used as the previous scheduling sequence corresponding to the next iteration time; if the current temperature corresponding to the current iteration time is greater than the termination temperature of the simulated annealing algorithm, the temperature of the next iteration time is obtained by multiplying the current temperature by an annealing coefficient, and then the step of perturbing the previous scheduling sequence corresponding to the previous iteration time by using the preset perturbation method to obtain the current scheduling sequence corresponding to the current iteration time is returned; if the current temperature corresponding to the current iteration time is less than or equal to the termination temperature, the iteration calculation is stopped.

2. The method of claim 1, wherein, the batched resource scheduling tasks are batched so that the number of resource scheduling tasks in each batch is not greater than the number of schedulable computing nodes, and the method comprises the following steps: parameter information of each resource scheduling task is analyzed to obtain a dependency relationship between the resource scheduling tasks, and each resource scheduling task is sorted according to a priority; each resource scheduling task is batched according to the sorting of each resource scheduling task, while ensuring that the resource scheduling tasks having a dependency relationship are allocated to different batches and the number of resource scheduling tasks in each batch is not greater than the number of schedulable computing nodes.

3. The method of claim 1, wherein, the batched resource scheduling tasks are batched so that the number of resource scheduling tasks in each batch is not greater than the number of schedulable computing nodes, and the method comprises the following steps: A remainder obtained by dividing the number of the resource scheduling tasks by the number of the schedulable computing nodes is a number of a last batch of the resource scheduling tasks, and a number of a remaining batch of the resource scheduling tasks is equal to the number of the schedulable computing nodes.

4. The method of claim 1, wherein, An objective function of the simulated annealing algorithm is: ； wherein, ； ； ； ； ； ； an objective function, an evaluation function of a generated scheduling sequence in the simulated annealing algorithm, an evaluation function of a CPU load condition corresponding to the scheduling sequence, a weight of the CPU load condition evaluation function, an evaluation function of a memory load condition corresponding to the scheduling sequence, a weight of the memory load condition evaluation function, an evaluation function of an IO load corresponding to the scheduling sequence, a weight of the IO load evaluation function, an evaluation function of an affinity matching condition between the resource scheduling task corresponding to the scheduling sequence and the schedulable computing node, a weight of the affinity matching condition evaluation function, an evaluation function of resource utilization rationality corresponding to the scheduling sequence, a weight of the resource utilization rationality evaluation function; a number of the schedulable computing nodes, a CPU resource occupied by the resource scheduling task allocated for the i-th schedulable computing node under the scheduling sequence, a CPU resource used by the i-th schedulable computing node, a total amount of CPU resources of the i-th schedulable computing node; memory resource occupied by the resource scheduling task allocated for the ith schedulable computing node in the scheduling sequence, memory resource used by the ith schedulable computing node, total memory resource of the ith schedulable computing node; the IO resource occupied by the resource scheduling task allocated for the ith schedulable computing node in the scheduling sequence, the IO resource used by the ith schedulable computing node, the total amount of IO resource of the ith schedulable computing node; an affinity matching function of the resource scheduled task assigned to the ith schedulable computing node under the scheduling sequence, an affinity of the resource scheduled task assigned to the ith schedulable computing node under the scheduling sequence, an affinity of the ith schedulable computing node when satisfies affinity = 0, when does not satisfy affinity = 1; Let be the resource utilization rationality evaluation function for the i-th schedulable computing node under the scheduling sequence. The resource usage of the resource scheduling task allocated to the i-th schedulable computing node under the scheduling sequence. Let be the total resource amount of the i-th schedulable computing node, when hour =0, when hour =1.

5. The method of claim 1, wherein, After the stop iteration calculation, further comprising: If resource occupation of each of the schedulable computing nodes after being allocated corresponding resource scheduling tasks in a result scheduling sequence corresponding to the last iteration does not exceed total resources of the schedulable computing nodes, taking the result scheduling sequence as the scheduling strategy of the batch of resource scheduling tasks; If resource occupation of at least one of the schedulable computing nodes after being allocated corresponding resource scheduling tasks in the result scheduling sequence exceeds total resources of the schedulable computing nodes, returning to the step of dividing the resource scheduling tasks into batches after reducing the number of the schedulable computing nodes until a first result scheduling sequence of the batch of resource scheduling tasks is obtained by re-allocation; meanwhile, enabling preemption scheduling, exchanging high-priority tasks in the batch of resource scheduling tasks with low-priority tasks running in the schedulable computing nodes, and returning to the step of obtaining the scheduling strategy of the batch by applying the simulated annealing algorithm to obtain a second result scheduling sequence of the batch of resource scheduling tasks; Selecting the scheduling strategy of the batch of resource scheduling tasks in the first result scheduling sequence and the second result scheduling sequence with a smaller value of the evaluation function.

6. The method of claim 1, wherein, Further comprising: When the number of the resource scheduling tasks is less than the number of the schedulable computing nodes, sorting each of the schedulable computing nodes according to node load, and selecting the schedulable computing node with smaller node load to allocate the resource scheduling tasks.

7. A resource dynamic scheduling apparatus, characterized by comprising: Comprising: a batching unit configured to divide the resource scheduling tasks into batches when the number of the resource scheduling tasks of a user is greater than the number of schedulable computing nodes, so that the number of the resource scheduling tasks in each batch is not greater than the number of the schedulable computing nodes; an optimization unit configured to, from a first batch of the resource scheduling tasks, take the global resource utilization rate of each of the schedulable computing nodes as the optimization target after the batch of resource scheduling tasks is put into the schedulable computing nodes for scheduling, and apply the simulated annealing algorithm to obtain the scheduling strategy of the batch until the scheduling strategy of each batch is obtained; a scheduling unit configured to execute scheduling of each of the resource scheduling tasks according to the scheduling strategy of each batch; In one of the scheduling strategies, each of the schedulable computing nodes is allocated at most one of the resource scheduling tasks. The application of the simulated annealing algorithm to obtain the scheduling strategy of the batch specifically comprises: Starting from an initial scheduling sequence, in each iteration calculation, a preset perturbation method is used to perturb a previous scheduling sequence corresponding to a previous iteration time to obtain a current scheduling sequence corresponding to a current iteration time; if the value of the evaluation function corresponding to the current scheduling sequence is less than the value of the evaluation function corresponding to the previous scheduling sequence, then the current scheduling sequence is the previous scheduling sequence corresponding to the next iteration number; if the value of the evaluation function corresponding to the current scheduling sequence is greater than or equal to the value of the evaluation function corresponding to the previous scheduling sequence, then a Metropolis judgment is made; if the current scheduling sequence passes the Metropolis judgment compared to the previous scheduling sequence, then the current scheduling sequence is the previous scheduling sequence corresponding to the next iteration number; if the current scheduling sequence does not pass the Metropolis judgment compared to the previous scheduling sequence, then the previous scheduling sequence is the previous scheduling sequence corresponding to the next iteration number; if the current temperature corresponding to the current iteration number is greater than the termination temperature of the simulated annealing algorithm, then the current temperature is multiplied by an annealing coefficient to obtain the temperature of the next iteration number, and the step of perturbing the previous scheduling sequence corresponding to the previous iteration number using the preset perturbation method to obtain the current scheduling sequence corresponding to the current iteration number is returned; if the current temperature corresponding to the current iteration number is less than or equal to the termination temperature, then the iteration calculation is stopped.

8. A resource dynamic scheduling device, characterized by, The method comprises: a memory for storing a computer program; a processor for executing the computer program, wherein the computer program, when executed by the processor, implements the steps of the resource dynamic scheduling method according to any one of claims 1 to 6.

9. A computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program, when executed by the processor, implements the steps of the resource dynamic scheduling method according to any one of claims 1 to 6.

Images