Resource scheduling method, device, equipment, medium and program product

Abstract

The present application provides a resource scheduling method, device, equipment, medium and program product, which relates to the field of resource scheduling technology to solve the problem of poor effect of related resource scheduling. The method includes: identifying the corresponding instance identification ID, shard ID and Pod ID label for each Pod in multiple instances; determining the target node corresponding to the Pod to be scheduled in the Pod of the multiple instances, and the target node is the node with the highest reward value among the multiple nodes determined based on the reward function; wherein the reward function is used to determine the reward values ​​of multiple nodes based on the labels corresponding to each Pod, and the multiple nodes include the target node. The present application can improve the effect of resource scheduling.

Description

Technical Field

[0001] The present application relates to the technical field of resource scheduling, and in particular to a resource scheduling method, device, equipment, medium and program product. Background Art

[0002] In the field of cloud computing, Kubernetes has become the de facto standard in the field of container orchestration, and the Kubernetes scheduler is an important part of container orchestration, that is, it selects a suitable node for the container to run, which provides a guarantee for the normal operation of the container. When Redis is deployed in a containerized manner, a Redis usually has a master-slave and multi-shard situation, that is, an instance contains multiple Pods, and the distribution of Pods in an instance also has a relatively large impact on the performance of the entire instance. However, the scheduling process in related technologies is based on a single Pod, which leads to poor resource scheduling effects. Summary of the invention

[0003] The embodiments of the present application provide a resource scheduling method, apparatus, device, medium and program product to solve the problem of poor effect of related resource scheduling.

[0004] To solve the above technical problems, this application is implemented as follows:

[0005] In a first aspect, an embodiment of the present application provides a resource scheduling method, including:

[0006] Identify the corresponding instance ID, shard ID, and Pod ID label for each Pod in multiple instances;

[0007] Determine a target node corresponding to the Pod to be scheduled in the Pods of the multiple instances, wherein the target node is a node with the highest reward value among the multiple nodes determined based on the reward function;

[0008] The reward function is used to determine reward values ​​of multiple nodes based on the labels corresponding to each Pod, and the multiple nodes include the target node.

[0009] Optionally, determining a target node corresponding to a Pod to be scheduled in the Pods of the multiple instances includes:

[0010] Build a neural network based on deep reinforcement learning algorithm;

[0011] Iteratively training the neural network to obtain a target neural network model;

[0012] Determining reward values ​​of multiple nodes based on a reward function in the target neural network model;

[0013] A target node corresponding to the Pod to be scheduled is determined based on the reward values ​​of the multiple nodes.

[0014] Optionally, determining the reward values ​​of the plurality of nodes based on the reward function in the target neural network model comprises:

[0015] Determine a first vector based on the total allocatable amount of resources of the i-th node and the sum of the occupancy of the resources of the deployed Pods on the i-th node, where the first vector is used to indicate the resource usage rate of the i-th node, the i-th node is any one of the multiple nodes, and i is an integer greater than 0;

[0016] Determine a second vector based on the resource demand of the Pod to be scheduled and the total amount of resources that can be allocated, where the second vector is used to indicate the ratio of the resource demand of the Pod to be scheduled to the total amount of resources that can be allocated;

[0017] Determine a difference value between the first vector and the second vector, where the difference value is used to indicate a degree of resource conflict between the Pod to be scheduled and the Pod deployed on the i-th node;

[0018] Determine a first value of the i-th node based on the label corresponding to each Pod;

[0019] Based on the difference value and the first value, a reward value of the i-th node is determined.

[0020] Optionally, determining the reward value of the i-th node based on the difference value and the first value includes:

[0021] Determine a second value based on the difference value of the same type of Pods corresponding to the Pod to be scheduled, where the second value is used to indicate the degree of imbalance of the difference of the same type of Pods;

[0022] Determine a third value based on resource usage of multiple nodes in the cluster, wherein the third value is used to indicate a degree of cluster load balancing;

[0023] A reward value of the i-th node is determined based on the difference value, the first value, the second value, and the third value.

[0024] Optionally, the iterative training of the neural network to obtain a target neural network model includes:

[0025] Running the schedule a first preset number of times using the neural network;

[0026] Determine the cluster state, action, action probability and reward corresponding to each dispatch;

[0027] The advantage value was calculated using the generalized advantage estimation (GAE) method;

[0028] updating the neural network a second preset number of times based on the cluster state, actions, action probabilities, rewards, and advantage values;

[0029] When the neural network is used to run the third preset number of schedulings, the training of the neural network is completed to obtain the target neural network model.

[0030] Optionally, the method further comprises:

[0031] The action probability corresponding to the invalid action in the output of the neural network is set to a preset value.

[0032] Optionally, the cluster status includes at least one of the following:

[0033] The remaining allocatable CPU and memory resources of the nodes in the cluster, the remaining available rates of disk inflow and outflow IO, the remaining available rates of network inflow and outflow IO, and the resource usage of the nodes.

[0034] In a second aspect, an embodiment of the present application provides a resource scheduling device, including:

[0035] An identification module is used to identify the corresponding instance identification ID, shard ID, and Pod ID label for each Pod in multiple instances;

[0036] A determination module, used to determine a target node corresponding to the Pod to be scheduled in the Pods of the multiple instances, wherein the target node is a node with the highest reward value among the multiple nodes determined based on the reward function;

[0037] The reward function is used to determine reward values ​​of multiple nodes based on the labels corresponding to each Pod, and the multiple nodes include the target node.

[0038] Optionally, the determining module includes:

[0039] Building unit, used to build neural networks based on deep reinforcement learning algorithms;

[0040] A training unit, used for iteratively training the neural network to obtain a target neural network model;

[0041] A first determining unit, configured to determine reward values ​​of a plurality of nodes based on a reward function in the target neural network model;

[0042] The second determining unit is used to determine the target node corresponding to the Pod to be scheduled based on the reward values ​​of the multiple nodes.

[0043] Optionally, the first determining unit is specifically configured to:

[0044] Determine a first vector based on the total allocatable amount of resources of the i-th node and the sum of the occupancy of the resources of the deployed Pods on the i-th node, where the first vector is used to indicate the resource usage rate of the i-th node, the i-th node is any one of the multiple nodes, and i is an integer greater than 0;

[0045] Determine a second vector based on the resource demand of the Pod to be scheduled and the total amount of resources that can be allocated, where the second vector is used to indicate the ratio of the resource demand of the Pod to be scheduled to the total amount of resources that can be allocated;

[0046] Determine a difference value between the first vector and the second vector, where the difference value is used to indicate a degree of resource conflict between the Pod to be scheduled and the Pod deployed on the i-th node;

[0047] Determine a first value of the i-th node based on the label corresponding to each Pod;

[0048] Based on the difference value and the first value, a reward value of the i-th node is determined.

[0049] Optionally, determining the reward value of the i-th node based on the difference value and the first value includes:

[0050] Determine a second value based on the difference value of the same type of Pods corresponding to the Pod to be scheduled, where the second value is used to indicate the degree of imbalance of the difference of the same type of Pods;

[0051] Determine a third value based on resource usage of multiple nodes in the cluster, wherein the third value is used to indicate a degree of cluster load balancing;

[0052] A reward value of the i-th node is determined based on the difference value, the first value, the second value, and the third value.

[0053] Optionally, the training unit is specifically used for:

[0054] Running the schedule a first preset number of times using the neural network;

[0055] Determine the cluster state, action, action probability and reward corresponding to each dispatch;

[0056] The advantage value was calculated using the generalized advantage estimation (GAE) method;

[0057] updating the neural network a second preset number of times based on the cluster state, actions, action probabilities, rewards, and advantage values;

[0058] When the neural network is used to run the third preset number of schedulings, the training of the neural network is completed to obtain the target neural network model.

[0059] Optionally, the training unit is further used for:

[0060] The action probability corresponding to the invalid action in the output of the neural network is set to a preset value.

[0061] Optionally, the cluster status includes at least one of the following:

[0062] The remaining allocatable CPU and memory resources of the nodes in the cluster, the remaining available rates of disk inflow and outflow IO, the remaining available rates of network inflow and outflow IO, and the resource usage of the nodes.

[0063] In a third aspect, an embodiment of the present application provides an electronic device, including a transceiver and a processor, wherein the processor is configured to:

[0064] Identify the corresponding instance ID, shard ID, and Pod ID label for each Pod in multiple instances;

[0065] Determine a target node corresponding to the Pod to be scheduled in the Pods of the multiple instances, wherein the target node is a node with the highest reward value among the multiple nodes determined based on the reward function;

[0066] The reward function is used to determine reward values ​​of multiple nodes based on the labels corresponding to each Pod, and the multiple nodes include the target node.

[0067] Optionally, the processor is specifically configured to:

[0068] Build a neural network based on deep reinforcement learning algorithm;

[0069] Iteratively training the neural network to obtain a target neural network model;

[0070] Determining reward values ​​of multiple nodes based on a reward function in the target neural network model;

[0071] A target node corresponding to the Pod to be scheduled is determined based on the reward values ​​of the multiple nodes.

[0072] Optionally, the processor is specifically configured to:

[0073] Determine a first vector based on the total allocatable amount of resources of the i-th node and the sum of the occupancy of the resources of the deployed Pods on the i-th node, where the first vector is used to indicate the resource usage rate of the i-th node, the i-th node is any one of the multiple nodes, and i is an integer greater than 0;

[0074] Determine a second vector based on the resource demand of the Pod to be scheduled and the total amount of resources that can be allocated, where the second vector is used to indicate the ratio of the resource demand of the Pod to be scheduled to the total amount of resources that can be allocated;

[0075] Determine a difference value between the first vector and the second vector, where the difference value is used to indicate a degree of resource conflict between the Pod to be scheduled and the Pod deployed on the i-th node;

[0076] Determine a first value of the i-th node based on the label corresponding to each Pod;

[0077] Based on the difference value and the first value, a reward value of the i-th node is determined.

[0078] Optionally, the processor is specifically configured to:

[0079] Determine a second value based on the difference value of the same type of Pods corresponding to the Pod to be scheduled, where the second value is used to indicate the degree of imbalance of the difference of the same type of Pods;

[0080] Determine a third value based on resource usage of multiple nodes in the cluster, wherein the third value is used to indicate a degree of cluster load balancing;

[0081] A reward value of the i-th node is determined based on the difference value, the first value, the second value, and the third value.

[0082] Optionally, the processor is specifically configured to:

[0083] Running the schedule a first preset number of times using the neural network;

[0084] Determine the cluster state, action, action probability and reward corresponding to each dispatch;

[0085] The advantage value was calculated using the generalized advantage estimation (GAE) method;

[0086] updating the neural network a second preset number of times based on the cluster state, actions, action probabilities, rewards, and advantage values;

[0087] When the neural network is used to run the third preset number of schedulings, the training of the neural network is completed to obtain the target neural network model.

[0088] Optionally, the processor is specifically configured to:

[0089] The action probability corresponding to the invalid action in the output of the neural network is set to a preset value.

[0090] Optionally, the cluster status includes at least one of the following:

[0091] The remaining allocatable CPU and memory resources of the nodes in the cluster, the remaining available rates of disk inflow and outflow IO, the remaining available rates of network inflow and outflow IO, and the resource usage of the nodes.

[0092] In a fourth aspect, an embodiment of the present application provides an electronic device, comprising: a processor, a memory, and a program stored in the memory and executable on the processor, wherein when the program is executed by the processor, the steps of the resource scheduling method described in the first aspect above are implemented.

[0093] In a fifth aspect, an embodiment of the present application provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the resource scheduling method described in the first aspect above are implemented.

[0094] In a sixth aspect, an embodiment of the present application provides a computer program product, comprising computer instructions, which, when executed by a processor, implement the steps of the resource scheduling method as described in the first aspect above.

[0095] In an embodiment of the present application, the above-mentioned resource scheduling method can respectively identify the corresponding instance identification ID, shard ID and Pod ID label for each Pod in multiple instances, and determine the target node corresponding to the Pod to be scheduled in the Pod of the multiple instances, wherein the target node is the node with the highest reward value among the multiple nodes determined based on the reward function, and the reward function is used to determine the reward values ​​of the multiple nodes based on the labels corresponding to each Pod, and the multiple nodes include the target node, so that the relationship between multiple Pods of a Redis instance can be taken into account during the scheduling process. It is an instance-level scheduling, and the scheduling result is more suitable for Redis instances, thereby improving the effect of resource scheduling. BRIEF DESCRIPTION OF THE DRAWINGS

[0096] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the description of the embodiments of the present application will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0097] Figure 1 This is a schematic diagram of deploying a Redis instance in a Kubernetes cluster provided by an embodiment of the present application;

[0098] Figure 2 is a flow chart of a resource scheduling method provided in an embodiment of the present application;

[0099] Figure 3 It is an application flow chart of a resource scheduling method provided in an embodiment of the present application;

[0100] Figure 4 It is a structural diagram of a resource scheduling device provided in an embodiment of the present application;

[0101] Figure 5 It is a structural schematic diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0102] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0103] For ease of understanding, some contents involved in the embodiments of the present application are described below:

[0104] The default scheduler of Kubernetes is still the most widely used one at present. The whole scheduling process is divided into the pre-selection and optimization stages. The pre-selection stage is to exclude some nodes that do not meet the conditions according to a series of filter conditions configured by the user, and the pre-selection rules form an extensible filter chain by combining multiple filter functions. There are currently dozens of registered filter functions available in Kubernetes. The filter functions are mainly divided into four categories: storage-related, Pod and node matching-related, Pod and Pod matching-related, and Pod scattering-related. The optimization stage is to prioritize each pre-selected node. Each node will calculate the score according to a series of scoring algorithms. The node with the highest score is the node where the Pod is finally deployed. When setting the scoring algorithm, it is necessary to consider issues such as resource fragmentation, disaster recovery, resource water level, affinity and anti-affinity of nodes or Pods, so the scoring functions are also mainly divided into node resource water level-related, Pod scattering-related, node affinity-related, and Pod affinity-related. When considering the node resource situation, the commonly used scoring algorithms are Least Requested Priority and Balanced Resource Allocation.

[0105] The LeastRequestedPriority algorithm schedules the Pod to the node with the most remaining CPU and memory resources:

[0106] The strategy adopted by the BalancedResourceAllocation algorithm is to select nodes with more balanced central processing unit (CPU) and memory usage after the Pod is deployed:

[0107]

[0108] Among them, R represents the node resource usage, and C is the total amount of node resources that can be allocated.

[0109] The scoring process is to sum up multiple influencing factors with certain weights. It lacks fine-grained management and cannot accurately perceive the problem of resource preemption or resource shortage. The service performance of Redis is determined by the Pod with the smallest resource occupation, so it eventually becomes the bottleneck of the entire service performance. Once fine-grained control is considered, the entire problem becomes a multi-objective optimization problem, and the difficulty of problem solving increases accordingly. The Redis instance itself has many nodes. If the resource dimension increases, its solution space will also increase, making it more difficult to effectively solve the problem.

[0110] With the development of machine learning, a method based on deep reinforcement learning has been proposed to solve the resource scheduling problem. The state space of this method includes the machine state and the state of the tasks to be scheduled in the cluster, and the action refers to selecting a node for the task. The actions can be executed in parallel. The optimization target of the reward function is the average weighted turnaround time of the task. The reward function is set as:

[0111]

[0112] Among them, t is the time interval between two scheduling, d j is the actual execution time of j.

[0113] In the embodiments of the present application, a resource scheduling method, apparatus, device, medium and program product are proposed to solve the problem of poor effect of related resource scheduling.

[0114] As an in-memory database, Redis consumes resources other than CPU and memory. In the persistence process, disk input / output (IO) and network IO consumed by Redis internal communication are also indicators that need to be focused on. However, the above solutions do not reflect the focus on IO. Considering the fine-grained control of multi-dimensional resource usage, Redis instances have many shards, many replicas, many resource dimensions, and a large solution space, making it difficult to effectively solve the problem.

[0115] In the above scheduling scheme, the scheduling process is considered based on a single Pod. When Redis is deployed in a containerized manner, a Redis instance usually has a master-slave and multi-shard situation, that is, an instance contains multiple Pods. The distribution of Pods in an instance also has a relatively large impact on the performance of the entire instance. Therefore, it is more meaningful to consider Pod scheduling at the instance level. Figure 1 FIG. 1 is a schematic diagram of a deployment of a Redis instance in a Kubernetes cluster provided in an embodiment of the present application. Figure 1 As shown:

[0116] During the entire scheduling process, all scoring strategies are calculated and then added together according to fixed weights to calculate the scores. However, the focus on testing will change with the different cluster states. The weights of the first technical solution are fixed and difficult to modify. The second technical solution only optimizes the task running time. In the Pod scheduling process, there are many aspects that need to be considered, and single-goal optimization is bound to have a greater impact on other aspects.

[0117] See also Figure 2 , Figure 2 is a flow chart of a resource scheduling method provided in an embodiment of the present application, such as Figure 2 As shown, the method comprises the following steps:

[0118] Step 201: Identify the corresponding instance ID, shard ID and Pod ID label for each Pod in multiple instances.

[0119] Specifically, the above instance can be an independent running unit of a deployed application or service, for example, a Redis instance, which is specifically an execution instance of a Redis database. The above Pod is the basic deployment unit in Kubernetes, which is a group of one or more containers running on a cluster.

[0120] It is understandable that an instance can correspond to multiple Pods. Specifically, an instance can correspond to multiple shards, a shard can correspond to multiple Pods, and a shard can also include a master node and one or more slave nodes. The above instance ID can be understood as the instance ID of the Pod, the above shard ID can be understood as the shard ID of the Pod, and the above PodID can identify the type of the Pod.

[0121] It should be noted that, the corresponding master node or slave node ID can also be identified for each Pod in multiple instances according to needs.

[0122] Step 202: determine a target node corresponding to the Pod to be scheduled among the Pods of the multiple instances, wherein the target node is a node with the highest reward value among the multiple nodes determined based on a reward function; wherein the reward function is used to determine the reward values ​​of the multiple nodes based on the labels corresponding to each Pod, and the multiple nodes include the target node.

[0123] Specifically, the Pod to be scheduled may be a Pod that has been created but has not yet been assigned to any node, and the Kubernetes scheduler selects a suitable node to run the Pod to be scheduled. The reward value may be used to indicate the degree of node resource conflict determined by assigning the Pod to be scheduled to any node.

[0124] In an embodiment of the present application, the above-mentioned resource scheduling method can respectively identify the corresponding instance identification ID, shard ID and Pod ID label for each Pod in multiple instances, and determine the target node corresponding to the Pod to be scheduled in the Pod of the multiple instances, wherein the target node is the node with the highest reward value among the multiple nodes determined based on the reward function, and the reward function is used to determine the reward values ​​of the multiple nodes based on the labels corresponding to each Pod, and the multiple nodes include the target node, so that the relationship between multiple Pods of a Redis instance can be taken into account during the scheduling process. It is an instance-level scheduling, and the scheduling result is more suitable for Redis instances, thereby improving the effect of resource scheduling.

[0125] Optionally, determining a target node corresponding to a Pod to be scheduled in the Pods of the multiple instances includes:

[0126] Build a neural network based on deep reinforcement learning algorithm;

[0127] Iteratively training the neural network to obtain a target neural network model;

[0128] Determining reward values ​​of multiple nodes based on a reward function in the target neural network model;

[0129] A target node corresponding to the Pod to be scheduled is determined based on the reward values ​​of the multiple nodes.

[0130] Specifically, the above-mentioned deep reinforcement learning algorithm can adopt a proximal policy optimization (PPO) algorithm.

[0131] It is understandable that the mathematical foundation and modeling tool of deep reinforcement learning is the Markov decision process, which is composed of the environment, the agent, the action, and the t , Reward t , status tTherefore, in order to use the reinforcement learning algorithm to solve the Redis instance scheduling problem, it is necessary to model the problem as a Markov decision process.

[0132] The environment is the optional cluster environment of Redis, which is the Kubernetes cluster. It describes the state of the environment at time t. In this scheduling problem, it represents the resource usage of each node in the cluster at time t and the distribution of Pods on each node. The intelligent agent is the selected deep neural network, which is used to t Select the action corresponding to the maximum reward; action a t Indicates the action taken by the agent when faced with a problem. In this problem, the node where the current Pod finally runs is given; the reward is for executing action a t The feedback given after the action is used to modify the agent's strategy for selecting actions.

[0133] Figure 3 is an application flow chart of a resource scheduling method provided in an embodiment of the present application, such as Figure 3 As shown: The overall process of the scheduling process is to give the deep neural network cluster state s t , the neural network gives a decision t , schedule the current Pod to the decision node, and after the Pod successfully runs on the selected node, calculate a reward value r t , and the current state s t+1 The rewards are returned to the neural network, which continuously adjusts the strategy for selecting actions based on the rewards, and eventually forms a selection strategy that can maximize the reward value each time.

[0134] In this implementation, the above-mentioned resource scheduling method can build a neural network based on a deep reinforcement learning algorithm, iteratively train the neural network to obtain a target neural network model, determine the reward values ​​of multiple nodes based on the reward function in the target neural network model, and determine the target nodes corresponding to the Pod to be scheduled based on the reward values ​​of the multiple nodes, so that the deep reinforcement learning algorithm can be used to solve the complex solution space, thereby improving the efficiency of resource scheduling.

[0135] Optionally, determining the reward values ​​of the plurality of nodes based on the reward function in the target neural network model comprises:

[0136] Determine a first vector based on the total allocatable amount of resources of the i-th node and the sum of the occupancy of the resources of the deployed Pods on the i-th node, where the first vector is used to indicate the resource usage rate of the i-th node, the i-th node is any one of the multiple nodes, and i is an integer greater than 0;

[0137] Determine a second vector based on the resource demand of the Pod to be scheduled and the total amount of resources that can be allocated, where the second vector is used to indicate the ratio of the resource demand of the Pod to be scheduled to the total amount of resources that can be allocated;

[0138] Determine a difference value between the first vector and the second vector, where the difference value is used to indicate a degree of resource conflict between the Pod to be scheduled and the Pod deployed on the i-th node;

[0139] Determine a first value of the i-th node based on the label corresponding to each Pod;

[0140] Based on the difference value and the first value, a reward value of the i-th node is determined.

[0141] It is understandable that when setting the reward function, the optimization goal is to find nodes with large differences between the resource requirements of the Pod to be scheduled and the total resource consumption of all Pods already running on the node, and try to avoid scheduling Pods that consume the same type of resources to the same node, thereby solving the problem that multi-dimensional resource scheduling causes a resource conflict to become a service performance bottleneck. A service contains multiple Pods, and the Pod with the smallest resource occupancy determines the performance of certain services. Therefore, the balance of resources occupied by the same type of Pods in an instance should also be considered, so as to alleviate the problem of resource waste caused by excessive differences in the resources occupied by Pods in an instance. In addition, the impact of the current scheduling result on the balance of resources in the entire cluster also needs to be considered. The reward reflects the quality of the selected action and guides the agent to choose the action with high reward. According to the above analysis of the optimization goal, when designing the reward function, it is necessary to consider the conflict between the resources occupied by the Pod to be scheduled and all deployed Pods on the selected node, the balance of resource occupancy between Pods in the same instance, and the balance of resources in the entire cluster.

[0142] Exemplarily, the specific process of determining the difference value between the first vector and the second vector is as follows:

[0143] When calculating the resource conflicts between the Pod to be scheduled and all deployed Pods on the node, it is assumed that the i-th node in the cluster is selected, R i =[r i,1 r i,2 r i,3 r i,4 r i,5 r i,6 ] is the remaining allocatable resources of the i-th node in the cluster, U i =[u i,1 u i,2 u i,3 u i,4 u i,5 ui,6 ] is the total occupancy of all Pod resources on the i-th node at time t, P = [p1 p2 p3p4 p5 p6] is the demand for Pod resources to be scheduled, and U i +R i Approximately represents the total amount of resources that can be allocated to the i-th node, Represents the resource usage rate of the i-th node. As a memory database, Redis pays special attention to memory resources. To increase the attention to memory resources, the memory resource usage rate is multiplied by a factor of 2. It represents the ratio of Pod application to the total resources of the i-th node. The difference between the two is calculated using the following formula:

[0144]

[0145] The cosine theorem is used to calculate the similarity of two vectors, and the similarity is subtracted from 1. The lower the similarity between the two, the higher the score, and the less likely the resources are to conflict.

[0146] Specifically, the first value of the i-th node determined based on the label corresponding to each Pod can be obtained by scoring according to the distribution of the Pod to be scheduled and the Pod allocated to the instance on the node, and the first value can be used as a penalty item of the reward function.

[0147] Exemplarily, when the Pod to be scheduled and the Pod already allocated on the node belong to the same instance, the node score is increased by 1; when the Pod to be scheduled and the Pod already allocated on the node belong to the same instance and the same shard, the node score is increased by 2. The maximum value of the first value is set to 10, recorded as w.

[0148] In this implementation, the above-mentioned resource scheduling method can balance the node resource occupancy rate in the same cluster by representing the relationship between multiple Pods of the same instance through the average difference of the Pod resource demand ratio, so as to reduce or avoid the strategy of scheduling Pods consuming the same resource type to the same node, thereby improving the instance service quality and further improving the effect of resource scheduling.

[0149] Optionally, determining the reward value of the i-th node based on the difference value and the first value includes:

[0150] Determine a second value based on the difference value of the same type of Pods corresponding to the Pod to be scheduled, where the second value is used to indicate the degree of imbalance of the difference of the same type of Pods;

[0151] Determine a third value based on resource usage of multiple nodes in the cluster, wherein the third value is used to indicate a degree of cluster load balancing;

[0152] A reward value of the i-th node is determined based on the difference value, the first value, the second value, and the third value.

[0153] Specifically, the second value and the third value can be used as penalty items of the reward function. The specific process of determining the second value based on the difference value of the same type of Pod corresponding to the Pod to be scheduled is as follows:

[0154] Define V = [v1v2…v] as the difference value of the scheduled Pods of the same type of service. This value is differentiated according to the Label field value in the Pod resource configuration file. When V is empty, a penalty value is randomly generated from the range of [5,15]. When it is not empty, the second value is the standard deviation of the values ​​in V:

[0155]

[0156] It should be noted that, in order to limit the influence of the second value on the final score, it can be set that when the second value is greater than or equal to 15, the value of the second value is 15.

[0157] The method for determining the third value based on the resource utilization of multiple nodes in the cluster may be to first calculate the resource utilization of the cluster O=[O1O2...O m ] T , the matrix O is normalized according to the columns to represent the ratio of the occupancy rate of each type of resource of each node to the occupancy rate of the same type of resources in the cluster. The standard deviation of the normalized matrix O is expressed as the third value l, which is recorded as a measure of the load balancing situation of the cluster.

[0158] The difference value, the first value, the second value and the third value determine the reward value of the i-th node, and the first value, the second value and the third value may be used as penalty terms of the reward function, that is, the final reward value is vblw.

[0159] In this implementation, the above-mentioned resource scheduling method can affect the reward value by setting a penalty item in the process of setting the reward function, taking into account the degree of imbalance of differences among similar Pods and the degree of cluster load balance. In addition, since the weights of multiple optimization objectives in the deep reinforcement learning algorithm will change dynamically when facing scenarios in different states, it can ultimately achieve a compromise among cluster resource load, node resource conflict, and service performance, thereby further improving the effect of resource scheduling.

[0160] Optionally, the iterative training of the neural network to obtain a target neural network model includes:

[0161] Running the schedule a first preset number of times using the neural network;

[0162] Determine the cluster state, action, action probability and reward corresponding to each dispatch;

[0163] The advantage value was calculated using the generalized advantage estimation (GAE) method;

[0164] updating the neural network a second preset number of times based on the cluster state, actions, action probabilities, rewards, and advantage values;

[0165] When the neural network is used to run the third preset number of schedulings, the training of the neural network is completed to obtain the target neural network model.

[0166] For example, two neural networks Actor and Critic are first constructed. The neural network consists of six linear layers, the number of neurons in the middle layer is 256, and the activation function is the Tanh function. The Actor network inputs the state of the environment and the output dimension is the same as the action space dimension. The Critic inputs the state of the environment and the output dimension is 1.

[0167] At the beginning of algorithm training, hyperparameters need to be set, where total_steps represents the total number of steps in the entire training process, and target_steps represents the number of steps after which the neural network is updated. The cluster environment used in the training process has approximately 50 steps per round, the reward-to-go calculation parameter is set to 0.96, the clipping factor is 0.2, and the advantage value calculation is set to 0.98.

[0168] Then, the current Actor network is used to run target_steps scheduling, and the cluster state s, action a, probability last_prob of all actions, and reward r generated during each scheduling process are collected. When there is no node in the cluster that meets the current Pod requirements, a round of scheduling is completed, and all Pods deployed in this round in the cluster environment will be cleared.

[0169] By formula Calculate rewards-to-go, where T represents the number of steps per round. When calculating the advantage value, the formula Different approximations of the Q value in will lead to imbalanced deviation and variance, making the algorithm difficult to converge or causing large errors. k Represents the parameters of the k-th round of Critic network. In order to alleviate this problem, the GAE method can be used to calculate the advantage value. The formula is as follows:

[0170]

[0171] δ t =γ t +γv(s t+1 )-v(st )

[0172] Among them, v(s t ) represents the state value at time t, δ t It represents the error between the state value at time t+1 and at time t, γ represents the discount factor, and λ is used to balance the deviation and variance. Its value range is [0,1]. The larger the value, the smaller the deviation and the larger the variance.

[0173] After normalizing the advantage value, the algorithm performance will be improved. The formula for normalizing the advantage value is as follows:

[0174]

[0175] in, is the mean of the advantage values, represents the standard deviation of the odds values.

[0176] After collecting and processing the data, we start to use the above data to update the neural network repeat_times times, and the value of repeat_times is set to 6. First, we randomly select a batch of data from the collected data set, and set the value to 128. Then, we put the selected state into the Actor network to calculate the logarithm of the action probability under the current Actor network, and then update the parameters by gradient ascent after deriving the loss function of the Actor network. On the basis of the original loss function, we add the incentive term policy entropy H(π(·|s t )) to optimize the policy entropy in the direction of increasing, thereby encouraging algorithm exploration. θ represents the current policy network parameters, θ k represents the policy network parameters in the kth round, which are the parameters of the policy network when interacting with the environment to generate this training data, π θ represents the policy network using the current policy network function θ, r t (θ) represents the ratio between the new and old strategies. ε and β are hyperparameters used to ensure that the gap between the new and old strategies is not too large when the same batch of data is used for multiple strategy updates. The clip operation limits the ratio (the ratio of the new strategy to the old strategy) to a range of [1-ε, 1+ε]. It represents the advantage value in the range of [1-ε,1+ε]. The specific formula is as follows:

[0177] Loss(θ)=E t [min(r t (θ)A t ,g(ε,A t ))]+β·E s [H(S;θ)]

[0178]

[0179]

[0180]

[0181] When updating the Critic network, the selected state is first put into the Critic network to obtain the current Critic network's estimate of the state value. The estimated value and rewards-to-go are updated by gradient descent through the SmoothL1Loss function.

[0182] After executing repeat_times neural network parameter updates, the target_steps data used for parameter updates will be discarded, and the training data set will be saved again in the subsequent interaction process. At the same time, an evaluator is defined to evaluate the optimized neural network. First, 4 rounds of real scheduling will be performed, and the mean reward in the 4 rounds of scheduling will be calculated. When the mean reward exceeds the current maximum mean, the parameters of the current neural network are saved.

[0183] Execute the above loops in sequence until the algorithm ends after reaching total_times scheduling times.

[0184] It is understandable that the above training process targets the dynamic changes of optional nodes and the complexity of the training process faced in the Kubernetes scheduling process, and makes targeted modifications to the deep reinforcement learning algorithm in the scenario of Kubernetes scheduling Redis instances, clarifies the training process of the entire algorithm, and ultimately achieves the effectiveness of the training results.

[0185] In this implementation, the resource scheduling method may use the neural network to run a first preset number of schedulings, determine the cluster state, action, action probability and reward corresponding to each scheduling, calculate the advantage value using the generalized advantage estimation GAE method, and update the neural network a second preset number of times based on the cluster state, action, action probability, reward and advantage value. After the neural network is used to run a third preset number of schedulings, the training of the neural network is completed to obtain the target neural network model, so that the deep reinforcement learning algorithm can be optimized, thereby reducing the convergence difficulty and error of the algorithm, thereby further improving the effect of resource scheduling.

[0186] Optionally, the method further comprises:

[0187] The action probability corresponding to the invalid action in the output of the neural network is set to a preset value.

[0188] During each scheduling process, the nodes in the cluster that meet the Pod configuration conditions will change, so the Pod's optional action space will also change. s(l(s)) i It is defined as setting the value corresponding to the invalid action in the output l(s) of the Actor network to a very small negative number M. For example, during the training process, the value of M is set to 1×10 -8 , this process is recorded as π new The specific formula is as follows:

[0189] π′(·|s t ;θ)=softmax(inv s (l(s)))

[0190]

[0191] In this implementation, the resource scheduling method can set the action probability corresponding to the invalid action in the output of the neural network to a preset value. After normalization by the softmax function, the probability corresponding to the invalid action becomes zero, thereby shielding the invalid action, thereby avoiding calculation errors or adverse consequences caused by selecting invalid actions, thereby improving the rationality of strategy determination.

[0192] Optionally, the cluster status includes at least one of the following:

[0193] The remaining allocatable CPU and memory resources of the nodes in the cluster, the remaining available rates of disk inflow and outflow IO, the remaining available rates of network inflow and outflow IO, and the resource usage of the nodes.

[0194] Specifically, the above-mentioned remaining allocatable central processing unit CPU can be understood as the amount of CPU resources on the node that are not currently used by Pods and can be used for new Pods, the above-mentioned memory resources can be understood as the amount of memory resources on the node that are not currently used by Pods and can be used for new Pods, the above-mentioned remaining available rate of disk inflow and outflow IO can be understood as the disk read and write performance that is not currently in use on the node, the above-mentioned remaining available rate of network inflow and outflow IO can be understood as the remaining bandwidth on the node that can still be used to transmit data, and the above-mentioned node resource usage is used to indicate how much resources are used on the current node, including CPU, memory, storage, and network, etc.

[0195] For example, in order to have a more comprehensive description of the resources required by Redis in the cluster, the CPU, memory, disk IO and network IO in the cluster are taken as the focus objects, so the state st, that is, the state s at time t, is expressed as:

[0196]

[0197] Among them, r n,1 、r n,2Indicates the remaining CPU and memory resources that can be allocated to Pods on the nth node, r n,3 、r n,4 is the remaining available rate of disk IO inflow and outflow, r n,5 、r n,6 is the remaining available rate for network inflow and outflow, u n,k Indicates the usage of the corresponding resource on the current node. It can be obtained by comparing the maximum value of the sum of the allocated Pod resources on the nth node and the sum of the actual usage of the Pod operation, so as to obtain the dynamic resource consumption of the service and reserve relatively sufficient resources for the service operation. The actual resource usage can be taken as the average within 5 seconds.

[0198] In this implementation, the cluster status includes at least one of the following: the remaining allocatable central processing unit CPU and memory resources of the nodes in the cluster, the remaining available rate of disk inflow and outflow IO, the remaining available rate of network inflow and outflow IO, and the resource usage of the node, so that the above-mentioned resource scheduling method takes disk IO and network IO into consideration during the scheduling process, solves the problem of few resource dimensions considered by the current scheduler, and thus can improve the effect of resource scheduling.

[0199] See also Figure 4 , Figure 4 is a schematic diagram of a resource scheduling device provided in an embodiment of the present application. Figure 4 As shown, the resource scheduling device 400 includes:

[0200] An identification module 401 is used to identify the corresponding instance identification ID, shard ID and Pod ID label for each Pod in multiple instances;

[0201] A determination module 402 is used to determine a target node corresponding to the Pod to be scheduled in the Pods of the multiple instances, wherein the target node is a node with the highest reward value among the multiple nodes determined based on the reward function;

[0202] The reward function is used to determine reward values ​​of multiple nodes based on the labels corresponding to each Pod, and the multiple nodes include the target node.

[0203] Optionally, the determining module includes:

[0204] Building unit, used to build neural networks based on deep reinforcement learning algorithms;

[0205] A training unit, used for iteratively training the neural network to obtain a target neural network model;

[0206] A first determining unit, configured to determine reward values ​​of a plurality of nodes based on a reward function in the target neural network model;

[0207] The second determining unit is used to determine the target node corresponding to the Pod to be scheduled based on the reward values ​​of the multiple nodes.

[0208] Optionally, the first determining unit is specifically configured to:

[0209] Determine a first vector based on the total allocatable amount of resources of the i-th node and the sum of the occupancy of the resources of the deployed Pods on the i-th node, where the first vector is used to indicate the resource usage rate of the i-th node, the i-th node is any one of the multiple nodes, and i is an integer greater than 0;

[0210] Determine a second vector based on the resource demand of the Pod to be scheduled and the total amount of resources that can be allocated, where the second vector is used to indicate the ratio of the resource demand of the Pod to be scheduled to the total amount of resources that can be allocated;

[0211] Determine a difference value between the first vector and the second vector, where the difference value is used to indicate a degree of resource conflict between the Pod to be scheduled and the Pod deployed on the i-th node;

[0212] Determine a first value of the i-th node based on the label corresponding to each Pod;

[0213] Based on the difference value and the first value, a reward value of the i-th node is determined.

[0214] Optionally, determining the reward value of the i-th node based on the difference value and the first value includes:

[0215] Determine a second value based on the difference value of the same type of Pods corresponding to the Pod to be scheduled, where the second value is used to indicate the degree of imbalance of the difference of the same type of Pods;

[0216] Determine a third value based on resource usage of multiple nodes in the cluster, wherein the third value is used to indicate a degree of cluster load balancing;

[0217] A reward value of the i-th node is determined based on the difference value, the first value, the second value, and the third value.

[0218] Optionally, the training unit is specifically used for:

[0219] Running the schedule a first preset number of times using the neural network;

[0220] Determine the cluster state, action, action probability and reward corresponding to each dispatch;

[0221] The advantage value was calculated using the generalized advantage estimation (GAE) method;

[0222] updating the neural network a second preset number of times based on the cluster state, actions, action probabilities, rewards, and advantage values;

[0223] When the neural network is used to run the third preset number of schedulings, the training of the neural network is completed to obtain the target neural network model.

[0224] Optionally, the training unit is further used for:

[0225] The action probability corresponding to the invalid action in the output of the neural network is set to a preset value.

[0226] Optionally, the cluster status includes at least one of the following:

[0227] The remaining allocatable CPU and memory resources of the nodes in the cluster, the remaining available rates of disk inflow and outflow IO, the remaining available rates of network inflow and outflow IO, and the resource usage of the nodes.

[0228] It should be noted that the resource scheduling device provided in the embodiment of the present application is a device capable of executing the above resource scheduling method, and all implementations in the above resource scheduling method embodiment are applicable to the device and can achieve the same or similar beneficial effects. To avoid repeated description, this embodiment will not be repeated.

[0229] For details, see Figure 5 As shown, an embodiment of the present application further provides an electronic device, including a bus 501, a transceiver 502, an antenna 503, a bus interface 504, a processor 505 and a memory 506.

[0230] The processor 505 is configured to:

[0231] Identify the corresponding instance ID, shard ID, and Pod ID label for each Pod in multiple instances;

[0232] Determine a target node corresponding to the Pod to be scheduled in the Pods of the multiple instances, wherein the target node is a node with the highest reward value among the multiple nodes determined based on the reward function;

[0233] The reward function is used to determine reward values ​​of multiple nodes based on the labels corresponding to each Pod, and the multiple nodes include the target node.

[0234] exist Figure 5In the embodiment, the bus architecture (represented by bus 501) is shown, and bus 501 may include any number of interconnected buses and bridges, and bus 501 links various circuits including one or more processors represented by processor 505 and memory represented by memory 506. Bus 501 may also link various other circuits such as peripherals, voltage regulators, and power management circuits, which are well known in the art and are therefore not further described herein. Bus interface 504 provides an interface between bus 501 and transceiver 502. Transceiver 502 may be one element or multiple elements, such as multiple receivers and transmitters, providing a unit for communicating with various other devices on a transmission medium. Data processed by processor 505 is transmitted on a wireless medium via antenna 503, and further, antenna 503 also receives data and transmits the data to processor 505.

[0235] The processor 505 is responsible for managing the bus 501 and general processing, and can also provide various functions, including timing, peripheral interfaces, voltage regulation, power management and other control functions. The memory 506 can be used to store data used by the processor 505 when performing operations.

[0236] Optionally, the processor 505 may be a central processing unit (CPU), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or a complex programmable logic device (CPLD).

[0237] Optionally, the processor 505 is specifically configured to:

[0238] Build a neural network based on deep reinforcement learning algorithm;

[0239] Iteratively training the neural network to obtain a target neural network model;

[0240] Determining reward values ​​of multiple nodes based on a reward function in the target neural network model;

[0241] A target node corresponding to the Pod to be scheduled is determined based on the reward values ​​of the multiple nodes.

[0242] Optionally, the processor 505 is specifically configured to:

[0243] Determine a first vector based on the total allocatable amount of resources of the i-th node and the sum of the occupancy of the resources of the deployed Pods on the i-th node, where the first vector is used to indicate the resource usage rate of the i-th node, the i-th node is any one of the multiple nodes, and i is an integer greater than 0;

[0244] Determine a second vector based on the resource demand of the Pod to be scheduled and the total amount of resources that can be allocated, where the second vector is used to indicate the ratio of the resource demand of the Pod to be scheduled to the total amount of resources that can be allocated;

[0245] Determine a difference value between the first vector and the second vector, where the difference value is used to indicate a degree of resource conflict between the Pod to be scheduled and the Pod deployed on the i-th node;

[0246] Determine a first value of the i-th node based on the label corresponding to each Pod;

[0247] Based on the difference value and the first value, a reward value of the i-th node is determined.

[0248] Optionally, the processor 505 is specifically configured to:

[0249] Determine a second value based on the difference value of the same type of Pods corresponding to the Pod to be scheduled, where the second value is used to indicate the degree of imbalance of the difference of the same type of Pods;

[0250] Determine a third value based on resource usage of multiple nodes in the cluster, wherein the third value is used to indicate a degree of cluster load balancing;

[0251] A reward value of the i-th node is determined based on the difference value, the first value, the second value, and the third value.

[0252] Optionally, the processor 505 is specifically configured to:

[0253] Running the schedule a first preset number of times using the neural network;

[0254] Determine the cluster state, action, action probability and reward corresponding to each dispatch;

[0255] The advantage value was calculated using the generalized advantage estimation (GAE) method;

[0256] updating the neural network a second preset number of times based on the cluster state, actions, action probabilities, rewards, and advantage values;

[0257] When the neural network is used to run the third preset number of schedulings, the training of the neural network is completed to obtain the target neural network model.

[0258] Optionally, the processor 505 is specifically configured to:

[0259] The action probability corresponding to the invalid action in the output of the neural network is set to a preset value.

[0260] Optionally, the cluster status includes at least one of the following:

[0261] The remaining allocatable CPU and memory resources of the nodes in the cluster, the remaining available rates of disk inflow and outflow IO, the remaining available rates of network inflow and outflow IO, and the resource usage of the nodes.

[0262] It should be noted that the electronic device provided in the embodiment of the present application is a device capable of executing the above resource scheduling method, and all implementation methods in the above resource scheduling method embodiment are applicable to the electronic device and can achieve the same or similar beneficial effects. To avoid repeated description, this embodiment will not be repeated.

[0263] An embodiment of the present application also provides an electronic device, including: a processor, a memory, and a program stored in the memory and executable on the processor. When the program is executed by the processor, the various processes of the above-mentioned resource scheduling method embodiment are implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0264] The embodiment of the present application also provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, each process of the above-mentioned resource scheduling method embodiment is implemented, and the same technical effect can be achieved. To avoid repetition, it is not repeated here. The computer-readable storage medium is, for example, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0265] The embodiment of the present application also provides a computer program product, including computer instructions. When the computer instructions are executed by a processor, the various processes of the above-mentioned resource scheduling method embodiment are implemented and the same technical effect can be achieved. To avoid repetition, they are not repeated here.

[0266] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element.

[0267] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, a magnetic disk, or an optical disk), and includes a number of instructions for a terminal (which can be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) to execute the methods described in each embodiment of the present application.

[0268] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present application, ordinary technicians in this field can also make many forms without departing from the purpose of the present application and the scope of protection of the claims, all of which are within the protection of the present application.

Claims

1. A resource scheduling method, characterized in that: include: Identify the corresponding instance ID, shard ID, and Pod ID label for each Pod in multiple instances; Determine a target node corresponding to the Pod to be scheduled in the Pods of the multiple instances, wherein the target node is a node with the highest reward value among the multiple nodes determined based on the reward function; The reward function is used to determine reward values ​​of multiple nodes based on the labels corresponding to each Pod, and the multiple nodes include the target node.

2. The method according to claim 1, characterized in that The determining a target node corresponding to a Pod to be scheduled in the Pods of the multiple instances includes: Build a neural network based on deep reinforcement learning algorithm; Iteratively training the neural network to obtain a target neural network model; Determining reward values ​​of multiple nodes based on a reward function in the target neural network model; A target node corresponding to the Pod to be scheduled is determined based on the reward values ​​of the multiple nodes.

3. The method according to claim 2, characterized in that The determining reward values ​​of multiple nodes based on the reward function in the target neural network model includes: Determine a first vector based on the total allocatable amount of resources of the i-th node and the sum of the occupancy of the resources of the deployed Pods on the i-th node, where the first vector is used to indicate the resource usage rate of the i-th node, the i-th node is any one of the multiple nodes, and i is an integer greater than 0; Determine a second vector based on the resource demand of the Pod to be scheduled and the total amount of resources that can be allocated, where the second vector is used to indicate the ratio of the resource demand of the Pod to be scheduled to the total amount of resources that can be allocated; Determine a difference value between the first vector and the second vector, where the difference value is used to indicate a degree of resource conflict between the Pod to be scheduled and the Pod deployed on the i-th node; Determine a first value of the i-th node based on the label corresponding to each Pod; Based on the difference value and the first value, a reward value of the i-th node is determined.

4. The method according to claim 3, characterized in that The step of determining the reward value of the i-th node based on the difference value and the first value includes: Determine a second value based on the difference value of the same type of Pods corresponding to the Pod to be scheduled, where the second value is used to indicate the degree of imbalance of the difference of the same type of Pods; Determine a third value based on resource usage of multiple nodes in the cluster, wherein the third value is used to indicate a degree of cluster load balancing; A reward value of the i-th node is determined based on the difference value, the first value, the second value, and the third value.

5. The method according to claim 2, characterized in that: The iterative training of the neural network to obtain a target neural network model includes: Running the schedule a first preset number of times using the neural network; Determine the cluster state, action, action probability and reward corresponding to each dispatch; The advantage value was calculated using the generalized advantage estimation (GAE) method; updating the neural network a second preset number of times based on the cluster state, actions, action probabilities, rewards, and advantage values; When the neural network is used to run the third preset number of schedulings, the training of the neural network is completed to obtain the target neural network model.

6. The method according to claim 5, characterized in that The method further comprises: The action probability corresponding to the invalid action in the output of the neural network is set to a preset value.

7. The method according to claim 5, characterized in that The cluster status includes at least one of the following: The remaining allocatable CPU and memory resources of the nodes in the cluster, the remaining available rates of disk inflow and outflow IO, the remaining available rates of network inflow and outflow IO, and the resource usage of the nodes.

8. A resource scheduling device, characterized in that: include: The identification module is used to identify the corresponding instance identification ID, shard ID and PodID label for each Pod in multiple instances; A determination module, used to determine a target node corresponding to the Pod to be scheduled in the Pods of the multiple instances, wherein the target node is a node with the highest reward value among the multiple nodes determined based on the reward function; The reward function is used to determine reward values ​​of multiple nodes based on the labels corresponding to each Pod, and the multiple nodes include the target node.

9. An electronic device, characterized in that: The invention comprises a transceiver and a processor, wherein the processor is configured to: Identify the corresponding instance ID, shard ID, and Pod ID label for each Pod in multiple instances; Determine a target node corresponding to the Pod to be scheduled in the Pods of the multiple instances, wherein the target node is a node with the highest reward value among the multiple nodes determined based on the reward function; The reward function is used to determine reward values ​​of multiple nodes based on the labels corresponding to each Pod, and the multiple nodes include the target node.

10. An electronic device, characterized in that: include: A processor, a memory, and a program stored in the memory and executable on the processor, wherein the program, when executed by the processor, implements the steps of the resource scheduling method according to any one of claims 1 to 7.

11. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the resource scheduling method according to any one of claims 1 to 7 are implemented.

12. A computer program product, characterized in that The method comprises computer instructions, which, when executed by a processor, implement the steps of the resource scheduling method according to any one of claims 1 to 7.

Images