A resource scheduling method, device, equipment and medium based on immune algorithm

Abstract

The present invention provides a resource scheduling method, device, equipment and medium based on an immune algorithm, maps all K8s Pod sets required for a resource scheduling task to a set of working nodes, thereby constructing a target strategy matrix; constructing a Pod node affinity objective function according to the target strategy matrix, and constructing a resource loss rate objective function; defining the objective function and constraint conditions for resource scheduling of the Pod, and using the Pareto optimal frontier to construct a resource scheduling target model; defining an antigen as an optimization problem, an antibody as a problem solution, and the number of antigens as the number of optimized sub-goals, defining the fitness calculation, antibody concentration calculation and antibody mutation probability of the antibody in the immune algorithm, and using the immune algorithm to solve the resource scheduling target model. The advantages of the present invention are: it can achieve better allocation of resources under limited resources, improve resource utilization and system throughput.

Description

Technical Field

[0001] The present invention relates to the technical field of resource scheduling, and in particular to a resource scheduling method, device, equipment and medium based on immune algorithm. Background Art

[0002] Immune algorithm is a computational method that simulates the mechanism of biological immune system. It draws on the interaction principle between antibodies and antigens in the immune system of organisms. In the biological immune system, antibodies can recognize and neutralize foreign antigens (such as viruses, bacteria, etc.). This process involves biological phenomena such as antibody production, selection, cloning and mutation. Immune algorithm abstracts these biological principles into algorithmic operations to solve optimization problems.

[0003] In the field of resource scheduling, especially in the application scenarios of video middleware, video surveillance and image processing tasks are increasing, and the demand for computing resources is also rising. In the actual allocation of resources, how to achieve better allocation of resources under limited resources and improve resource utilization and system throughput is an urgent problem to be solved. In view of the above-mentioned problems, the inventor of this case conducted an in-depth study on this issue, which led to the creation of this case. Summary of the invention

[0004] The technical problem to be solved by the present invention is to provide a resource scheduling method, device, equipment and medium based on an immune algorithm, which can achieve better allocation of resources under limited resources and improve resource utilization and system throughput.

[0005] In a first aspect, the present invention provides a resource scheduling method based on an immune algorithm, the method comprising the following steps:

[0006] Map all K8s Pod sets required for resource scheduling tasks to the set of working nodes to build a target policy matrix;

[0007] A Pod node affinity objective function and a resource loss rate objective function are constructed according to the target strategy matrix;

[0008] Define the objective function and constraints for Pod resource scheduling, and use the Pareto optimal frontier to build a resource scheduling target model; define the antigen as the optimization problem, the antibody as the problem solution, the number of antigens as the number of optimization sub-goals, define the fitness calculation, antibody concentration calculation and antibody mutation probability of the antibody in the immune algorithm, and use the immune algorithm to solve the resource scheduling target model.

[0009] In a second aspect, the present invention provides a resource scheduling device based on an immune algorithm, the device comprising a matrix building module, an objective function building module and a resource scheduling module;

[0010] The matrix construction module is used to map all K8s Pod sets required for resource scheduling tasks to work node sets, thereby constructing a target policy matrix;

[0011] The objective function construction module is used to construct a Pod node affinity objective function and a resource loss rate objective function according to the target strategy matrix;

[0012] The resource scheduling module is used to define the objective function and constraints for resource scheduling of Pods, and use the Pareto optimal frontier to build a resource scheduling target model; antigens are defined as optimization problems, antibodies are defined as problem solutions, the number of antigens is the number of optimization sub-goals, the fitness calculation, antibody concentration calculation and antibody mutation probability of antibodies in the immune algorithm are defined, and the immune algorithm is used to solve the resource scheduling target model.

[0013] In a third aspect, the present invention provides an electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the method described in the first aspect when executing the program.

[0014] In a fourth aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method described in the first aspect.

[0015] One or more technical solutions provided in the embodiments of the present invention have at least the following technical effects or advantages: by adopting the Pareto optimal frontier to integrate the two evaluation indicators of the Pod node affinity objective function and the resource loss rate objective function, and combining the defined constraints to construct a resource scheduling target model; at the same time, the basic idea of ​​the immune algorithm is used to transform the optimization problem into a process of interaction between antibodies and antigens, and the immune algorithm is used to solve the resource scheduling target model, so as to obtain the Pareto optimal solution in resource scheduling. Because the immune algorithm has the advantages of adaptability, strong global search capability, and fast convergence speed, by adopting the technical solution of the present invention, it is possible to achieve better allocation of resources under limited resources, and while ensuring high availability and load balancing of tasks, improve resource utilization and system throughput, thereby effectively improving the efficiency and performance of the entire resource scheduling system.

[0016] The above description is only an overview of the technical solution of the present invention. In order to more clearly understand the technical means of the present invention, it can be implemented according to the contents of the specification. In order to make the above and other purposes, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The present invention will be further described below in conjunction with embodiments with reference to the accompanying drawings.

[0018] Figure 1 is a flowchart of the execution of the method in Embodiment 1 of the present invention;

[0019] Figure 2 It is a structural schematic diagram of the device in the second embodiment of the present invention;

[0020] Figure 3 This is a schematic diagram of the structure of an electronic device in Embodiment 3 of the present invention;

[0021] Figure 4 Schematic diagram of the structure of the medium in the fourth embodiment of the present invention. DETAILED DESCRIPTION

[0022] Embodiment 1

[0023] This embodiment provides a resource scheduling method based on immune algorithm, such as Figure 1 As shown, the method comprises the following steps:

[0024] Step S1: Map all K8s Pod sets required for resource scheduling tasks to the set of working nodes, thereby constructing a target policy matrix; wherein:

[0025] K8s (Kubernetes) is Google's open source container cluster management system. Based on Docker technology, it provides a series of complete functions such as deployment and operation, resource scheduling, service discovery and dynamic scaling for containerized applications, which improves the convenience of large-scale container cluster management. Pod is a combination of several related containers running on the Node node. The containers contained in the Pod run on the same host machine, use the same network namespace, IP address and port, and can communicate through localhost; Pod is the smallest unit for Kubernetes to create, schedule and manage. It provides a higher level of abstraction than containers, making deployment and management more flexible. A Pod can contain one container or multiple related containers;

[0026] Step S2: constructing a Pod node affinity objective function and a resource loss rate objective function according to the target strategy matrix;

[0027] Since specific business characteristics need to be considered during Pod scheduling, two optimization goals are mainly focused on. One is the maximum resource utilization of the working node, and the other is the Pod affinity on the working node. That is, the scheduling must meet the principle of minimizing the use of resources for the specified business goals. This optimization goal requires maximizing the Pod affinity on the working node. For this reason, the present invention proposes a Pod node affinity objective function, which is an evaluation indicator for evaluating resource usage.

[0028] At the same time, since the main cause of resource loss is the unusable fragments generated during the resource allocation process, when the resource ratios of different resources differ greatly from the resource ratios used by containers, the possibility of fragment generation will also increase. Therefore, one of the major goals of container allocation is to improve resource utilization. To this end, the present invention proposes a resource loss rate objective function, which is an evaluation indicator for evaluating resource utilization.

[0029] Step S3, define the objective function and constraints for resource scheduling of Pod, and use the Pareto optimal frontier to build a resource scheduling target model; define the antigen as the optimization problem, the antibody as the problem solution, the number of antigens as the number of optimization sub-goals, define the fitness calculation, antibody concentration calculation and antibody mutation probability of the antibody in the immune algorithm, and use the immune algorithm to solve the resource scheduling target model.

[0030] The present invention adopts the Pareto optimal frontier to integrate the two evaluation indicators of the Pod node affinity objective function and the resource loss rate objective function, and constructs a resource scheduling target model in combination with the defined constraints; at the same time, the basic idea of ​​the immune algorithm is used to transform the optimization problem into a process of interaction between antibodies and antigens, and the immune algorithm is used to solve the resource scheduling target model, so as to obtain the Pareto optimal solution in resource scheduling. Since the immune algorithm has the advantages of adaptability, strong global search capability, fast convergence speed, etc., by adopting the technical solution of the present invention, it is possible to achieve better allocation of resources under limited resources, and while ensuring high availability and load balancing of tasks, improve resource utilization and system throughput, thereby effectively improving the efficiency and performance of the entire resource scheduling system.

[0031] In an embodiment of the present invention, the mapping of all K8s Pod sets required for resource scheduling tasks to a set of working nodes to construct a target policy matrix specifically includes:

[0032] Map all K8s Pod sets required for resource scheduling tasks to the set of working nodes to construct the target strategy matrix STRATEGY. The specific definition of the target strategy matrix STRATEGY is as follows:

[0033]

[0034] In formula (1), S NP Represents the relationship between the working node N and the Pod P node. When S NP 1 means that Pod P node is allocated to worker node N. NPA value of 0 indicates that there is no allocation relationship between worker node N and Pod node P; N indicates that there are N worker nodes in total, and n indicates any worker node between 1 and N; P indicates that there are P Pods in total, and p indicates any Pod node between 1 and P.

[0035] When the present invention is implemented specifically, the main goal that the resource scheduling algorithm needs to achieve is to optimize the target strategy matrix STRATEGY, that is, to find the optimal solution in the solution space; when solving the target strategy matrix STRATEGY, the following requirements need to be met: the resources on each working node are limited, and the total resources of the resident and running Pod need to be constrained.

[0036] In an embodiment of the present invention, constructing a Pod node affinity objective function according to the target strategy matrix specifically includes:

[0037] When allocating Pods for solving the resource scheduling task, the affinity between Pods is defined according to the target strategy matrix STRATEGY as follows (2):

[0038]

[0039] In formula (2), x ij Represents the relationship between the i-th Pod and the j-th Pod in the Pod relationship matrix, where the Pod relationship matrix refers to the S in the target strategy matrix. NP ; P represents the number of Pods; AFFINITY represents the value of normalizing the affinity value. Because the result of the affinity sum will vary around a value greater than 1, it is necessary to use normalization to adjust the value to between 0 and 1; f(x ij )’s affinity calculation rules are as follows:

[0040] The affinity of homogeneous Pod nodes on the same worker node is -1, the affinity of associated Pod nodes on different worker nodes is 2, and the affinity between Pod nodes in other cases is 0.

[0041] For example, suppose that resource scheduling tasks A, B, and C require m, n, and l Pod containers to provide services respectively, among which resource scheduling task B needs to call the service of A; according to the principle of breaking up homogeneous Pods and aggregating related Pods, f(x ij )’s affinity calculation rules are as follows:

[0042] (1) Homogeneous Pod nodes belonging to the same resource scheduling task A or B, if they are on the same working node, are as follows (2.1):

[0043]

[0044] (2) If the associated Pod nodes of resource scheduling tasks A and B are located on different working nodes, then the following formula (2.2) is used:

[0045]

[0046] (3) The affinity between Pod nodes in other cases is as follows (2.3):

[0047] f(x ij )=0,f(x ji )=0(2.3)

[0048] In the present invention, the resource loss rate objective function represents the resource utilization rate by defining the resource loss degree. Therefore, the construction of the resource loss rate objective function specifically includes:

[0049] When allocating Pods to solve the resource scheduling task, the resource utilization rate is converted into the node resource loss rate. The calculation formula of the node resource loss rate is as follows (3):

[0050]

[0051] In formula (3), ULOST n Indicates the resource loss rate on working node n; Indicates the amount of resource loss on working node n; Represents the total amount of resources on the working node n; the calculation of the node resource loss rate depends on the resource utilization on the node. When there are unusable resource fragments on the node, the resource loss rate will increase, thus reflecting the waste of resources.

[0052] In the specific implementation of the present invention, the judgment standard for whether it is resource fragmentation is to use the minimum resource of the Pod as the threshold. For example, the minimum resource of the Pod is 2G. If the memory remaining of the current node is less than 2G, then the remaining resources of the current node are resource fragments. At this time, the resource fragments are marked as unusable.

[0053] In a multi-objective optimization problem, the Pareto optimal frontier is a set of all optimal solutions. The present invention constructs a resource scheduling target model by multiplying the multi-objective function. The resource scheduling target model can be described by a mathematical formula to ensure that resource utilization and efficiency are maximized while satisfying resource constraints. To construct a resource scheduling target model, it is necessary to first define the resource scheduling target and constraint conditions, and transform the resource scheduling problem into an optimization problem model, where the specific target and constraint conditions are as follows (15):

[0054]

[0055] In formula (15), Indicates that j is an integer, j represents the jth working node; N represents the total number of working nodes; i represents the i-th Pod; m represents the number of Pods on working node j; resource(S jp ) represents the resources required by the Podp node residing on the working node j; resource(Node j ) represents the total resources on worker node j.

[0056] Based on the above analysis, the objective function and constraints for resource scheduling of Pods are defined, and the resource scheduling target model is constructed using the Pareto optimal frontier, which specifically includes:

[0057] The objective function and constraints for resource scheduling of Pod are defined as follows:

[0058]

[0059] f2(x)=-(∑∑AFFINITY ij ) (5)

[0060] minF(x)=min(f1(x),f2(x)) (6)

[0061]

[0062] In equations (4) to (7), ULOST n AFFINITY represents the resource loss rate on worker node n, where N represents the number of worker nodes; ij represents the affinity between Pod node i and Pod node j; x = (x1, x2, ..., x k ), represents the decision vector x consisting of k decision variables; θ is the decision space, θ∈STRATEGY np ; Represents the resource units occupied by k Pods residing on worker node n, NR n is the total resource unit owned by worker node n, n represents worker node, and m represents Pod node; f1(x) is the first objective function, which represents the average resource loss rate of all worker nodes; f2(x) is the second objective function, which represents the inverse of the affinity of all worker nodes; F(x) represents the final optimization objective function; minF(x) represents the optimal objective solution before converting the minimum value problem into a maximum value problem;

[0063] The Pareto optimal frontier is used to multiply multiple objective functions to construct a resource scheduling target model. In order to reduce the large differences between the objective functions, resulting in the small impact of other objective functions on the final result, the present invention adopts normalization processing of the relevant objective functions, as shown in the following formulas (8) to (10):

[0064]

[0065] In equations (8)-(10), D1 and D2 are both constants. D1 and D2 are used to convert the minimum problem into the maximum problem. The values ​​of D1 and D2 are determined by observing the values ​​in the experiment so that the optimal values ​​of the two optimization objectives are not negative; x represents the decision vector; β is a parameter for adjusting the weight, which is used to ensure that the extreme value of the affinity target is less than 1 and is an experimental parameter; f1(x) is the first objective function, which represents the average resource loss rate of all working nodes; f2(x) is the second objective function, which represents the inversion of the affinity of all working nodes; represents the normalized value of the first objective function, represents the normalized value of the second objective function; F(x) in formula (10) and F(x) in formula (6) both represent the final optimization objective function, where F(x) in formula (10) is the final optimization objective function after Pareto optimal frontier transformation, and F(x) in formula (6) is the final optimization objective function before Pareto optimal frontier transformation; maxF(x) represents the optimal objective solution.

[0066] In an embodiment of the present invention, the definition of the fitness calculation, antibody concentration calculation and antibody mutation probability of the antibody in the immune algorithm specifically includes:

[0067] Assume that there are H antibodies, each antibody has M gene positions, and each gene position has S discrete values ​​to choose from (that is, each gene position has k1, k2, ..., k s There are S discrete values ​​to choose from), then the fitness of the antibody is defined as the following formula (11):

[0068] fit j =((D1-f1(x i ))*0.5)*(D2-f2(x i )) (11)

[0069] In formula (11), j = {1,...H}, i = {1,...S}; in the immune algorithm iteration, fit j The maximum value of is taken as the solution; D1, D2, f1(x) and f2(x) have the same meanings as those in the above equations (8)-(10).

[0070] The concentration of the jth antibody is defined as the following formula (12):

[0071]

[0072] In formula (12), H represents H antibodies, X jv represents the density of the j-th antibody, |X jv |2 represents the second-order norm of antibody v in the jth class of antibodies;

[0073] The concentration of an antibody refers to the number or density of a certain type of antibody in an antibody population, that is, the frequency of the solution appearing in the search space.

[0074] Assume that the algorithm algebra is g, antibody X i The gene position Φ i The mutation probability of is controlled by the following equations (13) and (14):

[0075]

[0076] In equations (13) and (14), P i Represents antibody X i The mutation probability; α represents a constant; random represents a random number, and others represents a conditional judgment; i = {1,...,M}, when Φ(i) is 1, it means that the gene position is selected for mutation, and the mutation constraint is in, Represents the resource units occupied by k Pods residing on worker node n, NR n It represents the total resource units owned by the working node n, n represents the working node, m represents the Pod node, PR represents the resources occupied by the Pod, and NR represents the total resources of the working node; It should be noted that: In formula (13), 1. indicates that this is a floating point number;

[0077] The mutation probability of an antibody refers to the probability that an antibody mutates when a new antibody is generated; antibody mutation refers to the generation of a new antibody based on an antibody by introducing some random changes; mutation usually includes operations such as gene mutation and gene recombination, which can make the newly generated antibody have better fitness, thereby promoting the convergence of the search algorithm.

[0078] In an embodiment of the present invention, the use of an immune algorithm to solve the resource scheduling target model specifically includes:

[0079] First, initialization: randomly generate a set of initial solutions as the population, and calculate the fitness of each solution;

[0080] Second, selection: Calculate the antibody concentration according to the fitness of each solution and sort them, and select a part of the solutions with the best fitness as antibodies for the subsequent evolution process;

[0081] Third, cloning: cloning the selected antibodies to produce a group of cloned antibodies;

[0082] Fourth, mutation: Perform random mutation operations on cloned antibodies to generate new solutions, where the random mutation operations include at least one of the following: node selectors and labels mutation, affinity and anti-affinity rule mutation, taints and tolerances mutation, resource requests and limits mutation;

[0083] Fifth, optimization: According to the fitness of each solution, select some excellent solutions to add to the next generation population;

[0084] Sixth, termination condition: When the preset number of iterations or target fitness is reached, the immune algorithm stops and the optimal solution is output. The specific implementation code is as follows:

[0085] Initialize the population;

[0086] Initialize the iteration counter L and set the target fitness Q;

[0087] Calculate the fitness value fit;

[0088] While(L>0or fit <Q)

[0089] { / / Iteration starts

[0090] Calculate the affinity of the Pod node;

[0091] / *Calculate antibody concentration* /

[0092] Calculate antibody concentration based on fitness;

[0093] Sort and get the population sort list new_pods;

[0094] / *Select operation* /

[0095] Select multiple optimal solutions from new_pods and clone and evolve;

[0096] / *Mutation operation* /

[0097] According to a certain probability Pi, individuals are mutated;

[0098] / *Evaluation operation* /

[0099] Calculate the individual's fitness fit;

[0100] / *Immune selection operation* /

[0101] Compare the fitness of an individual with the fitness of similar individuals;

[0102] According to the rules, take the maximum value as the generation;

[0103] / *Update operation* /

[0104] Update the population and generate the next generation;

[0105] L = L-1;

[0106] } / / Iteration ends;

[0107] Output the optimal solution.

[0108] Based on the same inventive concept, the present application also provides a device corresponding to the method in Example 1, see Example 2 for details.

[0109] Embodiment 2

[0110] In this embodiment, a resource scheduling device based on an immune algorithm is provided. Figure 2 As shown, the device includes a matrix construction module, an objective function construction module and a resource scheduling module;

[0111] The matrix construction module is used to map all K8s Pod sets required for resource scheduling tasks to the work node set, thereby constructing a target policy matrix; wherein:

[0112] K8s (Kubernetes) is Google's open source container cluster management system. Based on Docker technology, it provides a series of complete functions such as deployment and operation, resource scheduling, service discovery and dynamic scaling for containerized applications, which improves the convenience of large-scale container cluster management. Pod is a combination of several related containers running on the Node node. The containers contained in the Pod run on the same host machine, use the same network namespace, IP address and port, and can communicate through localhost; Pod is the smallest unit for Kubernetes to create, schedule and manage. It provides a higher level of abstraction than containers, making deployment and management more flexible. A Pod can contain one container or multiple related containers;

[0113] The objective function construction module is used to construct a Pod node affinity objective function and a resource loss rate objective function according to the target strategy matrix;

[0114] Since specific business characteristics need to be considered during Pod scheduling, two optimization goals are mainly focused on. One is the maximum resource utilization of the working node, and the other is the Pod affinity on the working node. That is, the scheduling must meet the principle of minimizing the use of resources for the specified business goals. This optimization goal requires maximizing the Pod affinity on the working node. For this reason, the present invention proposes a Pod node affinity objective function, which is an evaluation indicator for evaluating resource usage.

[0115] At the same time, since the main cause of resource loss is the unusable fragments generated during the resource allocation process, when the resource ratios of different resources differ greatly from the resource ratios used by containers, the possibility of fragment generation will also increase. Therefore, one of the major goals of container allocation is to improve resource utilization. To this end, the present invention proposes a resource loss rate objective function, which is an evaluation indicator for evaluating resource utilization.

[0116] The resource scheduling module is used to define the objective function and constraints for resource scheduling of Pods, and use the Pareto optimal frontier to build a resource scheduling target model; antigens are defined as optimization problems, antibodies are defined as problem solutions, the number of antigens is the number of optimization sub-goals, the fitness calculation, antibody concentration calculation and antibody mutation probability of antibodies in the immune algorithm are defined, and the immune algorithm is used to solve the resource scheduling target model.

[0117] The present invention adopts the Pareto optimal frontier to integrate the two evaluation indicators of the Pod node affinity objective function and the resource loss rate objective function, and constructs a resource scheduling target model in combination with the defined constraints; at the same time, the basic idea of ​​the immune algorithm is used to transform the optimization problem into a process of interaction between antibodies and antigens, and the immune algorithm is used to solve the resource scheduling target model, so as to obtain the Pareto optimal solution in resource scheduling. Since the immune algorithm has the advantages of adaptability, strong global search capability, fast convergence speed, etc., by adopting the technical solution of the present invention, it is possible to achieve better allocation of resources under limited resources, and while ensuring high availability and load balancing of tasks, improve resource utilization and system throughput, thereby effectively improving the efficiency and performance of the entire resource scheduling system.

[0118] In an embodiment of the present invention, the matrix construction module is specifically used for:

[0119] Map all K8s Pod sets required for resource scheduling tasks to the set of working nodes to construct the target strategy matrix STRATEGY. The specific definition of the target strategy matrix STRATEGY is as follows:

[0120]

[0121] In formula (1), S NPRepresents the relationship between the working node N and the Pod P node. When S NP 1 means that Pod P node is allocated to worker node N. NP A value of 0 indicates that there is no allocation relationship between worker node N and Pod node P; N indicates that there are N worker nodes in total, and n indicates any worker node between 1 and N; P indicates that there are P Pods in total, and p indicates any Pod node between 1 and P.

[0122] When the present invention is implemented specifically, the main goal that the resource scheduling algorithm needs to achieve is to optimize the target strategy matrix STRATEGY, that is, to find the optimal solution in the solution space; when solving the target strategy matrix STRATEGY, the following requirements need to be met: the resources on each working node are limited, and the total resources of the resident and running Pod need to be constrained.

[0123] In an embodiment of the present invention, in the objective function construction module, constructing the Pod node affinity objective function according to the target strategy matrix specifically includes:

[0124] When allocating Pods for solving the resource scheduling task, the affinity between Pods is defined according to the target strategy matrix STRATEGY as follows (2):

[0125]

[0126] In formula (2), x ij Represents the relationship between the i-th Pod and the j-th Pod in the Pod relationship matrix, where the Pod relationship matrix refers to the S in the target strategy matrix. NP ; P represents the number of Pods; AFFINITY represents the value of normalizing the affinity value. Because the result of the affinity sum will vary around a value greater than 1, it is necessary to use normalization to adjust the value to between 0 and 1; f(x ij )’s affinity calculation rules are as follows:

[0127] The affinity of homogeneous Pod nodes on the same worker node is -1, the affinity of associated Pod nodes on different worker nodes is 2, and the affinity between Pod nodes in other cases is 0.

[0128] For example, suppose that resource scheduling tasks A, B, and C require m, n, and l Pod containers to provide services respectively, among which resource scheduling task B needs to call the service of A; according to the principle of breaking up homogeneous Pods and aggregating related Pods, f(x ij )’s affinity calculation rules are as follows:

[0129] (1) Homogeneous Pod nodes belonging to the same resource scheduling task A or B, if they are on the same working node, are as follows (2.1):

[0130]

[0131] (2) If the associated Pod nodes of resource scheduling tasks A and B are located on different working nodes, then the following formula (2.2) is used:

[0132]

[0133] (3) The affinity between Pod nodes in other cases is as follows (2.3):

[0134] f(x ij )=0,f(x ji )=0(2.3)

[0135] In the present invention, in the objective function construction module, the resource loss rate objective function represents the resource utilization rate by defining the resource loss degree. Therefore, the construction of the resource loss rate objective function specifically includes:

[0136] When allocating Pods to solve the resource scheduling task, the resource utilization rate is converted into the node resource loss rate. The calculation formula of the node resource loss rate is as follows (3):

[0137]

[0138] In formula (3), ULOST n Indicates the resource loss rate on working node n; Indicates the amount of resource loss on working node n; Represents the total amount of resources on the working node n; the calculation of the node resource loss rate depends on the resource utilization on the node. When there are unusable resource fragments on the node, the resource loss rate will increase, thus reflecting the waste of resources.

[0139] In the specific implementation of the present invention, the judgment standard for whether it is resource fragmentation is to use the minimum resource of the Pod as the threshold. For example, the minimum resource of the Pod is 2G. If the memory remaining of the current node is less than 2G, then the remaining resources of the current node are resource fragments. At this time, the resource fragments are marked as unusable.

[0140] In a multi-objective optimization problem, the Pareto optimal frontier is a set of all optimal solutions. The present invention constructs a resource scheduling target model by multiplying the multi-objective function. The resource scheduling target model can be described by a mathematical formula to ensure that resource utilization and efficiency are maximized while satisfying resource constraints. To construct a resource scheduling target model, it is necessary to first define the resource scheduling target and constraint conditions, and transform the resource scheduling problem into an optimization problem model, where the specific target and constraint conditions are as follows (15):

[0141]

[0142] In formula (15), Indicates that j is an integer, j represents the jth working node; N represents the total number of working nodes; i represents the i-th Pod; m represents the number of Pods on working node j; resource(S jp ) represents the resources required by the Podp node residing on the working node j; resource(Node j ) represents the total resources on worker node j.

[0143] Based on the above analysis, in the resource scheduling module, the objective function and constraints for defining resource scheduling for Pods, and the resource scheduling target model constructed using the Pareto optimal frontier specifically include:

[0144] The objective function and constraints for resource scheduling of Pod are defined as follows:

[0145]

[0146] f2(x)=-(∑∑AFFINITY ij ) (5)

[0147] minF(x)=min(f1(x),f2(x)) (6)

[0148]

[0149] In equations (4) to (7), ULOST n AFFINITY represents the resource loss rate on worker node n, where N represents the number of worker nodes; ij represents the affinity between Pod node i and Pod node j; x = (x1, x2, ..., x k ), represents the decision vector x consisting of k decision variables; θ is the decision space, θ∈STRATEGY np ; Represents the resource units occupied by k Pods residing on worker node n, NRn is the total resource unit owned by worker node n, n represents worker node, and m represents Pod node; f1(x) is the first objective function, which represents the average resource loss rate of all worker nodes; f2(x) is the second objective function, which represents the inverse of the affinity of all worker nodes; F(x) represents the final optimization objective function; minF(x) represents the optimal objective solution before converting the minimum value problem into a maximum value problem;

[0150] The Pareto optimal frontier is used to multiply multiple objective functions to construct a resource scheduling target model. In order to reduce the large differences between the objective functions, resulting in the small impact of other objective functions on the final result, the present invention adopts normalization processing of the relevant objective functions, as shown in the following formulas (8) to (10):

[0151]

[0152] In equations (8)-(10), D1 and D2 are both constants. D1 and D2 are used to convert the minimum problem into the maximum problem. The values ​​of D1 and D2 are determined by observing the values ​​in the experiment so that the optimal values ​​of the two optimization objectives are not negative; x represents the decision vector; β is a parameter for adjusting the weight, which is used to ensure that the extreme value of the affinity target is less than 1 and is an experimental parameter; f1(x) is the first objective function, which represents the average resource loss rate of all working nodes; f2(x) is the second objective function, which represents the inversion of the affinity of all working nodes; represents the normalized value of the first objective function, represents the normalized value of the second objective function; F(x) in formula (10) and F(x) in formula (6) both represent the final optimization objective function, where F(x) in formula (10) is the final optimization objective function after Pareto optimal frontier transformation, and F(x) in formula (6) is the final optimization objective function before Pareto optimal frontier transformation; maxF(x) represents the optimal objective solution.

[0153] In an embodiment of the present invention, in the resource scheduling module, the definition of the fitness calculation, antibody concentration calculation and antibody mutation probability of the antibody in the immune algorithm specifically includes:

[0154] Assume that there are H antibodies, each antibody has M gene positions, and each gene position has S discrete values ​​to choose from (that is, each gene position has k1, k2, ..., k s There are S discrete values ​​to choose from), then the fitness of the antibody is defined as the following formula (11):

[0155] fit j =((D1-f1(x i))*0.5)*(D2-f2(x i )) (11)

[0156] In formula (11), j = {1,...H}, i = {1,...S}; in the immune algorithm iteration, fit j The maximum value of is taken as the solution; D1, D2, f1(x) and f2(x) have the same meanings as those in the above equations (8)-(10).

[0157] The concentration of the jth antibody is defined as the following formula (12):

[0158]

[0159] In formula (12), H represents H antibodies, X jv represents the density of the j-th antibody, |X jv |2 represents the second-order norm of antibody v in the jth class of antibodies;

[0160] The concentration of an antibody refers to the number or density of a certain type of antibody in an antibody population, that is, the frequency of the solution appearing in the search space.

[0161] Assume that the algorithm algebra is g, antibody X i The gene position Φ i The mutation probability of is controlled by the following equations (13) and (14):

[0162]

[0163]

[0164] In equations (13) and (14), P i Represents antibody X i The mutation probability; α represents a constant; random represents a random number, and others represents a conditional judgment; i = {1,...,M}, when Φ(i) is 1, it means that the gene position is selected for mutation, and the mutation constraint is in, Represents the resource units occupied by k Pods residing on worker node n, NR n It represents the total resource units owned by the working node n, n represents the working node, m represents the Pod node, PR represents the resources occupied by the Pod, and NR represents the total resources of the working node; It should be noted that: In formula (13), 1. indicates that this is a floating point number;

[0165] The mutation probability of an antibody refers to the probability that an antibody mutates when a new antibody is generated; antibody mutation refers to the generation of a new antibody based on an antibody by introducing some random changes; mutation usually includes operations such as gene mutation and gene recombination, which can make the newly generated antibody have better fitness, thereby promoting the convergence of the search algorithm.

[0166] In an embodiment of the present invention, in the resource scheduling module, the use of the immune algorithm to solve the resource scheduling target model specifically includes:

[0167] First, initialization: randomly generate a set of initial solutions as the population, and calculate the fitness of each solution;

[0168] Second, selection: Calculate the antibody concentration according to the fitness of each solution and sort them, and select a part of the solutions with the best fitness as antibodies for the subsequent evolution process;

[0169] Third, cloning: cloning the selected antibodies to produce a group of cloned antibodies;

[0170] Fourth, mutation: Perform random mutation operations on cloned antibodies to generate new solutions, where the random mutation operations include at least one of the following: node selectors and labels mutation, affinity and anti-affinity rule mutation, taints and tolerances mutation, resource requests and limits mutation;

[0171] Fifth, optimization: According to the fitness of each solution, select some excellent solutions to add to the next generation population;

[0172] Sixth, termination condition: When the preset number of iterations or target fitness is reached, the immune algorithm stops and the optimal solution is output. The specific implementation code is as follows:

[0173] Initialize the population;

[0174] Initialize the iteration counter L and set the target fitness Q;

[0175] Calculate the fitness value fit;

[0176] While(L>0or fit <Q)

[0177] { / / Iteration starts

[0178] Calculate the affinity of the Pod node;

[0179] / *Calculate antibody concentration* /

[0180] Calculate antibody concentration based on fitness;

[0181] Sort and get the population sort list new_pods;

[0182] / *Select operation* /

[0183] Select multiple optimal solutions from new_pods and clone and evolve;

[0184] / *Mutation operation* /

[0185] According to a certain probability Pi, individuals are mutated;

[0186] / *Evaluation operation* /

[0187] Calculate the individual's fitness fit;

[0188] / *Immune selection operation* /

[0189] Compare the fitness of an individual with the fitness of similar individuals;

[0190] According to the rules, take the maximum value as the generation;

[0191] / *Update operation* /

[0192] Update the population and generate the next generation;

[0193] L = L-1;

[0194] } / / Iteration ends;

[0195] Output the optimal solution.

[0196] Based on the same inventive concept, the present application provides an electronic device embodiment corresponding to the first embodiment, see the third embodiment for details.

[0197] Embodiment 3

[0198] This embodiment provides an electronic device, such as Figure 3 As shown, it includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, any implementation method in the first embodiment can be implemented.

[0199] Since the electronic device introduced in this embodiment is a device used to implement the method in the first embodiment of the present application, based on the method introduced in the first embodiment of the present application, a person skilled in the art can understand the specific implementation of the electronic device of the present embodiment and its various variations, so how the electronic device implements the method in the embodiment of the present application is not described in detail here. As long as a person skilled in the art implements the device used by the method in the embodiment of the present application, it belongs to the scope of protection of the present application.

[0200] Based on the same inventive concept, the present application provides a storage medium corresponding to the first embodiment, see the fourth embodiment for details.

[0201] Embodiment 4

[0202] This embodiment provides a computer-readable storage medium, such as Figure 4 As shown, a computer program is stored thereon, and when the computer program is executed by a processor, any implementation method in Example 1 can be implemented.

[0203] It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as methods, devices, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0204] The present invention is described with reference to flowcharts and / or block diagrams of methods, apparatuses, and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as a combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the process in the flowchart. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0205] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0206] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1A step that specifies a function in one or more boxes.

[0207] Although the specific implementation modes of the present invention are described above, those skilled in the art should understand that the specific implementation modes described are only illustrative and are not intended to limit the scope of the present invention. Equivalent modifications and changes made by those skilled in the art in accordance with the spirit of the present invention should be included in the scope of protection of the claims of the present invention.

Claims

1. A resource scheduling method based on immune algorithm, characterized in that: The method comprises the following steps: Map all K8s Pod sets required for resource scheduling tasks to the set of working nodes, so as to construct a target strategy matrix. Specifically, map all K8s Pod sets required for resource scheduling tasks to the set of working nodes, so as to construct a target strategy matrix STRATEGY. The specific definition of the target strategy matrix STRATEGY is as follows: In formula (1), S NP Represents the relationship between the working node N and the Pod P node. When S NP 1 means that Pod P node is allocated to worker node N. NP 0 means that there is no allocation relationship between working node N and Pod P node; N means there are N working nodes in total, n means any working node between 1 and N; P means there are P Pods in total, p means any Pod node between 1 and P; According to the target strategy matrix, a Pod node affinity objective function and a resource loss rate objective function are constructed, which specifically include: when solving the resource scheduling task and allocating Pods, according to the target strategy matrix, the affinity between Pods is defined as the following formula (2): In formula (2), x ij Represents the relationship between the i-th Pod and the j-th Pod in the Pod relationship matrix, where the Pod relationship matrix refers to the S in the target strategy matrix. NP ; P represents the number of Pods; f(x ij )’s affinity calculation rules are as follows: The affinity of homogeneous Pod nodes on the same worker node is -1, the affinity of associated Pod nodes on different worker nodes is 2, and the affinity between Pod nodes in other cases is 0; When allocating Pods to solve the resource scheduling task, the resource utilization rate is converted into the node resource loss rate. The calculation formula of the node resource loss rate is as follows (3): In formula (3), ULOST n Indicates the resource loss rate on working node n; Resource lostn Indicates the amount of resource loss on working node n; Resource totaln Indicates the total amount of resources on working node n; Define the objective function and constraints for Pod resource scheduling, and use the Pareto optimal frontier to build a resource scheduling target model; define the antigen as the optimization problem, the antibody as the problem solution, the number of antigens as the number of optimization sub-goals, define the fitness calculation, antibody concentration calculation and antibody mutation probability of the antibody in the immune algorithm, and use the immune algorithm to solve the resource scheduling target model.

2. The resource scheduling method based on immune algorithm according to claim 1, characterized in that: The objective function and constraints for resource scheduling of Pods are defined, and the resource scheduling objective model is constructed using the Pareto optimal frontier, specifically including: The objective function and constraints for resource scheduling of Pod are defined as follows: f2(x)=-(∑∑AFFINITY ij ) (5) min F(x)=min(f1(x),f2(x)) (6) In equations (4) to (7), ULOST n AFFINITY represents the resource loss rate on worker node n, where N represents the number of worker nodes; ij represents the affinity between Pod node i and Pod node j; x = (x1, x2, ..., x k ), represents the decision vector x consisting of k decision variables; θ is the decision space, θ∈STRATEGY np ; Represents the resource units occupied by k Pods residing on worker node n, NR n is the total resource unit owned by worker node n, where n represents worker node and m represents Pod node; f1(x) is the first objective function, which represents the average resource loss rate of all worker nodes; f2(x) is the second objective function, which represents the inverse of the affinity of all worker nodes; F(x) represents the final optimization objective function; The Pareto optimal frontier is used to multiply the multi-objective functions to construct a resource scheduling target model, as shown in the following formulas (8) to (10): In equations (8)-(10), D1 and D2 are both constants. D1 and D2 are used to convert the minimum problem into the maximum problem. The values ​​of D1 and D2 are determined by observing the values ​​in the experiment so that the optimal values ​​of the two optimization objectives are not negative; x represents the decision vector; β is a parameter for adjusting the weight, which is used to ensure that the extreme value of the affinity target is less than 1 and is an experimental parameter; f1(x) is the first objective function, which represents the average resource loss rate of all working nodes; f2(x) is the second objective function, which represents the inversion of the affinity of all working nodes; represents the normalized value of the first objective function, represents the normalized value of the second objective function; F(x) represents the final optimization objective function, and maxF(x) represents the optimal objective solution.

3. The resource scheduling method based on immune algorithm according to claim 2 is characterized in that: The definition of the fitness calculation, antibody concentration calculation and antibody mutation probability of the antibody in the immune algorithm specifically includes: Assuming that there are H antibodies, each antibody has M gene positions, and each gene position has S discrete values ​​to choose from, the fitness of the antibody is defined as the following formula (11): fit j =((D1-f1(x i ))*0.5)*(D2-f2(x i )) (11) In formula (11), j = {1,...H}, i = {1,...S}; in the immune algorithm iteration, fit j The maximum value of is taken as the solution; The concentration of the jth antibody is defined as the following formula (12): In formula (12), H represents H antibodies, X jv represents the density of the j-th antibody, |X jv |2 represents the second-order norm of antibody v in the jth class of antibodies; Assume that the algorithm algebra is g, antibody X i The gene position Φ i The mutation probability of is controlled by the following equations (13) and (14): In equations (13) and (14), P i Represents antibody X i The mutation probability; α represents a constant; random represents a random number, and others represents a conditional judgment; i = {1,...,M}, when Φ(i) is 1, it means that the gene position is selected for mutation, and the mutation constraint is in, Represents the resource units occupied by k Pods residing on worker node n, NR n Represents the total resource units owned by worker node n, where n represents the worker node, m represents the Pod node, PR represents the resources occupied by the Pod, and NR represents the total resources of the worker node.

4. The resource scheduling method based on immune algorithm according to claim 3 is characterized in that: The method of solving the resource scheduling target model by using the immune algorithm specifically includes: First, initialization: randomly generate a set of initial solutions as the population, and calculate the fitness of each solution; Second, selection: Calculate the antibody concentration according to the fitness of each solution and sort them, and select a part of the solutions with the best fitness as antibodies for the subsequent evolution process; Third, cloning: cloning the selected antibodies to produce a group of cloned antibodies; Fourth, mutation: Perform random mutation operations on the cloned antibodies to generate new solutions, where the random mutation operations include at least one of the following: node selector and label mutation, affinity and anti-affinity rule mutation, stain and tolerance mutation, resource request and restriction mutation; Fifth, optimization: According to the fitness of each solution, select some excellent solutions to add to the next generation population; Sixth, termination condition: When the preset number of iterations or target fitness is reached, the immune algorithm is stopped.

5. A resource scheduling device based on immune algorithm, characterized in that: The device includes a matrix construction module, an objective function construction module and a resource scheduling module; The matrix construction module is used to map all K8s Pod sets required for the resource scheduling task to the work node set, so as to construct a target strategy matrix, specifically including: mapping all K8s Pod sets required for the resource scheduling task to the work node set, so as to construct a target strategy matrix STRATEGY. The specific definition of the target strategy matrix STRATEGY is as follows: In formula (1), S NP Represents the relationship between the working node N and the Pod P node. When S NP 1 means that Pod P node is allocated to worker node N. NP 0 means that there is no allocation relationship between working node N and Pod P node; N means there are N working nodes in total, n means any working node between 1 and N; P means there are P Pods in total, p means any Pod node between 1 and P; The objective function construction module is used to construct a Pod node affinity objective function and a resource loss rate objective function according to the objective strategy matrix, specifically including: when solving the resource scheduling task and allocating Pods, according to the objective strategy matrix, defining the affinity between Pods as shown in the following formula (2): In formula (2), x ij Represents the relationship between the i-th Pod and the j-th Pod in the Pod relationship matrix, where the Pod relationship matrix refers to the S in the target strategy matrix. NP ; P represents the number of Pods; f(x ij )’s affinity calculation rules are as follows: The affinity of homogeneous Pod nodes on the same worker node is -1, the affinity of associated Pod nodes on different worker nodes is 2, and the affinity between Pod nodes in other cases is 0; When allocating Pods to solve the resource scheduling task, the resource utilization rate is converted into the node resource loss rate. The calculation formula of the node resource loss rate is as follows (3): In formula (3), ULOST n Indicates the resource loss rate on working node n; Indicates the amount of resource loss on working node n; Indicates the total amount of resources on working node n; The resource scheduling module is used to define the objective function and constraints for resource scheduling of Pods, and use the Pareto optimal frontier to build a resource scheduling target model; antigens are defined as optimization problems, antibodies are defined as problem solutions, the number of antigens is the number of optimization sub-goals, the fitness calculation, antibody concentration calculation and antibody mutation probability of antibodies in the immune algorithm are defined, and the immune algorithm is used to solve the resource scheduling target model.

6. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the method according to any one of claims 1 to 4 is implemented.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 4 is implemented.

Images