Resource scheduling and load balancing method and system in private cloud environment

Abstract

The present invention relates to the field of cloud computing, and specifically discloses a resource scheduling and load balancing method and system in a private cloud environment, the method comprising: defining a resource allocation matrix and an objective function, as well as resource and demand constraints; introducing replica variables of the resource allocation matrix, and adding new constraints to maintain consistency between the original variables and the replica variables; constructing an augmented Lagrangian function, converting resource and demand constraints into penalty terms in the objective function; applying an alternating direction multiplier method, alternately optimizing the original variables and the replica variables, and updating the dual variables and penalty parameters at the same time; decomposing the original large-scale optimization problem into multiple small-scale sub-problems that can be processed in parallel; solving multiple sub-problems in parallel; iteratively updating the dual variables and penalty parameters in the augmented Lagrangian function until convergence; outputting the optimal solution, extracting and outputting the optimal resource allocation plan. The present invention can improve the speed and quality of resource allocation in a private cloud environment.

Description

Technical Field

[0001] The present invention relates to the field of cloud computing technology, and more specifically, to a resource scheduling and load balancing method and system in a private cloud environment. Background Art

[0002] A private cloud environment provides a flexible and secure cloud computing model designed for a specific organization or enterprise. It allows organizations to deploy services on a dedicated infrastructure while maintaining full control over data and operations. However, as business needs grow and workloads change dynamically, resource scheduling and load balancing issues in a private cloud environment become increasingly complex.

[0003] The main problems faced by existing private cloud environments include:

[0004] Uneven resource utilization: Different virtual machines and applications may have different resource requirements and usage patterns, resulting in uneven resource utilization. Some resources may be overused while others are idle.

[0005] Scalability issues: As business expands, private clouds need to be able to quickly adjust resource allocation to accommodate new workloads, but traditional resource scheduling methods often have difficulty responding to such changes quickly.

[0006] Challenges in a multi-tenant environment: In a multi-tenant private cloud environment, it is necessary to ensure fairness in resource allocation to prevent some tenants from occupying too many resources and affecting the performance of other tenants.

[0007] Cost-effectiveness: Enterprises need to reduce resource costs as much as possible while ensuring service quality. Resource scheduling and load balancing strategies need to find a balance between performance and cost.

[0008] Dynamic workload: The workload in a private cloud often changes dynamically, which requires the load balancing strategy to respond to workload changes in real time and dynamically adjust resource allocation.

[0009] How to solve the resource scheduling and load balancing problems in private cloud environments is crucial to maintaining efficient, stable and economical cloud services. With the development of technology, new solutions are needed to meet these challenges, improve resource utilization, reduce costs, and ensure service quality. Summary of the invention

[0010] The purpose of the present invention is to propose a resource scheduling and load balancing method and system in a private cloud environment, which improves the speed and quality of resource allocation by decoupling and decomposing large-scale resource allocation problems and utilizing parallel computing technology, thus providing an effective solution to the resource scheduling and load balancing problems of private clouds.

[0011] To achieve the above object, the present invention proposes a resource scheduling and load balancing method in a private cloud environment, comprising:

[0012] S1: The resource allocation and load balancing problem of the private cloud is expressed as an optimization problem, and the resource allocation matrix and objective function, as well as the resource and demand constraints, are defined;

[0013] S2: Introduce replica variables of the resource allocation matrix to decouple resource and demand constraints, and add new constraints to maintain consistency between the original variables and the replica variables;

[0014] S3: Construct an augmented Lagrangian function to convert resource and demand constraints into penalty terms in the objective function;

[0015] S4: Apply the alternating direction multiplier method to alternately optimize the original variables and the replica variables, while updating the dual variables and penalty parameters;

[0016] S5: Decompose the original large-scale optimization problem into multiple small-scale sub-problems that can be processed in parallel, each sub-problem corresponding to the allocation of a resource or demand;

[0017] S6: Solve multiple sub-problems in parallel;

[0018] S7: Iteratively update the dual variables and penalty parameters in the augmented Lagrangian function according to the alternating direction multiplier method until convergence;

[0019] S8: Output the optimal solution, extract and output the optimal resource allocation plan, and perform resource scheduling and load balancing based on the optimal resource allocation plan.

[0020] Optionally, step S1 specifically includes:

[0021] Define the resource allocation matrix x, which is an n×m matrix, where n is the number of resources, m is the number of requirements, and the elements x in x are ij represents the amount of resource i allocated to demand j;

[0022] Formulate the objective function:

[0023] ;

[0024] Identify the constraints:

[0025] ;

[0026] ;

[0027] in, is the utility function of resource i, is the i-th row of x, representing the allocation vector of resource i; is the utility function of demand j, is the jth column of x, representing the allocation vector of demand j; X represents the feasible domain or feasible solution space of resource allocation, which is a set of resource allocation matrices or vectors that meet all constraint conditions, and is used to limit the value range of the optimization variable x to ensure that the allocation solution is feasible; in the objective function, x is the optimization target, and the value of the objective function is minimized by adjusting the value of x; R i and D j are the constraint matrices of resource i and demand j, r i and d j are the capacity of resource i and the demand of demand j respectively.

[0028] Optionally, step S2 specifically includes:

[0029] Introduce a replica variable z of the resource allocation matrix x, which is used to decouple resource and demand constraints;

[0030] Add a new constraint x−z=0 to keep x and z equal.

[0031] Optionally, the augmented Lagrangian function constructed in step S3 is:

[0032]

[0033] Where Lρ is the augmented Lagrangian function, ρ is the penalty parameter, α, β and λ are the Lagrangian multipliers of resource i, demand j and x=z constraint respectively, α i is the constraint condition with resource i The associated dual variable, β j is the constraint condition for demand j The dual variable associated with x=z constraint, λ is the dual variable associated with x=z constraint; represents the jth column of the z matrix, i.e., the allocation vector of demand j, Represents the utility function of demand j, which is used to evaluate the allocation plan of demand j quality; represents the L2 norm square of the vector, Represents the squared Frobenius norm of a matrix.

[0034] Optionally, step S4 specifically includes:

[0035] Apply the alternating direction multiplier method to optimize x and z alternately. In each iteration, first fix z and optimize x, then fix x and optimize z, and update the dual variable α i , β jand λ, the update formula is:

[0036] ;

[0037] Where k is the number of iterations; := indicates assignment; x (k+1) is the resource allocation matrix x for the k+1th iteration; z (k+1) is the resource allocation matrix z of the k+1th iteration; α i (k+1) is the updated value of the dual variable of resource i in the k+1th iteration; α i (k) is the dual variable of resource i in the kth iteration; β j (k) is the dual variable of demand j at the kth iteration; β j (k+1) is the updated value of the dual variable of requirement j at the k+1th iteration; (k) is the dual variable of the x=z constraint at the kth iteration; (k+1) is the updated value of the dual variable of the x=z constraint at the k+1th iteration; α (k) is the dual variable vector of resource constraints in the kth iteration, β (k) is the dual variable vector of the demand constraints in the kth iteration; represents the allocation vector of resource i in the k+1th iteration; represents the allocation vector of demand j at the k+1th iteration.

[0038] Optionally, in step S5, the optimization problem is decomposed into sub-problems for each resource and each demand, and the calculation formula is:

[0039] ;

[0040] in, represents the allocation vector of resource i at the k+1th iteration; represents the allocation vector of demand j at the k+1th iteration; express The feasible domain of represents the current value of the allocation vector of resource i in the kth iteration; represents the current value of the dual variable associated with the x=z constraint at iteration k, specifically the part of resource i; represents the current value of the dual variable associated with the x=z constraint at iteration k, specifically the part of demand j.

[0041] Optionally, step S6 specifically includes:

[0042] Execute the solver in parallel on multiple CPU cores, with each core solving one or more subproblems;

[0043] After solving each sub-problem, the results of each computational unit are collected and merged to form a global optimal solution.

[0044] Optionally, step S7 specifically includes:

[0045] Based on the update formula in the alternating direction multiplier method, by iteratively updating the dual variables and penalty parameters, the algorithm can gradually meet all constraints and find a feasible solution in the optimization process. In each iteration, the dual variables and penalty parameters in the alternating direction multiplier method are updated until the convergence conditions are met.

[0046] The penalty parameter ρ is usually kept constant in each iteration. If it is found that the constraint violation persists, the value of ρ is increased to improve the convergence performance of the algorithm.

[0047] Optionally, step S8 specifically includes:

[0048] After the alternating direction multiplier method converges, the optimal resource allocation matrix x is output, which describes the optimal allocation relationship between resources and demands;

[0049] Based on the obtained optimal resource allocation solution, corresponding resource scheduling and load balancing strategies are executed in private cloud computing resources.

[0050] The present invention also provides a resource scheduling and load balancing system in a private cloud environment, comprising:

[0051] The optimization problem building module is used to express the resource allocation and load balancing problem of the private cloud as an optimization problem, define the resource allocation matrix and objective function, as well as the resource and demand constraints.

[0052] A decoupling and constrained replication module, which is used to introduce replica variables of the resource allocation matrix to decouple resource and demand constraints, and add new constraints to maintain consistency between the original variables and the replica variables;

[0053] The augmented Lagrangian construction module is used to construct the augmented Lagrangian function and convert the resource and demand constraints into penalty terms in the objective function;

[0054] An alternating optimization module, which is used to apply the alternating direction multiplier method to alternately optimize the original variables and the replica variables while updating the dual variables and penalty parameters;

[0055] The optimization problem decomposition module is used to decompose the original large-scale optimization problem into multiple small-scale sub-problems that can be processed in parallel, each of which corresponds to the allocation of a resource or demand;

[0056] Parallel solving module, used to solve multiple sub-problems in parallel;

[0057] An iterative update module, used for iteratively updating the dual variables and penalty parameters in the augmented Lagrangian function according to the alternating direction multiplier method until convergence;

[0058] The output module is used to output the optimal solution, extract and output the optimal resource allocation plan, and perform resource scheduling and load balancing based on the optimal resource allocation plan.

[0059] The beneficial effects of the present invention are:

[0060] The present invention significantly improves the speed of resource allocation through the "decoupling and decomposition" method, making it possible to quickly respond to resource allocation in a large-scale cloud system, thereby improving the response speed and user experience of private cloud services. The present invention can handle optimization problems involving a large number of variables, effectively solve the scalability crisis encountered by traditional private cloud resource scheduling when handling large-scale problems, and enable resource allocation schemes to adapt to the rapid expansion and diversification of private cloud environments. While improving the speed of resource allocation, the present invention can also maintain or approach the optimal resource allocation quality, ensuring the efficiency and effectiveness of resource scheduling and load balancing. The present invention does not rely on assumptions about specific problems or workloads, and provides a general resource allocation solution that is applicable to a variety of different resource allocation scenarios, enhancing the applicability and flexibility of the algorithm. Utilizing the multi-core characteristics of modern computing systems, the present invention realizes true parallel computing, improves solution efficiency, and significantly reduces computing time, especially on multi-core processors. Compared with methods that require a lot of manual adjustments and specific domain knowledge, the present invention provides a more automated solution, reduces the need for manual intervention, and reduces operational complexity. The present invention can adapt to different workload types and changes, does not rely on granular assumptions such as resource interchangeability, enhances the robustness of the algorithm, and ensures stability and reliability under various conditions. Through more sophisticated resource allocation, the present invention helps to improve resource utilization and reduce resource waste, thereby reducing operating costs and improving economic benefits. The present invention can quickly adapt to dynamic changes in the cloud environment, such as demand changes, workload fluctuations, etc., maintain service continuity and quality, and improve the flexibility and adaptability of the system.

[0061] The system of the present invention has other characteristics and advantages, which will be apparent from the drawings incorporated herein and the following detailed description, or will be described in detail in the drawings incorporated herein and the following detailed description, which together serve to explain the specific principles of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0062] The above and other objects, features and advantages of the present invention will become more apparent through a more detailed description of exemplary embodiments of the present invention in conjunction with the accompanying drawings, in which like reference numerals generally represent like components.

[0063] Figure 1 A flow chart showing the steps of a resource scheduling and load balancing method in a private cloud environment according to the present invention is shown. DETAILED DESCRIPTION

[0064] The present invention will be described in more detail below with reference to the accompanying drawings. Although preferred embodiments of the present invention are shown in the accompanying drawings, it should be understood that the present invention can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided to make the present invention more thorough and complete, and to fully convey the scope of the present invention to those skilled in the art.

[0065] Example 1

[0066] like Figure 1 As shown, this embodiment provides a resource scheduling and load balancing method in a private cloud environment, including the following steps:

[0067] S1: The resource allocation and load balancing problem of the private cloud is expressed as an optimization problem, and the resource allocation matrix and objective function, as well as the resource and demand constraints, are defined;

[0068] Step S1 specifically includes:

[0069] Define the resource allocation matrix x, which is an n×m matrix, where n is the number of resources, m is the number of requirements, and the elements x in x are ij represents the amount of resource i allocated to demand j;

[0070] Formulate the objective function:

[0071] ;

[0072] Identify the constraints:

[0073] ;

[0074] ;

[0075] in, is the utility function of resource i, is the i-th row of x, representing the allocation vector of resource i; is the utility function of demand j, is the jth column of x, representing the allocation vector of demand j; X represents the feasible domain or feasible solution space of resource allocation, which is a set of resource allocation matrices or vectors that meet all constraint conditions, and is used to limit the value range of the optimization variable x to ensure that the allocation solution is feasible; in the objective function, x is the optimization target, and the value of the objective function is minimized by adjusting the value of x; R i and D j are the constraint matrices of resource i and demand j, r i and d j are the capacity of resource i and the demand of demand j respectively.

[0076] Specifically, this step expresses the resource allocation and load balancing problem as an optimization problem, including an objective function and resource / demand constraints, where the objective function is the sum of the resource and demand allocation utilities, and the constraints ensure that the allocation of resources and demands does not exceed their capacity and demand.

[0077] S2: Introduce replica variables of the resource allocation matrix to decouple resource and demand constraints, and add new constraints to maintain consistency between the original variables and the replica variables;

[0078] Step S2 specifically includes:

[0079] Introduce a replica variable z of the resource allocation matrix x, which is used to decouple resource and demand constraints;

[0080] Add a new constraint x−z=0 to keep x and z equal.

[0081] This step introduces a copy z of the resource allocation matrix x and replaces x with z in the demand constraints and objective function, that is, by introducing a new variable z to decouple the resource and demand constraints, so that resources and demand can be optimized separately. At the same time, a new constraint x=z is added to keep x and z equal, in preparation for the subsequent optimization steps.

[0082] S3: Construct an augmented Lagrangian function to convert resource and demand constraints into penalty terms in the objective function;

[0083] The augmented Lagrangian function constructed in step S3 is:

[0084]

[0085] Where Lρ is the augmented Lagrangian function, ρ is the penalty parameter, α, β and λ are the Lagrangian multipliers of resource i, demand j and x=z constraint respectively, α represents the set or vector of dual variables of all resource constraints, α i represents an element in the α vector, specifically the dual variable of the i-th resource constraint, and β contains the set or vector of all demand constraint dual variables, where βj is its jth element; at the same time α i is the constraint condition with resource i The associated dual variable, β j is the constraint condition with demand j The dual variable associated with x=z constraint, λ is the dual variable associated with x=z constraint; represents the jth column of the z matrix, i.e., the allocation vector of demand j, Represents the utility function of demand j, which is used to evaluate the allocation plan of demand j quality; represents the L2 norm square of the vector, Represents the squared Frobenius norm of a matrix.

[0086] Specifically, this step uses the augmented Lagrangian method to convert the constraints into penalty terms in the objective function so that the constraints can be satisfied by optimizing the objective function. and Converted into a penalty term and added to the objective function.

[0087] S4: Apply the alternating direction multiplier method to alternately optimize the original variables and the replica variables, while updating the dual variables and penalty parameters;

[0088] Step S4 specifically includes:

[0089] Apply the alternating direction multiplier method to optimize x and z alternately. In each iteration, first fix z and optimize x, then fix x and optimize z, and update the dual variable α i , β j and λ, the update formula is:

[0090] ;

[0091] Where k is the number of iterations; := indicates assignment; x (k+1) is the resource allocation matrix x for the k+1th iteration; z (k+1) is the resource allocation matrix z of the k+1th iteration; α i (k+1) is the updated value of the dual variable of resource i in the k+1th iteration; α i (k) is the dual variable of resource i in the kth iteration; β j (k) is the dual variable of demand j at the kth iteration; β j (k+1) is the updated value of the dual variable of requirement j at the k+1th iteration;(k) is the dual variable of the x=z constraint at the kth iteration; (k+1) is the updated value of the dual variable of the x=z constraint at the k+1th iteration; α (k) is the dual variable vector of resource constraints in the kth iteration, β (k) is the dual variable vector of the demand constraints in the kth iteration; represents the allocation vector of resource i in the k+1th iteration; represents the allocation vector of demand j at the k+1th iteration;

[0092] Among them, the formula The role of is: when z, α, β and λ are fixed, optimize x to minimize the augmented Lagrangian function;

[0093] formula The role of is: when x, α, β and λ are fixed, optimize z to minimize the augmented Lagrangian function;

[0094] formula The role of is: to update the dual variable α according to the degree of violation of the resource constraint;

[0095] formula The role of is: to update the dual variable β according to the degree of violation of the demand constraint;

[0096] formula The role of is to update the dual variable λ according to the difference between x and z.

[0097] Specifically, this step applies the alternating direction multiplier method to alternately optimize x and z, where the optimization of x is: fix z, optimize x to satisfy the resource constraint and x=z. The optimization of z is: fix x, optimize z to satisfy the demand constraint and x=z. The alternating direction multiplier method is an iterative algorithm that solves the optimization problem by alternately optimizing two sets of variables (x and z). In each iteration, first fix z and optimize x, then fix x and optimize z, and update the dual variables α, β, and λ.

[0098] S5: Decompose the original large-scale optimization problem into multiple small-scale sub-problems that can be processed in parallel, each sub-problem corresponding to the allocation of a resource or demand;

[0099] In step S5, the optimization problem is decomposed into sub-problems for each resource and each demand, and the calculation formula is:

[0100] ;

[0101] in, represents the allocation vector of resource i at the k+1th iteration; represents the allocation vector of demand j at the k+1th iteration; express The feasible domain of represents the current value of the allocation vector of resource i in the kth iteration; represents the current value of the dual variable associated with the x=z constraint at iteration k, specifically the part of resource i; represents the current value of the dual variable associated with the x=z constraint at iteration k, specifically the part of demand j.

[0102] Specifically, this step decomposes the optimization problem of x and z into independent sub-problems for each resource and each demand, and each sub-problem only involves the allocation of one resource or one demand, so that it can be solved in parallel.

[0103] S6: Solve multiple sub-problems in parallel;

[0104] Step S6 specifically includes:

[0105] Execute the solver in parallel on multiple CPU cores, with each core solving one or more subproblems;

[0106] After solving each sub-problem, the results of each computational unit are collected and merged to form a global optimal solution.

[0107] In this step, you can use an off-the-shelf solver (such as cvxpy) to solve each subproblem in parallel, taking advantage of all available CPU cores, that is, executing the cvxpy solver in parallel on multiple CPU cores. This improves the efficiency of solving resource allocation problems by taking advantage of the computing power of multi-core processors. By solving each subproblem in parallel, you can significantly reduce the computing time and improve the scalability of the algorithm.

[0108] S7: Iteratively update the dual variables and penalty parameters in the augmented Lagrangian function according to the alternating direction multiplier method until convergence;

[0109] Step S7 specifically includes:

[0110] Based on the update formula in the alternating direction multiplier method, by iteratively updating the dual variables and penalty parameters, the algorithm can gradually meet all constraints and find a feasible solution in the optimization process. In each iteration, the dual variables and penalty parameters in the alternating direction multiplier method are updated until the convergence conditions are met.

[0111] The penalty parameter ρ is usually kept constant in each iteration. If it is found that the constraint violation persists, the value of ρ is increased to improve the convergence performance of the algorithm.

[0112] Specifically, in each iteration, the dual variables and penalty parameters in the alternating direction multiplier method are updated until the convergence condition is met. By continuously updating the dual variables and penalty parameters, the difference between x and z is gradually reduced, and finally convergence is achieved.

[0113] S8: Output the optimal solution, extract and output the optimal resource allocation plan, and perform resource scheduling and load balancing based on the optimal resource allocation plan.

[0114] Step S8 specifically includes:

[0115] After the alternating direction multiplier method converges, the optimal resource allocation matrix x is output, which describes the optimal allocation relationship between resources and demands;

[0116] Based on the obtained optimal resource allocation solution, corresponding resource scheduling and load balancing strategies are executed in private cloud computing resources.

[0117] Specifically, once the algorithm converges, the optimal resource allocation matrix x is output, and the final x matrix describes the optimal allocation relationship between resources and requirements. Through these steps, private clouds can effectively handle large-scale resource allocation problems while maintaining high computing efficiency and good scalability.

[0118] During the specific implementation process, the method of the present invention can be constructed based on cvxpy and the specific functions of the method of the present invention can be developed and designed through Python language.

[0119] In summary, the resource scheduling and load balancing method in a private cloud environment of the present invention achieves the following beneficial technical effects:

[0120] (1) Improve resource allocation speed:

[0121] The present invention significantly improves the speed of resource allocation through the "decoupling and decomposition" method, making it possible to quickly respond to resource allocation in a large-scale cloud system, thereby improving the response speed of cloud services and user experience.

[0122] (2) Enhance system scalability:

[0123] The present invention can handle optimization problems involving millions of variables, effectively solving the scalability crisis encountered by traditional commercial solvers when dealing with large-scale problems, and enables resource allocation solutions to adapt to the rapid expansion and diversification of cloud environments.

[0124] (3) Maintain and optimize the quality of resource allocation:

[0125] While improving the resource allocation speed, the present invention can also maintain or approach the optimal resource allocation quality, thereby ensuring the high efficiency and effectiveness of resource scheduling and load balancing.

[0126] (4) Improving versatility and applicability:

[0127] The present invention does not rely on assumptions about specific problems or workloads, and provides a universal resource allocation solution that is applicable to a variety of different resource allocation scenarios, thereby enhancing the applicability and flexibility of the algorithm.

[0128] (5) Realizing true parallel computing:

[0129] By utilizing the multi-core characteristics of modern computing systems, the present invention realizes true parallel computing and improves solution efficiency, especially on multi-core processors, significantly reducing computing time.

[0130] (6) Reduce manual intervention and customization requirements:

[0131] Compared to methods that require a lot of manual adjustments and domain-specific knowledge, the present invention provides a more automated solution, reducing the need for manual intervention and reducing operational complexity.

[0132] (7) Enhanced robustness and adaptability:

[0133] The present invention can adapt to different workload types and changes, does not rely on granularity assumptions such as resource interchangeability, enhances the robustness of the algorithm, and ensures stability and reliability under various conditions.

[0134] (8) Optimize resource utilization and reduce costs:

[0135] Through more refined resource allocation, the present invention helps to improve resource utilization and reduce resource waste, thereby reducing operating costs and improving economic benefits.

[0136] (9) Simplify the implementation and integration process:

[0137] The present invention can be implemented as a Python package and is compatible with the open source solver cvxpy, making integration and implementation simple, easy for developers to use and expand, and accelerating development speed and deployment efficiency.

[0138] (10) Quickly adapt to dynamic changes in the cloud environment:

[0139] The present invention can quickly adapt to dynamic changes in the cloud environment, such as demand changes, workload fluctuations, etc., maintain service continuity and quality, and improve the flexibility and adaptability of the system.

[0140] In summary, the method of the present invention not only improves the speed and quality of resource allocation, but also enhances the scalability, versatility and robustness of the system, bringing significant improvements and economic benefits to the resource management of private cloud systems.

[0141] Example 2

[0142] This embodiment provides a resource scheduling and load balancing system in a private cloud environment, including:

[0143] The optimization problem building module is used to express the resource allocation and load balancing problem of the private cloud as an optimization problem, define the resource allocation matrix and objective function, as well as the resource and demand constraints.

[0144] A decoupling and constrained replication module, which is used to introduce replica variables of the resource allocation matrix to decouple resource and demand constraints, and add new constraints to maintain consistency between the original variables and the replica variables;

[0145] The augmented Lagrangian construction module is used to construct the augmented Lagrangian function and convert the resource and demand constraints into penalty terms in the objective function;

[0146] An alternating optimization module, which is used to apply the alternating direction multiplier method to alternately optimize the original variables and the replica variables while updating the dual variables and penalty parameters;

[0147] The optimization problem decomposition module is used to decompose the original large-scale optimization problem into multiple small-scale sub-problems that can be processed in parallel, each of which corresponds to the allocation of a resource or demand;

[0148] Parallel solving module, used to solve multiple sub-problems in parallel;

[0149] An iterative update module, used for iteratively updating the dual variables and penalty parameters in the augmented Lagrangian function according to the alternating direction multiplier method until convergence;

[0150] The output module is used to output the optimal solution, extract and output the optimal resource allocation plan, and perform resource scheduling and load balancing based on the optimal resource allocation plan.

[0151] For the specific functions of each functional module of the system in this embodiment, please refer to Example 1 and will not be repeated here.

[0152] The embodiments of the present invention have been described above, and the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and changes will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments.

Claims

1. A resource scheduling and load balancing method in a private cloud environment, characterized in that: include: S1: The resource allocation and load balancing problem of the private cloud is expressed as an optimization problem, and the resource allocation matrix and objective function, as well as the resource and demand constraints, are defined; The resource allocation matrix is ​​defined as x , x is a n × m The matrix of n is the amount of resources, m is the quantity demanded, x Elements in x ij Representation Resources i Assign to demand j The amount of S2: Introduce replica variables of the resource allocation matrix to decouple resource and demand constraints, and add new constraints to maintain consistency between the original variables and the replica variables, including: Introduce the resource allocation matrix x A copy variable z , z Used to decouple resource and demand constraints; and add new constraints x − z =0 to maintain x and z equal; S3: Construct an augmented Lagrangian function to convert resource and demand constraints into penalty terms in the objective function; S4: Apply the alternating direction multiplier method to alternately optimize the original variables and the replica variables, while updating the dual variables and penalty parameters; S5: Decompose the original large-scale optimization problem into multiple small-scale sub-problems that can be processed in parallel, each sub-problem corresponding to the allocation of a resource or demand; S6: Solve multiple sub-problems in parallel; S7: Iteratively update the dual variables and penalty parameters in the augmented Lagrangian function according to the alternating direction multiplier method until convergence; S8: Output the optimal solution, extract and output the optimal resource allocation plan, and perform resource scheduling and load balancing based on the optimal resource allocation plan; In step S1: The objective function is defined as: ； The defined constraints are: ； ； in, is the utility function of resource i, is the i-th row of x, representing the allocation vector of resource i; is the utility function of demand j, is the jth column of x, representing the allocation vector of demand j; X represents the feasible domain or feasible solution space of resource allocation, which is a set of resource allocation matrices or vectors that meet all constraint conditions, and is used to limit the value range of the optimization variable x to ensure that the allocation solution is feasible; in the objective function, x is the optimization target, and the value of the objective function is minimized by adjusting the value of x; R i and D j are the constraint matrices of resource i and demand j, r i and d j are the capacity of resource i and the demand of demand j respectively; The augmented Lagrangian function constructed in step S3 is: ； Where Lρ is the augmented Lagrangian function, ρ is the penalty parameter, α, β and λ are the Lagrangian multipliers of resource i, demand j and x=z constraint respectively, α i is the constraint condition with resource i The associated dual variable, β j is the constraint condition with demand j The dual variable associated with x=z constraint, λ is the dual variable associated with x=z constraint; represents the j-th column of the z matrix, i.e., the allocation vector for demand j, Represents the utility function of demand j, which is used to evaluate the allocation plan of demand j quality; represents the L2 norm square of the vector, represents the squared Frobenius norm of a matrix; Step S4 specifically includes: Apply the alternating direction multiplier method to optimize x and z alternately. In each iteration, first fix z and optimize x, then fix x and optimize z, and update the dual variable α i , β j and λ, the update formula is: ； Where k is the number of iterations; := indicates assignment; x (k+1) is the resource allocation matrix x for the k+1th iteration; z (k+1) is the resource allocation matrix z of the k+1th iteration; α i (k+1) is the updated value of the dual variable of resource i in the k+1th iteration; α i (k) is the dual variable of resource i in the kth iteration; β j (k) is the dual variable of demand j at the kth iteration; β j (k+1) is the updated value of the dual variable of requirement j at the k+1th iteration; (k) is the dual variable of the x=z constraint at the kth iteration; (k+1) is the updated value of the dual variable of the x=z constraint at the k+1th iteration; α (k) is the dual variable vector of resource constraints in the kth iteration, β (k) is the dual variable vector of the demand constraints in the kth iteration; represents the allocation vector of resource i at the k+1th iteration; represents the allocation vector of demand j at the k+1th iteration; In step S5, the optimization problem is decomposed into sub-problems for each resource and each demand, and the calculation formula is: ； in, represents the allocation vector of resource i at the k+1th iteration; represents the allocation vector of demand j at the k+1th iteration; express The feasible domain of represents the current value of the allocation vector of resource i in the kth iteration; represents the current value of the dual variable associated with the x=z constraint at iteration k, specifically the part of resource i; represents the current value of the dual variable associated with the x=z constraint at iteration k, specifically the part of demand j.

2. The method according to claim 1, characterized in that Step S6 specifically includes: Execute the solver in parallel on multiple CPU cores, with each core solving one or more subproblems; After solving each sub-problem, the results of each computational unit are collected and merged to form a global optimal solution.

3. The method according to claim 2, characterized in that Step S7 specifically includes: Based on the update formula in the alternating direction multiplier method, by iteratively updating the dual variables and penalty parameters, the algorithm can gradually meet all constraints and find a feasible solution in the optimization process. In each iteration, the dual variables and penalty parameters in the alternating direction multiplier method are updated until the convergence conditions are met. The penalty parameter ρ is usually kept constant in each iteration. If it is found that the constraint violation persists, the value of ρ is increased to improve the convergence performance of the algorithm.

4. The method according to claim 3, characterized in that Step S8 specifically includes: After the alternating direction multiplier method converges, the optimal resource allocation matrix x is output, which describes the optimal allocation relationship between resources and demands; Based on the obtained optimal resource allocation solution, corresponding resource scheduling and load balancing strategies are executed in private cloud computing resources.

5. A system for implementing the resource scheduling and load balancing method in a private cloud environment as described in any one of claims 1 to 4, characterized in that: include: The optimization problem building module is used to express the resource allocation and load balancing problem of the private cloud as an optimization problem, define the resource allocation matrix and objective function, as well as the resource and demand constraints. The resource allocation matrix is ​​defined as x , x is a n × m The matrix of n is the amount of resources, m is the quantity demanded, x Elements in x ij Representation Resources i Assign to demand j The amount of Decoupling and Constrained Replication module, which is used to introduce replica variables of the resource allocation matrix to decouple resource and demand constraints, and add new constraints to maintain consistency between the original variables and the replica variables; Includes: Introducing the resource allocation matrix x A copy variable z , z Used to decouple resource and demand constraints; and add new constraints x − z =0 to maintain x and z equal; The augmented Lagrangian construction module is used to construct the augmented Lagrangian function and convert the resource and demand constraints into penalty terms in the objective function; An alternating optimization module, which is used to apply the alternating direction multiplier method to alternately optimize the original variables and the replica variables while updating the dual variables and the penalty parameters; The optimization problem decomposition module is used to decompose the original large-scale optimization problem into multiple small-scale sub-problems that can be processed in parallel, each of which corresponds to the allocation of a resource or demand; Parallel solving module, used to solve multiple sub-problems in parallel; An iterative update module, used for iteratively updating the dual variables and penalty parameters in the augmented Lagrangian function according to the alternating direction multiplier method until convergence; The output module is used to output the optimal solution, extract and output the optimal resource allocation plan, and perform resource scheduling and load balancing based on the optimal resource allocation plan.

Images