Virtual machine arrangement scheduling method and equipment

Abstract

The embodiment of the invention provides a virtual machine arrangement and scheduling method and equipment, and the method comprises the steps: creating a first virtual machine planning model according to the deployment attribute characteristics of an initial virtual machine deployed on a physical machine in response to a secondary scheduling request of a virtual machine, and determining a first deployment scheme enabling the total scheduling cost to be lowest; comprising the following steps: on the premise of not considering the anti-affinity constraint of the virtual machines, predicting the deployment number of each type of initial virtual machines in each physical machine, and determining to-be-migrated initial virtual machines in each type of initial virtual machines in each physical machine; creating a second virtual machine planning model for the target to-be-emigrated initial virtual machine with the anti-affinity constraint, and determining a second deployment scheme enabling the scheduling cost of the target to-be-emigrated initial virtual machine to be the lowest, including a target physical machine to which each target to-be-emigrated initial virtual machine is to be deployed on the premise of considering the anti-affinity constraint of the virtual machine; and performing virtual machine scheduling. Through a multi-stage scheduling simplified model, the scheduling efficiency and success rate of the virtual machine are improved.

Description

Technical Field

[0001] The embodiments of the present disclosure relate to the field of computer and network communication technology, and more particularly to a virtual machine scheduling method and device. Background Art

[0002] With the development of technology, the scale of computing resources required by some enterprises is increasing. In the cloud computing environment, the problem of resource fragmentation is becoming increasingly prominent. A large number of resource fragments occupy physical machines, resulting in a shortage of large-scale inventory, and small-scale fragment resources are basically idle and difficult to be used. In order to solve the problem of resource fragmentation and improve resource utilization, it is hoped that the virtual machines that have been deployed in each physical machine in the cloud environment can be re-scheduled to re-arrange the virtual machines, thereby freeing up empty machines or large-scale inventory.

[0003] In the existing technology, traditional heuristic algorithms are usually used to re-schedule the virtual machines deployed in each physical machine, which often fall into the local optimal solution. Compared with the global optimal solution, there is a significant gap in the orchestration quality. Due to the complexity of scheduling constraints, if the global optimal solution is to be solved, a global optimization modeling solution of "which physical machine each virtual machine is deployed on" is required, which often takes a long time to solve, is costly and inefficient. Summary of the invention

[0004] The embodiments of the present disclosure provide a virtual machine scheduling method and device to improve the virtual machine scheduling efficiency.

[0005] In a first aspect, an embodiment of the present disclosure provides a virtual machine scheduling method, including:

[0006] In response to the virtual machine secondary scheduling request, a first virtual machine planning model is created according to the deployment attribute characteristics of the initial virtual machine deployed on the physical machine, and a first deployment scheme that minimizes the total scheduling cost is determined according to the first virtual machine planning model, wherein the first deployment scheme includes a predicted deployment quantity of each type of initial virtual machine in each physical machine without considering the virtual machine anti-affinity constraint;

[0007] Determine, according to the initial deployment status of each type of initial virtual machines in each physical machine and the first deployment scheme, the initial virtual machines to be migrated out of each type of initial virtual machines in each physical machine;

[0008] Determine a target initial virtual machine to be migrated that has an anti-affinity constraint among the initial virtual machines to be migrated, create a second virtual machine planning model for the target initial virtual machine to be migrated, and determine a second deployment plan that minimizes the scheduling cost of the target initial virtual machine to be migrated according to the second virtual machine planning model, wherein the second deployment plan includes a target physical machine to which each target initial virtual machine to be migrated is to be deployed under the premise of considering the anti-affinity constraint of the virtual machine;

[0009] Virtual machine scheduling is performed according to the first deployment scheme and the second deployment scheme.

[0010] In a second aspect, an embodiment of the present disclosure provides a virtual machine scheduling device, including:

[0011] A first orchestration unit is configured to create a first virtual machine planning model in response to a virtual machine secondary scheduling request according to deployment attribute characteristics of an initial virtual machine deployed on a physical machine, and determine a first deployment plan that minimizes a total scheduling cost according to the first virtual machine planning model, wherein the first deployment plan includes a predicted number of initial virtual machines of each type in each physical machine without considering anti-affinity constraints of the virtual machines; and determine an initial virtual machine to be migrated out of each type of initial virtual machine in each physical machine according to an initial deployment status of each type of initial virtual machine in each physical machine and the first deployment plan;

[0012] A second orchestration unit is configured to determine a target initial virtual machine to be migrated that has an anti-affinity constraint among the initial virtual machines to be migrated, and to create a second virtual machine planning model for the target initial virtual machine to be migrated, and to determine a second deployment plan that minimizes the scheduling cost of the target initial virtual machine to be migrated according to the second virtual machine planning model, wherein the second deployment plan includes a target physical machine to which each target initial virtual machine to be migrated is to be deployed under the premise of considering the anti-affinity constraint of the virtual machine;

[0013] A scheduling unit is used to schedule virtual machines according to the first deployment scheme and the second deployment scheme.

[0014] In a third aspect, an embodiment of the present disclosure provides an electronic device, including: a processor and a memory;

[0015] The memory stores computer-executable instructions;

[0016] The processor executes the computer-executable instructions stored in the memory, so that the at least one processor executes the virtual machine orchestration and scheduling method described in the first aspect and various possible designs of the first aspect.

[0017] In a fourth aspect, an embodiment of the present disclosure provides a computer-readable storage medium, in which computer execution instructions are stored. When a processor executes the computer execution instructions, the virtual machine orchestration and scheduling method described in the first aspect and various possible designs of the first aspect is implemented.

[0018] In a fifth aspect, an embodiment of the present disclosure provides a computer program product, including a computer program, which, when executed by a processor, implements the virtual machine orchestration and scheduling method as described in the first aspect and various possible designs of the first aspect.

[0019] The virtual machine orchestration scheduling method and device provided by the embodiments of the present disclosure create a first virtual machine planning model in response to a virtual machine secondary scheduling request according to the deployment attribute characteristics of the initial virtual machine deployed on the physical machine, determine a first deployment plan that minimizes the total scheduling cost according to the first virtual machine planning model, wherein the first deployment plan includes a predicted deployment quantity of each type of initial virtual machines in each physical machine without considering the virtual machine anti-affinity constraint; determine the initial virtual machines to be migrated out of each type of initial virtual machines in each physical machine according to the initial deployment status of each type of initial virtual machines in each physical machine and the first deployment plan; determine the target initial virtual machines to be migrated out with anti-affinity constraints in the initial virtual machines to be migrated, and create a second virtual machine planning model for the target initial virtual machines to be migrated; determine a second deployment plan that minimizes the scheduling cost of the target initial virtual machines to be migrated according to the second virtual machine planning model, wherein the second deployment plan includes the target physical machine to which each target initial virtual machine to be migrated is to be deployed under the premise of considering the virtual machine anti-affinity constraint; and schedule the virtual machines according to the first deployment plan and the second deployment plan. This embodiment can simplify the model through multi-stage orchestration and scheduling, accurately and efficiently solve the deployment plan of virtual machine orchestration and scheduling with complex constraints, improve the efficiency and success rate of virtual machine orchestration and scheduling, improve resource utilization, and reduce orchestration and scheduling costs. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the embodiments of the present disclosure or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0021] Figure 1 A schematic diagram of a scenario of a virtual machine scheduling method provided by an embodiment of the present disclosure;

[0022] Figure 2 A schematic diagram of a virtual machine scheduling method flow chart provided in an embodiment of the present disclosure;

[0023] Figure 3 A structural block diagram of a virtual machine scheduling device provided in an embodiment of the present disclosure;

[0024] Figure 4 A schematic diagram of the hardware structure of a virtual machine orchestration and scheduling device provided in one embodiment of the present disclosure. DETAILED DESCRIPTION

[0025] In order to make the purpose, technical solution and advantages of the embodiments of the present disclosure clearer, the technical solution in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are part of the embodiments of the present disclosure, not all of the embodiments. Based on the embodiments in the present disclosure, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present disclosure.

[0026] In the existing technology, traditional heuristic algorithms are usually used to re-schedule the virtual machines deployed in each physical machine, which often fall into the local optimal solution. Compared with the global optimal solution, there is a significant gap in the orchestration quality. Due to the complexity of scheduling constraints, if the global optimal solution is to be solved, a global optimization modeling solution of "which physical machine each virtual machine is deployed on" is required, which often takes a long time to solve, is costly and inefficient.

[0027] In order to solve the above technical problems, the embodiment of the present disclosure provides a virtual machine orchestration and scheduling method, which responds to a virtual machine secondary scheduling request, creates a first virtual machine planning model according to the deployment attribute characteristics of the initial virtual machine deployed on the physical machine, and determines a first deployment plan that minimizes the total scheduling cost according to the first virtual machine planning model, wherein the first deployment plan includes the predicted deployment number of each type of initial virtual machines in each physical machine without considering the virtual machine anti-affinity constraint; determines the initial virtual machines to be migrated in each type of initial virtual machines in each physical machine according to the initial deployment status of each type of initial virtual machines in each physical machine and the first deployment plan; determines the target initial virtual machines to be migrated that have anti-affinity constraints in the initial virtual machines to be migrated, and creates a second virtual machine planning model for the target initial virtual machines to be migrated, and determines a second deployment plan that minimizes the scheduling cost of the target initial virtual machines to be migrated according to the second virtual machine planning model, wherein the second deployment plan includes the target physical machine to which each target initial virtual machine to be migrated is to be deployed under the premise of considering the virtual machine anti-affinity constraint; and schedules the virtual machine according to the first deployment plan and the second deployment plan. The disclosed embodiments can simplify the model through multi-stage orchestration and scheduling, accurately and efficiently solve the deployment plan of virtual machine orchestration and scheduling with complex constraints, improve the efficiency and success rate of virtual machine orchestration and scheduling, improve resource utilization, and reduce orchestration and scheduling costs.

[0028] The application scenarios of the virtual machine scheduling method of the embodiment of the present disclosure are as follows: Figure 1 As shown, it may include three stages, wherein in the first stage, virtual machines are divided into types, and a first virtual machine planning model is created without considering the virtual machine anti-affinity constraint, and the solution is performed with the virtual machine type as the granularity to determine the predicted deployment quantity of each type of initial virtual machines in each physical machine as the first deployment plan, and according to the initial deployment status of each type of initial virtual machines in each physical machine and the first deployment plan, the initial virtual machines to be migrated out of each type of initial virtual machines in each physical machine are determined;

[0029] In the second stage, for the target initial virtual machines to be migrated that have anti-affinity constraints among the initial virtual machines to be migrated, a second virtual machine planning model is created to determine the target physical machine to be deployed to each of the target initial virtual machines to be migrated under the premise of considering the anti-affinity constraints of the virtual machines as the second deployment plan; the first deployment plan is updated according to the second deployment plan, that is, the second deployment plan is merged with the first deployment plan;

[0030] In the third stage, virtual machine scheduling is performed according to the first deployment scheme and the second deployment scheme.

[0031] The virtual machine scheduling method disclosed in the present invention will be described in detail below in conjunction with specific embodiments.

[0032] refer to Figure 2 , Figure 2 A flowchart of a virtual machine scheduling method provided by an embodiment of the present disclosure. The method of this embodiment can be applied to any electronic device such as a terminal device or a server, and the virtual machine scheduling method includes:

[0033] S201. In response to a secondary scheduling request of a virtual machine, a first virtual machine planning model is created according to deployment attribute characteristics of an initial virtual machine deployed on a physical machine, and a first deployment plan that minimizes a total scheduling cost is determined according to the first virtual machine planning model, wherein the first deployment plan includes a predicted number of initial virtual machines of each type in each physical machine without considering anti-affinity constraints of the virtual machine.

[0034] In this embodiment, in the scenario of secondary scheduling of virtual machines in response to secondary scheduling requests of virtual machines, it is necessary to reschedule the initial virtual machines deployed in each physical machine, thereby freeing up the entire machine or large-scale space and improving the sales rate. In order to achieve the rescheduling of the initial virtual machines deployed in each physical machine, in this embodiment, a first virtual machine planning model can be created according to the deployment attribute characteristics of the initial virtual machines deployed on the physical machine, and then the deployment plan of the virtual machine can be determined based on the first virtual machine planning model. The first virtual machine planning model is a model that solves the virtual machine deployment plan with the virtual machine type as the granularity without considering the virtual machine anti-affinity constraint. Specifically, it can be a solution model represented by a functional relationship (including model optimization objectives, model optimization conditions, and solution decision variables), or it can be a model built by any tool or a machine learning model (which can input model optimization objectives and model optimization conditions and output decision variables).

[0035] Specifically, the goal of the first virtual machine planning model is to determine the predicted deployment quantity of each type of virtual machine in each physical machine without considering the virtual machine anti-affinity constraint, so as to minimize the total scheduling cost and obtain the first deployment plan. Anti-affinity constraint means that two virtual machines are deployed as separately as possible, including different situations such as not being allowed to be deployed on the same physical machine, not being allowed to be deployed on the physical machine of the same rack, and not being allowed to be deployed on the physical machine of the same switch. In this embodiment, the first virtual machine planning model does not consider the virtual machine anti-affinity constraint in order to simplify the model, reduce the model complexity, and improve the processing efficiency. In the subsequent consideration of the virtual machine anti-affinity constraint, a good scheduling effect can still be achieved. In addition, in this embodiment, the virtual machine is scheduled and scheduled based on the category as the granularity, which can simplify the first virtual machine planning model and reduce the scale of the model solution. The types of virtual machines can adopt equivalent classes, and the classification of virtual machine types can be achieved through some equivalent methods or rules, such as classification based on the usage of one or more resources by the virtual machine, or any other method can be used to classify the virtual machines.

[0036] Optionally, the total scheduling cost in this embodiment may include but is not limited to one or more of the migration cost of the initial virtual machine to be migrated, the cost of resource fragments generated after the remaining resources of each physical machine are used to create a virtual machine of the target type to be built, and the physical machine occupancy cost, wherein the migration cost of the initial virtual machine to be migrated may take into account factors such as technical cost, time cost, economic cost, and the migration cost of each type of initial virtual machine may be different; and the cost of resource fragments generated after the remaining resources of each physical machine are used to create a virtual machine of the target type to be built, that is, when the remaining resources of each physical machine are subsequently sold, one or more virtual machines of a certain target type to be built can be specified, until the remaining resources of the physical machine can no longer create the virtual machine of the target type to be built, at which time the remaining resources of the physical machine are resource fragments, and when different deployment schemes are adopted, the predicted deployment number of each type of virtual machine deployed on the physical machine is different, and the physical machine resource fragmentation rate will also be different, so resource fragments can also be considered as a cost in the total scheduling cost; the physical machine occupancy cost, that is, there will be operation and maintenance costs, energy consumption costs, etc. for each physical machine occupied.

[0037] The first virtual machine planning model may be an integer programming model.

[0038] S202: Determine the initial virtual machines to be migrated out of each type of initial virtual machines in each physical machine according to the initial deployment status of each type of initial virtual machines in each physical machine and the first deployment scheme.

[0039] In this embodiment, after determining the first deployment plan, since the first deployment plan uses the category of virtual machines as the granularity for deployment, that is, only the predicted deployment quantity of each type of initial virtual machines in each physical machine can be determined, or further, the number of initial virtual machines to be migrated out of each type of initial virtual machines in each physical machine can be determined, but the specific initial virtual machine to be migrated is not determined. Therefore, it is necessary to further determine the specific initial virtual machines to be migrated out of each type of initial virtual machines in each physical machine based on the initial deployment status of each type of initial virtual machines in each physical machine and the first deployment plan.

[0040] In a specific implementation, the target number of initial virtual machines of each type to be migrated from each physical machine is determined based on the initial deployment status of the initial virtual machines of each type in each physical machine and the first deployment plan. For example, assuming that there are 10 initial virtual machines of the i-th type deployed in the k-th physical machine, and the predicted deployment number of initial virtual machines of the i-th type in the k-th physical machine in the first deployment plan is 8, then the target number of initial virtual machines of the i-th type to be migrated from the k-th physical machine is 10-8=2; further, the initial virtual machines to be migrated can be determined from the initial virtual machines of each type in each physical machine based on the target number of initial virtual machines of each type to be migrated from each physical machine.

[0041] The initial virtual machine to be migrated out of each type of initial virtual machine on each physical machine may be determined in any manner or rule, such as by a heuristic solution or by random selection.

[0042] Optionally, initial virtual machines without anti-affinity constraints are preferentially selected from each type of initial virtual machines in each physical machine. If the number of initial virtual machines without anti-affinity constraints in each type of initial virtual machines in each physical machine is less than the corresponding target number, initial virtual machines with anti-affinity constraints are selected. The initial virtual machines without anti-affinity constraints are preferentially selected in order to reduce the number of initial virtual machines with anti-affinity constraints, thereby reducing the amount of calculation in the following S203 and improving efficiency.

[0043] Of course, when the initial virtual machines without anti-affinity constraints are preferentially selected from the initial virtual machines of each type in each physical machine, there may be a variety of different situations. For example, if the number of initial virtual machines without anti-affinity constraints in the i-th type of initial virtual machines in the k-th physical machine is greater than or equal to the target number of initial virtual machines to be migrated of the i-th type in the k-th physical machine, then the initial virtual machines selected are all initial virtual machines without anti-affinity constraints; if the number of initial virtual machines without anti-affinity constraints in the i-th type of initial virtual machines in the k-th physical machine is less than the target number of initial virtual machines to be migrated of the i-th type in the k-th physical machine, then the initial virtual machines selected are all initial virtual machines without anti-affinity constraints. If the target number of initial virtual machines of the i-th type to be migrated is greater than 0, all the initial virtual machines without anti-affinity constraints among the initial virtual machines of the i-th type in the k-th physical machine will be selected. However, the target number is less than the target number at this time, and the insufficient number will be selected from the initial virtual machines with anti-affinity constraints among the initial virtual machines of the i-th type in the k-th physical machine. If the number of initial virtual machines without anti-affinity constraints among the initial virtual machines of the i-th type in the k-th physical machine is 0, all of them need to be selected from the initial virtual machines with anti-affinity constraints among the initial virtual machines of the i-th type in the k-th physical machine.

[0044] S203, determining a target initial virtual machine to be migrated that has an anti-affinity constraint among the initial virtual machines to be migrated, and creating a second virtual machine planning model for the target initial virtual machine to be migrated, and determining a second deployment plan that minimizes the scheduling cost of the target initial virtual machine to be migrated according to the second virtual machine planning model, wherein the second deployment plan includes a target physical machine to which each target initial virtual machine to be migrated is to be deployed under the premise of considering the anti-affinity constraint of the virtual machine.

[0045] In this embodiment, after determining the virtual machines to be migrated, the target initial virtual machines to be migrated with anti-affinity constraints can be determined from the virtual machines to be migrated, and a second virtual machine planning model can be created for the scheduling process of the target initial virtual machines to be migrated. The second virtual machine planning model needs to consider the virtual machine anti-affinity constraints. Under the premise of considering the virtual machine anti-affinity constraints, the target physical machine to which each target initial virtual machine to be migrated with anti-affinity constraints is to be deployed is determined, so that after each target initial virtual machine to be migrated with anti-affinity constraints is deployed to the target physical machine, there is no anti-affinity with other virtual machines in the target physical machine, including but not limited to one or more different anti-affinity situations such as not being allowed to be deployed on the same physical machine with another virtual machine, not being allowed to be deployed on a physical machine in the same rack, not being allowed to be deployed on a physical machine on the same switch, etc. Optionally, the second virtual machine planning model is a model that solves the virtual machine deployment plan for the target initial virtual machine to be migrated with anti-affinity constraints under the premise of considering the virtual machine anti-affinity constraints. Specifically, it can be a solution model represented by a functional relationship (including model optimization objectives, model optimization conditions, and solution decision variables), or it can be a model built using any tool or a machine learning model (which can input model optimization objectives, model optimization conditions, and output decision variables).

[0046] Of course, if all the initial virtual machines to be migrated are target initial virtual machines to be migrated without anti-affinity constraints, S203 can be skipped.

[0047] The second virtual machine planning model may be an integer programming model.

[0048] S204. Schedule virtual machines according to the first deployment plan and the second deployment plan.

[0049] In this embodiment, after determining the first deployment scheme and the second deployment scheme, the first deployment scheme and the second deployment scheme can be integrated, and virtual machine scheduling is performed according to the first deployment scheme and the second deployment scheme. Specifically, each target initial virtual machine to be migrated with anti-affinity constraints is deployed to the corresponding target physical machine, and the target initial virtual machine to be migrated without anti-affinity constraints is migrated to a physical machine where the predicted deployment number of virtual machines of the corresponding type is insufficient. Alternatively, the predicted migration number of each type of virtual machine for each physical machine can be further determined, and then the target initial virtual machine to be migrated without anti-affinity constraints is migrated according to the predicted migration number and the target physical machine to which each target initial virtual machine to be migrated with anti-affinity constraints is to be deployed. Finally, the number of each type of initial virtual machine in each physical machine meets the corresponding predicted deployment number.

[0050] Optionally, when scheduling virtual machines according to the first deployment plan and the second deployment plan, considering the limited bandwidth, it is impossible to complete the migration of all the initial virtual machines to be migrated at one time. Therefore, it is necessary to schedule and execute the migration tasks for all the initial virtual machines to be migrated according to the updated deployment plan, especially to schedule the migration batches and sequence of each initial virtual machine to be migrated, so that the migration task satisfies: when each initial virtual machine to be migrated is migrated, it is ensured that the target physical machine to be migrated has sufficient resources to accommodate the initial virtual machine to be migrated and satisfies the anti-affinity constraint conditions, that is, at any time during the migration process, each virtual machine in each physical machine (including the initial virtual machines to be migrated that have not yet been migrated, the initial virtual machines that do not need to be migrated, and the virtual machines that have already been migrated) needs to satisfy the anti-affinity constraint, so as to ensure that no constraint conflicts occur during the migration process, thereby improving the overall migration efficiency and success rate.

[0051] Optionally, when scheduling the migration tasks for all the initial virtual machines to be migrated, any heuristic method or any other feasible method may be used for scheduling, which is not limited here.

[0052] The virtual machine orchestration and scheduling method provided in the present embodiment creates a first virtual machine planning model according to the deployment attribute characteristics of the initial virtual machine deployed on the physical machine in response to the virtual machine secondary scheduling request, determines a first deployment plan that minimizes the total scheduling cost according to the first virtual machine planning model, wherein the first deployment plan includes the predicted deployment quantity of each type of initial virtual machines in each physical machine without considering the virtual machine anti-affinity constraint; determines the initial virtual machines to be migrated out of each type of initial virtual machines in each physical machine according to the initial deployment status of each type of initial virtual machines in each physical machine and the first deployment plan; determines the target initial virtual machines to be migrated out with anti-affinity constraints among the initial virtual machines to be migrated, and creates a second virtual machine planning model for the target initial virtual machines to be migrated; determines a second deployment plan that minimizes the scheduling cost of the target initial virtual machines to be migrated according to the second virtual machine planning model, wherein the second deployment plan includes the target physical machine to which each target initial virtual machine to be migrated is to be deployed under the premise of considering the virtual machine anti-affinity constraint; and schedules the virtual machines according to the first deployment plan and the second deployment plan. This embodiment can simplify the model through multi-stage orchestration and scheduling, accurately and efficiently solve the deployment plan of virtual machine orchestration and scheduling with complex constraints, improve the efficiency and success rate of virtual machine orchestration and scheduling, improve resource utilization, and reduce orchestration and scheduling costs.

[0053] On the basis of any of the above embodiments, a first decision variable to be solved is used to construct a first model optimization objective and a first model constraint condition, wherein the first decision variable includes the predicted deployment quantity of each type of initial virtual machines in each physical machine, the predicted creation quantity of the target type of virtual machines to be built, and the predicted occupancy status of each physical machine; the first model optimization objective is the total scheduling cost, and the total scheduling cost is the weighted sum of the migration cost of the initial virtual machines to be migrated, the cost of resource fragments generated after the remaining resources of each physical machine are used to create the target type of virtual machines to be built, and the physical machine occupancy cost; the first model constraint condition is the constraint condition related to the first decision variable without considering the anti-affinity constraint of the virtual machine, wherein the first model constraint condition may include but is not limited to constraints related to the number of virtual machines of various types, constraints related to occupied resources, constraints on the number of migrations, etc., but does not include anti-affinity constraints.

[0054] Furthermore, the first decision variable that minimizes the optimization objective of the first model while satisfying the constraints of the first model can be determined as the first deployment plan, that is, the predicted deployment quantity of each type of initial virtual machines in each physical machine, the predicted creation quantity of additional target type of virtual machines to be created for each physical machine, and the predicted occupancy status of each physical machine are solved as the first deployment plan.

[0055] Of course, the predicted deployment number of each type of initial virtual machines in each physical machine to be solved, the predicted creation number of virtual machines of the target type to be built, and the predicted occupancy status of each physical machine are the main decision variables that need to be solved by the first virtual machine planning model. Of course, other decision variables can also be included.

[0056] Optionally, each physical machine may include two non-uniform memory access (NUMA) nodes. In the NUMA architecture, the CPU of the physical machine is usually divided into NUMA nodes. Each NUMA node has its own independent memory space and PCIE (Peripheral Component Interconnect Express) bus system, and each CPU communicates with each other through the QPI (Intel QuickPath Interconnect) bus. Some types of virtual machines need to be deployed on a single NUMA node of a physical machine, and some types of virtual machines need to be deployed on two NUMA nodes of a physical machine. Therefore, the above issues also need to be considered when determining the first deployment plan, that is, the first decision variable also includes the predicted deployment number of each type of initial virtual machines in each NUMA node of each physical machine, and the constraints related to the physical machine NUMA can be added to the constraints of the first objective function. Then, when solving the first objective function based on the constraints of the first objective function, the predicted deployment number of each type of initial virtual machines in each physical machine, the predicted deployment number of each type of initial virtual machines in each NUMA node of each physical machine, the predicted creation number of additional target type of virtual machines to be created for each physical machine, and the predicted occupancy status of each physical machine can be determined, so as to obtain a more accurate first deployment plan.

[0057] Based on the above embodiment, the first model constraint condition may include but is not limited to one or more of the following:

[0058] 1) The constraints on the deployment of the dual-NUMA initial virtual machine and the single-NUMA initial virtual machine on the two NUMA nodes of the physical machine. Specifically, the dual-NUMA initial virtual machine needs to be deployed on both NUMA nodes of the physical machine, and is essentially the same virtual machine, while the single-NUMA initial virtual machine can only be deployed on one NUMA node of the physical machine;

[0059] 2) The constraint that the initial virtual machine to be migrated out must be migrated in and out of two NUMA nodes of the physical machine at the same time, i.e., it must avoid migration to different NUMA nodes on the same physical machine when migrating in or out;

[0060] 3) The constraint that the number of initial virtual machines of each type remains unchanged before and after scheduling;

[0061] 4) Constraints on the resources occupied by each type of initial virtual machine in each physical machine, that is, the total amount of resources occupied by each type of initial virtual machine in each physical machine cannot exceed the amount of resources of the physical machine. Optionally, the total amount of resources occupied by virtual machines deployed on each NUMA node of the physical machine cannot exceed the amount of resources of the NUMA node.

[0062] 5) The constraint condition on the relationship between the predicted number of initial virtual machines of each type in each physical machine and the predicted occupancy status of each physical machine, that is, if the predicted number of initial virtual machines of any type in each physical machine is not 0, the physical machine is occupied;

[0063] 6) Constraints on the relationship between the predicted number of additional virtual machines of the target type to be built in each physical machine and the remaining resources of each physical machine, that is, the resources occupied by the predicted number of additional virtual machines of the target type to be built in each physical machine cannot exceed the remaining resources of the physical machine. Optionally, the resources occupied by the predicted number of additional virtual machines of the target type to be built on each NUMA node of the physical machine cannot exceed the remaining resources of the NUMA node.

[0064] 7) The constraint condition of the upper limit of the total number of migrations in the scheduling process, that is, to minimize the total number of migrations, the upper limit of the total number of migrations can be set in advance to avoid the total number of migrations in the scheduling process exceeding the upper limit of the total number of migrations.

[0065] 8) The constraint that a specific initial virtual machine is not allowed to migrate, that is, some virtual machines are not allowed to migrate.

[0066] Based on the above embodiment, this embodiment provides a specific first virtual machine planning model, as shown below.

[0067] The first decision variable to be solved in the first virtual machine planning model includes:

[0068] x i,k , represents the predicted number of deployments of the i-th type of initial virtual machines in the k-th physical machine;

[0069] represents the predicted number of deployments of the i-th type of initial virtual machines in the j-th NUMA in the k-th physical machine;

[0070] Indicates the number of initial virtual machines of type i that need to be migrated to the k-th physical machine;

[0071] s k , represents the predicted occupancy status of the kth physical machine, and takes a value of 0 or 1, 0 means unoccupied, and 1 means occupied;

[0072] e i,k, represents the predicted number of virtual machines of the i-th type to be created by the k-th physical machine;

[0073] signal i,k , indicating whether there is a migration operation for the i-th type of initial virtual machine in the k-th physical machine, 1 indicates migration, and 0 indicates migration;

[0074] The parameters in the first virtual machine planning model include:

[0075] y i,k , represents the initial deployment number of the i-th type of initial virtual machines on the k-th physical machine in the initial deployment situation;

[0076] represents the initial deployment number of the i-th type of initial virtual machines in the j-th NUMA unit of the k-th physical machine in the initial deployment situation;

[0077] d i , represents the total number of initial virtual machines of type i;

[0078] represents the usage of the rth resource by the i-th type of initial virtual machine;

[0079] Indicates the resource amount of the rth resource of the kth physical machine;

[0080] c i , represents the migration cost of the i-th type of initial virtual machine;

[0081] l i,k,j , represents the number of virtual machines of type i in the j-th NUMA of the k-th physical machine that are not allowed to migrate;

[0082] f i , represents the fragmentation weight of the i-th type of virtual machine, where if only the target type of virtual machine to be built is created, the fragmentation weight of the target type of virtual machine to be built is 1, and the fragmentation weights of the other types of virtual machines are 0;

[0083] M represents a larger natural number, which may be, for example, larger than the total number of initial virtual machines, or other values;

[0084] MI, represents the upper limit of the total number of migrations in the scheduling process;

[0085] numa_d, which indicates the mapping between NUMA number and virtual machine type, for example, numa_d = {1:[i1,i2,…,i n1 ],2:[i n1+1 ,i n1+2 ,…,i n2 ]}, where i1,i2,…,in1 The virtual machine type is a single NUMA virtual machine, the i n1+1 ,i n1+2 ,…,i n2 The type of virtual machine is a dual-NUMA virtual machine;

[0086] w r , represents the migration loss weight, that is, the weight of the migration cost of the initial virtual machine to be migrated;

[0087] w f , represents the fragmentation rate loss weight, that is, the weight of the cost of resource fragmentation generated after the remaining resources of each physical machine are used to create the target type of virtual machine to be built;

[0088] w p , represents the weight of the physical machine usage loss, that is, the weight of the physical machine occupancy cost.

[0089] Furthermore, a first virtual machine planning model is created:

[0090] The migration cost of the initial virtual machine to be migrated is:

[0091]

[0092] The cost of resource fragmentation generated after the remaining resources of each physical machine are used to create the target type of virtual machine to be built is:

[0093]

[0094] Among them, d t Indicates the number of deployed virtual machines of type t among all physical machines. represents the usage of the rth resource by the deployed virtual machine of the tth type; the denominator represents the total remaining resources of all physical machines, and the numerator represents the resource fragmentation generated after the target type of virtual machine to be built is created based on the total remaining resources. i is 1, and the f of other types of virtual machines i is 0, so this formula represents the resource fragmentation rate after only creating the target type of virtual machines to be built.

[0095] Number of physical machines occupied:

[0096]

[0097] Therefore, the optimization objective of the first model is:

[0098]

[0099] That is, the weighted sum of the migration cost of the initial virtual machine to be migrated, the cost of resource fragmentation generated after the remaining resources of each physical machine are used to create the target type of virtual machine to be built, and the physical machine occupancy cost.

[0100] The first model constraints are as follows:

[0101] 1) NUMA constraint: Since the initial virtual machine of dual-NUMA needs to be deployed on both NUMA nodes of the physical machine, and the initial virtual machine of single-NUMA can only be deployed on one NUMA node of the physical machine, it is expressed as follows:

[0102]

[0103] That is, for i∈numa d[2] The initial virtual machine of dual-NUMA, that is, the predicted deployment number of the same type of initial virtual machines in the two NUMA units in the k-th physical machine is the same and equal to the predicted deployment number of the i-th type of virtual machines in the k-th physical machine; and for i∈≤numa d[1] The initial virtual machine, that is, the initial virtual machine of a single NUMA, in the two NUMA units in the k-th physical machine, the sum of the predicted deployment numbers of the same type of initial virtual machines is equal to the predicted deployment number of the i-th type of virtual machines in the k-th physical machine.

[0104] 2) The constraint that the initial virtual machine to be migrated out must be migrated in and out of the two NUMA nodes of the physical machine at the same time, to avoid the migration of virtual machines between different NUMA nodes of the same physical machine. Therefore, an intermediate variable is established to indicate the migration in / out operation for the i-th initial virtual machine:

[0105]

[0106] Among them, on the kth physical machine, for the i-th type of initial virtual machine, if it is migrated, the first NUMA unit and the second NUMA unit of the kth physical machine are all negative values; if it is migration, then the first NUMA unit and the second NUMA unit of the kth physical machine All are positive values.

[0107] 3) The constraint that the number of initial virtual machines of each type remains unchanged before and after scheduling:

[0108]

[0109] 4) Constraints on the resources occupied by each type of initial virtual machine in each physical machine. The total amount of resources occupied by the virtual machines deployed on each NUMA node of each physical machine cannot exceed the amount of resources of the NUMA node:

[0110]

[0111] 5) The constraint condition on the relationship between the predicted number of initial virtual machines of each type in each physical machine and the predicted occupancy status of each physical machine, that is, if the predicted number of initial virtual machines of any type in each physical machine is not 0, then the physical machine is occupied. The following inequality can be used to determine whether the kth physical machine is occupied. If the physical machine is not occupied, the right side of the inequality is 0:

[0112] x i,k ≤Ms k ,i=1,…,N;k=1,…,K

[0113] 6) Constraints that do not allow migration of specific initial virtual machines:

[0114]

[0115] 7) The number of virtual machines of type i migrated from the kth physical machine needs to satisfy the following inequality to limit the number of migrations to not be a negative number:

[0116]

[0117] 8) Constraints on the relationship between the predicted number of additional virtual machines of the target type to be built created by the k-th physical machine and the remaining resources of each physical machine, that is, the predicted number of additional virtual machines of the target type to be built created on each NUMA node of the k-th physical machine depends on the remaining resources of the NUMA node and the resources occupied by the virtual machines of the target type to be built (different formulas are used for the virtual machines of the target type to be built of single NUMA and dual NUMA):

[0118]

[0119] 9) Constraints on the upper limit of the total number of migrations during the scheduling process:

[0120]

[0121] By optimizing the first model optimization target based on the first model constraint condition, a first decision variable that minimizes the first model optimization target while satisfying the first model constraint condition can be determined as the first deployment solution. The solution method is not limited in this embodiment.

[0122] It should be noted that if each physical machine includes more than two NUMA nodes, the above-mentioned related first decision variables, first model constraints, etc. can be adaptively adjusted according to the number of NUMA nodes.

[0123] Based on any of the above embodiments, constructing a second virtual machine planning model includes: using the second decision variable to be solved to construct a second model optimization target and a second model constraint condition, wherein the second decision variable includes the target physical machine to which each target initial virtual machine to be migrated is to be deployed; the second model optimization target is the scheduling cost of the target initial virtual machine to be migrated; the second model constraint condition is the constraint condition related to the second decision variable under the premise of considering the anti-affinity constraint of the virtual machine, wherein the second model constraint condition may include but is not limited to constraints related to the number of various types of virtual machines, constraints related to occupied resources, and anti-affinity constraints, etc.

[0124] Furthermore, when creating the second virtual machine planning model, since only the target initial virtual machine to be migrated with an anti-affinity constraint exists among the initial virtual machines to be migrated in the first deployment plan, the scheduling cost of the target initial virtual machine to be migrated in the second virtual machine planning model can only consider the migration cost of the target initial virtual machine to be migrated, that is, a second model optimization target can be created based on the target physical machine to which each target initial virtual machine to be migrated is to be deployed, and the second model optimization target represents the scheduling cost of the target initial virtual machine to be migrated.

[0125] Then, a second decision variable that minimizes the optimization target of the second model while satisfying the constraint condition of the second model can be determined as the second deployment solution.

[0126] Of course, the target physical machine to which each target initial virtual machine to be migrated out is to be deployed is the decision variable that the second virtual machine planning model mainly needs to solve, and of course other decision variables may also be included.

[0127] Optionally, in the case that each physical machine may include two NUMA nodes, the second decision variable also includes the target NUMA node to which each target initial virtual machine to be migrated is to be deployed, and the constraints related to the physical machine NUMA can be added to the constraints of the second objective function. Then, when the second model optimization objective is solved based on the second model constraints, the target physical machine and the target NUMA node to which each target initial virtual machine to be migrated is to be deployed can be determined as the second deployment plan.

[0128] Based on the above embodiment, the second model constraint condition may include but is not limited to one or more of the following:

[0129] 1) The constraint that each target initial virtual machine to be migrated can only be deployed on one target physical machine;

[0130] 2) a constraint condition for whether migration is allowed or not pre-configured for each target initial virtual machine to be migrated, that is, whether migration is allowed or not for a target initial virtual machine to be migrated can be pre-configured;

[0131] 3) Constraints on the resources occupied by each target initial virtual machine to be migrated out, that is, the total amount of resources occupied by all target initial virtual machines to be migrated out of each physical machine after they are migrated into the physical machine cannot exceed the amount of resources of the physical machine. More specifically, the total amount of resources occupied by all target initial virtual machines to be migrated out of any NUMA unit in each physical machine after they are migrated into the NUMA unit cannot exceed the amount of resources of the NUMA unit.

[0132] 4) The constraints on the deployment of the dual-NUMA target initial virtual machine to be migrated and the single-NUMA target initial virtual machine to be migrated on the two NUMA nodes of the physical machine, that is, the dual-NUMA target initial virtual machine to be migrated needs to be deployed on both NUMA nodes of the physical machine, and is essentially the same virtual machine, while the single-NUMA target initial virtual machine to be migrated can only be deployed on one NUMA node of the physical machine;

[0133] 5) The constraint condition that the target initial virtual machine to be migrated out is migrated in and out of two NUMA nodes of the physical machine at the same time, that is, it is avoided to migrate on different NUMA nodes on the same physical machine when migrating in or out;

[0134] 6) The target initial virtual machine to be migrated has at least one anti-affinity constraint condition in the physical machine range, rack range, and switch range. In the physical machine range, two target initial virtual machines to be migrated with anti-affinity cannot be deployed in the same physical machine, in the rack range, two target initial virtual machines to be migrated with anti-affinity cannot be deployed in the same rack, and in the switch range, two target initial virtual machines to be migrated with anti-affinity cannot be deployed in the same switch.

[0135] Based on the above embodiments, this embodiment provides a specific second virtual machine planning model (in which each decision variable and parameter are redefined, and the decision variables and parameters repeated in the first virtual machine planning model are based on the definitions in this embodiment), as shown below.

[0136] The second decision variables to be solved in the second virtual machine planning model include:

[0137] x i,k , indicating whether the i-th target initial virtual machine to be migrated is deployed in the k-th physical machine, 1 for yes, 0 for no;

[0138] Indicates whether the i-th target initial virtual machine to be migrated is deployed in the j-th NUMA unit of the k-th physical machine, 1 for yes, 0 for no;

[0139] Indicates whether the i-th target to be migrated initial virtual machine is migrated to the k-th physical machine, 1 means yes, 0 means no;

[0140] signal i,k , indicating the migration operation type of the i-th target initial virtual machine to be migrated out in the k-th physical machine, 1 indicates migration in, and 0 indicates migration out.

[0141] The parameters in the second virtual machine planning model include:

[0142] y i,k , indicating whether the i-th target initial virtual machine to be migrated is deployed in the k-th physical machine in the initial deployment situation, 1 indicates yes, 0 indicates no;

[0143] Indicates whether the i-th target to be migrated initial virtual machine is deployed in the j-th NUMA unit of the k-th physical machine in the initial deployment situation, 1 means yes, 0 means no;

[0144] represents the usage of the rth resource by the i-th target initial virtual machine to be migrated;

[0145] Indicates the resource amount of the rth resource of the kth physical machine;

[0146] c i , represents the migration cost of the i-th target initial virtual machine to be migrated;

[0147] l i , indicating whether the i-th target to be migrated out of the initial virtual machine is allowed to migrate;

[0148] M represents a larger natural number, for example, it may be larger than the total number of initial virtual machines to be migrated, or other values;

[0149] numa_d, which indicates the mapping between the NUMA number and the virtual machine type, for example, numa_d = {1:[i1,i2,…,i n1 ],2:[i n1+1 ,i n1+2 ,…,i n2 ]}, where i1,i2,…,i n1 The initial virtual machine to be migrated is a single NUMA virtual machine. n1+1 ,i n1+2 ,…,i n2 The initial virtual machine to be migrated is a dual-NUMA virtual machine;

[0150] Φn , represents the anti-affinity virtual machine set at the nth physical machine granularity, for example Represents a virtual machine Two nodes cannot be deployed on the same physical machine.

[0151] Π m , represents the anti-affinity virtual machine set at the mth rack granularity, for example Represents a virtual machine Two servers cannot be deployed in the same rack.

[0152] Ψ s , represents the sth anti-affinity virtual machine set at the switch granularity, for example Represents a virtual machine Two switches cannot be deployed on the same switch.

[0153] Ω p , represents the set of physical machines included in the p-th rack, for example Represents a physical machine Belong to the same rack p;

[0154] Θ q , represents the set of physical machines included in the qth switch, for example Represents a physical machine Belong to the same switch q;

[0155] Furthermore, a second model optimization target of the second virtual machine planning model is created, that is, the migration cost of the target initial virtual machine to be migrated out:

[0156]

[0157] The constraints of the second model are as follows:

[0158] 1) Constraints that each target initial virtual machine to be migrated can only be deployed on one target physical machine:

[0159]

[0160] 2) The pre-configured constraints for each target initial virtual machine to be migrated to determine whether migration is allowed:

[0161]

[0162] 3) Constraints on the resources occupied by each target initial virtual machine to be migrated out, that is, the total amount of resources occupied by all target initial virtual machines to be migrated out in any NUMA unit of each physical machine after migrating into the NUMA unit cannot exceed the amount of resources of the NUMA unit:

[0163]

[0164] 4) NUMA constraints, that is, the constraints on the deployment of the dual-NUMA target initial virtual machine to be migrated and the single-NUMA target initial virtual machine to be migrated on the two NUMA nodes of the physical machine. Since the dual-NUMA initial virtual machine needs to be deployed on both NUMA nodes of the physical machine, and the single-NUMA initial virtual machine can only be deployed on one NUMA node of the physical machine, it is expressed as follows:

[0165]

[0166] 5) The constraint condition that the target initial virtual machine to be migrated is migrated in and out of two NUMA nodes of the physical machine at the same time, that is, it is avoided to migrate on different NUMA nodes on the same physical machine when migrating in or out, which is expressed as follows:

[0167]

[0168] 6) Anti-affinity constraints of at least one of the physical machine range, rack range, and switch range of the target initial virtual machine to be migrated:

[0169] The anti-affinity constraint in the physical machine range is as follows, that is, in the kth physical machine i∈Φ n The total number of deployed virtual machines does not exceed 1:

[0170]

[0171] Where NumAniAffineMachine represents the number of anti-affinity virtual machine sets at the physical machine granularity;

[0172] The anti-affinity constraint in the rack scope is as follows, that is, in the mth physical machine i∈Π m The total number of deployed virtual machines does not exceed 1:

[0173]

[0174] Where NumRack represents the number of racks; NumAniAffineRack represents the number of anti-affinity virtual machine sets at the rack granularity;

[0175] The switch-wide anti-affinity constraint is as follows, i∈Ψ in the sth switch s The total number of deployed virtual machines does not exceed 1:

[0176]

[0177] NumSwitch represents the number of switches, and NumAniAffineSwitch represents the number of anti-affinity virtual machine sets at the switch granularity.

[0178] 7) The number of initial virtual machines to be migrated out of the k-th physical machine to the i-th target needs to satisfy the following inequality, which is used to limit the number of initial virtual machines to be migrated out of the i-th target to be not negative:

[0179]

[0180] By optimizing the model optimization target based on the second model constraint, a second decision variable that minimizes the second model optimization target while satisfying the second model constraint can be determined as the first deployment solution. The solution method is not limited in this embodiment.

[0181] It should be noted that if each physical machine includes more than two NUMA nodes, the above-mentioned related second decision variables, second model constraints, etc. can be adaptively adjusted according to the number of NUMA nodes.

[0182] Corresponding to the virtual machine scheduling method of the above embodiment, Figure 3 This is a structural block diagram of a virtual machine scheduling device provided by an embodiment of the present disclosure. For ease of explanation, only the parts related to the embodiment of the present disclosure are shown. Figure 3 The virtual machine orchestration and scheduling device 300 includes: a first orchestration unit 301, a second orchestration unit 302, and a scheduling unit 303.

[0183] The first orchestration unit 301 is used to create a first virtual machine planning model in response to a virtual machine secondary scheduling request according to the deployment attribute characteristics of the initial virtual machine deployed on the physical machine, and determine a first deployment plan that minimizes the total scheduling cost according to the first virtual machine planning model, wherein the first deployment plan includes a predicted deployment quantity of each type of initial virtual machine in each physical machine without considering the anti-affinity constraint of the virtual machine; and determine the initial virtual machine to be migrated in each type of initial virtual machine in each physical machine according to the initial deployment status of each type of initial virtual machine in each physical machine and the first deployment plan;

[0184] The second orchestration unit 302 is used to determine a target initial virtual machine to be migrated that has an anti-affinity constraint among the initial virtual machines to be migrated, and to create a second virtual machine planning model for the scheduling process of the target initial virtual machine to be migrated, and to determine a second deployment plan that minimizes the scheduling cost of the target initial virtual machine to be migrated according to the second virtual machine planning model, wherein the second deployment plan includes a target physical machine to which each target initial virtual machine to be migrated is to be deployed under the premise of considering the anti-affinity constraint of the virtual machine;

[0185] The scheduling unit 303 is used to schedule the virtual machine according to the first deployment scheme and the second deployment scheme.

[0186] In one or more embodiments of the present disclosure, when creating the first virtual machine planning model for the virtual machine scheduling process, the first orchestration unit 301 is used to:

[0187] Constructing the first virtual machine planning model includes: constructing the first model optimization target and the first model constraint condition by using the first decision variable to be solved, wherein the first decision variable includes the predicted deployment quantity of each type of initial virtual machines in each physical machine, the predicted creation quantity of the target type of virtual machines to be built, and the predicted occupancy status of each physical machine; the first model optimization target is the total scheduling cost, and the total scheduling cost is the weighted sum of the migration cost of the initial virtual machines to be migrated, the cost of resource fragments generated after the remaining resources of each physical machine are used to create the target type of virtual machines to be built, and the physical machine occupancy cost; the first model constraint condition is the constraint condition related to the first decision variable without considering the virtual machine anti-affinity constraint;

[0188] Accordingly, when determining the first deployment solution with the lowest total scheduling cost according to the first virtual machine planning model, the first orchestration unit 301 is configured to:

[0189] A first decision variable that minimizes the optimization objective of the first model while satisfying the first model constraint condition is determined as the first deployment solution.

[0190] In one or more embodiments of the present disclosure, each physical machine includes two non-uniform memory access NUMA nodes; the first decision variable also includes the predicted deployment quantity of each type of initial virtual machines in each NUMA node of each physical machine.

[0191] In one or more embodiments of the present disclosure, the first model constraint condition includes one or more of the following:

[0192] The constraints on the deployment of dual-NUMA initial virtual machines and single-NUMA initial virtual machines on two NUMA nodes of a physical machine;

[0193] The constraint that the initial virtual machine to be migrated is migrated in and out of the two NUMA nodes of the physical machine at the same time;

[0194] The constraint that the number of initial virtual machines of each type remains unchanged before and after scheduling;

[0195] Constraints on the resources occupied by each type of initial virtual machine in each physical machine;

[0196] A constraint on the relationship between the predicted number of initial virtual machines of each type deployed in each physical machine and the predicted occupancy state of each physical machine;

[0197] A constraint condition on the relationship between the predicted number of virtual machines of the target type to be created in each physical machine and the remaining resources of each physical machine;

[0198] The constraint condition of the upper limit of the total number of migrations in the scheduling process;

[0199] A constraint that does not allow migration of a specific initial virtual machine.

[0200] In one or more embodiments of the present disclosure, when determining the initial virtual machines to be migrated out of each type of initial virtual machines in each physical machine according to the initial deployment status of each type of initial virtual machines in each physical machine and the first deployment scheme, the first orchestration unit 301 is configured to:

[0201] Determine a target number of each type of initial virtual machines to be migrated from each physical machine according to the initial deployment status of each type of initial virtual machines in each physical machine and the first deployment plan;

[0202] According to the target number of each type of initial virtual machines to be migrated from each physical machine, the initial virtual machines to be migrated are determined from each type of initial virtual machines in each physical machine.

[0203] In one or more embodiments of the present disclosure, when determining the initial virtual machines to be migrated from each type of initial virtual machines in each physical machine according to the target number of each type of initial virtual machines to be migrated from each physical machine, the second orchestration unit 302 is configured to:

[0204] Initial virtual machines without anti-affinity constraints are preferentially selected from each type of initial virtual machines in each physical machine. If the number of initial virtual machines without anti-affinity constraints in each type of initial virtual machines in each physical machine is less than the corresponding target number, initial virtual machines with anti-affinity constraints are selected.

[0205] In one or more embodiments of the present disclosure, when creating the second virtual machine planning model for the target initial virtual machine to be migrated, the second orchestration unit 302 is configured to:

[0206] Constructing the second virtual machine planning model includes: constructing a second model optimization target and a second model constraint condition by using a second decision variable to be solved, wherein the second decision variable includes a target physical machine to which each target initial virtual machine to be migrated is to be deployed; the second model optimization target is the scheduling cost of the target initial virtual machine to be migrated; the second model constraint condition is a constraint condition related to the second decision variable under the premise of considering the virtual machine anti-affinity constraint;

[0207] Accordingly, when the second scheduling unit 302 determines the second deployment solution that minimizes the scheduling cost of the target initial virtual machine to be migrated according to the second virtual machine planning model, it is configured to:

[0208] A second decision variable that minimizes the optimization objective of the second model while satisfying the constraints of the second model is determined as the second deployment solution.

[0209] In one or more embodiments of the present disclosure, each of the physical machines includes two NUMA nodes; the second decision variable also includes a target NUMA node to which each of the target initial virtual machines to be migrated is to be deployed.

[0210] In one or more embodiments of the present disclosure, the second model constraint condition includes one or more of the following:

[0211] A constraint condition that each of the target initial virtual machines to be migrated can only be deployed on one target physical machine;

[0212] The constraints of whether migration is allowed pre-configured for each of the target initial virtual machines to be migrated;

[0213] Constraints on resources occupied by each of the target initial virtual machines to be migrated;

[0214] The constraints of the deployment of the dual-NUMA target initial virtual machine to be migrated and the single-NUMA target initial virtual machine to be migrated on two NUMA nodes of the physical machine;

[0215] The constraint condition that the target initial virtual machine to be migrated is migrated in and out of two NUMA nodes of the physical machine at the same time;

[0216] The target to-be-migrated initial virtual machine has an anti-affinity constraint condition in at least one range of a physical machine range, a rack range, and a switch range.

[0217] In one or more embodiments of the present disclosure, when the scheduling unit 303 performs virtual machine scheduling according to the first deployment scheme and the second deployment scheme, it is used to:

[0218] According to the first deployment plan and the second deployment plan, the migration task of the virtual machine to be migrated is determined, and the migration task meets the following conditions: the real-time remaining resources of the target physical machine to be migrated to when each initial virtual machine to be migrated meets the resource usage of the initial virtual machine to be migrated, and the anti-affinity constraint condition is met between the initial virtual machine to be migrated and the currently deployed virtual machine in the target physical machine; the migration task is executed.

[0219] The device provided in this embodiment can be used to execute the technical solution of the above method embodiment. Its implementation principle and technical effect are similar, and this embodiment will not be repeated here.

[0220] In order to implement the above embodiment, the embodiment of the present disclosure also provides an electronic device.

[0221] refer to Figure 4 , which shows a schematic diagram of the structure of an electronic device 400 suitable for implementing the embodiment of the present disclosure, and the electronic device 400 may be a terminal device or a server. The terminal device may include but is not limited to mobile terminals such as mobile phones, notebook computers, digital broadcast receivers, personal digital assistants (PDAs), tablet computers, portable multimedia players (PMPs), vehicle terminals (such as vehicle navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 4 The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0222] like Figure 4 As shown, the electronic device 400 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 401, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 402 or a program loaded from a storage device 408 to a random access memory (RAM) 403. Various programs and data required for the operation of the electronic device 400 are also stored in the RAM 403. The processing device 401, the ROM 402, and the RAM 403 are connected to each other via a bus 404. An input / output (I / O) interface 405 is also connected to the bus 404.

[0223] Typically, the following devices may be connected to the I / O interface 405: an input device 406 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 407 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 408 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 409. The communication device 409 may allow the electronic device 400 to communicate with other devices wirelessly or by wire to exchange data. Although Figure 4The electronic device 400 is shown with various devices, but it should be understood that it is not required to implement or possess all the devices shown. More or fewer devices may be implemented or possessed instead.

[0224] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program contains program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication device 409, or installed from the storage device 408, or installed from the ROM 402. When the computer program is executed by the processing device 401, the above-mentioned functions defined in the method of the embodiment of the present disclosure are executed.

[0225] It should be noted that the computer-readable medium disclosed above may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in combination with an instruction execution system, device or device. In the present disclosure, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, in which a computer-readable program code is carried. This propagated data signal may take a variety of forms, including but not limited to an electromagnetic signal, an optical signal, or any suitable combination of the above. The computer readable signal medium may also be any computer readable medium other than a computer readable storage medium, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer readable medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0226] The computer-readable medium may be included in the electronic device, or may exist independently without being incorporated into the electronic device.

[0227] The computer-readable medium carries one or more programs. When the one or more programs are executed by the electronic device, the electronic device executes the method shown in the above embodiment.

[0228] Computer program code for performing the operations of the present disclosure may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as an independent software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0229] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some implementations as replacements, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0230] The units involved in the embodiments described in the present disclosure may be implemented by software or hardware. The name of a unit does not limit the unit itself in some cases. For example, the first acquisition unit may also be described as a "unit for acquiring at least two Internet Protocol addresses".

[0231] The functions described above herein may be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), and the like.

[0232] In a first aspect, according to one or more embodiments of the present disclosure, a virtual machine scheduling method is provided, comprising:

[0233] In response to the virtual machine secondary scheduling request, a first virtual machine planning model is created according to the deployment attribute characteristics of the initial virtual machine deployed on the physical machine, and a first deployment scheme that minimizes the total scheduling cost is determined according to the first virtual machine planning model, wherein the first deployment scheme includes a predicted deployment quantity of each type of initial virtual machine in each physical machine without considering the virtual machine anti-affinity constraint;

[0234] Determine, according to the initial deployment status of each type of initial virtual machines in each physical machine and the first deployment scheme, the initial virtual machines to be migrated out of each type of initial virtual machines in each physical machine;

[0235] Determine a target initial virtual machine to be migrated that has an anti-affinity constraint among the initial virtual machines to be migrated, create a second virtual machine planning model for the target initial virtual machine to be migrated, and determine a second deployment plan that minimizes the scheduling cost of the target initial virtual machine to be migrated according to the second virtual machine planning model, wherein the second deployment plan includes a target physical machine to which each target initial virtual machine to be migrated is to be deployed under the premise of considering the anti-affinity constraint of the virtual machine;

[0236] Virtual machine scheduling is performed according to the first deployment scheme and the second deployment scheme.

[0237] According to one or more embodiments of the present disclosure, the step of creating a first virtual machine planning model for a virtual machine scheduling process includes:

[0238] Constructing the first virtual machine planning model includes: constructing the first model optimization target and the first model constraint condition by using the first decision variable to be solved, wherein the first decision variable includes the predicted deployment quantity of each type of initial virtual machines in each physical machine, the predicted creation quantity of the target type of virtual machines to be built, and the predicted occupancy status of each physical machine; the first model optimization target is the total scheduling cost, and the total scheduling cost is the weighted sum of the migration cost of the initial virtual machines to be migrated, the cost of resource fragments generated after the remaining resources of each physical machine are used to create the target type of virtual machines to be built, and the physical machine occupancy cost; the first model constraint condition is the constraint condition related to the first decision variable without considering the virtual machine anti-affinity constraint;

[0239] Accordingly, determining a first deployment solution that minimizes the total scheduling cost according to the first virtual machine planning model includes:

[0240] A first decision variable that minimizes the optimization objective of the first model while satisfying the first model constraint condition is determined as the first deployment solution.

[0241] According to one or more embodiments of the present disclosure, each physical machine includes two non-uniform memory access NUMA nodes; the first decision variable also includes the predicted deployment quantity of each type of initial virtual machines in each NUMA node of each physical machine.

[0242] According to one or more embodiments of the present disclosure, the first model constraint condition includes one or more of the following:

[0243] The constraints on the deployment of dual-NUMA initial virtual machines and single-NUMA initial virtual machines on two NUMA nodes of a physical machine;

[0244] The constraint that the initial virtual machine to be migrated is migrated in and out of the two NUMA nodes of the physical machine at the same time;

[0245] The constraint that the number of initial virtual machines of each type remains unchanged before and after scheduling;

[0246] Constraints on the resources occupied by each type of initial virtual machine in each physical machine;

[0247] A constraint on the relationship between the predicted number of initial virtual machines of each type deployed in each physical machine and the predicted occupancy state of each physical machine;

[0248] A constraint condition on the relationship between the predicted number of virtual machines of the target type to be created in each physical machine and the remaining resources of each physical machine;

[0249] The constraint condition of the upper limit of the total number of migrations in the scheduling process;

[0250] A constraint that does not allow migration of a specific initial virtual machine.

[0251] According to one or more embodiments of the present disclosure, determining the initial virtual machines to be migrated in each type of initial virtual machines on each physical machine according to the initial deployment status of each type of initial virtual machines on each physical machine and the first deployment scheme includes:

[0252] Determine a target number of each type of initial virtual machines to be migrated from each physical machine according to the initial deployment status of each type of initial virtual machines in each physical machine and the first deployment plan;

[0253] According to the target number of each type of initial virtual machines to be migrated from each physical machine, the initial virtual machines to be migrated are determined from each type of initial virtual machines in each physical machine.

[0254] According to one or more embodiments of the present disclosure, determining the initial virtual machines to be migrated from each type of initial virtual machines in each physical machine according to the target number of each type of initial virtual machines to be migrated from each physical machine includes:

[0255] Initial virtual machines without anti-affinity constraints are preferentially selected from each type of initial virtual machines in each physical machine. If the number of initial virtual machines without anti-affinity constraints in each type of initial virtual machines in each physical machine is less than the corresponding target number, initial virtual machines with anti-affinity constraints are selected.

[0256] According to one or more embodiments of the present disclosure, the step of creating a second virtual machine planning model for the target initial virtual machine to be migrated includes:

[0257] Constructing the second virtual machine planning model includes: constructing a second model optimization target and a second model constraint condition by using a second decision variable to be solved, wherein the second decision variable includes a target physical machine to which each target initial virtual machine to be migrated is to be deployed; the second model optimization target is the scheduling cost of the target initial virtual machine to be migrated; the second model constraint condition is a constraint condition related to the second decision variable under the premise of considering the virtual machine anti-affinity constraint;

[0258] Accordingly, the second deployment scheme that minimizes the scheduling cost of the target initial virtual machine to be migrated is determined according to the second virtual machine planning model, including:

[0259] A second decision variable that minimizes the optimization objective of the second model while satisfying the constraints of the second model is determined as the second deployment solution.

[0260] According to one or more embodiments of the present disclosure, each of the physical machines includes two NUMA nodes; the second decision variable also includes a target NUMA node to which each of the target initial virtual machines to be migrated is to be deployed.

[0261] According to one or more embodiments of the present disclosure, the second model constraint condition includes one or more of the following:

[0262] A constraint condition that each of the target initial virtual machines to be migrated can only be deployed on one target physical machine;

[0263] The constraints of whether migration is allowed pre-configured for each of the target initial virtual machines to be migrated;

[0264] Constraints on resources occupied by each of the target initial virtual machines to be migrated;

[0265] The constraints of the deployment of the dual-NUMA target initial virtual machine to be migrated and the single-NUMA target initial virtual machine to be migrated on two NUMA nodes of the physical machine;

[0266] The constraint condition that the target initial virtual machine to be migrated is migrated in and out of two NUMA nodes of the physical machine at the same time;

[0267] The target to-be-migrated initial virtual machine has an anti-affinity constraint condition in at least one range of a physical machine range, a rack range, and a switch range.

[0268] According to one or more embodiments of the present disclosure, the scheduling of virtual machines according to the first deployment scheme and the second deployment scheme includes:

[0269] According to the first deployment plan and the second deployment plan, the migration task of the virtual machine to be migrated is determined, and the migration task meets the following conditions: the real-time remaining resources of the target physical machine to be migrated to when each initial virtual machine to be migrated meets the resource usage of the initial virtual machine to be migrated, and the anti-affinity constraint condition is met between the initial virtual machine to be migrated and the currently deployed virtual machine in the target physical machine; the migration task is executed.

[0270] In a second aspect, according to one or more embodiments of the present disclosure, a virtual machine scheduling device is provided, including:

[0271] A first orchestration unit is configured to create a first virtual machine planning model in response to a virtual machine secondary scheduling request according to deployment attribute characteristics of an initial virtual machine deployed on a physical machine, and determine a first deployment plan that minimizes a total scheduling cost according to the first virtual machine planning model, wherein the first deployment plan includes a predicted number of initial virtual machines of each type in each physical machine without considering anti-affinity constraints of the virtual machines; and determine an initial virtual machine to be migrated out of each type of initial virtual machine in each physical machine according to an initial deployment status of each type of initial virtual machine in each physical machine and the first deployment plan;

[0272] A second orchestration unit is configured to determine a target initial virtual machine to be migrated that has an anti-affinity constraint among the initial virtual machines to be migrated, and to create a second virtual machine planning model for the scheduling process of the target initial virtual machine to be migrated, and to determine a second deployment plan that minimizes the scheduling cost of the target initial virtual machine to be migrated according to the second virtual machine planning model, wherein the second deployment plan includes a target physical machine to which each target initial virtual machine to be migrated is to be deployed under the premise of considering the anti-affinity constraint of the virtual machine;

[0273] A scheduling unit is used to schedule virtual machines according to the first deployment scheme and the second deployment scheme.

[0274] According to one or more embodiments of the present disclosure, when the first orchestration unit creates the first virtual machine planning model for the virtual machine scheduling process, it is used to:

[0275] Constructing the first virtual machine planning model includes: constructing the first model optimization target and the first model constraint condition by using the first decision variable to be solved, wherein the first decision variable includes the predicted deployment quantity of each type of initial virtual machines in each physical machine, the predicted creation quantity of the target type of virtual machines to be built, and the predicted occupancy status of each physical machine; the first model optimization target is the total scheduling cost, and the total scheduling cost is the weighted sum of the migration cost of the initial virtual machines to be migrated, the cost of resource fragments generated after the remaining resources of each physical machine are used to create the target type of virtual machines to be built, and the physical machine occupancy cost; the first model constraint condition is the constraint condition related to the first decision variable without considering the virtual machine anti-affinity constraint;

[0276] Accordingly, when the first orchestration unit determines the first deployment solution that minimizes the total scheduling cost according to the first virtual machine planning model, it is configured to:

[0277] A first decision variable that minimizes the optimization objective of the first model while satisfying the first model constraint condition is determined as the first deployment solution.

[0278] According to one or more embodiments of the present disclosure, each physical machine includes two non-uniform memory access NUMA nodes; the first decision variable also includes the predicted deployment quantity of each type of initial virtual machines in each NUMA node of each physical machine.

[0279] According to one or more embodiments of the present disclosure, the first model constraint condition includes one or more of the following:

[0280] The constraints on the deployment of dual-NUMA initial virtual machines and single-NUMA initial virtual machines on two NUMA nodes of a physical machine;

[0281] The constraint that the initial virtual machine to be migrated is migrated in and out of the two NUMA nodes of the physical machine at the same time;

[0282] The constraint that the number of initial virtual machines of each type remains unchanged before and after scheduling;

[0283] Constraints on the resources occupied by each type of initial virtual machine in each physical machine;

[0284] A constraint on the relationship between the predicted number of initial virtual machines of each type deployed in each physical machine and the predicted occupancy state of each physical machine;

[0285] A constraint condition on the relationship between the predicted number of virtual machines of the target type to be created in each physical machine and the remaining resources of each physical machine;

[0286] The constraint condition of the upper limit of the total number of migrations in the scheduling process;

[0287] A constraint that does not allow migration of a specific initial virtual machine.

[0288] According to one or more embodiments of the present disclosure, when the first orchestration unit determines the initial virtual machines to be migrated out of each type of initial virtual machines on each physical machine according to the initial deployment status of each type of initial virtual machines on each physical machine and the first deployment scheme, it is configured to:

[0289] Determine a target number of each type of initial virtual machines to be migrated from each physical machine according to the initial deployment status of each type of initial virtual machines in each physical machine and the first deployment plan;

[0290] According to the target number of each type of initial virtual machines to be migrated from each physical machine, the initial virtual machines to be migrated are determined from each type of initial virtual machines in each physical machine.

[0291] According to one or more embodiments of the present disclosure, when the second orchestration unit determines the initial virtual machines to be migrated from each type of initial virtual machines in each physical machine according to the target number of each type of initial virtual machines to be migrated from each physical machine, it is configured to:

[0292] Initial virtual machines without anti-affinity constraints are preferentially selected from each type of initial virtual machines in each physical machine. If the number of initial virtual machines without anti-affinity constraints in each type of initial virtual machines in each physical machine is less than the corresponding target number, initial virtual machines with anti-affinity constraints are selected.

[0293] According to one or more embodiments of the present disclosure, when the second orchestration unit creates the second virtual machine planning model for the target initial virtual machine to be migrated, it is configured to:

[0294] Constructing the second virtual machine planning model includes: constructing a second model optimization target and a second model constraint condition by using a second decision variable to be solved, wherein the second decision variable includes a target physical machine to which each target initial virtual machine to be migrated is to be deployed; the second model optimization target is the scheduling cost of the target initial virtual machine to be migrated; the second model constraint condition is a constraint condition related to the second decision variable under the premise of considering the virtual machine anti-affinity constraint;

[0295] Accordingly, when the second orchestration unit determines the second deployment solution that minimizes the scheduling cost of the target initial virtual machine to be migrated according to the second virtual machine planning model, it is configured to:

[0296] A second decision variable that minimizes the optimization objective of the second model while satisfying the constraints of the second model is determined as the second deployment solution.

[0297] According to one or more embodiments of the present disclosure, each of the physical machines includes two NUMA nodes; the second decision variable also includes a target NUMA node to which each of the target initial virtual machines to be migrated is to be deployed.

[0298] According to one or more embodiments of the present disclosure, the second model constraint condition includes one or more of the following:

[0299] A constraint condition that each of the target initial virtual machines to be migrated can only be deployed on one target physical machine;

[0300] The constraints of whether migration is allowed pre-configured for each of the target initial virtual machines to be migrated;

[0301] Constraints on resources occupied by each of the target initial virtual machines to be migrated;

[0302] The constraints of the deployment of the dual-NUMA target initial virtual machine to be migrated and the single-NUMA target initial virtual machine to be migrated on two NUMA nodes of the physical machine;

[0303] The constraint condition that the target initial virtual machine to be migrated is migrated in and out of two NUMA nodes of the physical machine at the same time;

[0304] The target to-be-migrated initial virtual machine has an anti-affinity constraint condition in at least one range of a physical machine range, a rack range, and a switch range.

[0305] According to one or more embodiments of the present disclosure, when the scheduling unit performs virtual machine scheduling according to the first deployment scheme and the second deployment scheme, it is used to:

[0306] According to the first deployment plan and the second deployment plan, the migration task of the virtual machine to be migrated is determined, and the migration task meets the following conditions: the real-time remaining resources of the target physical machine to be migrated to when each initial virtual machine to be migrated meets the resource usage of the initial virtual machine to be migrated, and the anti-affinity constraint condition is met between the initial virtual machine to be migrated and the currently deployed virtual machine in the target physical machine; the migration task is executed.

[0307] In a third aspect, according to one or more embodiments of the present disclosure, there is provided an electronic device, comprising: at least one processor and a memory;

[0308] The memory stores computer-executable instructions;

[0309] The at least one processor executes the computer-executable instructions stored in the memory, so that the at least one processor executes the virtual machine orchestration and scheduling method described in the first aspect and various possible designs of the first aspect.

[0310] In a fourth aspect, according to one or more embodiments of the present disclosure, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer execution instructions. When a processor executes the computer execution instructions, the virtual machine orchestration and scheduling method described in the first aspect and various possible designs of the first aspect is implemented.

[0311] In a fifth aspect, according to one or more embodiments of the present disclosure, a computer program product is provided, including a computer program, wherein when the computer program is executed by a processor, the virtual machine scheduling method described in the first aspect and various possible designs of the first aspect is implemented.

[0312] The above description is only a preferred embodiment of the present disclosure and an explanation of the technical principles used. Those skilled in the art should understand that the scope of disclosure involved in the present disclosure is not limited to the technical solutions formed by a specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above disclosed concept. For example, the above features are replaced with the technical features with similar functions disclosed in the present disclosure (but not limited to) by each other to form a technical solution.

[0313] In addition, although each operation is described in a specific order, this should not be understood as requiring these operations to be performed in the specific order shown or in a sequential order. Under certain circumstances, multitasking and parallel processing may be advantageous. Similarly, although some specific implementation details are included in the above discussion, these should not be interpreted as limiting the scope of the present disclosure. Some features described in the context of a separate embodiment can also be implemented in a single embodiment in combination. On the contrary, the various features described in the context of a single embodiment can also be implemented in multiple embodiments individually or in any suitable sub-combination mode.

[0314] Although the subject matter has been described in language specific to structural features and / or methodological logical actions, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. On the contrary, the specific features and actions described above are merely example forms of implementing the claims.

Claims

1. A virtual machine scheduling method, characterized in that: include: In response to the virtual machine secondary scheduling request, a first virtual machine planning model is created according to the deployment attribute characteristics of the initial virtual machine deployed on the physical machine, and a first deployment scheme that minimizes the total scheduling cost is determined according to the first virtual machine planning model, wherein the first deployment scheme includes a predicted deployment quantity of each type of initial virtual machine in each physical machine without considering the virtual machine anti-affinity constraint; Determine, according to the initial deployment status of each type of initial virtual machines in each physical machine and the first deployment scheme, the initial virtual machines to be migrated out of each type of initial virtual machines in each physical machine; Determine a target initial virtual machine to be migrated that has an anti-affinity constraint among the initial virtual machines to be migrated, create a second virtual machine planning model for the target initial virtual machine to be migrated, and determine a second deployment plan that minimizes the scheduling cost of the target initial virtual machine to be migrated according to the second virtual machine planning model, wherein the second deployment plan includes a target physical machine to which each target initial virtual machine to be migrated is to be deployed under the premise of considering the anti-affinity constraint of the virtual machine; Virtual machine scheduling is performed according to the first deployment scheme and the second deployment scheme.

2. The method according to claim 1, characterized in that: The step of creating a first virtual machine planning model for a virtual machine scheduling process includes: Constructing the first virtual machine planning model includes: constructing the first model optimization target and the first model constraint condition by using the first decision variable to be solved, wherein the first decision variable includes the predicted deployment quantity of each type of initial virtual machines in each physical machine, the predicted creation quantity of the target type of virtual machines to be built, and the predicted occupancy status of each physical machine; the first model optimization target is the total scheduling cost, and the total scheduling cost is the weighted sum of the migration cost of the initial virtual machines to be migrated, the cost of resource fragments generated after the remaining resources of each physical machine are used to create the target type of virtual machines to be built, and the physical machine occupancy cost; the first model constraint condition is the constraint condition related to the first decision variable without considering the virtual machine anti-affinity constraint; Accordingly, determining the first deployment solution that minimizes the total scheduling cost according to the first virtual machine planning model includes: A first decision variable that minimizes the optimization objective of the first model while satisfying the first model constraint condition is determined as the first deployment solution.

3. The method according to claim 2, characterized in that Each physical machine includes two non-uniform memory access NUMA nodes; the first decision variable also includes the predicted deployment quantity of each type of initial virtual machines in each NUMA node of each physical machine.

4. The method according to claim 3, characterized in that The first model constraint condition includes one or more of the following: The constraints on the deployment of dual-NUMA initial virtual machines and single-NUMA initial virtual machines on two NUMA nodes of a physical machine; The constraint that the initial virtual machine to be migrated is migrated in and out of the two NUMA nodes of the physical machine at the same time; The constraint that the number of initial virtual machines of each type remains unchanged before and after scheduling; Constraints on the resources occupied by each type of initial virtual machine in each physical machine; A constraint on the relationship between the predicted number of initial virtual machines of each type deployed in each physical machine and the predicted occupancy state of each physical machine; A constraint condition on the relationship between the predicted number of virtual machines of the target type to be created in each physical machine and the remaining resources of each physical machine; The constraint condition of the upper limit of the total number of migrations in the scheduling process; A constraint that does not allow migration of a specific initial virtual machine.

5. The method according to any one of claims 1 to 4, characterized in that: The step of determining the initial virtual machines to be migrated out of each type of initial virtual machines in each physical machine according to the initial deployment status of each type of initial virtual machines in each physical machine and the first deployment scheme includes: Determine a target number of each type of initial virtual machines to be migrated from each physical machine according to the initial deployment status of each type of initial virtual machines in each physical machine and the first deployment plan; According to the target number of each type of initial virtual machines to be migrated from each physical machine, the initial virtual machines to be migrated are determined from each type of initial virtual machines in each physical machine.

6. The method according to claim 5, characterized in that The step of determining the initial virtual machines to be migrated from each type of initial virtual machines in each physical machine according to the target number of each type of initial virtual machines to be migrated from each physical machine includes: Initial virtual machines without anti-affinity constraints are preferentially selected from each type of initial virtual machines in each physical machine. If the number of initial virtual machines without anti-affinity constraints in each type of initial virtual machines in each physical machine is less than the corresponding target number, initial virtual machines with anti-affinity constraints are selected.

7. The method according to claim 1, characterized in that The step of creating a second virtual machine planning model for the target initial virtual machine to be migrated includes: Constructing the second virtual machine planning model includes: constructing a second model optimization target and a second model constraint condition by using a second decision variable to be solved, wherein the second decision variable includes a target physical machine to which each target initial virtual machine to be migrated is to be deployed; the second model optimization target is the scheduling cost of the target initial virtual machine to be migrated; the second model constraint condition is a constraint condition related to the second decision variable under the premise of considering the virtual machine anti-affinity constraint; Accordingly, the second deployment scheme that minimizes the scheduling cost of the target initial virtual machine to be migrated is determined according to the second virtual machine planning model, including: A second decision variable that minimizes the optimization objective of the second model while satisfying the constraints of the second model is determined as the second deployment solution.

8. The method according to claim 7, characterized in that Each of the physical machines includes two NUMA nodes; the second decision variable also includes a target NUMA node to which each of the target initial virtual machines to be migrated is to be deployed.

9. The method according to claim 8, characterized in that The second model constraint condition includes one or more of the following: A constraint condition that each of the target initial virtual machines to be migrated can only be deployed on one target physical machine; The constraints of whether migration is allowed pre-configured for each of the target initial virtual machines to be migrated; Constraints on resources occupied by each of the target initial virtual machines to be migrated; The constraints of the deployment of the dual-NUMA target initial virtual machine to be migrated and the single-NUMA target initial virtual machine to be migrated on two NUMA nodes of the physical machine; The constraint condition that the target initial virtual machine to be migrated is migrated in and out of two NUMA nodes of the physical machine at the same time; The target to-be-migrated initial virtual machine has an anti-affinity constraint condition in at least one range of a physical machine range, a rack range, and a switch range.

10. The method according to claim 1, characterized in that The performing virtual machine scheduling according to the first deployment scheme and the second deployment scheme includes: Determine the migration task of the virtual machine to be migrated according to the first deployment scheme and the second deployment scheme, and the migration task satisfies the following conditions: the real-time remaining resource amount of the target physical machine to be migrated to when each initial virtual machine to be migrated satisfies the resource usage of the initial virtual machine to be migrated, and the initial virtual machine to be migrated satisfies the anti-affinity constraint condition with the currently deployed virtual machine in the target physical machine; Execute the migration task.

11. A virtual machine scheduling device, characterized in that: include: A first orchestration unit is configured to create a first virtual machine planning model in response to a virtual machine secondary scheduling request according to deployment attribute characteristics of an initial virtual machine deployed on a physical machine, and determine a first deployment plan that minimizes a total scheduling cost according to the first virtual machine planning model, wherein the first deployment plan includes a predicted number of initial virtual machines of each type in each physical machine without considering anti-affinity constraints of the virtual machines; and determine an initial virtual machine to be migrated out of each type of initial virtual machine in each physical machine according to an initial deployment status of each type of initial virtual machine in each physical machine and the first deployment plan; A second orchestration unit is configured to determine a target initial virtual machine to be migrated that has an anti-affinity constraint among the initial virtual machines to be migrated, and to create a second virtual machine planning model for the target initial virtual machine to be migrated, and to determine a second deployment plan that minimizes the scheduling cost of the target initial virtual machine to be migrated according to the second virtual machine planning model, wherein the second deployment plan includes a target physical machine to which each target initial virtual machine to be migrated is to be deployed under the premise of considering the anti-affinity constraint of the virtual machine; A scheduling unit is used to schedule virtual machines according to the first deployment scheme and the second deployment scheme.

12. An electronic device, characterized in that: include: Processor and memory; The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory, so that the processor performs the method according to any one of claims 1 to 10.

13. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, and when the processor executes the computer-executable instructions, the method according to any one of claims 1 to 10 is implemented.

14. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the method according to any one of claims 1 to 10 is implemented.

Images