NUMA Granularity Defragmentation Method

Abstract

The present invention provides a NUMA granularity defragmentation method, comprising: defragmenting the host machine's resources at the NUMA granularity based on the host machine's resource allocation information; sorting the host machines based on supply and demand matching during the defragmentation, and then sorting the NUMA within the host machine; scheduling virtual machines based on the supply and demand matching, searching for the optimal destination host for the selected virtual machines or virtual machine combinations, and determining a migration strategy for the virtual machines or virtual machine combinations; and optimizing and executing the migration strategy for the virtual machines or virtual machine combinations. The present invention defragments the resource pool through the defragmentation function, thereby increasing the sellable number of virtual machines of the specified specification, improving resource utilization in the resource pool, and avoiding the costs associated with resource expansion.

Description

Technical Field

[0001] The present invention relates to the technical field of cloud computing, and in particular to a NUMA granularity defragmentation method. Background Art

[0002] Existing networks have a large number of resource pools with surplus resources, but customers are unable to provision VMs of specified specifications. Consequently, the number of VMs available for sale is insufficient to meet customer demand. Although each host machine has surplus resources, insufficient CPU resources, insufficient memory resources, or both prevent the provision of VMs of specified specifications. Ultimately, the total remaining resources of all hosts in the resource pool are sufficient, but no more VMs can be sold, resulting in resource fragmentation and waste.

[0003] Currently, there is no effective method for resource defragmentation. Existing technologies mainly focus on the resource utilization of virtual machines and the host machine, and reduce resource fragmentation by scheduling virtual machines to balance the CPU and memory resource utilization of the host machine. However, this method does not consider the NUMA information of the host machine and only defragments at the host machine granularity. It is not applicable when the host machine is multi-NUMA, and it cannot meet the specifications required by users. Summary of the Invention

[0004] In view of this, the present invention proposes a NUMA granularity defragmentation method, which aims to solve the problem of high resource fragmentation rate and low resource pool utilization in the resource pool.

[0005] In one aspect, the present invention provides a NUMA granularity defragmentation method, comprising:

[0006] Based on the host's resource allocation information, the host's resources are defragmented at the NUMA granularity.

[0007] When defragmenting resources on the host machine, after sorting the host machine based on supply and demand matching, the NUMA in the host machine is sorted;

[0008] Perform virtual machine scheduling based on supply and demand matching, find the best destination host for the selected virtual machine or virtual machine combination, and determine the migration strategy for the virtual machine or virtual machine combination;

[0009] Optimize and execute the migration strategy of the virtual machine or combination of virtual machines.

[0010] Furthermore, when defragmenting resources of the host machine, the process includes:

[0011] Fill each host machine with placeholder virtual machines until the host machine is unable to load a placeholder virtual machine; wherein the placeholder virtual machine is the input virtual machine specification flavor;

[0012] After the host machine is filled, a combination of virtual machines is found, and the virtual machine combination is migrated to the remaining suitable host machine so that the host machine can start at least one virtual machine of the flavor.

[0013] Furthermore, after the hosts are sorted based on supply and demand matching, the NUMA within the hosts is sorted, including:

[0014] Based on the remaining memory and CPU resources of the host machine, the remaining resources are scored and evaluated to perform host scoring;

[0015] Based on the host scoring results, the hosts are sorted in ascending order, with the resource allocation rates of the sorted hosts increasing in sequence and the remaining resource amounts of the hosts decreasing in sequence, and a host with more remaining resources is found;

[0016] Based on the sorting results of the host machines, the first host machine is determined, the CPU and memory resource scarcity is calculated according to the remaining CPU and memory resources of each NUMA of the first host machine, each NUMA in the host machine is scored according to the resource scarcity, and the NUMAs in the host machine are sorted in ascending order according to the scores.

[0017] Furthermore, when scoring and evaluating the remaining resources based on the remaining memory and CPU resources of the host machine, the following formula is used for calculation:

[0018]

[0019]

[0020]

[0021] Among them, S C is the sum of the CPU allocation rates of all hosts, S M is the sum of the memory allocation rates of all hosts, is the CPU allocation rate of the i-th host, is the memory allocation rate of the i-th host, n is the number of hosts, Score i The score of the i-th host.

[0022] Furthermore, the CPU resource scarcity is calculated according to the following formula:

[0023]

[0024] Calculate the memory resource scarcity according to the following formula

[0025]

[0026] in, is the remaining amount of CPU resources of the i-th NUMA in the host machine, is the remaining amount of memory of the i-th NUMA in the host;

[0027] Score each NUMA in the host machine according to the following formula:

[0028]

[0029]

[0030]

[0031] Where n is the number of NUMA in the host machine, is the sum of the CPU scarcity of all NUMAs in the host machine, is the sum of the memory scarcity of all NUMAs in the host machine, Score of the i-th NUMA in the host.

[0032] Furthermore, when virtual machine scheduling is performed based on supply and demand matching, it includes:

[0033] Determine the optimal virtual machine or combination of virtual machines to meet the resource requirements of the host machine to start another virtual machine of a specified flavor.

[0034] Furthermore, when determining the optimal virtual machine or combination of virtual machines to meet the resource requirements of starting another virtual machine of a specified flavor on the host machine, the following steps are included:

[0035] Select the first f numa[numa1,numa2,…,numa f ], numa f Represents the fth numa, and calculates the amount of resources required to open another virtual machine on each numa in:

[0036]

[0037]

[0038] Among them, V c The CPU size of the flavor to be sorted, in units of cores, V m The memory size of the flavor to be sorted, in GB. The amount of CPU resources required for a virtual machine for the i-th numa, The amount of memory resources required for a virtual machine for the i-th numa, numa i The CPU remaining amount is the CPU remaining amount of numa i, numa i The remaining memory is the remaining memory of the ith numa;

[0039] when and When it is less than or equal to 0, it is judged that numa i The remaining resources meet the resource requirements. All NUMAs whose resources meet the requirements are removed from the f NUMA sequences, and the number of remaining NUMAs is determined to be f'. The virtual machines that are not related to the f' NUMAs are removed.

[0040] Furthermore, after removing the virtual machines that are not related to f' numas, the following are included:

[0041] The remaining virtual machines are sorted in ascending order according to their resource usage on each numa. When the average CPU shortage of f numas is greater than or equal to the average memory shortage, the virtual machines are sorted according to the average CPU usage on each numa. Otherwise, the virtual machines are sorted according to the average memory usage on each numa.

[0042] Furthermore, when the remaining virtual machines are sorted in ascending order according to their resource occupancy on each numa, it includes:

[0043] When sorting, the similarity between the virtual machine and f' numas is added as an influencing factor for sorting.

[0044] Furthermore, when searching for the best destination host for the selected virtual machine or combination of virtual machines, the following steps are included:

[0045] A virtual machine or a combination of virtual machines that meets the resource requirements of the host machine is selected from the sorted virtual machines. The virtual machine or combination of virtual machines is selected by double pointer selection, specifically:

[0046] The two pointers start pointing to the first virtual machine after sorting at the same time;

[0047] One of the pointers slides backward until the total resources of the virtual machine are greater than the resource requirements of the host machine and then stops;

[0048] The other pointer also slides backward until the total resources of the virtual machine equals the resource requirements of the host machine.

[0049] Get the virtual machine or combination of virtual machines in the range pointed to by the double pointer.

[0050] Furthermore, when searching for the best destination host for the selected virtual machine or combination of virtual machines, the following steps are also included:

[0051] Among the hosts after the host is sorted, it is traversed from the back to the front first, and the remaining resources of each numa of the host are determined, and the number of numas that can satisfy the resources occupied by the virtual machine in each numa is calculated;

[0052] When the number of numa is less than the number of virtual machine cross-numa, continue to traverse the next host;

[0053] When the number of NUMAs is greater than or equal to the number of NUMAs spanned by the virtual machine, all NUMAs in the host machine that meet the resource requirements of the virtual machine are optimally screened. If the virtual machine spans j NUMAs, the j NUMAs are sorted according to similarity and the optimal first j NUMAs are selected as the NUMA positions of the virtual machine in the target host machine.

[0054] Furthermore, when optimizing and executing the migration strategy of the virtual machine or the combination of virtual machines, redundancy is removed from the number of migrated virtual machines, and causal sorting is performed on the migrated virtual machines.

[0055] Furthermore, when defragmenting the host's resources at the NUMA granularity based on the host's resource allocation information, the fragmentation rate is measured by the memory fragmentation rate, CPU fragmentation rate, and specification fragmentation rate. The calculation method is as follows:

[0056] Remaining memory of a single host = (host memory - host reserved memory) * host memory over-provisioning ratio - host used memory

[0057] The number of remaining CPU cores on a single host = (the number of CPU cores on the host - the number of CPU cores reserved on the host) * the over-provisioning ratio of the number of CPU cores on the host - the number of CPU cores used on the host

[0058]

[0059]

[0060]

[0061] Compared with the existing technology, the beneficial effect of the present invention lies in that the present invention provides a migration plan through resource analysis based on the supply and demand matching idea from the host machine NUMA level, and migrates virtual machines according to the migration plan, with the purpose of reducing resource fragmentation so that the resource pool can provide more virtual machines of specified specifications. In particular, when the resource pool still has remaining resources but can no longer provide virtual machines of the customer's specified specifications, the resource pool is defragmented through the defragmentation function to increase the saleable quantity of virtual machines of this specification, improve the resource utilization of the resource pool, and avoid the cost brought by resource expansion. BRIEF DESCRIPTION OF THE DRAWINGS

[0062] Various other advantages and benefits will become apparent to those skilled in the art upon reading the detailed description of the preferred embodiment below. The accompanying drawings are for illustration purposes only and are not to be considered as limiting the present invention. The same reference symbols are used throughout the drawings to represent the same components. In the drawings:

[0063] Figure 1 A functional block diagram of a defragmentation system provided by an embodiment of the present invention;

[0064] Figure 2 A flowchart of a NUMA granularity defragmentation method provided by an embodiment of the present invention;

[0065] Figure 3 A schematic diagram of a flow chart of a NUMA granularity defragmentation method provided by an embodiment of the present invention;

[0066] Figure 4 A schematic diagram illustrating the remaining resources of a host machine before and after virtual machine migration provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0067] The exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided to enable a more thorough understanding of the present disclosure and to fully convey the scope of the present disclosure to those skilled in the art. It should be noted that, unless there is a conflict, the embodiments of the present disclosure and the features described in the embodiments can be combined with each other. The present invention will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0068] This embodiment proposes a NUMA granularity defragmentation method based on the supply and demand matching concept. From the host machine NUMA level, a migration plan is given through resource analysis based on the supply and demand matching concept, and virtual machine migration is performed according to the migration plan. The purpose is to reduce resource fragmentation so that the resource pool can provide more virtual machines of specified specifications. In particular, when the resource pool still has remaining resources but can no longer provide virtual machines of the customer's specified specifications, the resource pool is defragmented through the defragmentation function to increase the saleable number of virtual machines of the specification, improve the resource utilization of the resource pool, and avoid the cost of resource expansion.

[0069] See Figure 1As shown, this embodiment also proposes a defragmentation system for executing a NUMA granularity defragmentation method, wherein the defragmentation system includes a front-end display module, a data service, an algorithm service, a scheduling service, a Region-level API forwarding service, and a network-wide API forwarding service, wherein the front-end display module issues defragmentation requests and displays the effects and benefits before and after defragmentation. The network-wide API forwarding service manages all resource pools and is responsible for forwarding front-end requests to a corresponding specific resource pool. The Region-level API forwarding service manages all services of a resource pool and is responsible for forwarding requests to specific services. The data service collects and pre-processes resource data such as virtual machine and host memory (including large page memory), CPU, and NUMA-related data. The scheduling service schedules virtual machines based on the correspondence between the virtual machine and the target host to be migrated given by the algorithm service. The algorithm service makes scheduling strategy decisions through the defragmentation algorithm based on the information collected by the data service, and selects the optimal scheduling solution for the scheduling service.

[0070] See Figure 2 As shown, a NUMA granularity defragmentation method provided in this embodiment includes the following steps:

[0071] Step S100: Defragmenting the host's resources at the NUMA granularity based on the host's resource allocation information;

[0072] Step S200: When defragmenting the host machine's resources, sort the host machines based on supply and demand matching, and then sort the NUMAs within the host machines;

[0073] Step S300: scheduling virtual machines based on supply and demand matching, finding the best destination host for the selected virtual machines or virtual machine combinations, and determining migration strategies for the virtual machines or virtual machine combinations;

[0074] Step S400: Optimize and execute the migration strategy of the virtual machine or the combination of virtual machines.

[0075] In this embodiment, a migration plan is given through resource analysis at the host machine NUMA level based on the supply and demand matching concept, and virtual machine migration is performed according to the migration plan. The purpose is to reduce resource fragmentation so that the resource pool can provide more virtual machines of specified specifications. In particular, when the resource pool still has remaining resources but can no longer provide virtual machines of the specifications specified by the customer, the resource pool is defragmented through the defragmentation function to increase the saleable number of virtual machines of this specification, improve the resource utilization of the resource pool, and avoid the cost of resource expansion.

[0076] In the above step S100, when defragmenting the host's resources at the NUMA granularity based on the host's resource allocation information, the fragmentation rate is measured by the memory fragmentation rate, the CPU fragmentation rate, and the specification fragmentation rate, and the calculation method is as follows:

[0077] Remaining memory of a single host = (host memory - host reserved memory) * host memory over-provisioning ratio - host used memory

[0078] The number of remaining CPU cores on a single host = (the number of CPU cores on the host - the number of CPU cores reserved on the host) * the over-provisioning ratio of the number of CPU cores on the host - the number of CPU cores used on the host

[0079]

[0080]

[0081]

[0082] Specifically, the above embodiment measures the resource fragmentation rate of the host machine using three indicators: memory fragmentation rate, CPU fragmentation rate, and specification fragmentation rate. The obtained fragmentation rate is used to measure the effect of defragmentation, and this indicator can effectively evaluate the defragmentation effect of the method of this embodiment. For example, the fragmentation rate before defragmentation is what, and the fragmentation rate after defragmentation is what, and the degree of reduction in the fragmentation rate is used to measure the defragmentation effect.

[0083] In some embodiments of the present application, Figure 3 As shown, when defragmenting host resources, each host is populated with placeholder virtual machines until the host is unable to accommodate a single placeholder virtual machine. The placeholder virtual machines are input virtual machine flavors. After the host is populated, a combination of virtual machines is found and then migrated to a suitable remaining host, enabling the host to provision at least one virtual machine of that flavor. In the above embodiment, resource consolidation is performed based on the principle of matching supply and demand to reduce resource fragmentation.

[0084] In some embodiments of the present application, after sorting the host machines based on supply and demand matching, sorting the NUMAs within the host machines includes:

[0085] Based on the remaining memory and CPU resources of the host machine, the remaining resources are scored and evaluated to perform host scoring;

[0086] Sort the hosts in ascending order based on their scores. The hosts with higher resource allocation rates and lower remaining resources are ranked. The host with more remaining resources is found.

[0087] Based on the host machine sorting results, the first host machine is determined. The CPU and memory resource scarcity is calculated based on the remaining CPU and memory resources of each NUMA of the first host machine. Each NUMA in the host machine is scored according to the resource scarcity, and the NUMAs in the host machine are sorted in ascending order based on the scores.

[0088] In some embodiments of the present application, when scoring and evaluating the remaining resources based on the remaining memory and CPU resources of the host machine, the calculation is performed according to the following formula:

[0089]

[0090]

[0091]

[0092] Among them, S C is the sum of the CPU allocation rates of all hosts, S M is the sum of the memory allocation rates of all hosts, is the CPU allocation rate of the i-th host, is the memory allocation rate of the i-th host, n is the number of hosts, Score i The score of the i-th host.

[0093] Specifically, after obtaining the host machine scores, the hosts are ranked and the remaining resources are scored and evaluated by comprehensively considering the remaining memory and CPU resources of the host machines.

[0094] In some embodiments of the present application, numa sorting is performed to find the numa with more resources in the host. After sorting the hosts, the first host after sorting is sorted according to the remaining CPU and memory (large page memory) resources of each numa. Calculate the resource scarcity of CPU and memory. The greater the scarcity, the more resources are required to allocate another virtual machine:

[0095] Specifically, the CPU resource scarcity is calculated according to the following formula:

[0096]

[0097] Calculate the memory resource scarcity according to the following formula

[0098]

[0099] in, is the remaining amount of CPU resources of the i-th NUMA in the host machine, is the remaining amount of memory in the i-th NUMA in the host.

[0100] Specifically, each NUMA in the host machine is scored according to the following formula:

[0101]

[0102]

[0103]

[0104] Where n is the number of NUMA in the host machine, is the sum of the CPU scarcity of all NUMAs in the host machine, is the sum of the memory scarcity of all NUMAs in the host machine, Score of the i-th NUMA in the host.

[0105] The NUMA in the host machine is sorted in ascending order according to the scores. After sorting, the resources required for allocating a virtual machine to the NUMA in the host machine are ranked in ascending order.

[0106] In some embodiments of the present application, when virtual machine scheduling is performed based on supply and demand matching, the best virtual machine or combination of virtual machines is determined to meet the resource requirements of the host machine to start another virtual machine of a specified flavor.

[0107] Specifically, when determining the best virtual machine or combination of virtual machines to meet the resource requirements of the host machine to start another virtual machine of a specified flavor, set the number of numas required by the flavor to f and the number of numas contained in the host machine to h. Specifically:

[0108] Select the first f numa[numa1,numa2,…,numa f ], numa f Represents the fth numa, and calculates the amount of resources required to open another virtual machine on each numa in:

[0109]

[0110]

[0111] Among them, V c The CPU size of the flavor to be sorted, in units of cores, V m The memory size of the flavor to be sorted, in GB. The amount of CPU resources required for a virtual machine for the i-th numa, The amount of memory resources required for a virtual machine for the i-th numa, numa i The CPU remaining amount is the CPU remaining amount of numa i, numa i The remaining memory is the remaining memory of the ith numa;

[0112] when and When it is less than or equal to 0, it is judged that numa i The remaining resources meet the resource requirements. All NUMAs that meet the resource requirements are removed from the f NUMA sequence, and the number of remaining NUMAs is determined to be f'. The virtual machines that are not related to the f' NUMAs are removed, that is, the virtual machines that are not related to these f' NUMAs, that is, the virtual machines that are not on the f' NUMAs, thereby removing invalid virtual machines and freeing up resources.

[0113] In some embodiments of the present application, after removing virtual machines that are not related to f' numas, the following steps are included:

[0114] The remaining virtual machines are sorted in ascending order according to their resource usage on each numa. When the average CPU shortage of f numas is greater than or equal to the average memory shortage, the virtual machines are sorted according to the average CPU usage on each numa. Otherwise, the virtual machines are sorted according to the average memory usage on each numa.

[0115] In some embodiments of the present application, when the remaining virtual machines are sorted in ascending order according to their resource occupancy on each numa, the method includes adding the similarity between the virtual machine and f' numas as an influencing factor in the sorting.

[0116] Specifically, the remaining virtual machines are sorted in ascending order according to their resource occupancy on each NUMA (virtual machine resources are evenly distributed to different NUMAs). If the average CPU shortage of f NUMAs is greater than or equal to the average memory shortage, the virtual machines are sorted according to the average CPU occupied by each NUMA; otherwise, they are sorted according to the average memory occupied by each NUMA.

[0117] When sorting, the similarity between the virtual machine and f' numas is added as an influencing factor to participate in the sorting. The similarity is calculated by the resource vector of the virtual machine. The resource vector required to create another virtual machine with the host The cosine similarity measure of vm The resources corresponding to the numa not occupied by the virtual machine are 0,v h ostExcept for f' NUMA, the resources on the remaining NUMA are set to 0 by default. Remove the dimensions where the resource requirements on the first f' NUMA on the host are 0 to avoid affecting the COS similarity calculation. Similarly, v vm Also remove v from the vector h ost The corresponding dimension removed, the elements involved in the sorting of each virtual machine are:

[0118] v=CPU or memory-0.9*cos( <v vm ,v h ost >)

[0119] The purpose of multiplying the scaling factor by 0.9 is to make cos( <v vm ,v host >) is less than 1, ensuring that affinity only affects the sorting of virtual machines with the same resources, cos( <v vm ,v h ost >) is used to generate cos( <v vm ,v h ost The larger the value of >), the closer the numa occupied by the virtual machine is to f numa, and the higher the ranking. Sort numa in ascending order based on the values ​​of the above formula.

[0120] In a specific example based on the above embodiment, assuming that CPU resources are relatively scarce, virtual machines are prioritized by CPU, the host machine has 2 numas, and the remaining resources of each numa are (2C, 6G) and (1C, 5G) respectively. The specifications of the newly opened virtual machine are (8C, 8G), spanning 2 numas, then the resources occupied by each numa of the virtual machine are (4C, 4G), and the resources required for each numa of another virtual machine are [(2C, 0G), (3C, 0G)] respectively. If there are 3 virtual machines on the host machine, the resources are as follows:

[0121] vm1: occupies 2 numa [(1C, 2G), (1C, 2G)]

[0122] vm2: occupies 2 numa [(2C, 4G), (2C, 4G)]

[0123] vm3: occupies 2 numa [(2C, 4G), (0C, 0G)]

[0124] After removing resources with a demand of 0:

[0125] v h ost =[2,3]

[0126]

[0127]

[0128]

[0129] After vm is sorted: vm1, vm2, vm3, cos( <v vm ,v h ost >) When each numa occupies the same number of cores as vm2 and vm3, put vm2 in front, so that the combination of virtual machines selected for removal is [vm1,vm2].

[0130] In some embodiments of the present application, when searching for the best destination host machine for the selected virtual machine or combination of virtual machines, the process includes:

[0131] A virtual machine or combination of virtual machines that meets the host machine's resource requirements is selected from the sorted virtual machines. The selection of the virtual machine or combination of virtual machines is performed using a double pointer, specifically:

[0132] The two pointers start pointing to the first virtual machine after sorting;

[0133] One of the pointers slides backward until the total resources of the virtual machine are greater than the resource requirements of the host machine, and then stops;

[0134] The other pointer also slides backward until the total resources of the virtual machine equals the resource requirements of the host machine.

[0135] Get the virtual machine or combination of virtual machines in the range pointed to by the double pointer.

[0136] In some embodiments of the present application, when searching for the best destination host for the selected virtual machine or combination of virtual machines, the process further includes:

[0137] Among the hosts after the host is sorted, it is traversed from the back to the front first, and the remaining resources of each numa of the host are determined, and the number of numas that can satisfy the resources occupied by the virtual machine in each numa is calculated;

[0138] When the number of numa is less than the number of virtual machine cross-numa, continue to traverse the next host;

[0139] When the number of NUMAs is greater than or equal to the number of NUMAs spanned by the virtual machine, all NUMAs in the host machine that meet the resource requirements of the virtual machine are optimally screened. If the virtual machine spans j NUMAs, the j NUMAs are sorted according to similarity and the optimal first j NUMAs are selected as the NUMA positions of the virtual machine in the target host machine.

[0140] Specifically, for each selected virtual machine or combination of virtual machines, search for the target host from the remaining hosts in reverse order:

[0141] Step a: Traverse the hosts from back to front according to the host sorting priority, that is, start from the host with the least remaining resources, determine the remaining resources of each NUMA of the host, and calculate the number of NUMAs that can meet the resources occupied by the virtual machine in each NUMA. If the number is less than the number of NUMAs across the virtual machine, continue to traverse the next host, otherwise execute step b:

[0142] Step b: Optimize all numa in the host machine that meet the virtual machine resource requirements: Calculate the resource remaining vector [numa i CPU remaining amount, numa i Memory remaining] and the average resource usage vector of each numa of the virtual machine Calculate the similarity (Euclidean distance). Assuming that the virtual machine spans j NUMAs, then select the best first j NUMAs after sorting by similarity as the NUMA position of the virtual machine in the target host.

[0143] Specifically, the host machines from which the virtual machines have been migrated are removed, and the remaining host machines are defragmented according to the steps in the above embodiment until all host machines are traversed.

[0144] In some embodiments of the present application, when optimizing and executing the migration policy of a virtual machine or a combination of virtual machines, redundancy is removed from the number of migrated virtual machines, and causal sorting is performed on the migrated virtual machines.

[0145] Specifically, when removing redundant migration, for example, if two virtual machines V1 and V2 with the same specifications are on two host machines H1 and H2 respectively, and there is also a host machine H3, and the migration strategy is V2—>H3 and V1—>H2, they can be directly merged into V1—>H3, reducing the number of virtual machines to be migrated.

[0146] When performing causal sorting, causal sorting means that there is a temporal relationship between migration strategies. For example, two virtual machines V1 and V2 with different specifications are on two hosts H1 and H2 respectively. There is also a host H3. The migration strategy is V1->H2, V2->H3. However, V2 must be pulled out of H2 first before H2 can accommodate V1. Therefore, V2->H3 must be executed first, and then V1->H2.

[0147] It is understood that in the above embodiments, defragmentation is performed at the NUMA granularity based on the host's resource allocation, supporting not only the case of a single NUMA host but also the case of multiple NUMA hosts. Furthermore, defragmentation can be performed by user-specified flavor specifications. This is particularly true when there are remaining resources in the resource pool but the customer cannot provision VMs of the required specifications or the quantity does not meet the customer's needs. By performing defragmentation based on the customer's specifications in the above embodiments, not only can the available resources be increased, but urgent customer needs can also be addressed, while avoiding the time and cost associated with capacity expansion.

[0148] Furthermore, in the above embodiments, when defragmenting resources, the idea of ​​supply and demand matching is proposed, by finding the best virtual machine combination that meets the resource requirements of the source host and migrating it to the most suitable destination host to integrate resources, thereby improving resource utilization and reducing resource fragmentation.

[0149] When the defragmentation methods of the above embodiments are applied to the online Shanghai 7 and Kunming 2 resource pools, the application results are shown in Tables 1 and 2. It can be seen from the application results that the fragmentation rate is significantly reduced:

[0150] Table 1 Defragmentation effect of Shanghai 7 resource pool

[0151]

[0152] Table 2 Kunming 2 resource pool defragmentation effect

[0153]

[0154]

[0155] In a specific example based on the above embodiment,

[0156] 1. Sample Data

[0157] (1) Flavor information:

[0158] The flavor of the defragmentation is (16C, 32G), the number of span numas is 2, and each numa is (8C, 16G)

[0159] (2) Host information:

[0160] After the host filling algorithm steps in the above embodiment, there are 4 non-empty host machines, each with a numa quantity of 4. The host machine resource status is as follows:

[0161] Host1:

[0162] Total resources: [numa1(16C,32G),numa2(16C,32G),numa3(16C,32G),numa4(16C,32G)];

[0163] Remaining resources: [numa1(6C,8G),numa2(6C,8G),numa3(8C,4G),numa4(8C,4G)];

[0164] Resource usage of included virtual machines:

[0165] instance1: [numa1(1C,4G),numa2(1C,4G),numa3(0C,0G),numa4(0C,0G)];

[0166] instance2: [numa11C,4G),numa2(1C,4G),numa3(0C,0G),numa4(0C,0G)];

[0167] instance3: [numa1(2C,8G),numa2(0C,0G),numa3(0C,0G),numa4(0C,0G)];

[0168] instance4: [numa1(0C,0G),numa2(2C,4G),numa3(0C,0G),numa4(0C,0G)];

[0169] instance5: [numa1(2C,4G),numa2(2C,4G),numa3(0C,0G),numa4(0C,0G)];

[0170] instance6: [numa1(4C,4G),numa2(0C,0G),numa3(0C,0G),numa4(0C,0G)];

[0171] instance7: [numa1(0C,0G),numa2(4C,8G),numa3(0C,0G),numa4(0C,0G)];

[0172] instance8: [numa1(0C,0G),numa2(0C,0G),numa3(4C,16G),numa4(4C,16G)];

[0173] instance9: [numa1(0C,0G),numa2(0C,0G),numa3(4C,12G),numa4(4C,12G)];

[0174] Host2:

[0175] Total resources: [numa1(16C,32G),numa2(16C,32G),numa3(16C,32G),numa4(16C,32G)];

[0176] Remaining resources: [numa1(6C,8G),numa2(6C,8G),numa1(6C,4G),numa2(6C,4G)];

[0177] Resource usage of included virtual machines:

[0178] instance1: [numa1(2C,4G),numa2(2C,4G),numa3(0C,0G),numa4(0C,0G)];

[0179] instance2: [numa1(2C,4G),numa2(2C,4G),numa3(0C,0G),numa4(0C,0G)];

[0180] instance3: [numa1(2C,8G),numa2(2C,8G),numa3(2C,8G),numa4(2C,8G)];

[0181] instance4: [numa1(4C,8G),numa2(4C,8G),numa3(4C,8G),numa4(4C,8G)];

[0182] instance5: [numa1(0C,0G),numa2(0C,0G),numa3(4C,12G),numa4(4C,12G)];

[0183] Host3:

[0184] Total resources: [numa1(16C,32G),numa2(16C,32G),numa3(16C,32G),numa4(16C,32G)];

[0185] Remaining resources: [numa1 (1C, 5G), numa2 (1C, 8G), numa3 (3C, 8G), numa4 (4C, 8G)];

[0186] Host4:

[0187] Total resources: [numa1(16C,32G),numa2(16C,32G),numa3(16C,32G),numa4(16C,32G)];

[0188] Remaining resources: [numa1(1C,4G),numa2(1C,5G),numa3(3C,8G),numa4(4C,8G)].

[0189] 2. Defragmentation

[0190] As shown in the previous data, the algorithm steps after the host population have been completed. Ignoring the migration plan optimization algorithm steps, the remaining defragmentation steps are analyzed as follows:

[0191] Host sorting

[0192] After sorting the hosts according to the host sorting method in the supply and demand matching during defragmentation, the order of the hosts is: Host1, Host2, Host3, Host4.

[0193] numa sort

[0194] The numa resource scarcity is calculated according to the numa scarcity calculation formula in the above embodiment, and the numa are sorted according to the final score of resource scarcity, with the lower scarcity being placed in front. In this way, the front numa will be preferentially selected for optimization later. The sorted numa order is: numa1, numa2, numa3, numa4.

[0195] Schedule virtual machines based on supply and demand matching

[0196] Because Host1 is ranked first, Host1 is sorted first:

[0197] Find virtual machine combinations:

[0198] According to the resource scarcity, memory resources are relatively tight. Therefore, according to the average memory usage of each numa in the virtual machine and cos( <v vm ,v host >)*0.9, and remove the virtual machines that are not on numa1 and numa2. The resource vectors of the host and virtual machines are:

[0199] v host =[2,8,2,8]

[0200]

[0201]

[0202]

[0203]

[0204]

[0205]

[0206]

[0207] cos( <v vm ,v host The values ​​of >)*0.9 are:

[0208] 0.9,0.9,0.6364,0.6212,0.8786,0.5457,0.6212;

[0209] Memory per numa on the virtual machine - cos( <v vm ,v host The values ​​of >)*0.9 are:

[0210] 3.1,3.1,7.3636,3.3788,3.1214,3.454,7.3788;

[0211] Therefore, the order of virtual machines on Host1 is:

[0212] instance1: [numa1(1C,4G),numa2(1C,4G),numa3(0C,0G),numa4(0C,0G)];

[0213] instance2: [numa1 1C,4G),numa2(1C,4G),numa3(0C,0G),numa4(0C,0G)];

[0214] instance5: [numa1(2C,4G),numa2(2C,4G),numa3(0C,0G),numa4(0C,0G)];

[0215] instance4: [numa1(0C,0G),numa2(2C,4G),numa3(0C,0G),numa4(0C,0G)];

[0216] instance6: [numa1(4C,4G),numa2(0C,0G),numa3(0C,0G),numa4(0C,0G)];

[0217] instance3: [numa1(2C,8G),numa2(0C,0G),numa3(0C,0G),numa4(0C,0G)];

[0218] instance7: [numa1(0C,0G),numa2(4C,8G),numa3(0C,0G),numa4(0C,0G)].

[0219] The virtual machine combination finally selected by the algorithm is [instance1,instance2].

[0220] Find the target host for the virtual machine combination, starting from Host4 according to the sorting priority. The resource usage on each NUMA will be calculated according to the algorithm to calculate a similarity with the remaining resources of each NUMA of the host. Then, based on the similarity, the best two NUMAs with the most similar resources are selected to load the migrated virtual machine:

[0221] The remaining resources for each numa on Host4 are:

[0222] numa1(1C,4G),numa2(1C,5G),numa3(3C,8G),numa4(4C,8G)

[0223] Resources occupied by the virtual machine on each numa:

[0224] instance1:(1C,4G)

[0225] instance2:(1C,4G)

[0226] The root enumeration algorithm calculates the Euclidean distances between vector [1,4] and vectors [1,4], [1,5], [3,8].[4,8] as 0.0, 1.0, 4.4721, and 5.0 respectively;

[0227] Based on similarity and resource availability, we can finally:

[0228] instance1 migrates to numa1 and numa2;

[0229] Instance2 is migrated to numa3 and numa4.

[0230] The remaining resources of Host4 are [numa1(0C,0G),numa2(0C,1G),numa3(2C,4G),numa4(3C,4G)].

[0231] After Host1 is optimized and removed, continue to optimize the remaining hosts by returning to the previous steps according to the defragmentation method: At this time, optimize Host2 again. You need to migrate instance1 and instance2. You can load a virtual machine of Flavor (16C, 32G). Select instance1 to migrate to numa3, numa4 of Host4, and instance2 to migrate to numa3, numa4 of Host3. This can organize the space for Host2 to load a virtual machine of Flavor (16C, 32G).

[0232] After defragmentation, the algorithm gives the following migration plan:

[0233] Instance 1 of host1 is migrated to numa1 and numa2 of host4, and instance 2 is migrated to numa3 and numa4 of host4.

[0234] Instance 1 of host2 is migrated to numa3 and numa4 of host4, and instance 2 is migrated to numa3 and numa4 of host3;

[0235] Finally, host1 and host2 can be re-allocated to 2 virtual machines (16C, 32G).

[0236] See Figure 4 As shown, the remaining resources of the four host machines before and after the virtual machine migration plan is carried out according to the above steps. It can be seen that the above embodiment proposes the idea of ​​supply and demand matching during fragmentation, and improves resource utilization and reduces resource fragmentation by finding the best virtual machine combination that meets the resource requirements of the source host machine and migrating it to the most suitable destination host machine for resource integration.

[0237] Those skilled in the art will appreciate that the embodiments of the present application can be provided as methods, systems, or computer program products. Therefore, the present application can adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application can adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0238] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the steps in the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0239] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0240] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 The steps for the function specified in one or more boxes.

[0241] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit it. Although the present invention has been described in detail with reference to the above embodiments, ordinary technicians in the field should understand that the specific implementation methods of the present invention can still be modified or replaced by equivalents. Any modification or equivalent replacement that does not depart from the spirit and scope of the present invention should be covered by the scope of protection of the claims of the present invention.

Claims

1. A NUMA granularity defragmentation method, characterized in that: include: Based on the host's resource allocation information, the host's resources are defragmented at the NUMA granularity. When defragmenting resources on the host machine, after sorting the host machine based on supply and demand matching, the NUMA in the host machine is sorted; Perform virtual machine scheduling based on supply and demand matching, find the best destination host for the selected virtual machine or virtual machine combination, and determine the migration strategy for the virtual machine or virtual machine combination; Optimizing and executing a migration strategy for the virtual machine or combination of virtual machines; When searching for the best destination host for the selected virtual machine or combination of virtual machines, the following steps are performed: A virtual machine or a combination of virtual machines that meets the resource requirements of the host machine is selected from the sorted virtual machines. The virtual machine or combination of virtual machines is selected by double pointer selection, specifically: The two pointers start pointing to the first virtual machine after sorting at the same time; One of the pointers slides backward until the total resources of the virtual machine are greater than the resource requirements of the host machine and then stops; The other pointer also slides backward until the total resources of the virtual machine equals the resource requirements of the host machine. Get the virtual machine or combination of virtual machines in the range pointed to by the double pointer.

2. The NUMA granularity defragmentation method according to claim 1, wherein: When defragmenting resources of the host machine, the process includes: Fill each host machine with placeholder virtual machines until the host machine is unable to load a placeholder virtual machine; wherein the placeholder virtual machine is the input virtual machine specification flavor; After the host machine is filled, a combination of virtual machines is found, and the virtual machine combination is migrated to the remaining suitable host machine so that the host machine can start at least one virtual machine of the flavor.

3. The NUMA granularity defragmentation method according to claim 1, wherein: After sorting hosts based on supply and demand matching, sorting NUMA within hosts includes: Based on the remaining memory and CPU resources of the host machine, the remaining resources are scored and evaluated to perform host scoring; Based on the host scoring results, the hosts are sorted in ascending order, with the resource allocation rates of the sorted hosts increasing in sequence and the remaining resource amounts of the hosts decreasing in sequence, and a host with more remaining resources is found; Based on the sorting results of the host machines, the first host machine is determined, the CPU and memory resource scarcity is calculated according to the remaining CPU and memory resources of each NUMA of the first host machine, each NUMA in the host machine is scored according to the resource scarcity, and the NUMAs in the host machine are sorted in ascending order according to the scores.

4. The NUMA granularity defragmentation method according to claim 3, wherein: When scoring and evaluating the remaining resources based on the remaining memory and CPU resources of the host machine, the following formula is used for calculation: Among them, S C is the sum of the CPU allocation rates of all hosts, S M is the sum of the memory allocation rates of all hosts, is the CPU allocation rate of the i-th host, is the memory allocation rate of the i-th host, n is the number of hosts, is the score of the i-th host.

5. The NUMA granularity defragmentation method according to claim 4, characterized in that: Calculate the CPU resources according to the following formula Source scarcity Calculate the memory resource scarcity according to the following formula in, is the remaining amount of CPU resources of the i-th NUMA in the host machine, is the remaining amount of memory of the i-th NUMA in the host; Score each NUMA in the host machine according to the following formula: Where n is the number of NUMA in the host machine, is the sum of the CPU scarcity of all NUMAs in the host machine, is the sum of the memory scarcity of all NUMAs in the host machine, Score of the i-th NUMA in the host.

6. The NUMA granularity defragmentation method according to claim 1, wherein: When scheduling virtual machines based on supply and demand matching, it includes: Determine the optimal virtual machine or combination of virtual machines to meet the resource requirements of the host machine to start another virtual machine of a specified flavor.

7. The NUMA granularity defragmentation method according to claim 6, wherein: Determining the optimal virtual machine or combination of virtual machines to meet the resource requirements of a host machine to start a virtual machine of a specific flavor includes: Select the first f sorted numas [numa1, numa2, …, numaf], where numaf represents the fth numa, and calculate the amount of resources (CPUnuma, MEMnuma) required to provision a virtual machine on each numa, where: Among them, V c The flavorCPU size to be sorted, in units of cores, The memory size of the flavor to be sorted, in GB, CPUnuma i The amount of CPU resources required for a virtual machine for the i-th numa, MEMnuma i The amount of memory resources required for a virtual machine for the i-th numa, The remaining amount is the CPU remaining amount of numa i, The remaining memory is the remaining memory of the ith numa; WhenCPUnuma i and MEMnuma i When it is less than or equal to 0, it is judged that numa i The remaining resources meet the resource requirements. All NUMAs whose resources meet the requirements are removed from the f NUMA sequences, and the number of remaining NUMAs is determined to be f'. The virtual machines that are not related to the f' NUMAs are removed.

8. The NUMA granularity defragmentation method according to claim 7, wherein: After removing the virtual machines that are not related to f' numa, including: Sort the remaining virtual machines in ascending order according to their resource usage on each numa. When the average CPU shortage of f numas is greater than or equal to the average memory shortage, sort them according to the average CPU usage on each numa. Otherwise, sort them according to the average memory usage on each numa. When the remaining virtual machines are sorted in ascending order according to their resource occupation on each numa, the method includes: adding the similarity between the virtual machine and the f' numas as an influencing factor in the sorting.

9. The NUMA granularity defragmentation method according to claim 1, wherein: When searching for the best destination host for the selected virtual machine or combination of virtual machines, the following steps are also included: Among the hosts after the host is sorted, it is traversed from the back to the front first, and the remaining resources of each numa of the host are determined, and the number of numas that can satisfy the resources occupied by the virtual machine in each numa is calculated; When the number of numa is less than the number of virtual machine cross-numa, continue to traverse the next host; When the number of NUMAs is greater than or equal to the number of NUMAs spanned by the virtual machine, all NUMAs in the host machine that meet the resource requirements of the virtual machine are optimally screened. If the virtual machine spans j NUMAs, the j NUMAs are sorted according to similarity and the optimal first j NUMAs are selected as the NUMA positions of the virtual machine in the target host machine.

Images