Improved ant colony virtual machine optimization data system based on dimension extension
By introducing energy efficiency as an optimization dimension in the virtual machine merging algorithm, and optimizing virtual machine deployment using the Ant Swarm Optimization Algorithm, the problem of maximizing energy efficiency in virtual machine merging is solved, and the efficient migration of virtual machines from high load to idle servers is achieved, reducing the efficiency of virtual machines. energy consumption.
Patent Information
- Application Number
- CN202410968978.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-18
- Publication Date
- 2024-11-12
AI Technical Summary
The existing virtual machine merging algorithms cannot effectively regard energy efficiency as the optimization goal, and cannot accurately weight each goal, resulting in the inability to maximize energy efficiency in virtual machine optimization.
Adopt an improved Ant colony virtual machine optimization data system based on dimension expansion, through data preprocessing, objective function definition and algorithm implementation, energy efficiency is introduced as an additional optimization dimension, and the deployment of virtual machine is optimized by Ant algorithm to maximize resource utilization and Minimize operating costs.
Effectively guide virtual machines to migrate from high-load servers to idle servers, improve overall energy efficiency, reduce energy consumption, and meet the energy efficiency needs of virtual machines to optimize data.
Smart Images

Figure SMS_1 
Figure SMS_2 
Figure SMS_3
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of big data processing, and in particular to an improved ant colony virtual machine optimization data system based on dimensional expansion. Background Art
[0002] Data centers are composed of tens of thousands of physical hosts (PMs). These physical resources will cause a lot of energy consumption, which will not only increase operating costs, but also lead to huge carbon emissions. At the same time, the high energy consumption of data centers is mainly caused by insufficient resource utilization. Virtualization enables virtual machines (VMs) with independent performance to share the same host to improve resource utilization, while virtual machine consolidation and the hibernation or low-power conversion of idle hosts can further improve resource utilization and energy consumption. Virtual machine consolidation is the most effective way to improve the energy efficiency of data centers. The dynamic virtual machine migration consolidation technology can deploy virtual machines on fewer hosts. Virtual machine consolidation is mainly achieved through migration. Virtual machine migration between different hosts can shut down idle hosts. The performance measurement indicators of virtual machine consolidation mainly include: the number of released hosts (need to be maximized), the number of virtual machine migrations (need to be minimized), the algorithm running time (need to be minimized), and energy efficiency (need to be maximized). The virtual machine consolidation problem itself is an NP problem. The solution technology is such as the mixed integer linear regularization method, but its optimization time reaches exponential level. The second is the meta-heuristic method, which can effectively search the feasible solution space within the polynomial time complexity.
[0003] The adaptive meta-heuristic ant colony intelligent optimization method is an effective method for solving virtual machine merging. Existing research has mostly focused on combining multi-objective networks through the aggregate objective function AOF of a single objective and a single group. The advantage of the AOF method is that it can reduce time complexity and improve the algorithm's operating efficiency by limiting the subspace search of feasible solutions. However, the inappropriateness of this method is that it cannot accurately assign the weights that need to be considered to each objective, and can only be assigned through subjective consciousness. And AOF may not be able to combine optimization objectives in an appropriate way. The AOF of the existing ant colony virtual machine merging algorithm has not prioritized the optimization objectives.
[0004] The inventors of this team have designed a virtual machine migration method that maximizes the host release and minimizes the virtual machine migration to achieve virtual machine merging. Its main contributions are: 1) It realizes multi-objective and multi-group optimization. The algorithm expands the existing virtual machine merging single-objective and single-group optimization idea to achieve multi-objective and multi-group ant colony optimization. 2) It reduces the search space. The algorithm sets constraints to exclude some migration solutions in the merging process without reducing the solution quality, so that the algorithm converges faster and the scalability is enhanced.
[0005] However, the specific deployment results of the virtual machine will also have an impact on energy efficiency. At this time, it is necessary to calculate the energy efficiency of the specific deployment, and then design a deployment algorithm with higher energy efficiency. Therefore, it is necessary to further introduce the concept of dimension expansion and use energy efficiency as an additional optimization dimension to meet the demand for reducing energy efficiency in virtual machine optimization data.
[0006] In summary, it is particularly important to build an improved ant colony virtual machine optimization data system based on dimensional expansion. Summary of the invention
[0007] The technical problem to be solved by the present invention is to provide an improved ant colony virtual machine optimization data system based on dimensional expansion, in order to further introduce the concept of dimensional expansion, take energy efficiency as an additional optimization dimension, and meet the demand for reducing energy efficiency in virtual machine optimization data.
[0008] In order to solve the above technical problems, the technical solutions adopted by the present invention are as follows:
[0009] The improved ant colony virtual machine optimization data system based on dimensional expansion is characterized by including three major links: data preprocessing, objective function definition and algorithm implementation. The anomalies are initially handled by collecting and cleaning the resource data of the virtual machine and the physical host, and then the optimization goals are set, including maximizing the host release amount, minimizing the migration amount and improving energy efficiency. By initializing the pheromone and heuristic value matrix, the ant algorithm is used to select the migration plan and update the pheromone to ensure the efficiency and cost-effectiveness of the migration process.
[0010] Furthermore, by optimizing virtual machine deployment through advanced algorithms, resource utilization is maximized and operating costs are minimized, effectively guiding virtual machines to migrate from high-load servers to idle servers, thereby improving overall energy efficiency and reducing energy consumption.
[0011] Furthermore, the following data contents are included:
[0012] a. Data preprocessing;
[0013] b. Objective function definition, including 3 objective functions, namely:
[0014] 1) Maximize the host release:
[0015]
[0016] Among them, V p represents the set of virtual machines running on host p;
[0017] 2) Minimize the amount of virtual machine migration:
[0018]
[0019] Where T represents the set of all possible migration tuples, MigrationCost(v,p src ,p dst ) indicates that the virtual machine v is from the source host p src Migrate to the target host p dst The cost
[0020] 3) Maximize energy efficiency:
[0021]
[0022] Among them, Energy pv represents the energy efficiency of virtual machine v on host p;
[0023] c. Algorithm, comprising the following steps:
[0024] 1) Initialization
[0025] Initialize the pheromone matrix τ and the heuristic value matrix η:
[0026] The pheromone matrix τ is initialized to a small positive number;
[0027] The heuristic value matrix η is calculated based on the initial data;
[0028] Ant search process
[0029] Step 1: Main loop: Iterate until the stopping condition, that is, all remaining hosts are fully utilized or the performance cannot be further improved when a given number of consecutive iterations is reached;
[0030] Step 2: ACS|PR| group optimizes the host release amount: each ant selects a migration plan based on pheromones and heuristic values, calculates the host release amount; updates the pheromone matrix τ and the heuristic value matrix η;
[0031] Step 3: ACSnM group optimizes the amount of virtual machine migration: each ant selects a migration plan based on pheromones and heuristic values, calculates the amount of virtual machine migration, and updates the pheromone matrix τ and the heuristic value matrix η;
[0032] Step 4: ACS|Energy|Group Optimization Energy Efficiency: Each ant selects a migration plan based on pheromones and heuristic values, and calculates energy efficiency; updates the pheromone matrix τ and the heuristic value matrix η;
[0033] Step 5: Update the global optimal migration plan: If ACS|PR| finds a migration plan with a higher host release amount, update Φ + ; If ACSnM finds a lower VM migration amount and is consistent with Φ + The migration scheme with at least the same host release amount in the update Φ + ; If ACS|Energy| finds a higher energy efficiency and is equal to Φ+ The migration scheme with at least the same host release amount and virtual machine migration amount in the update Φ + .
[0034] Furthermore, the data preprocessing includes data collection, data cleaning, energy efficiency data standardization, and initial settings.
[0035] Furthermore, the data collection includes virtual machine and physical host resource utilization data: data set D contains resource utilization information of all physical hosts P and virtual machines V; data format:
[0036] D={(p i , v j , CPU ij , Memory ij ,Energy ij )|p i ∈P,v j ∈V}
[0037] Among them, CPU ij 、Memory ij and Energy ij Represents virtual machine v j On the host p i CPU, memory utilization and energy efficiency on the host; Energy efficiency data collection: Collect energy efficiency data of each virtual machine on different hosts, expressed as Energy ij ;Energy efficiency data can include power consumption, cooling requirements, etc.;
[0038] The data cleaning includes missing values, outliers, standardization processing, etc.; specifically includes:
[0039]
[0040] If |xi-μ|>Kσ, then xi is an outlier;
[0041] Among them, μ is the mean, σ is the standard deviation, and k is a constant;
[0042]
[0043] Among them, min(x) and max(x) represent the minimum and maximum values of the data set x respectively;
[0044] The energy efficiency data standardization includes normalizing the energy efficiency data to ensure that the energy efficiency data is comparable with other resource utilization data; the normalization formula is as follows:
[0045]
[0046] The initial setting includes initialization of the pheromone matrix τ and the heuristic value matrix η: the pheromone matrix τ is initialized to a small positive number τ0;
[0047]
[0048] The heuristic value matrix η is calculated based on the initial data:
[0049]
[0050] Among them, d ij For virtual machine v i From the host p src Migrate to host p dst The specific calculation method is as follows:
[0051] d ij =w1·CPU ij +w2·Memory ij +w3·Energy ij
[0052] Among them, w1, w2 and w3 are weight parameters used to balance the impact of different resources; energy efficiency weight setting: according to the specific needs of the data center, set the weight w3 of energy efficiency in the total migration cost.
[0053] Furthermore, the pheromone update formula is:
[0054] τ ij =(1-ρ)τ ij +Δτ ij
[0055] Among them, ρ is the pheromone volatility coefficient, Δτ ij is the pheromone increment, defined as:
[0056]
[0057] Among them, Q is a constant, L best is the optimal path length in the current iteration.
[0058] Furthermore, the heuristic value is defined as:
[0059]
[0060] Among them, d ij For virtual machine v i From the host p src Migrate to host p dst The cost.
[0061] Furthermore, each ant selects a migration scheme according to the pheromone and the heuristic value, using the following probability formula:
[0062]
[0063] Among them, α and β are the pheromone importance and heuristic value importance parameters respectively, and allowed is the set of migration schemes that ants can choose.
[0064] Furthermore, the migration cost includes comprehensive consideration of CPU memory and energy efficiency, and the migration cost calculation formula is:
[0065] d ij =w1·CPU ij +w2·Memory ij +w3·Energy ij
[0066] Among them, w1, w2 and w3 are weight parameters used to balance the impact of different resources.
[0067] Furthermore, the updating of the global optimal migration solution includes updating the global optimal solution Φ according to the currently found optimal solution at the end of each iteration. + :
[0068]
[0069] Among them, λ1 and λ2 are weight parameters used to balance the influence of different objective functions.
[0070] The beneficial effects of adopting the above technical solution are:
[0071] The present invention designs a completely new data system structure, and updates the pheromone update data algorithm paradigm, the heuristic value data algorithm paradigm, the ant selection migration scheme probability data algorithm paradigm, the migration cost data algorithm paradigm, and the global optimal scheme update data algorithm paradigm;
[0072] The present invention further introduces the concept of dimension expansion, taking energy efficiency as an additional optimization dimension, and meets the demand for reducing energy efficiency in virtual machine optimization data. DETAILED DESCRIPTION
[0073] The following examples describe the present invention in detail. In the description of the following examples, specific details such as specific system structures, technologies, etc. are proposed for illustration rather than for limitation, so as to thoroughly understand the present application embodiments. However, it should be clear to those skilled in the art that the present application can also be implemented in other embodiments without these specific details. In other cases, the detailed description of well-known systems, devices, circuits and methods is omitted to prevent unnecessary details from hindering the description of the present application. It should be understood that when used in the present application specification and the attached claims, the term "comprising" indicates the presence of the described features, wholes, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or their collections. It should also be understood that the term "and / or" used in the present application specification and the attached claims refers to any combination of one or more of the items listed in association and all possible combinations, and includes these combinations.
[0074] In addition, in the description of the present application specification and the attached claims, the terms "first", "second", "third", etc. are only used to distinguish the description and cannot be understood as indicating or implying relative importance. The reference to "one embodiment" or "some embodiments" described in the present application specification means that one or more embodiments of the present application include specific features, structures or characteristics described in conjunction with the embodiment. Therefore, the statements "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. that appear in the different parts of this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways. The terms "including", "comprising", "having" and their variations all mean "including but not limited to", unless otherwise specifically emphasized in other ways.
[0075] This application is based on the previous research results of the inventor's team and further introduces the concept of dimensional expansion, developing energy efficiency as an additional optimization dimension.
[0076] The AC0 heuristic method for ant colony optimization is inspired by the foraging behavior of ants. The food transport from the food source to the ant nest is tracked by chemicals such as pheromones on the path. The method allows direct communication between ants to find the optimal path from the starting point. Although each ant can find a feasible solution, the high-quality solution is the result of indirect communication and global collaboration between ants. The types of AC0 algorithms include ant system AS, ant colony system ACS, and maximum minimum ant system MMAS, among which ACS is the current optimal solution. In the existing virtual machine deployment and merging research, ACO is used to solve the nonlinear resource allocation problem and obtain the optimal allocation of tasks under limited resources. The improved ACO is used to solve the multi-objective resource allocation. The single-objective single-colony MMAS algorithm is used to solve the virtual machine merging problem. The vector algebra mechanism is integrated into ACS to optimize resource utilization. The single-objective single-colony ACS algorithm is designed to solve the virtual machine merging problem. The multi-objective ACS is used to solve two objective optimization problems: minimum energy consumption and minimum resource waste, but the method does not perform virtual machine migration and only considers the optimization objectives in the initial deployment.
[0077] The difference between the existing AC0 virtual machine merging method and an algorithm previously studied by the team of the present invention is that the existing method tends to use the single-objective single-group ACO algorithm of AOF to try to combine multiple objectives, while an algorithm previously studied by the team of the present invention directly uses two independent ant colonies to design a multi-objective ACS algorithm. Colony one maximizes the number of released hosts, and colony two minimizes the amount of virtual machine migration. And the search in the solution space is optimized by designing three constraints, which greatly reduces the search time for the optimal solution. However, the specific deployment results of the virtual machine will also have an impact on energy efficiency. At this time, it is necessary to calculate the energy efficiency problem during the specific deployment, and then design a deployment algorithm with higher energy efficiency. Therefore, it is necessary to further introduce the concept of dimensional expansion and use energy efficiency as an additional optimization dimension to meet the demand for reducing energy efficiency in virtual machine optimization data.
[0078] Example 1
[0079] The improved ant colony virtual machine optimization data system based on dimensional expansion includes three major links: data preprocessing, objective function definition and algorithm implementation. It initially handles anomalies by collecting and cleaning the resource data of virtual machines and physical hosts, and then sets optimization goals, including maximizing the host release, minimizing the migration amount and improving energy efficiency. By initializing the pheromone and heuristic value matrix, the ant algorithm is used to select the migration plan and update the pheromone to ensure the efficiency and cost-effectiveness of the migration process.
[0080] The specific data content is as follows:
[0081] a. Data preprocessing: data collection, data cleaning, energy efficiency data standardization, initial settings; data collection includes virtual machine and physical host resource utilization data: data set D contains resource utilization information of all physical hosts P and virtual machines V; data format:
[0082] D={(p i ,v j ,CPU ij ,Memory ij ,Energy ij )|p i ∈P,v j ∈V}
[0083] Among them, CPU ij 、Memory ij and Energy ij Represents virtual machine v j On the host p i CPU, memory utilization and energy efficiency on the host; Energy efficiency data collection: Collect energy efficiency data of each virtual machine on different hosts, expressed as Energy ij ;Energy efficiency data can include power consumption, cooling requirements, etc.;
[0084] The data cleaning includes missing values, outliers, standardization processing, etc.; specifically includes:
[0085]
[0086]
[0087] If |x i -μ|>Kσ, then x i is an outlier;
[0088] Among them, μ is the mean, σ is the standard deviation, and k is a constant;
[0089]
[0090] Among them, min(x) and max(x) represent the minimum and maximum values of the data set x respectively;
[0091] The energy efficiency data standardization includes normalizing the energy efficiency data to ensure that the energy efficiency data is comparable with other resource utilization data; the normalization formula is as follows:
[0092]
[0093] The initial setting includes initialization of the pheromone matrix τ and the heuristic value matrix η: the pheromone matrix τ is initialized to a small positive number τ0;
[0094]
[0095] The heuristic value matrix η is calculated based on the initial data:
[0096]
[0097] Among them, d ij For virtual machine v i From the host p src Migrate to host p dst The specific calculation method is as follows:
[0098] d ij =w1·CPU ij +w2·Memory ij +w3·Energy ij
[0099] Among them, w1, w2 and w3 are weight parameters used to balance the impact of different resources; Energy efficiency weight setting: according to the specific needs of the data center, set the weight w3 of energy efficiency in the total migration cost
[0100] b. Objective function definition, including 3 objective functions, namely:
[0101] 1) Maximize the host release:
[0102]
[0103] Among them, V p represents the set of virtual machines running on host p;
[0104] 2) Minimize the amount of virtual machine migration:
[0105]
[0106] Where T represents the set of all possible migration tuples, MigrationCost(v,p src ,p dst ) indicates that the virtual machine v is from the source host p src Migrate to the target host p dst The cost
[0107] 3) Maximize energy efficiency:
[0108]
[0109] Among them, Energy pv represents the energy efficiency of virtual machine v on host p;
[0110] c. Algorithm, comprising the following steps:
[0111] 1) Initialization
[0112] Initialize the pheromone matrix τ and the heuristic value matrix η:
[0113] The pheromone matrix τ is initialized to a small positive number;
[0114] The heuristic value matrix η is calculated based on the initial data;
[0115] 3) Ant search process
[0116] Step 1: Main loop: Iterate until the stopping condition, that is, all remaining hosts are fully utilized or the performance cannot be further improved when a given number of consecutive iterations is reached;
[0117] Step 2: ACS|PR| group optimizes the host release amount: each ant selects a migration plan based on pheromones and heuristic values, calculates the host release amount; updates the pheromone matrix τ and the heuristic value matrix η;
[0118] Step 3: ACSnM group optimizes the amount of virtual machine migration: each ant selects a migration plan based on pheromones and heuristic values, calculates the amount of virtual machine migration, and updates the pheromone matrix τ and the heuristic value matrix η;
[0119] Step 4: ACS|Energy|Group Optimization Energy Efficiency: Each ant selects a migration plan based on pheromones and heuristic values, and calculates energy efficiency; updates the pheromone matrix τ and the heuristic value matrix η;
[0120] Step 5: Update the global optimal migration plan: If ACS|PR| finds a migration plan with a higher host release amount, update Φ + ; If ACSnM finds a lower VM migration amount and is consistent with Φ + The migration scheme with at least the same host release amount in the update Φ + ; If ACS|Energy| finds a higher energy efficiency and is equal to Φ + The migration scheme with at least the same host release amount and virtual machine migration amount in the update Φ + .
[0121] This involves updating the following data paradigms:
[0122] (1) Pheromone update rules, pheromone update formula:
[0123] τ ij =(1-ρ)τ ij +Δτ ij
[0124] Among them, ρ is the pheromone volatility coefficient, Δτ ij is the pheromone increment, defined as:
[0125]
[0126] Among them, Q is a constant, L best is the optimal path length in the current iteration.
[0127] (2) Heuristic value calculation, heuristic value definition:
[0128]
[0129] Among them, d ij For virtual machine v i From the host p src Migrate to host p dst The cost.
[0130] (3) Ants choose migration plans - Each ant uses the following probability formula when choosing a migration plan:
[0131]
[0132] Among them, α and β are the pheromone importance and heuristic value importance parameters respectively, and allowed is the set of migration schemes that ants can choose.
[0133] (4) Migration cost calculation: Migration cost includes comprehensive consideration of CPU, memory, and energy efficiency:
[0134] d ij =w1·CPU ij +w2·Memory ij +w3·Energy ij
[0135] Among them, w1, w2 and w3 are weight parameters used to balance the impact of different resources.
[0136] (5) Global optimal solution update - At the end of each iteration, the global optimal solution Φ is updated according to the currently found optimal solution + :
[0137]
[0138] Among them, λ1 and λ2 are weight parameters used to balance the influence of different objective functions.
[0139] Example 2
[0140] Description of symbols involved in the embodiments of the present invention:
[0141] M-virtual machine migration solution collection
[0142] P - physical host collection
[0143] P R-The set of hosts released when executing the migration plan Φ
[0144] T K -The set of tuples not traversed by ant k
[0145] V-Virtual Machine Collection
[0146] -Target host P de The overall capability vector
[0147] N-Neighbors of the host
[0148] P de -Destination host in tuple
[0149] P SO - Source host in tuple
[0150] q-uniformly distributed random variable
[0151] S crk -Best score of Ant K so far
[0152] - The target host p has exploited capability vector
[0153] - The capability vector that the source host p has exploited
[0154] U v - Virtual machine v's exploited capability vector
[0155] η-heuristic value
[0156] τ - number of pheromones
[0157] τ0-initial pheromone quantity
[0158] Φ-Virtual machine migration solution
[0159] Φ + -Global optimal virtual machine migration solution
[0160] -ACS nM The best migration solution so far
[0161] The best migration solution so far
[0162] Φ K -Migration plan defined by Ant K
[0163] -Temporary migration plan defined by Ant K
[0164] The number of additional pheromones in the tuple
[0165] The number of additional pheromones in the tuple
[0166] β - parameter that determines the relative importance of η
[0167] ρ - Decay parameter of pheromone in local update rule
[0168] q0 - parameter that determines the relative importance of the search
[0169] nA - the number of ants that simultaneously build migration plans
[0170] nI - number of ant generations
[0171] Example 3: Embedded Algorithm Example a
[0172] 1.While until a stopping criterion is met do
[0173] 2. / / Global optimal migration solution assigned to empty set
[0174] 3. / / get Migration solution of the algorithm
[0175] 4. / / Get ACS nM Migration solution of the algorithm
[0176] 5. PM release is dominant
[0177] 6. / / Update Φ +
[0178] 7. / / ACS nM The PM release is at least dominant
[0179] 8. / / ACS nM Solution: VM migration volume is dominant
[0180] 9. / / Update Φ +
[0181] 10.consolidate VMs according to Φ + and terminate released PMs
[0182] 11.end while
[0183] This algorithm achieves the simultaneous optimization of the final dual objectives by coordinating the operation of two different single-objective optimization populations. Optimize the amount of virtual machine migration through group ACS nM The two groups operate independently and use independent pheromones and heuristic values, but the two groups will collaborate to produce the global optimal migration plan Φ + .
[0184] In virtual machine merging, each host p can host multiple virtual machines v. A host that hosts at least one virtual machine is called a potential source host. Both the host and the virtual machine have resource utilization attributes, including CPU utilization and memory utilization. The MOACS algorithm also introduces the concept of host neighbors. Neighbors represent mutually exclusive subsets of P. A neighbor is an abstract entity that represents a set of hosts located in adjacent locations in a data center. Then, the hosts in a data center can constitute a neighbor of the host. A virtual machine can be migrated to other hosts in any neighbor N. Therefore, other hosts within and outside the source host's neighbors are also a potential target host and also have resource utilization attributes. Based on this, the MOACS algorithm establishes a tuple set T, each tuple t∈T consists of three elements: source host P SO , virtual machine v and target host P de .
[0185] T=(P SO ,v,P de )(1); max|T|=|P||V|(|N|-1)(2);
[0186] Initially, the global optimal migration solution Φ + Unknown, Φ + is an empty set (step 2). The main loop process of step 1 iterates until the stopping condition, that is, all remaining hosts are fully utilized or the performance cannot be further improved when the given number of consecutive iterations is reached. In each iteration, the two groups and ACS nM Try to find the global optimal migration solution Φ according to their respective goals + In step 3, Find a migration solution with higher host release capacity. In step 4, ACS nM Find a migration solution with less virtual machine migration. When an improved migration solution is found, the global optimal migration solution Φ + is updated. If Find a migration solution with higher host release (step 5), Φ + according to The best migration plan so far obtained is updated, that is, (Step 6). Similarly, when ACS nM Find a VM with lower VM migration volume and Φ + When there is at least the same migration plan for the host release amount (step 7-step 8), Φ + According to ACS nM The best migration plan so far obtained is updated, that is, (Step 9). Finally, at the end of each iteration in the main loop, the virtual machine will migrate according to the global optimal migration solution Φ + The merge is performed and the released hosts are terminated, i.e. step 10.
[0187] Embodiment 4, embedded algorithm example b
[0188] 1. / / initialization
[0189] 2.t∈T|τ t =τ0 / / Tuple pheromone initialization
[0190] 3.for i∈[1,nI]do
[0191] 4. for k∈[1, nA] do
[0192] 5.
[0193] 6. S crk =0
[0194] 7. compute prob s by using(7) / / Calculate the probability of ants crossing
[0195] 8.choose a tuple t∈Tk to traverse by using(6)
[0196] 9. / / Update the temporary migration plan
[0197] 10.apply local update rule in(11) on t / / update tuple
[0198] 11.if the migration in t does not overload destination PM P de then
[0199] / / The migration of tuples did not cause host overload
[0200] 12.update used capacity vectors and int
[0201] 13.
[0202] 14.
[0203] 15.Φ K =Φ K ∪{t} / / Update the migration plan of ant k
[0204] 16.M=M∪{Φ K}
[0205] 17. / / Based on formula (3), we can get the global optimal solution
[0206] 18.apply global update rule in (9) on all s∈T / / Update all tuples
[0207] 19.return
[0208] The goal is to optimize the host release amount |P R |, that is, the algorithm objective function is: maxf(Φ)=|P R |(3);
[0209] Since the primary goal of VM consolidation is to minimize the number of active hosts, the objective function is defined in terms of the number of hosts released. And when executing the migration scheme, a constraint is used to minimize the number of active hosts by limiting the migration to only the released hosts P. R On hosts outside the collection to reduce the amount of virtual machine migration nM, that is:
[0210] In the algorithm, a host is considered to be released only after all virtual machines on it have been migrated. Therefore, the set of released hosts P R It can be defined as:
[0211] In the virtual machine merging problem, ants store pheromones according to the tuples defined in equation (1). Any ant among nA ants uses the random state transition rule to select the next tuple that can be passed through. The state transition rule in is the pseudo-random ratio rule. According to this rule, the rule for ant k to select the next tuple is:
[0212]
[0213] Where: argmax returns [τ][η] β The tuple with the maximum value is obtained, q∈[0,1] is a uniformly distributed random variable, q0∈[0,1] is a parameter, S is a random variable selected according to the probability distribution of formula (7), and the probability prob that ant k chooses tuple s to traverse is s Defined as:
[0214]
[0215] The heuristic value η of tuple s s Defined as:
[0216]
[0217] It can be seen that the target host with the least unutilized capacity will receive the highest heuristic value. Therefore, the heuristic value is biased towards virtual machine migration that leads to reduced host underutilization, while Prevents VM migration from causing the target host to de overload.
[0218] In addition to the random state transition rules, The global and local pheromone trail evaporation rules are also used. The global pheromone trail evaporation rule is used at the end of an iteration when all ants have completed their migration and is defined as:
[0219]
[0220] Where: α∈(0,1] is the pheromone attenuation parameter, For additional pheromone amount, only give The tuple in the best migration plan so far in , that is, Defined as:
[0221]
[0222] The local pheromone trail update rule is applied to the tuple when the ant traverses the tuple to make its own migration plan, and is defined as:
[0223] τ s =(1-ρ)·τ s +ρ·τ0 (11);
[0224] Where: ρ∈(0,1] is similar to α, τ0 represents the initial pheromone, which is calculated as the inverse value of the product of the number of hosts |P| and the approximate optimal solution |Φ|, that is: τ0=(|Φ|·|P|) -1 (12);
[0225] The algorithm generates a set of tuples T using equation (1) and sets the tuple pheromone to the initial pheromone τ0 using equation (12) (step 2). Then, the algorithm iterates nI times (step 3), and each iteration i establishes a new generation of nA ants and establishes their migration plan (steps 4 to 20). Each ant k iterates on |T| tuples (steps 6 to 18). The probability of selecting the next tuple to traverse is calculated using equation (7) (step 7). Then, based on the probability and random state transition rules calculated by equations (6) and (7), each ant selects a tuple t to traverse (step 8) and adds t to its temporary migration plan. (Step 9). Apply the local pheromone trace update rules of (11) and (12) on t (Step 10). If the migration represented by t does not make the target host P de If an overload occurs, the source host and the target host in t The utilized capability vector of will be updated according to the migration (step 12). Then, the objective function of equation (3) will be applied to If the target value obtained is higher than the ant’s optimal value S so far, crk (Step 13), then add t to the migration plan Φ defined by the ant K Then, when all ants complete the migration plan, the migration plan is added to M (step 16). K The objective function of formula (3) is used for evaluation, and the global optimal solution obtained so far is selected as Φ + (Step 17), and all tuples are updated using the global pheromone trace update rules in equations (9) and (10) (Step 18). Finally, when all iterations i are completed, Returns the best solution obtained so far (Step 19).
[0226] Example 5: Embedded Algorithm Example c
[0227] 1. / / initialization
[0228] 2.t∈T do τ t =τ0 / / Tuple pheromone initialization
[0229] 3.for i=1 to nI do / / iterative ant algebra
[0230] 4.for k=1 to nA do / / for each ant
[0231] 5. S crk=0 / / Initialize the temporary migration plan of the ant and the current optimal score
[0232] 6.
[0233] 7.compute probs using (7) / / Calculate the probability of ants crossing
[0234] 8.Choose a tuple t∈Tk to traverse using(6) / / Choose the next tuple
[0235] 9. / / Update the temporary migration plan of ants
[0236] 10.apply local update rule(11) on t / / Update tuple pheromone
[0237] 11.if the migration int does not overload destination PM then / / If the migration does not cause the destination host to be overloaded
[0238] 12.update used capacity vectors in t / / Update used capacity vector
[0239] 13.
[0240] 14. / / Update the current optimal score of the ant
[0241] 15.Φ K =Φ K ∪{t} / / Update the ant migration plan
[0242] 16.M=M∪{Φ K} / / Add the ant's migration plan to the collection
[0243] 17. / / Choose the best migration solution
[0244] 18.apply global update rule (14) on all s∈T / / Update the pheromone of all tuples
[0245] 19. / / Return the optimal migration plan
[0246] ACS nM The goal is to find a virtual machine with less migration and at least equal to Φ+ Medium host release amount P R The equivalent migration scheme, that is, the algorithm objective function is: Max g(Φ) = (nM) -1 (13);
[0247] Because virtual machine migration is a resource-intensive operation, ACS nM The objective function can be defined as the inverse of the migration amount.
[0248] ACS nM The ants in also use the pseudo-random ratio rule of equations (6) and (7) to select the next tuple to be traversed. And the heuristic value in equation (8) is more biased towards the virtual machine U with more utilized capability vectors v In this way, the virtual machine migration operation that is more likely to result in a reduction in the amount of virtual machine migration will receive a higher heuristic value. Therefore, the heuristic value in formula (8) supports ACS nM The objective function is defined in equation (13).
[0249] ACS nM The group also uses the local pheromone trail update rule in equation (11), however, the global pheromone trail update rule needs to be redefined as:
[0250]
[0251] Where: Indicates the amount of additional pheromone, only given to ACS nM The tuple in the best migration plan so far, that is, Defined as:
[0252]
[0253] ACS nM Algorithm steps and Similar (step 1 to step 12). Step 13 uses the host release amount objective function defined in equation (3) to replace the virtual machine migration amount objective function in equation (13). However, when step 17 is from ACS nM When choosing the best migration solution so far (i.e. ), all migration schemes defined by ants Φ K ∈M will first be evaluated using the host release objective function defined in formula (3), and then evaluated using the virtual machine migration objective function defined in formula (13). Therefore, the migration scheme with the most host release and less virtual machine migration will be selected as the optimal migration scheme. After that, the algorithm will use the global pheromone trace update rules in equations (14) and (15) to update pheromones on all tuples, i.e., step 18. Finally, the algorithm returns That is step 19.
[0254] The present invention designs a completely new data system structure, and updates the pheromone update data algorithm paradigm, the heuristic value data algorithm paradigm, the ant selection migration scheme probability data algorithm paradigm, the migration cost data algorithm paradigm, and the global optimal scheme update data algorithm paradigm;
[0255] The present invention further introduces the concept of dimension expansion, taking energy efficiency as an additional optimization dimension, and meets the demand for reducing energy efficiency in virtual machine optimization data.
[0256] Finally, it should be noted that the above are only preferred embodiments of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art can still modify the technical solutions described in the aforementioned embodiments or replace some of the technical features therein with equivalents. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the scope of protection of the claims of the present invention.
Claims
1. An improved ant colony virtual machine optimization data system based on dimension expansion, characterized in that: It includes three major steps: data preprocessing, objective function definition and algorithm implementation. It initially handles anomalies by collecting and cleaning the resource data of virtual machines and physical hosts, and then sets optimization goals, including maximizing the host release, minimizing the migration amount and improving energy efficiency. By initializing the pheromone and heuristic value matrix, the ant algorithm is used to select the migration plan and update the pheromone to ensure the efficiency and cost-effectiveness of the migration process.
2. The improved ant colony virtual machine optimization data system based on dimension expansion according to claim 1 is characterized in that: By optimizing virtual machine deployment through advanced algorithms, resource utilization is maximized and operating costs are minimized, effectively guiding virtual machines to migrate from high-load servers to idle servers, thereby improving overall energy efficiency and reducing energy consumption.
3. The improved ant colony virtual machine optimization data system based on dimension expansion according to claim 1 is characterized in that: The data includes the following: a. Data preprocessing; b. Objective function definition, including 3 objective functions, namely: 1) Maximize the host release: Among them, V p represents the set of virtual machines running on host p; 2) Minimize the amount of virtual machine migration: Where T represents the set of all possible migration tuples, MigrationCost(v,p src ,p dst ) indicates that the virtual machine v is from the source host p src Migrate to the target host p dst The cost 3) Maximize energy efficiency: Among them, Energy pv represents the energy efficiency of virtual machine v on host p; c. Algorithm, comprising the following steps: 1) Initialization Initialize the pheromone matrix τ and the heuristic value matrix η: The pheromone matrix τ is initialized to a small positive number; The heuristic value matrix η is calculated based on the initial data; 2) Ant search process Step 1: Main loop: Iterate until the stopping condition, that is, all remaining hosts are fully utilized or the performance cannot be further improved when a given number of consecutive iterations is reached; Step 2: ACS|PR| group optimizes the host release amount: each ant selects a migration plan based on pheromones and heuristic values, calculates the host release amount; updates the pheromone matrix τ and the heuristic value matrix η; Step 3: ACSnM group optimizes the amount of virtual machine migration: each ant selects a migration plan based on pheromones and heuristic values, calculates the amount of virtual machine migration, and updates the pheromone matrix τ and the heuristic value matrix η; Step 4: ACS|Energy|Group Optimization Energy Efficiency: Each ant selects a migration plan based on pheromones and heuristic values, and calculates energy efficiency; updates the pheromone matrix τ and the heuristic value matrix η; Step 5: Update the global optimal migration plan: If ACS|PR| finds a migration plan with a higher host release amount, update Φ + ; If ACSnM finds a lower VM migration amount and is consistent with Φ + The migration scheme with at least the same host release amount in the update Φ + ; If ACS|Energy| finds a higher energy efficiency and is equal to Φ + The migration scheme with at least the same host release amount and virtual machine migration amount in the update Φ + .
4. The improved ant colony virtual machine optimization data system based on dimension expansion according to claim 3 is characterized in that: The data preprocessing includes data collection, data cleaning, energy efficiency data standardization, and initial settings.
5. The improved ant colony virtual machine optimization data system based on dimension expansion according to claim 3 is characterized in that: The data collection includes virtual machine and physical host resource utilization data: data set D contains resource utilization information of all physical hosts P and virtual machines V; data format: D={(p i ,v j ,CPU ij ,Memory ij ,Energy ij )|p i ∈P,v j ∈V} Among them, CPU ij 、Memory ij and Energy ij Represents virtual machine v j On the host p i CPU, memory utilization and energy efficiency on the host; Energy efficiency data collection: Collect energy efficiency data of each virtual machine on different hosts, expressed as Energy ij ;Energy efficiency data can include power consumption, cooling requirements, etc.; The data cleaning includes missing values, outliers, standardization processing, etc.; specifically includes: If |x i -μ|>Kσ, then x i is an outlier; Among them, μ is the mean, σ is the standard deviation, and k is a constant; Among them, min(x) and max(x) represent the minimum and maximum values of the data set x respectively; The energy efficiency data standardization includes normalizing the energy efficiency data to ensure that the energy efficiency data is comparable with other resource utilization data; the normalization formula is as follows: The initial setting includes initialization of the pheromone matrix τ and the heuristic value matrix η: the pheromone matrix τ is initialized to a small positive number τ0; The heuristic value matrix η is calculated based on the initial data: Among them, d ij For virtual machine v i From the host p src Migrate to host p dst The specific calculation method is as follows: d ij =w1·CPU ij +w2·Memory ij +w3·Energy ij Among them, w1, w2 and w3 are weight parameters used to balance the impact of different resources; energy efficiency weight setting: according to the specific needs of the data center, set the weight w3 of energy efficiency in the total migration cost.
6. The improved ant colony virtual machine optimization data system based on dimension expansion according to claim 3 is characterized in that: The pheromone update formula is: t ij =(1-ρ)τ ij +Δt ij Among them, ρ is the pheromone volatility coefficient, Δτ ij is the pheromone increment, defined as: Among them, Q is a constant, L best is the optimal path length in the current iteration.
7. The improved ant colony virtual machine optimization data system based on dimension expansion according to claim 3 is characterized in that: The heuristic value is defined as: Among them, d ij For virtual machine v i From the host p src Migrate to host p dst The cost.
8. The improved ant colony virtual machine optimization data system based on dimension expansion according to claim 3 is characterized in that: Each ant selects a migration scheme based on pheromone and heuristic value, using the following probability formula: Among them, α and β are the pheromone importance and heuristic value importance parameters respectively, and allowed is the set of migration plans that ants can choose.
9. The improved ant colony virtual machine optimization data system based on dimension expansion according to claim 5 is characterized in that: The migration cost includes comprehensive consideration of CPU memory and energy efficiency. The migration cost calculation formula is: d ij =w1·CPU ij +w2·Memory ij +w3·Energy ij Among them, w1, w2 and w3 are weight parameters used to balance the impact of different resources.
10. The improved ant colony virtual machine optimization data system based on dimension expansion according to claim 3 is characterized in that: The updating of the global optimal migration solution includes updating the global optimal solution Φ according to the currently found optimal solution at the end of each iteration. + : Among them, λ1 and λ2 are weight parameters used to balance the influence of different objective functions.
