Cloud data center virtual machine adaptive integration method based on Autoformer and enhanced double-Q network

Abstract

The invention discloses a cloud data center virtual machine adaptive integration method based on Autoformer and an enhanced double-Q network, and relates to the technical field of cloud computing resource management and optimization, the resource utilization rate of each physical host is monitored in real time, historical load time sequence data is collected, a pre-trained Autoformer model is loaded, and double-Q network parameters and environment state variables are initialized; and predicting a host resource utilization rate based on an Autoformer model, and dividing an overload host, a low-load host and a normal host in combination with the current host resource utilization rate. According to the method, the Autoformer model is adopted for load prediction, trend terms and period terms are decomposed through an autocorrelation mechanism, the multi-scale periodicity in the cloud load is effectively captured, compared with a traditional LSTM model, the Autoformer is higher in modeling capacity for a complex time sequence mode, the misjudgment rate of an overload / low-load host can be remarkably reduced, a more accurate host state detection basis is provided for virtual machine integration, and the method has the advantages of being high in practicability and easy to popularize. Therefore, a resource allocation strategy is optimized.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of cloud computing resource management and optimization, and particularly relates to a cloud data center virtual machine adaptive integration method based on Autoformer and enhanced double Q network. BACKGROUND

[0002] With the rapid development of cloud computing, data centers, as the core infrastructure supporting big data analysis, artificial intelligence and Internet of Things applications, have experienced exponential growth in user demand, which has led to a significant expansion in data center scale. Such data centers are typically composed of thousands of physical servers. However, this expansion has brought a key problem: high energy consumption, with a significant portion of energy being wasted due to low resource utilization efficiency. High energy consumption not only increases the operating costs of data center operators but also exacerbates carbon emissions, which conflicts with global sustainable development and carbon neutralization goals. Therefore, reducing energy consumption while ensuring service quality has become an urgent challenge in the field of data center management.

[0003] To address the energy efficiency problem, virtual machine integration has emerged as a core strategy. Virtual machine integration optimizes resource allocation by dynamically migrating virtual machines between physical hosts (PMs). By integrating virtual machines onto fewer active physical hosts, idle or low-load servers can be shut down or placed into low-power mode, thereby reducing overall energy consumption and improving resource utilization. However, virtual machine integration is a complex decision-making process that requires balancing multiple objectives, such as minimizing energy consumption, reducing service level agreement violation rates, and reducing migration overhead. Achieving this balance requires accurate load prediction, reasonable virtual machine selection, and optimized virtual machine placement strategies.

[0004] Existing virtual machine integration methods mainly include heuristic algorithms, meta-heuristic algorithms, and reinforcement learning-based methods. However, they all have certain limitations. In addition, existing methods also include an adaptive DRL-based virtual machine integration framework (ADVMC), which provides an important reference for the field. ADVMC includes two core stages: (1) an IC-based VM selection algorithm (ICVMS), which calculates the similarity between VMs and hosts to preferentially migrate VMs that have the greatest impact on host overload, quickly relieving load pressure; (2) a PADRL-based DRL placement method (PADRL), which uses LSTM to predict future system states to accelerate DRL convergence and achieve energy-efficient VM placement.

[0005] Despite the significant progress made by ADVMC, it still has limitations:

[0006] (1) Load detection and prediction: PADRL relies on LSTM for state prediction, which can capture temporal dependencies, but has limited modeling capability for complex multi-scale periodicity in cloud load (such as intra-day and intra-week fluctuations), which may lead to misjudgment of overloaded / underloaded hosts;

[0007] (2) VM selection strategy: The influence coefficient of ICVMS only considers the resource correlation and usage of VM and host, without considering the actual overhead such as migration time, which may select large memory VMs and cause performance loss during migration;

[0008] (3) DRL placement mechanism: The DRL model of PADRL uses a fixed exploration probability and learning rate, which is difficult to balance "exploring new actions" and "using known optimal actions", and there is still room for improvement in convergence efficiency and decision accuracy in dynamic environments;

[0009] Therefore, how to use Autoformer for load prediction, decompose trend items and periodic items through autocorrelation mechanism, improve host state detection accuracy, and construct VM migration influence coefficient by combining Pearson correlation coefficient and migration time, balance load reduction effect and migration overhead, introduce upper confidence bound (UCB) and adaptive learning rate in double Q network, optimize exploration-exploitation balance and convergence speed of DRL, thereby reducing energy consumption and reducing SLAV, and provide a better solution for energy-saving cloud data centers, is the problem to be solved by the present application. To this end, a cloud data center virtual machine adaptive integration method based on Autoformer and enhanced double Q network is proposed. SUMMARY

[0010] The present application aims to provide a cloud data center virtual machine adaptive integration method based on Autoformer and enhanced double Q network to solve the problems raised in the background art.

[0011] To solve the above technical problems, the technical solution adopted by the present application is:

[0012] A cloud data center virtual machine adaptive integration method based on Autoformer and enhanced double Q network, comprising the following steps:

[0013] S1, real-time monitoring of resource utilization of each physical host, collection of historical load time series data, loading of pre-trained Autoformer model, initialization of double Q network parameters and environment state variables;

[0014] S2, predicting host resource utilization based on the Autoformer model, and dividing overloaded hosts, underloaded hosts and normal hosts according to the current host resource utilization;

[0015] S3, based on the Pearson correlation coefficient of the virtual machine and the physical host and the migration time, a migration influence coefficient is constructed to comprehensively evaluate the virtual machine migration value;

[0016] S4, select the virtual machine to be migrated according to the migration influence coefficient from high to low, and the list of virtual machines to be migrated, wherein, for the overload host, a greedy algorithm is used to select the migration virtual machine, and all virtual machines are marked for the low-load host;

[0017] S5, training the enhanced double Q network by using the experience replay mechanism, and regularly updating the target network parameters to optimize the decision strategy;

[0018] S6, through the enhanced double Q network, the optimal target physical host is selected to execute the virtual machine placement by combining the confidence upper bound strategy;

[0019] S7, after executing the virtual machine migration, the idle physical host with low resource utilization rate is closed to save energy.

[0020] The further improvement of the technical scheme of the application is that the S1 specifically comprises:

[0021] The resource utilization rate data of all physical hosts including CPU and memory is monitored in real time through the data center monitoring system, and the historical load time series data is stored according to the set time window;

[0022] The pre-trained Autoformer model is loaded, and the parameters of the double Q network are initialized, including the online Q network and the target Q network, and the environment state variable is defined;

[0023] According to the scale configuration of the data center, the hyperparameters including the time window size, the exploration coefficient and the discount factor are included, the experience replay buffer capacity and the training interval parameters are initialized.

[0024] The further improvement of the technical scheme of the application is that the S2 specifically comprises:

[0025] The current resource utilization rate of each physical host is obtained in real time, and the continuous resource utilization rate sequence is extracted according to the historical time window, the data of the resource utilization rate sequence is input into the pre-trained Autoformer model, and the resource utilization rate of the host is decomposed and predicted through the autocorrelation mechanism of the Autoformer model;

[0026] The current resource utilization rate and the predicted resource utilization rate of each physical host are compared with the preset overload threshold and low-load threshold respectively, if any resource utilization rate exceeds the overload threshold, the host is determined as an overload host, and the migration process is triggered to relieve the load pressure;

[0027] If the predicted resource utilization is lower than the low-load threshold, further check whether all resource utilization data in the historical load time series data are continuously lower than the low-load threshold, and only when the long-term load level is low, the low-load host is finally determined.

[0028] The further improvement of the technical scheme of the application is that the S3 specifically comprises:

[0029] Based on the historical resource utilization data of the virtual machine and the physical host, the Pearson correlation coefficient between them is calculated to describe the correlation degree of the resource utilization of the virtual machine and the host.

[0030] Based on the memory occupation of the virtual machine and the available network bandwidth of the host, the time cost modeling of the virtual machine migration from the host is performed to analyze the migration time, which directly reflects the time cost required to complete the migration, and the larger the memory or the lower the bandwidth, the greater the cost.

[0031] The Pearson correlation coefficient and the migration time are integrated to construct the migration influence coefficient of the virtual machine, and the migration value of the virtual machine is comprehensively evaluated.

[0032] The further improvement of the technical scheme of the application is that the Pearson correlation coefficient is a core index for measuring the linear correlation of two variables, which can accurately describe the correlation degree of the resource utilization of the virtual machine and the host in the virtual machine selection scene, and the formula is as follows:

[0033] ;

[0034] ;

[0035] ;

[0036] Among them, the Pearson correlation coefficient is represented by P, the virtual machine is represented by V, the physical host is represented by H, the length of the historical time window is represented by T, the CPU utilization of the virtual machine at time t is represented by V(t), the CPU utilization of the physical host at time t is represented by H(t), the mean of the CPU utilization time series of the virtual machine is represented by Vmean, the mean of the CPU utilization time series of the physical host is represented by Hmean.

[0037] The time cost modeling of the virtual machine migration from the physical host, that is, the formula of the migration time is:

[0038] ;

[0039] Among them,​​ for the virtual machine memory occupation, for the physical host available network bandwidth;

[0040] The migration influence coefficient of the virtual machine is obtained based on the Pearson correlation coefficient and the migration time, and the formula is as follows:

[0041] ;

[0042] wherein, is the migration influence coefficient, the square operation strengthens the influence of the high correlation virtual machine, and the denominator item punishes the migration of the large memory virtual machine.

[0043] Further improvement of the technical scheme of the application is that the S4 specifically comprises:

[0044] For all virtual machines to be evaluated, the migration influence coefficient is calculated according to the square of the Pearson correlation coefficient with the physical host and the migration time, and the virtual machines to be migrated are ranked from high to low according to the value of the migration influence coefficient, and a priority list of the virtual machines to be migrated is generated;

[0045] For the overloaded host, a greedy strategy is adopted to select the virtual machine with the highest migration influence coefficient from the priority list of the virtual machines to be migrated, and the migration is performed until the host load is reduced to below the safety threshold or there is no suitable virtual machine to be migrated, and the host load state needs to be re-evaluated after each migration;

[0046] For the low-load host, all virtual machines are directly marked as migratable objects and are not involved in priority ranking and are directly included in the list of virtual machines to be migrated, and then the virtual machines selected from the overloaded host and all virtual machines of the low-load host are merged to form a final list of virtual machines to be migrated.

[0047] Further improvement of the technical scheme of the application is that the S5 specifically comprises:

[0048] State transition samples are collected through the virtual machine migration interaction process, including the current environment state, the execution action, the immediate reward and the next state, and the data is stored in the experience replay buffer in time sequence to ensure sample independence and avoid overfitting;

[0049] A batch of samples is randomly sampled from the experience replay buffer, the online Q network parameter gradient is calculated based on the improved loss function (time difference error square), the update step is dynamically adjusted by combining the adaptive learning rate, the network weight is optimized through back propagation to approximate the optimal strategy, and the convergence and robustness of the reinforcement learning decision are improved.

[0050] The online Q network parameters are copied to the target Q network at a fixed interval to keep the target Q value calculation benchmark stable, avoid the estimation shock caused by parameter synchronization delay in traditional Q learning, and improve the convergence and robustness of the reinforcement learning decision, forming an enhanced double Q network.

[0051] Further improvement of the technical solution of the present application is that the enhanced double Q network comprises two Q networks with same structure but different parameter updating methods, namely an online Q network QNet( ) and a target Q network TargetQNet( ), wherein the online Q network is used to select an execution action according to a current environment state , and the target Q network is used to calculate a target Q value;

[0052] Representation of the current environment state: the current environment state comprises CPU utilization information of each physical host and CPU demand information of a virtual machine, and is defined as:

[0053] ;

[0054] wherein, represents CPU utilization of the i th physical host, represents CPU request information of the virtual machine to be placed; Selection of the execution action: the execution action

[0055] represents that the virtual machine to be selected is placed on a suitable physical host, in order to balance exploration and utilization, a confidence upper bound strategy is used to improve the action selection, for a certain environment state , the optimal execution action is selected according to the following formula:

[0056] ;

[0057] wherein, is a set of all execution actions that can be selected under the current environment state , is the current optimal execution action, is a confidence upper bound, and is defined as follows:

[0058] ;

[0059] wherein, is a value estimate of the online Q network for the action under the state , is an exploration coefficient, is a total number of times that the state is visited, is a number of times that the action is selected under the state ;

[0060] ​​​Instant reward: based on the multi-objective optimization problem in the system model, the reward function is expressed as a function of energy consumption, SLA violation rate and migration number, defined as follows:

[0061] ;

[0062] wherein, is a weight coefficient, satisfying ; is the total energy consumption, is the theoretical maximum energy consumption and the minimum energy consumption; is the service level agreement violation rate (Service Level Agreement Violation), is the theoretical maximum ; is the minimum ; represents the migration number, is the theoretical maximum migration number and the minimum migration number;

[0063] Loss function: the online Q network adopts the mean square error loss function to calculate the parameter gradient, and the formula is defined as follows:

[0064] ;

[0065] wherein, is the value estimation of the online Q network for the state under the action ;R is the reward obtained; is the discount factor; is the value estimation of the target Q network for the next state under the action ; is the optimal action selected by the target Q network for the next state;

[0066] Adaptive learning rate: the training of the online Q network adopts the adaptive learning rate, and the definition is as follows:

[0067] ;

[0068] wherein, is the learning rate, is the number of times of selecting the action under the state .

[0069] Further improvement of the technical scheme of the present application is that the S6 specifically comprises:

[0070] ​Based on the constructed enhanced double Q network including an online Q network and a target Q network, the input state includes physical host resource utilization, virtual machine demand and historical migration record, the network parameters are updated by using the time difference error through the experience replay mechanism, and the deterministic value and the exploratory value of the state-action are learned respectively, the overestimation bias is reduced, and the stability of action selection is improved.

[0071] In the action selection stage, the confidence upper bound (UCB) value of each candidate host is calculated combined with the output of the enhanced double Q network, and the host with high confidence upper bound (UCB) value is preferentially selected, and the high value action and the exploration of unknown environment are considered.

[0072] After completing N times of migration, the network parameters are soft-updated to the target Q network, the training process is stabilized, and based on the current state, the host with the highest confidence upper bound (UCB) value is selected by the greedy strategy to perform placement, and the global optimization of resource utilization, migration cost and QoS (quality of service) constraint is performed.

[0073] The further improvement of the technical scheme of the application is that the S7 specifically comprises:

[0074] According to the target host determined by the enhanced double Q network, the virtual machine to be migrated is migrated from the overloaded or low-load physical host to the selected host, and the resource allocation state of the physical host and the virtual machine is updated, so that the migration process meets the service quality constraint;

[0075] The resource utilization of each physical host after migration is monitored in real time, and the physical host which is continuously in a low-load state and has no task to be processed is selected as a candidate set of idle hosts which can be closed;

[0076] All idle physical hosts in the candidate set of idle hosts are closed through the power management interface, and the data center topology structure is updated, so that the remaining active hosts meet the current load demand, and finally the overall energy consumption is reduced.

[0077] Due to the adoption of the above technical scheme, the application has the following technical progress compared with the prior art:

[0078] 1、The application provides a cloud data center virtual machine adaptive integration method based on Autoformer and enhanced double Q network, adopts the Autoformer model for load prediction, decomposes the trend item and the periodic item through the autocorrelation mechanism, effectively captures the multi-scale periodicity in the cloud load, and compared with the traditional LSTM model, the Autoformer has stronger modeling capability for complex time sequence mode, can significantly reduce the misjudgment rate of overloaded / low-load hosts, provides more accurate host state detection basis for virtual machine integration, and thus optimizes the resource allocation strategy.

[0079] 2. The application provides a cloud data center virtual machine adaptive integration method based on Autoformer and enhanced double Q network, which constructs a migration influence coefficient by fusing Pearson correlation coefficient and migration time, comprehensively evaluates the virtual machine migration value, considers not only the resource correlation, but also quantifies the migration time overhead, avoids the performance loss caused by selecting a large memory virtual machine, and compared with the traditional influence coefficient algorithm, can balance the load reduction effect and migration cost, and improve the resource integration efficiency.

[0080] 3. The application provides a cloud data center virtual machine adaptive integration method based on Autoformer and enhanced double Q network, the enhanced double Q network reduces the overestimation bias of Q value through the double evaluator structure (online Q network and target Q network), combines the confidence upper bound strategy to optimize action selection, realizes dynamic balance between exploration and utilization, and adjusts the update step length according to the state-action pair access frequency through the adaptive learning rate mechanism, uses a smaller learning rate for high-frequency access pairs to ensure stability, uses a larger learning rate for low-frequency access pairs to accelerate exploration, and significantly improves the convergence speed and decision accuracy in complex cloud environment. BRIEF DESCRIPTION OF DRAWINGS

[0081] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed in the embodiments will be briefly introduced as follows. Obviously, the drawings described below are only some embodiments described in the present application, and other drawings can also be obtained by those skilled in the art based on these drawings.

[0082] Figure 1 The working process diagram of the cloud data center virtual machine adaptive integration method based on Autoformer and enhanced double Q network of the present application is shown in the figure.

[0083] Figure 2 The method flow diagram of the cloud data center virtual machine adaptive integration method based on Autoformer and enhanced double Q network of the present application is shown in the figure. DETAILED DESCRIPTION

[0084] In order to make the purpose, technical scheme and advantages of the embodiments of the present application more clear, the technical scheme in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0085] Embodiment 1, as Figure 1 , Figure 2As shown, the application provides a cloud data center virtual machine adaptive integration method based on Autoformer and enhanced double Q network, including the following steps:

[0086] S1, real-time monitoring of resource utilization of each physical host, collecting historical load time series data, and loading pre-trained Autoformer model, initializing double Q network parameters and environment state variables, real-time monitoring of resource utilization data of all physical hosts including CPU and memory through data center monitoring system, and storing historical load time series data according to the set time window, loading pre-trained Autoformer model, initializing double Q network parameters including online Q network and target Q network, and defining environment state variables, configuring hyperparameters including time window size, exploration coefficient and discount factor according to the scale of data center, initializing experience replay buffer capacity and training interval parameters;

[0087] The specific work content is: through the data center monitoring system, real-time collection of resource utilization data of all physical hosts, including CPU and memory usage, recording each resource utilization data at a fixed time interval, and storing historical load time series data according to the preset time window length, forming a continuous time series data set; load the pre-trained Autoformer model, the Autoformer model decomposes the load trend item and the periodic item through the autocorrelation mechanism, which is used for accurate prediction of host load, and the parameters of double Q network are initialized, including the weights of online Q network (QNet) and target Q network (TargetQNet), ensuring that the two structures are the same but the parameters are updated independently, and the environment state variable is defined as the combination of physical host CPU utilization and virtual machine CPU demand, which is used to represent the current resource allocation state; configure key hyperparameters according to the scale of data center, including time window size, exploration coefficient and discount factor, wherein the time window needs to cover the complete business cycle to capture multi-scale load mode, the exploration coefficient determines the trade-off between exploring new actions and using known optimal actions in reinforcement learning, too high may lead to slow convergence, too low is easy to fall into local optimum, and the discount factor reflects the influence degree of future reward on current decision, when initializing the experience replay buffer, determine its capacity and training interval parameters: the buffer capacity needs to be enough to store historical interaction data to avoid sample coverage; the training interval needs to balance real-time and computational overhead to ensure that the double Q network can continuously optimize the decision strategy;

[0088] S2, based on the Autoformer model, the host resource utilization is predicted, the current host resource utilization is combined to divide the overload host, the low-load host and the normal host, the current resource utilization of each physical host is obtained in real time, and the continuous resource utilization sequence is extracted according to the historical time window, the data of the resource utilization sequence is input into the pre-trained Autoformer model, the resource utilization of the host is decomposed and predicted through the autocorrelation mechanism, and the current resource utilization of each physical host is compared with the preset overload threshold and low-load threshold respectively, if any resource utilization exceeds the overload threshold, the host is determined as an overload host, and the migration process is triggered to relieve the load pressure, if the predicted resource utilization is lower than the low-load threshold, it is further verified whether all resource utilization data in the historical load time series data are continuously lower than the low-load threshold, and only when the host is in a low load level for a long time, the host is finally determined as a low-load host;

[0089] The specific work content is: through the data center monitoring system, the current resource utilization of each physical host is collected in real time, and the continuous resource utilization sequence is extracted according to the preset historical time window, the sequence is input into the pre-trained Autoformer model, the encoder decomposes the time series data into trend items (long-term changes) and periodic items (periodic fluctuations) through the autocorrelation mechanism, and the decoder double-branch structure processes two types of components and predicts the resource utilization respectively; the current resource utilization and the predicted resource utilization of each physical host are compared with the preset overload threshold in the system respectively, if any value exceeds the overload threshold, it is determined that the host is in an overload state, and is marked as an overload host, wherein, the overload state indicates that the host currently or in the future faces the risk of fierce resource competition and performance decline, the virtual machine migration process needs to be triggered immediately to reduce the load pressure, avoid service level agreement violation, and ensure the overall performance and stability of the system; if the predicted resource utilization of a physical host is lower than the preset low-load threshold, it is further verified whether all utilization data in the historical load time series data are continuously lower than the low-load threshold, and only when the host is in a low load level for a long time in the historical time window, the host is finally determined as a low-load host, the verification mechanism avoids misjudgment caused by short-term fluctuations (such as night valley being mistaken for continuous low load), ensures that the low-load host determination conforms to the actual resource demand, provides basis for subsequent virtual machine integration and idle host shutdown, and further effectively reduces the energy consumption of the data center;

[0090] S3, based on the Pearson correlation coefficient between the virtual machine and the physical host and the migration time, a migration influence coefficient is constructed, the value of the virtual machine migration is comprehensively evaluated, based on the historical resource utilization data of the virtual machine and the physical host, the Pearson correlation coefficient between the two is calculated, the correlation degree between the virtual machine and the host is described, based on the memory occupancy of the virtual machine and the available network bandwidth of the host, the time cost of the migration of the virtual machine from the host is modeled, the migration time is analyzed, and the time cost required to complete the migration is directly reflected, the larger the memory or the lower the bandwidth, the greater the cost, the Pearson correlation coefficient and the migration time are integrated to construct the migration influence coefficient of the virtual machine, and the migration value of the virtual machine is comprehensively evaluated;

[0091] The specific work content is: based on the historical resource utilization data of the virtual machine and the physical host, the linear correlation degree between the two is quantified by calculating the Pearson correlation coefficient, wherein, specifically, the CPU utilization time series of the virtual machine and the corresponding host in the same time window is collected, the sequence is standardized to eliminate the dimensional difference, and then the Pearson formula is used to calculate the Pearson correlation coefficient, the load coupling characteristics between the virtual machine and the host are identified by describing the resource correlation degree, the high correlation virtual machine may have a significant impact on the host load after migration, while the low correlation virtual machine has less disturbance to the host load, thereby assisting in judging the potential benefits and risks of migration; according to the memory occupancy of the virtual machine and the available network bandwidth of the target host, the migration time cost is modeled, the migration time is defined as the ratio of the memory occupancy of the virtual machine to the available network bandwidth of the host, which directly reflects the time cost required to complete one migration operation, the larger the memory occupancy of the virtual machine, the more data needs to be transmitted; the lower the available network bandwidth of the host, the slower the data transmission rate, both of which will significantly increase the migration time, as a key performance indicator, the migration time directly affects the continuity of the service and the system response capability, and longer migration time may cause performance degradation or service interruption; by integrating the Pearson correlation coefficient and the migration time cost, the migration influence coefficient is constructed, and the migration value of the virtual machine is comprehensively quantitatively evaluated, wherein, the square of the Pearson correlation coefficient is combined with the inverse of the migration time, which is mathematically represented as the inverse relationship between the correlation square and the migration time cost, a high correlation square value means that migrating the virtual machine can significantly alleviate the host resource contention, while a low migration time represents a low cost of migration operation, effectively balancing the benefits and costs of migration operation, and the virtual machine with high contribution to improving load distribution and low execution cost is preferentially selected for migration, thereby improving resource integration efficiency while ensuring service quality;

[0092] In addition, the Pearson correlation coefficient is a core indicator for measuring the linear correlation between two variables, which can accurately describe the correlation degree between the virtual machine and the host in the virtual machine selection scenario, and the formula is as follows:

[0093] ;

[0094] ;

[0095] ;

[0096] wherein, represents the Pearson correlation coefficient, represents a virtual machine, represents a physical host, represents a historical time window length, represents the CPU utilization of the virtual machine at time , represents the CPU utilization of the physical host at time , is the mean of the virtual machine CPU utilization time series, is the mean of the physical host CPU utilization time series;

[0097] The time overhead of the virtual machine migrating from the physical host, i.e. the formula of the migration time, is:

[0098] ;

[0099] wherein, is the memory occupancy of the virtual machine, is the available network bandwidth of the physical host;

[0100] The migration impact coefficient of the virtual machine is obtained based on the Pearson correlation coefficient and the migration time, and the formula is as follows:

[0101] ;

[0102] wherein, is the migration impact coefficient, the square operation strengthens the impact of the high correlation virtual machine, and the denominator item punishes the migration of the large memory virtual machine;

[0103] S4, select the virtual machines to be migrated according to the migration influence coefficient from high to low, and generate a list of virtual machines to be migrated, wherein, for the overloaded host, a greedy algorithm is applied to select the virtual machines to be migrated, and for the low-load host, all virtual machines are marked, for all virtual machines to be evaluated, the migration influence coefficient is calculated according to the square of the Pearson correlation coefficient of the virtual machine and the physical host and the migration time, and the virtual machines are sorted according to the value of the migration influence coefficient from high to low to generate a priority list of the virtual machines to be migrated, for the overloaded host, a greedy strategy is used to select the virtual machine with the highest migration influence coefficient from the priority list of the virtual machines to be migrated for migration, and the process is repeated until the load of the host is reduced to below the safety threshold or there is no suitable virtual machine to be migrated, and the load state of the host needs to be re-evaluated after each migration, for the low-load host, all virtual machines running thereon are marked as migratable objects and are directly included in the list of virtual machines to be migrated without priority sorting, and then the virtual machines selected from the overloaded host and all virtual machines from the low-load host are combined to form a final list of virtual machines to be migrated;

[0104] Specific work content: for all virtual machines to be evaluated, the Pearson correlation coefficient is calculated according to the historical resource utilization data of the virtual machine and the physical host, the linear correlation degree of the virtual machine and the host in load fluctuation is quantified, the square value is used to strengthen the importance of high correlation virtual machine, and the directionality of correlation is ignored, and the migration time cost is calculated according to the memory occupancy of the virtual machine and the available network bandwidth of the host, which directly reflects the data transmission cost required by the migration operation, and then the migration influence coefficient is constructed by combining the square of the Pearson correlation coefficient and the inverse of the migration time, which comprehensively measures the benefit and cost of the migration operation, and all virtual machines are sorted according to the migration influence coefficient from high to low to generate a priority list of virtual machines to be migrated; for the host in the overload state, a greedy strategy is used to select the virtual machine to be migrated from the priority list of the virtual machines to be migrated, in each iteration, the virtual machine with the highest migration influence coefficient in the current priority list of the virtual machines to be migrated is selected for migration, after each migration, the resource utilization state of the host is updated in real time, and whether the host is still in the overload state is re-evaluated, and the process is repeated until the load level of the host is reduced to below the preset safety threshold or there is no more suitable virtual machine to be migrated, which aims to quickly and effectively relieve the overload pressure of the host through the least number of migration operations, avoid performance bottleneck and service quality degradation; for the low-load host, all virtual machines running thereon are marked as migratable objects and are directly included in the list of virtual machines to be migrated without priority sorting, and then the virtual machines from the overloaded host are combined with all virtual machines from the low-load host to form a complete list of virtual machines to be migrated;

[0105] S5, training the enhanced double Q network using the experience replay mechanism, and updating the target network parameters regularly to optimize the decision strategy;

[0106] S6, selecting the optimal target physical host to execute the virtual machine placement by enhancing the double Q network and combining the confidence upper bound strategy;

[0107] S7, after executing the virtual machine migration, shutting down the idle physical host with the continuously low resource utilization to save energy.

[0108] Embodiment 2, as shown in the embodiment 1, the application provides a technical solution: preferably, the S5 specifically includes: Figure 1 、 Figure 2 The S5 specifically includes:

[0109] The state transition samples including the current environment state, the executed action, the instant reward and the next state are collected through the virtual machine migration interaction process, the data is stored in the experience replay buffer according to the time sequence, the sample independence is ensured and the overfitting is avoided, a batch of samples is randomly sampled from the experience replay buffer, the online Q network parameter gradient is calculated based on the improved loss function (the square of the time difference error), the update step is dynamically adjusted by combining the adaptive learning rate, the network weight is optimized through the back propagation to approximate the optimal strategy, the online Q network parameters are copied to the target Q network at a fixed interval, the stability of the target Q value calculation benchmark is maintained, the estimation oscillation caused by the parameter synchronization delay in the traditional Q learning is avoided, the convergence and the robustness of the reinforcement learning decision are improved, and the enhanced double Q network is formed;

[0110] The specific work content is: the four-tuple data (current state, execution action, immediate reward, next state) is continuously collected during the virtual machine migration process to form a training sample, wherein the current state includes continuous variables such as physical host CPU utilization, memory remaining amount and virtual machine resource demand; the execution action refers to the specific migration target host selection; the immediate reward is the weighted calculation of three targets including energy consumption reduction, service level agreement violation rate (SLAV) and migration overhead; the next state reflects the change of resource distribution after migration, all samples are stored in an experience replay buffer in chronological order, the time sequence correlation is broken through random sampling to avoid overfitting problem caused by sample continuity of reinforcement learning, the experience replay buffer adopts a circular queue structure, and a capacity threshold is set to balance the storage efficiency and sample diversity, and to ensure that the training data covers decision cases under different load scenarios; the online Q network parameter gradient is calculated based on the improved loss function (time difference error square), and the loss function is defined as the mean square error of the current Q value and the target Q value, wherein the target Q value is calculated by the estimation of the next state through the target network, and an adaptive learning rate mechanism is introduced in the training process, and the update step is dynamically adjusted according to the access frequency of the state-action pair: a smaller learning rate is used for high-frequency access pairs to ensure convergence stability, and a larger learning rate is used for low-frequency access pairs to accelerate strategy exploration; during back propagation, the gradient calculation is combined with the momentum term of the RMSProp optimizer to further suppress training shock; the parameter update efficiency is improved through batch sample training, and the network weight is updated synchronously after each batch of samples are processed, so that the Q value estimation gradually approaches the real action value under the optimal strategy; the parameters of the online Q network are copied to the target Q network at a fixed interval to keep the estimation benchmark of the target Q network stable, and the parameters of the online Q network are copied to the target network without loss every N time steps to avoid the overestimation problem of the estimation value caused by the parameter synchronization delay in traditional Q learning; in the double-network structure, the online network is responsible for action selection and real-time strategy update, and the target network provides a stable Q value calculation benchmark, and the decoupled design of the two networks effectively suppresses the strategy shock in the training process, and the synchronization interval is adaptively adjusted according to the environmental dynamics: the synchronization period is shortened to quickly respond to changes when the load fluctuates sharply, and the interval is lengthened to reduce the calculation overhead in the stable operation stage, thereby forming an enhanced double Q network framework with the online Q network, the target Q network, the experience replay and the adaptive learning rate, and improving the convergence speed and decision robustness of reinforcement learning in complex cloud environments.

[0111] Suppose the data center contains N physical servers , each physical server runs virtual machines . Virtual machine integration is to reasonably allocate a large number of virtual machines to physical servers, and the core goal is to optimize resource utilization and reduce total energy consumption under the premise of meeting service quality (QoS) and resource constraints.

[0112] Energy Consumption (EC) : The energy consumption of a physical server is represented by a linear model:

[0113] ;

[0114] wherein, EC represents the total energy consumption of a server, ECbase represents the base energy consumption in idle state, ECmax represents the theoretical maximum energy consumption, i.e. the maximum energy consumption when fully loaded, CPU represents the CPU utilization, and the total energy consumption of the data center is the sum of the energy consumption of all active servers:

[0115] ;

[0116] wherein, A represents the active state of a server,

[0117] Service Level Agreement Violation (SLAV) : SLAV, as a key indicator to measure whether the integration of virtual machines will affect the quality of service, refers to the number or proportion of times that the service level agreement (SLA) fails to meet the service quality specified in the agreement. Generally, SLAV depends on two measurement standards, one is the SLA conflict time (SLATAH) of each active host, and the other is the performance degradation caused by migration (PDM). SLAV can be represented as:

[0118] ;

[0119] wherein, SLATAH represents the SLA conflict time of each active host, i.e. the proportion of the violation time in the total running time of the service agreement of each active host, which is represented as:

[0120] ;

[0121] wherein, N represents the total number of all physical hosts, for any host , the total time length of violating the service level agreement due to overload is recorded as , and the total time length of the host in active state is ;

[0122] PDM is used to represent the proportion of performance degradation caused by virtual machine migration in the total performance demand, which is represented as:

[0123] ;

[0124] ​in, Represents the number of virtual machines. Indicates the resource type for any virtual machine. The performance degradation caused by migration is denoted as The total resource requirements of this virtual machine over its lifecycle are denoted as... ;

[0125] The enhanced dual-Q network comprises two Q-networks with identical structures but different parameter update methods: the online Q-network QNet (… ) and TargetQNet ( ), where the online Q network is used to determine the current environmental status. Select the action to perform. The target Q-network is used to calculate the target Q-value to avoid overestimation problems that may occur in traditional Q-learning, and to make the Q-value estimation more accurate.

[0126] Representation of the current environment state: Current environment state This includes CPU utilization information for each physical host and CPU demand information for virtual machines, defined as:

[0127] ;

[0128] in, Indicates the first CPU utilization of the physical host This indicates the CPU request information for the virtual machine to be placed.

[0129] Action selection: Action to be performed This indicates that the selected virtual machine will be placed on a suitable physical host. To balance exploration and exploitation, a confidence upper bound strategy is used to improve action selection for a given environment state. Select the optimal action to execute based on the following formula:

[0130] ;

[0131] in, In the current environmental state Below is a set of all selectable actions. The optimal action to take at present. The upper bound for confidence is defined as follows:

[0132] ;

[0133] in, Is the online Q network in status? Next action The value estimate, is the exploration coefficient, is the state is the total number of times visited, is the number of times the action is selected in state ; in this way, in the initial stage, more different VM placement actions can be explored, and with the accumulation of experience, the focus is gradually focused on the use of the better placement strategy found;

[0134] Instantaneous reward: based on the multi-objective optimization problem in the system model, the reward function is expressed as a function of energy consumption, SLA violation rate and migration times, defined as follows:

[0135] ;

[0136] wherein, is the weight coefficient, satisfying ; is the total energy consumption, and are the theoretical maximum energy consumption and minimum energy consumption; is the service level agreement violation rate (Service Level Agreement Violation), and are the theoretical maximum and minimum ; represents the number of migrations, and are the theoretical maximum migration number and minimum migration number;

[0137] Loss function: the online Q network uses the mean square error loss function to calculate the parameter gradient, and the formula is defined as follows:

[0138] ;

[0139] wherein, is the value estimate of the online Q network for the state performing action ; R is the reward obtained; is the discount factor; is the value estimate of the target Q network for the next state action ; is the optimal action selected by the target Q network for the next state;

[0140] Adaptive learning rate: the training of the online Q network uses an adaptive learning rate, defined as follows:

[0141] ;

[0142] wherein, is the learning rate, is the number of times the action is selected in state ;

[0143] When a certain state-action pair is rarely visited, ≈0, at this time the learning rate ≈1; a larger step size allows the online Q network parameters to be updated quickly, accelerating the learning of the value estimates of new decisions. As the same state-action pair is repeatedly visited, increases, the learning rate tends to 0 gradually, and a smaller step size avoids parameter oscillation due to excessive updates, allowing the Q value estimates to converge to stable values and ensuring accurate use of known high-quality strategies.

[0144] S6 specifically comprises:

[0145] Based on the enhanced double Q network comprising the online Q network and the target Q network, the input state comprises the physical host resource utilization, the virtual machine demand and the historical migration record. Through the experience replay mechanism, the network parameters are updated using the temporal difference error, and the deterministic value and the exploratory value of the state-action are learned respectively, the overestimation bias is reduced, and the stability of the action selection is improved. In the action selection stage, the confidence upper bound (UCB) value of each candidate host is calculated based on the output of the enhanced double Q network, the host with a high confidence upper bound (UCB) value is preferentially selected, the exploration of the high-value action and the unknown environment is considered, the network parameters are soft-updated to the target Q network after completing N migrations, the training process is stabilized, and based on the current state, the host with the highest confidence upper bound (UCB) value is selected by the greedy strategy to perform placement, and the global optimization of the resource utilization, the migration cost and the QoS (Quality of Service) constraint is performed.

[0146] The specific work content is: according to the constructed enhanced double Q network architecture, which is composed of two neural networks with the same structure but complementary functions, namely online Q network and target Q network, wherein the online Q network generates action value estimation according to the current environment state, and the target Q network is used to calculate stable target Q value to alleviate the overestimation problem in traditional Q learning, the input state is designed as a multi-dimensional vector, covering the real-time resource utilization of the physical host, the resource demand of the virtual machine to be migrated and the historical migration record, ensuring that the model perceives the influence of dynamic environment and historical behavior, stores historical migration experience through experience replay mechanism, updates network parameters using time difference error, the online Q network selects actions based on the current state, the target Q network calculates target value, the double network cooperates to learn deterministic value and exploratory value, effectively suppresses the overestimation bias in traditional Q learning, and improves the accuracy of value estimation; in the action selection stage, the output of the enhanced double Q network is used to calculate the confidence upper bound (UCB) value of each candidate physical host, and the basic estimation of the host value by the online Q network and an exploration reward item related to the access times are fused, so that the known high-value action and the potential optimal action that has not been fully explored are considered in the decision-making process, the physical host with the highest UCB value is selected as the migration target through priority selection, and an effective balance is achieved between "utilizing" existing knowledge and "exploring" unknown environment, avoiding falling into local optimal solution, adaptively adjusting the exploration intensity in the dynamic cloud environment, and improving the generalization ability and robustness of the strategy; periodic soft update mechanism is adopted to synchronize network parameters, after completing N times of migration, the parameters of the online Q network are soft updated to the target Q network with weight, the historical knowledge is retained while gradually absorbing new strategy, avoiding training shock caused by sudden change of target value, in the placement decision-making stage, the host with the highest UCB value is selected to perform migration through the greedy strategy based on the current state, and global optimization constraints are introduced, the resource utilization threshold of the physical host, the migration cost limit and the QoS constraint after migration are met, the actionable actions are selected through the constraint conditions, ensuring that the decision-making considers efficiency and safety, and finally realizing global resource optimization and service quality guarantee of virtual machine migration;

[0147] S7 specifically includes:

[0148] According to the target host determined by the enhanced double Q network, the virtual machine to be migrated is migrated from the overloaded or low-load physical host to the selected host, the resource allocation state of the physical host and the virtual machine is updated, it is ensured that the migration process meets the service quality constraint, the resource utilization of each physical host after migration is monitored in real time, the physical host which is continuously in a low-load state and has no pending task is selected and marked as a candidate set of idle host which can be turned off, all idle physical hosts in the candidate set of idle host are turned off through the power management interface, the data center topology structure is updated, it is ensured that the remaining active host meets the current load demand, and finally the overall energy consumption is reduced;

[0149] The specific work content is: according to the target host determined by the enhanced double Q network, the virtual machine to be migrated is migrated from the overloaded or low-load physical host to the selected target host, and the quality of service constraints and resource constraints need to be followed during the migration process, including network bandwidth guarantee, performance jitter upper limit and service continuity requirements, to ensure that the user business is not significantly affected, after the migration is completed, the resource allocation state of the physical host and the virtual machine is updated in real time, the resource pool metadata is synchronized and adjusted, and the affiliation and resource occupation of the virtual machine and the host are accurately reflected; the resource utilization of each physical host after migration is continuously monitored, the load trend is analyzed through a sliding time window, if the CPU and memory utilization of a host continuously below the preset low load threshold, and there is no task or key service to be processed on it, the physical host that meets the conditions is marked as a "shut down idle host candidate set", and its network topology position, power supply state and associated storage device information are recorded, at the same time, the power management interface availability of the candidate set host is verified through the data center management system; the power management interface is used to send a shutdown instruction to the idle host candidate set, and the power down operation is performed on the physical host in the idle host candidate set, at the same time, the data center resource topology structure is updated, the closed host is removed from the active host list, and the resource capacity and load distribution are recalculated, to ensure that the total amount of resources of the remaining active host can still meet the current and near future predicted load demand, avoid secondary overload caused by resource tightening, finally, through reducing the idle energy consumption, the overall energy consumption of the data center is significantly reduced, while maintaining the flexibility and availability of the resource pool.

[0150] The above is only a specific embodiment of the present application, but the protection scope of the present application is not limited thereto, any person skilled in the art can easily think of changes or replacements within the technical range disclosed in the present application, which should be covered within the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A cloud data center virtual machine adaptive integration method based on Autoformer and enhanced double Q network, characterized in that, The method comprises the following steps: S1, real-time monitoring of each physical host resource utilization, collecting historical load time series data, and loading a pre-trained Autoformer model, initializing double Q network parameters and environment state variables; S2, predicting host resource utilization based on the Autoformer model, and dividing overloaded hosts, low-load hosts and normal hosts according to the current host resource utilization; S3, based on the Pearson correlation coefficient between virtual machines and physical hosts and the migration time, a migration influence coefficient is constructed to comprehensively evaluate the virtual machine migration value; S4, select the virtual machine to be migrated according to the migration influence coefficient from high to low, and the list of virtual machines to be migrated, wherein for the overloaded host, the greedy algorithm is used to select the migration virtual machine, and all virtual machines are marked for the low-load host; S5, training and enhancing the double Q network by using the experience replay mechanism, and regularly updating the target network parameters to optimize the decision strategy; S6, selecting the optimal target physical host to execute virtual machine placement by combining the confidence upper bound strategy through the enhanced double Q network; S7, after executing the virtual machine migration, the idle physical host with low and continuous resource utilization is closed.

2. The cloud data center virtual machine adaptive integration method based on Autoformer and enhanced double Q network according to claim 1, characterized in that: The S1 specifically comprises: The resource utilization data of all physical hosts including CPU and memory is monitored in real time through the data center monitoring system, and the historical load time series data is stored according to the set time window; The pre-trained Autoformer model is loaded, and the parameters of the double Q network including the online Q network and the target Q network are initialized, and the environment state variables are defined; According to the scale configuration of the data center including the time window size, the exploration coefficient and the discount factor, the experience replay buffer capacity and the training interval parameters are initialized.

3. The cloud data center virtual machine adaptive integration method based on Autoformer and enhanced double Q network according to claim 1, characterized in that: The S2 specifically comprises: The current resource utilization of each physical host is obtained in real time, and the continuous resource utilization sequence is extracted according to the historical time window, and the data of the resource utilization sequence is input into the pre-trained Autoformer model, and the resource utilization of the host is decomposed and predicted through the autocorrelation mechanism; The current resource utilization of each physical host is compared with the predicted resource utilization, and the preset overload threshold and low-load threshold are compared, respectively, if any resource utilization exceeds the overload threshold, the host is determined as an overload host; If the predicted resource utilization is lower than the low-load threshold, further check whether all resource utilization data in the historical load time series data are continuously lower than the low-load threshold, only when the long-term load level is low, the host is finally determined as a low-load host.

4. The cloud data center virtual machine adaptive integration method based on Autoformer and enhanced double Q network according to claim 1, characterized in that: The S3 specifically comprises: Based on the historical resource utilization data of virtual machines and physical hosts, the Pearson correlation coefficient between them is calculated to describe the resource utilization correlation degree of virtual machines and hosts; Based on the virtual machine memory occupation and the available network bandwidth of the host, the time cost of virtual machine migration from the host is modeled; The Pearson correlation coefficient and the migration time are integrated to construct the migration influence coefficient of the virtual machine, and the migration value of the virtual machine is comprehensively evaluated.

5. The cloud data center virtual machine adaptive integration method based on Autoformer and enhanced double Q network according to claim 4, characterized in that: The formula of the Pearson correlation coefficient is as follows: ； ； ； wherein, denotes the Pearson correlation coefficient, denotes a virtual machine, denotes a physical host, denotes a historical time window length, denotes the CPU utilization of a virtual machine at time denotes the CPU utilization of a physical host at time is the mean of the virtual machine CPU utilization time series, is the mean of the physical host CPU utilization time series;​​ The time cost modeling of the virtual machine migration from the physical host, that is, the formula of the migration time is as follows: ； wherein, is the virtual machine memory occupancy, is the physical host available network bandwidth; The migration influence coefficient of the virtual machine is obtained based on the Pearson correlation coefficient and the migration time, and a formula is as follows: ； wherein is the migration influence coefficient.

6. The cloud data center virtual machine adaptive integration method based on Autoformer and enhanced double Q network according to claim 1, characterized in that: The S4 specifically comprises: For all the virtual machines to be evaluated, the migration influence coefficient is calculated according to the square of the Pearson correlation coefficient of the virtual machine and the physical host and the migration time, and the virtual machines to be migrated are ranked in descending order of the migration influence coefficient, and a priority list of the virtual machines to be migrated is generated; For the overloaded host, a greedy strategy is adopted to select the virtual machine with the highest migration influence coefficient from the priority list of the virtual machines to be migrated, and the migration is performed until the load of the host is reduced to below the safety threshold or no suitable virtual machine is available, and the load of the host is re-evaluated after each migration; For the low-load host, all the virtual machines of the low-load host are directly marked as migratable objects, and are directly included in the list of the virtual machines to be migrated without priority ranking, and then the virtual machines selected from the overloaded host and all the virtual machines of the low-load host are combined to form a final list of the virtual machines to be migrated.

7. The cloud data center virtual machine adaptive integration method based on Autoformer and enhanced double Q network according to claim 1, characterized in that: The S5 specifically comprises: State transition samples are collected through the virtual machine migration interaction process, including the current environment state, the executed action, the immediate reward and the next state, and the data is stored in the experience replay buffer in time sequence; A batch of samples is randomly sampled from the experience replay buffer, the parameter gradient of the online Q network is calculated based on the improved loss function, the update step is dynamically adjusted by combining the adaptive learning rate, the network weight is optimized by back propagation to approximate the optimal policy; The online Q network parameters are copied to the target Q network at a fixed interval to keep the target Q value calculation reference stable, and the enhanced double Q network is formed.

8. The cloud data center virtual machine adaptive integration method based on Autoformer and enhanced double Q network according to claim 7, characterized in that: The enhanced double Q network comprises two Q networks with the same structure but different parameter updating methods, namely an online Q network and a target Q network, wherein the online Q network is used for calculating a current Q value according to a current environment state Selecting to perform an action , and the target Q network is used for calculating a target Q value; Representation of current environment state: Current environment state The CPU utilization information of each physical host and the CPU demand information of the virtual machine are defined as: ； wherein, represents the first CPU utilization of the physical host of the server, represents the CPU request information of the virtual machine to be placed; Selection of action to perform: perform action represents the virtual machine to be selected and placed on a suitable physical host, the action selection is improved by using the upper confidence bound strategy, for a certain environment state , the optimal execution action is selected according to the following formula: ； wherein, is the set of all possible execution actions, under the current environmental state is the current optimal execution action, is the confidence upper bound, defined as follows: ； wherein, is the value estimate of the action by the online Q-network at state , is the exploration coefficient, is the state visited, is the total number of times is the number of times an action is selected at state ; Immediate reward: based on the multi-objective optimization problem in the system model, the reward function is expressed as a function of energy consumption, SLA violation rate and migration times, and is defined as follows: ； wherein, is a weight coefficient, satisfying is the total energy consumption, and are the theoretical maximum energy consumption and minimum energy consumption; is the service level agreement violation rate, and are the theoretical maximum and minimum migration times, respectively; represents the number of migrations, and are the theoretical maximum migration times and minimum migration times, respectively;​ Loss function: the online Q network uses the mean square error loss function to calculate the parameter gradient, and the formula is defined as follows: ； wherein, is the value estimate of the state by the online Q network ; R is the reward obtained; is a discount factor; is the value estimate of the next state by the target Q network ; a is the optimal action selected by the target Q network for the next state ; Adaptive learning rate: the training of the online Q network uses the adaptive learning rate, and is defined as follows: ； wherein, is a learning rate, is the number of times an action is selected in a state .

9. The cloud data center virtual machine adaptive integration method based on Autoformer and enhanced double Q network according to claim 8, characterized in that: The S6 specifically comprises: Based on the enhanced double Q network comprising the online Q network and the target Q network, the input state comprises the resource utilization of the physical host, the virtual machine demand and the historical migration record, the network parameters are updated by using the time difference error through the experience replay mechanism, and the deterministic value and the exploratory value of the state-action are learned respectively; In the action selection stage, the confidence upper bound value of each candidate host is calculated based on the output of the enhanced double Q network, the host with a high confidence upper bound value is selected, and the high value action and the exploration of the unknown environment are considered; After completing N migrations, the network parameters are soft-updated to the target Q network to stabilize the training process, the host with the highest confidence upper bound value is selected by the greedy strategy based on the current state to perform placement, and the global optimization of the resource utilization, the migration cost and the QoS constraint is performed.

10. The cloud data center virtual machine adaptive integration method based on Autoformer and enhanced double Q network according to claim 9, characterized in that: The S7 specifically comprises: According to the target host determined by the enhanced double Q network, the virtual machine to be migrated is migrated from the overloaded or low-load physical host to the selected host, and the resource allocation state of the physical host and the virtual machine is updated; The resource utilization of each physical host after migration is monitored in real time, the physical host which is in a low-load state and has no task to be processed is selected, and the physical host is marked as a candidate set of idle host to be closed; All idle physical hosts in the idle host candidate set are closed through the power management interface, and the data center topology is updated to ensure that the remaining active hosts meet the current load demand.

Images