A method and system for cross-region migration of computing tasks for cloud data centers

Abstract

A method and system for cross-region migration of computing tasks for cloud data centers. The cross-region migration process of computing tasks includes three steps: migration environment perception, migration decision, and migration synchronization. The migration environment perception step dynamically monitors the status of the cloud data center and obtains information such as the cloud data center load and power consumption in real time; the migration decision step uses an adaptive dynamic threshold method based on energy consumption level to determine the migration triggering time, comprehensively considers optimization goals such as low energy consumption and load balancing, and gives the optimal migration-out task set and migration-in destination host set; the migration synchronization step realizes the consistency of data and status before and after migration, as well as dynamic adaptation of network resource environment. The present invention can reduce the energy consumption of cloud data center systems, while meeting the requirements of system load balancing and application performance. The present invention helps to promote the development of cloud data center energy consumption management technology, and forms a reliable dynamic cross-region migration scheme for jobs in terms of cloud data center energy consumption optimization and job migration.

Description

Technical Field

[0001] The present invention belongs to the technical field of cloud computing, and in particular relates to a method and system for cross-region migration of computing tasks for a cloud data center. Background Art

[0002] In recent years, cloud computing technology has received more and more attention and application, the scale of cloud services has continued to expand, and the number and scale of cloud data centers have also increased significantly to support a large amount of data storage and user computing requests. With the continuous development of technology, information transmission and processing in cloud data centers will consume more electricity. How to use electricity more energy-efficiently and greenly is one of the most concerned issues for data center companies and even the whole society. According to statistics, the energy consumption of servers in an idle state is 50%-70% of the energy consumption of servers in a fully loaded state. In other words, idle static servers do not contribute to computing tasks, but consume a large part of the energy consumption. Therefore, some data center energy management solutions will monitor the utilization of servers to trigger virtual machine and container migration to integrate underutilized servers. In addition to energy consumption optimization scenarios, virtual machine and container migration technology will also be used in load balancing, server upgrades, and machine downtime maintenance scenarios. For example, virtual machines and containers on overloaded nodes are migrated to prevent the situation where node resource shortages cause application performance degradation on nodes, so as to achieve the goal of improving cluster load balancing and cluster resource utilization.

[0003] Selecting appropriate virtual machines and containers to transfer under target constraints and finding the most suitable migration target node is the key issue of computing task migration. At present, the migration scheduling process of virtual machines and containers is generally divided into three aspects: determining the migration timing (When), selecting the migrated virtual machines and containers (Which), and selecting the migration target physical host node (Where).

[0004] At present, the migration timing is mainly determined by setting the migration trigger threshold. When the host load exceeds the overload threshold, overload migration is triggered, and when the host load is lower than the underload threshold, underload migration is triggered. The migration trigger threshold is divided into static threshold and dynamic threshold. The static threshold directly sets the overload threshold and underload threshold manually, which is simple to implement, but the threshold will not change during the entire migration process, which is not flexible enough; the dynamic threshold will change dynamically according to time, resource conditions, and cluster load conditions, and has better adaptability and flexibility. However, in most current studies, the threshold dynamic change rules are only formulated based on resource and load conditions, and the impact of cluster energy consumption levels on thresholds is not considered. Formulating dynamic thresholds based on cluster energy consumption levels can speed up the response speed of system energy consumption optimization and improve the system energy consumption optimization effect. Therefore, it is meaningful to consider the impact of cluster energy consumption levels on thresholds.

[0005] Regarding the selection of migrated virtual machines and containers, there are mainly maximum relevance selection, minimum migration cost selection, minimum performance loss selection and other selection methods. These methods take into account one or more migration indicators such as resource utilization, load balancing, migration time, migration data volume, application performance, service quality, energy consumption, etc. However, there is a lack of methods that meet comprehensive requirements, and there is a lack of response measures when there are requirements for a specific reduction in host energy consumption.

[0006] At present, there are mainly heuristic algorithms, meta-heuristic algorithms and machine learning algorithms to determine the target physical host. Heuristic algorithms have the advantages of simple implementation and fast convergence speed, but they are easy to fall into local minimum and cannot achieve the best overall placement effect. Relatively speaking, meta-heuristic algorithms can find the global optimal solution well, but the convergence speed is slow, the algorithm parameters are too many, the reusability of the calculation results is poor, and the parameters cannot be quickly and effectively tuned. Machine learning algorithms have the characteristics of self-learning and self-adaptation, and have good global search capabilities, but poor comprehensibility and high dependence on training data. Most of the related research on determining the target physical host only considers finding the target host within the data center, and lacks consideration of cross-data center migration. When the energy consumption level of the data center is high and the overall resource utilization is saturated, it is necessary to find the target host across data centers. Summary of the invention

[0007] The problem to be solved by the present invention is: compared with the static threshold, the dynamic threshold has better adaptability and flexibility. How to formulate the dynamic change rules of the threshold according to the energy consumption level of the cloud data center? How to make a comprehensive and integrated consideration of multiple migration indicators such as energy consumption, resource utilization, load balancing, and service quality? When there is a requirement for a specific reduction in the host energy consumption, how to configure the response measures? When the energy consumption level of the cloud data center is high and the overall resource utilization is relatively saturated, how to find the target host across data centers to perform cross-region migration of computing tasks? These problems still need to be solved.

[0008] The technical solution of the present invention is: a method for cross-region migration of computing tasks for cloud data centers, taking into account the data management requirements and dynamic load balancing requirements of cloud data centers, and aiming to reduce system energy consumption, to perform cross-region migration of computing tasks, including the following steps:

[0009] Step 1: Migrate environment awareness: Dynamically monitor the status of the cloud data center and obtain the characteristic attributes and operation information of the cloud data center load, power consumption, and computing tasks in real time;

[0010] Step 2: Migration decision: Obtain the node information of the cloud data center through the migration environment perception step, determine the migration triggering time based on the adaptive dynamic threshold method of energy consumption level, comprehensively consider the optimization goals of low energy consumption, load balancing and QoS, determine the migration decision, and give the optimal migration task set and migration destination host set, including:

[0011] Step B1, determine the migration timing: set the migration trigger thresholds for underload and overload, and perform migration when the physical host load reaches the migration trigger threshold;

[0012] Step B2, providing the optimal migration task set: when underload migration is triggered, all computing tasks on the underloaded host are migrated; when overload migration is triggered, the migrated computing tasks are arranged according to the load correlation between the virtual machine and the host, the container and the host, and the comprehensive load of the computing tasks. If the migration requires a specific reduction in the host's energy consumption, then under the energy consumption reduction requirement, with the goal of minimizing the migration cost of the container and the virtual machine, the optimal migration task set is solved for all computing tasks on the overloaded host;

[0013] Step B3, providing the optimal migration destination host set: for hosts that meet the task migration conditions, comprehensively consider host energy consumption, load balancing, resource utilization, and user service quality, and use a heuristic multi-objective optimization method to select the optimal migration host set;

[0014] Step 3: Migration synchronization: Use incremental synchronization to synchronize data before and after virtual machine and container migration, and use task binding virtual IP addresses to enable applications to dynamically adapt to the network environment of the migration target host.

[0015] The migration environment perception step of the present invention is designed to dynamically monitor the status of server hosts, virtual machines, containers and computing tasks, and obtain the data center load, power consumption and characteristic attributes and operation information of computing tasks in real time. First, the load information of CPU, memory, IO and network is obtained respectively, and the node resource utilization is calculated using the load information. Then, intelligent learning methods such as decision trees, random forests, support vector machines SVM, association rules, deep neural networks, etc. are combined with multivariate polynomial regression methods to perform multi-level and multi-dimensional energy consumption quantitative analysis of physical hosts, virtual machines, containers, etc., and a refined model of power consumption measurement of virtual machines, containers and physical nodes is established. The power consumption of virtual machines, containers and physical nodes is obtained through the model and load data, and the energy consumption of the cloud data center is calculated.

[0016] The migration decision step uses an adaptive dynamic threshold method based on energy consumption level to determine the migration triggering time, comprehensively considers optimization goals such as low energy consumption and load balancing, and gives the optimal migration task set and migration destination host set. Define the host's low load threshold and overload threshold, and further dynamically and automatically adjust the migration trigger threshold according to different energy consumption levels. Monitor the host load through the migration environment perception step, and trigger low load migration or overload migration based on the host's load data over a period of time in combination with the low load threshold and overload threshold. When low load migration is triggered, all computing tasks on the low load host are migrated; when overload migration is triggered, computing tasks with higher load correlation with the physical node can be selected for migration. When the specific reduction in host energy consumption is required, the dynamic programming method can be used to give the set of minimum container and virtual machine migration costs that meet the energy consumption reduction requirements. The migration cost is a positively correlated function of the amount of migrated data. The selection of the migration destination host set comprehensively considers host energy consumption, load balancing, resource utilization, user service quality, etc., and is given by a heuristic multi-objective optimization method.

[0017] In migration synchronization, the main purpose of incremental synchronization migration is to minimize downtime. After the migration starts, the virtual machines and containers of the source host are not terminated immediately, but continue to run. The memory data of the source node is sent to the target node in the transmission iteration. In the first iteration, all memory page data are sent. In the subsequent iterations, only the modified memory page data is sent until the memory page modification rate drops below the threshold or reaches the upper limit of the number of iterations. The iteration process ends, the source node virtual machine and container are shut down, and the last round of dirty page data is transmitted. Then, the virtual machine and container of the destination node are restarted and started, and the entire incremental synchronization process is completed. In stateful migration behavior, it is necessary to maintain the running data and related configuration environment in the memory during the migration process to ensure that the service quality after the computing task migration will not be reduced. The most important point is the network environment. The present invention adopts the method of binding the task to the virtual IP address, and through the network address conversion, it is ensured that the task can dynamically adapt to the resource environment and network environment of the migration target host, thereby shielding the heterogeneity and differences of network resources in different data centers, so that the migrated virtual machines and containers can seamlessly adapt to the network in the new host environment.

[0018] Based on the above method, the present invention also provides a cloud data center system, including multiple data centers, and the cloud data center system is configured with a computer program, which implements the above-mentioned cross-region migration method of computing tasks when executed.

[0019] The present invention has the following beneficial effects:

[0020] The present invention studies and designs a cross-region migration strategy for computing tasks in cloud data centers. An adaptive dynamic threshold method based on energy consumption level is proposed, which can dynamically and automatically adjust the migration trigger threshold according to different energy consumption levels; comprehensively consider factors such as energy consumption, resource utilization, load balancing, and service quality to select the task set to be migrated, and when there is a requirement for a specific reduction in host energy consumption, a dynamic programming method is used to give a minimum migration job set that meets the energy consumption reduction requirements; select hosts that meet the requirements in computing power, memory space, and network performance as candidate hosts for migration, and then comprehensively consider host energy consumption, load balancing, resource utilization, and user service quality, and use a heuristic multi-objective optimization method to select the optimal migration host set, and when the cloud data center has a high energy consumption level and the overall resource utilization is relatively saturated, the target host is searched across data centers for cross-region migration of computing tasks; data synchronization and state consistency before and after migration are achieved to ensure that the task can dynamically adapt to the resource environment of the migration target host. The research results of this invention will be directly applied to cloud data centers, which will help fill the domestic technical gap of this type and promote the development of cloud data center energy consumption management technology; in terms of cloud data center energy consumption optimization and job migration, a reliable dynamic cross-regional job migration technology will be formed. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 It is a general architecture diagram of an implementation scheme of a method for cross-region migration of computing tasks for a cloud data center according to the present invention.

[0022] Figure 2 It is a flow chart of a method for cross-region migration of computing tasks for a cloud data center according to the present invention.

[0023] Figure 3 This is an architecture diagram of the cross-region transfer strategy for cloud data center computing tasks of the present invention.

[0024] Figure 4 It is an architecture diagram of the dynamic migration solution based on the migration synchronization technology of the present invention. DETAILED DESCRIPTION

[0025] The purpose of the present invention is to provide a cross-region migration technology for computing tasks for cloud data centers, which can reduce the energy consumption of cloud data center systems while meeting the requirements of system load balancing and application performance. The cross-region migration process of computing tasks includes three steps: migration environment perception, migration decision-making, and migration synchronization. The specific implementation of the present invention is described below.

[0026] The present invention provides a cross-region migration technology for computing tasks in cloud data centers, with the goal of reducing system energy consumption. The overall architecture of the implementation scheme is as follows: Figure 1As shown, based on the needs of reducing the energy consumption cost of cloud data centers, unified resource optimization management, dynamic load balancing, etc., the core technology of cross-region migration of cloud data center computing tasks is studied. The specific process of the present invention is as follows Figure 2 As shown, firstly, the migration environment perception step is performed, the status of the cloud data center is dynamically monitored, and the characteristic attributes and operation information of the cloud data center load, power consumption, and computing tasks are obtained in real time. Then, through the migration decision step, the migration triggering time is determined and the optimal migration task set and migration destination host set are given. Then, the data synchronization before and after the virtual machine and container migration is realized and the network environment of the target host is dynamically adapted. Finally, the computing task is started to run on the new host. The specific technical implementation scheme adopted by the present invention is as follows.

[0027] Step 1: Migration environment awareness: The migration environment awareness step is designed to dynamically monitor the status of server hosts, virtual machines, containers, and computing tasks, and obtain data center load, power consumption, and characteristic attributes and operation information of computing tasks in real time.

[0028] As a preferred implementation scheme, in step 1, A1: obtain the physical node load at regular intervals and calculate the node resource utilization U = {U CPU ,U MEM ,U IO ,U BW},U CPU is the CPU utilization, U MEM is the memory utilization, U IO is the IO utilization, U BW A2: Establish a refined model for the power consumption measurement of virtual machines, containers, and physical nodes, obtain the power consumption of virtual machines, containers, and physical nodes through the model and load data, and calculate the energy consumption of the cloud data center.

[0029] In A1, the top command can be used to obtain CPU load information. The CPU utilization is calculated as follows:

[0030]

[0031] in, The idle CPU percentage directly output by the top command.

[0032] The free command can be used to obtain memory load information. The formula for calculating memory utilization is as follows:

[0033]

[0034] Among them, M total is the total memory of the host, M free The free memory size.

[0035] Get the number of disk I / Os per second using the iostat command iops , the calculation formula of IO utilization is as follows:

[0036]

[0037] Among them, M iops It is the maximum number of IO operations that the disk can achieve per unit time.

[0038] The network load information can be obtained through the ifconfig command. The calculation formula for network bandwidth utilization is as follows:

[0039]

[0040] where F in is the amount of data flowing into the local machine per unit time, F out is the amount of data flowing out of the local machine per unit time, B net The host network card bandwidth.

[0041] In A2, in view of the diversity, on-demand, dynamic and shared characteristics of task requests and resource usage in cloud data centers, intelligent learning methods such as decision trees, random forests, support vector machines (SVMs), association rules, or deep neural networks are used in combination with multivariate polynomial regression methods to perform multi-level and multi-dimensional energy consumption quantitative analysis of physical hosts, virtual machines, containers, etc.

[0042] Physical host: A physical host is a physical node. Usually, the power consumption of a physical host is positively correlated with its CPU utilization, memory utilization, and bandwidth utilization. Taking the physical host as the modeling unit, the power consumption model generated by different hardware configurations, such as the number of CPUs, the number of cores, the amount of memory, the presence or absence of a GPU, and other conditions under different load states (high / medium / low) is quantified.

[0043] Virtual machine: Taking the virtual host as the modeling unit, establish a quantitative energy consumption model of virtual machines with different configurations under various load states (high / medium / low).

[0044] Container: Take the container as the modeling unit and establish a quantitative energy consumption model for containers with different configurations under different workload states (high / medium / low).

[0045] The present invention quantitatively analyzes multi-dimensional energy consumption. The energy consumption of the physical host = the static energy consumption of the host + the dynamic energy consumption generated by the virtual machines and containers running on the host. The energy consumption of the cloud data center can be expressed as the sum of the energy consumption of the host. The energy consumption model of the virtual machine and container is used to measure the energy consumption of the migrated objects (virtual machines, containers) in the subsequent migration decision. After establishing the power consumption / energy consumption model of the host, virtual machine and container, the existing data learning and analysis methods such as the aforementioned decision tree and random forest are used to quantify and analyze the energy consumption.

[0046] The energy consumption of a cloud data center is equal to the sum of the energy consumption of all physical nodes in the center, so the energy consumption of the center in the time interval from t1 to t2 is calculated as follows:

[0047]

[0048] Among them, H is the set of all physical nodes in the cloud data center, P h (t) is the power consumption of physical node h at time t, which is given by the power consumption model of the above physical host, t∈[t1,t2].

[0049] Step 2: Migration decision: The specific plan for migration decision is as follows: Figure 3 As shown in the figure, node information is obtained through the migration environment perception step, and the migration triggering time is determined based on the adaptive dynamic threshold method of energy consumption level. Taking into account the optimization goals such as energy consumption, load balancing, and QoS, the optimal migration task set and migration destination host set are given, as follows:

[0050] Step B1: Determine the migration timing: Configure two migration trigger thresholds and define the low load threshold Thr of the physical host in the cloud data center. low ={CPU low ,MEM low ,IO low ,BW low}, where CPU low The component is the lower limit threshold of the host CPU utilization, MEM low The component is the lower limit threshold of memory utilization, IO low The component is the lower limit threshold of IO utilization, BW low The components are the lower limit threshold of network bandwidth utilization and the overload threshold Thr high ={CPU high ,MEM high ,IO high ,BW high}, CPU high The component is the upper threshold of the host CPU utilization, MEM high The component is the upper threshold of memory utilization, IO high The component is the upper threshold of IO utilization, BW high The component is the upper threshold of network bandwidth utilization, where CPU high Dynamically adjust according to energy consumption level, configure corresponding CPU for different energy consumption levels high Through the migration environment perception step, the host load computing utilization is monitored, the migration trigger threshold corresponding to the current energy consumption level is determined, and adaptive dynamic migration triggering is realized. Within the time interval t, m groups of load data of a host are collected. When at least n groups (n<=m) of the m groups of data are less than Thrlow The host triggers low-load migration when at least n groups (n<=m) of data among m groups are larger than Thr high The adaptive dynamic threshold method based on energy consumption level of the present invention dynamically and automatically adjusts the migration trigger threshold according to different energy consumption levels.

[0051] As a preferred implementation scheme, in step B1, the present invention uses a three-color energy consumption representation method to describe the energy consumption level of a cloud data center, assuming that the maximum tolerable energy consumption of the center is E max , when the central energy consumption is 0.5E max When the energy consumption is below 0.5E, the energy consumption state is blue. max and 0.75E max Between, the state is green, at 0.75E max The above status is red. Overload threshold Thr high MEM high ,IO high and BW high The static threshold is used, and its value is always 0.8 regardless of the energy consumption level of the cloud data center. high CPU high The dynamic threshold is used for the component, and the specific calculation formula is:

[0052]

[0053] This realizes Thr high Adaptive dynamic adjustment based on energy consumption level, where CPU max is the maximum CPU resource of the physical host, step is an adjustable parameter of (0, 0.1], and α is an adjustable parameter of (0, 1). When the energy consumption level of the cloud data center is in the blue state, the CPU overload threshold high Remain unchanged, the value is 0.9×CPU max ; When the energy consumption level is in the green state, the CPU high Enter the fixed step size reduction process, the minimum value is 0.7×CPU max ; When the energy consumption level is in the red state, the CPU high Enter the constant coefficient reduction process, the minimum value is 0.6×CPU max The low load threshold adopts the static threshold method, and the four components remain unchanged and take the value of 0.2, that is, Thr low ={0.2, 0.2, 0.2, 0.2}.

[0054] Step B2, providing the optimal migration task set: when low-load migration is triggered, all computing tasks on the low-load host are migrated, and no optimal selection is involved; when overload migration is triggered, the present invention provides two optimal migration task selection methods, and the administrator can choose independently according to different needs.

[0055] Method 1: Select computing tasks with high load correlation and small comprehensive load value for migration, which can restore the host to normal load status relatively quickly, and the migration cost is relatively low.

[0056] First, calculate the load correlation. The reason for considering the load correlation is that the higher the correlation between the virtual machine or container load and the host load, the greater the possibility that the virtual machine or container will cause the host to overload. The main correlation measures include the Pearson correlation coefficient method, the Euclidean distance method, the angle cosine similarity method, etc. The present invention adopts the Pearson correlation coefficient method. The computing task is encapsulated in a virtual machine or container, that is, the load correlation between the virtual machine and the host, and the container and the host is calculated, and the historical load record of the virtual machine is recorded as {a1, a2, ..., a k}, the historical load record of the container is {b1,b2,…,b k}, the historical load record of the physical host is {c1,c2,…,c k}, the correlation coefficient calculation formula is as follows:

[0057]

[0058]

[0059] Among them, r ac is the correlation coefficient between the virtual machine and the physical host, r bc is the correlation coefficient between the container and the physical host, are the average values ​​of historical load data of virtual machines, containers, and physical hosts, respectively. The correlation coefficient r is between [-1, 1]. The larger the absolute value of r, the higher the correlation. When 0.8<|r|≤1, the virtual machines, containers, and hosts are extremely strongly correlated. When 0.6<|r|≤0.8, the virtual machines, containers, and hosts are strongly correlated. Virtual machines and containers that meet the extremely strong correlation and strong correlation are added to the candidate migration set.

[0060] Then calculate the comprehensive load value of the elements in the candidate migration set. The calculation formula is as follows:

[0061] Load l =0.5×CPU l +0.3×MEM l +0.1×IO l +0.1×BW l

[0062] Among them, Load l is the comprehensive load value of the lth element in the collection, CPU l is the CPU load value of element l, MEM l is the memory load value of element l, IO l is the IO load value of element l, BW l is the network bandwidth load value of element l. Sort the virtual machines and containers in the candidate migration set in descending order according to their comprehensive load values, and migrate them in sequence until the overloaded host returns to the normal load level.

[0063] Method 2: When a specific reduction in host energy consumption is required, a set of minimum migration operations that meet the specific requirements of energy consumption reduction is given. The present invention uses a dynamic programming method to give a set of minimum migration operations. Assuming that the specific reduction in energy consumption required is E d , add all computing tasks on the overloaded host to the candidate migration set, and obtain the energy consumption measurement values ​​of the candidate migration objects {e1,e2,e3,…,e N}, the candidate migration objects are all task objects on the overloaded host, where e j is the energy consumption of task j, and the selection problem of the outgoing task set is abstracted into the following optimization problem:

[0064]

[0065]

[0066] That is, the migration cost of containers and virtual machines is minimized while meeting the requirement of reducing energy consumption.

[0067] Where f(E d ) is the set of candidate migration tasks, p j is the migration cost of task j, and the migration cost is a positively correlated function of the amount of migrated data.

[0068] Step B3, give the optimal set of migration destination hosts: For the migrated virtual machines and containers, it is necessary to find a destination host for placement. In this process, the service quality of the cloud data center must be considered, and it cannot be significantly reduced due to migration and placement. In terms of determining the target physical host, it is essentially similar to the initial placement problem of virtual machines and containers, both of which are multi-objective combinatorial optimization problems.

[0069] As a preferred implementation scheme, in step B3, firstly, hosts that meet the requirements in computing power, memory space, and network performance are selected as candidate hosts for migration, and then the host energy consumption, load balancing, resource utilization, user service quality, etc. are comprehensively considered, and a heuristic multi-objective optimization method is adopted to select the optimal set of hosts for migration. In addition, the present invention selects different migration destinations according to the energy consumption level of the cloud data center. When the energy consumption level of the cloud data center is in the blue or green state, internal migration of the data center is performed, that is, the host to be migrated out and the host to be migrated in are in the same data center; when it is in the red state, cross-regional migration is performed, that is, the host to be migrated out and the host to be migrated in are in different data centers. First, a data center in the blue or green state is selected, and then the above-mentioned target host selection method is used in the data center to select the optimal destination host for migration.

[0070] Step 3: Migration synchronization: The goal of migration synchronization is to achieve data synchronization and status consistency before and after the migration of virtual machines and containers, shield the differences between the hardware resources of the cloud computing center, and ensure that the task can dynamically adapt to the resource environment of the migration target host.

[0071] As a preferred embodiment, Figure 4 As shown in the figure, in step 3, the migration of virtual machines and containers uses the incremental synchronization method. After the migration starts, the virtual machines and containers of the source host are not terminated immediately, but continue to run. The memory data of the source node is sent to the target node in the transmission iteration. In the first iteration, all memory page data are sent. In the subsequent iterations, only the modified memory page data is sent until the memory page modification rate drops below the threshold or reaches the upper limit of the number of iterations. The iteration process ends, the source node virtual machine and container are shut down, and the last round of dirty page data is transmitted. Then, the virtual machine or container of the destination node is resumed and started, and the entire incremental synchronization process is completed. The incremental synchronization method reduces the amount of data to be transmitted from all memory pages to the dirty pages of the last round of iterations through repeated iterations, greatly shortening the transmission time and thus reducing downtime.

[0072] At the same time, the present invention makes full use of the layered characteristics of the container, and migrates the read-only public image to the destination node first, and then only needs to synchronize the top read-write layer of the container, which can greatly reduce the amount of data, thereby reducing the data synchronization time. Among them, the container technology does not have a Hypervisor layer similar to the virtual machine technology. It can be said that the container is essentially a process with specific parameters. The migration is mainly implemented with the help of the process real-time migration tool CRIU. CRIU is the abbreviation of Checkpoint / Restore In User Space. It is a tool that can implement the checkpoint / recovery mechanism in the user space. Using CRIU, the running application can be frozen, and then the checkpoint mechanism can be used to dump its state information as a set of image files and store them on the hard disk. Finally, these image file sets can be used through the recovery mechanism to restore and re-run the application from the freezing point.

[0073] Stateful migration transfers the state of virtual machines and containers, i.e., memory pages, CPU states, registers, and disk contents, to the target server so that the virtual machines and containers at the target are the same as before the migration. Stateful migration behavior needs to maintain the running data in the memory and the related configuration environment during the migration process, and to ensure that the service quality after the computing task migration will not be reduced, and the most important point is the network environment. Virtual machines and containers usually communicate continuously with other endpoints, such as external clients and end-user applications. Usually, it is hoped that such communications will be retained after the migration, that is, the external client application continues to access the virtual machine or container in a seamless manner after the migration. This problem is called connection maintenance. In addition, the endpoints can communicate using a connection-oriented protocol (such as TCP), and the connection represents the state shared between the endpoints. In addition to retaining the connection between the endpoints, the connection migration also needs to retain the active connection state, so the endpoint does not need to establish a new connection after the migration. The present invention adopts the method of binding the task to the virtual IP address, and through the network address translation, it ensures that the task can dynamically adapt to the resource environment and network environment of the migration target host, thereby shielding the heterogeneity and differences of network resources in different data centers, so that the migrated virtual machine / container can seamlessly adapt to the network in the new host environment.

[0074] The above method is implemented based on a computer program and a cloud data system. The present invention also provides a cloud data center system, including multiple data centers. The cloud data center system is configured with a computer program. When the computer program is executed, the above method for migrating computing tasks across regions is implemented.

[0075] In summary, the present invention encapsulates computing and data processing services in virtual machines and containers for cross-regional migration, dynamically reallocates computing resources, fully utilizes the huge hardware resources of the data center, and reduces the energy consumption of the cloud data center system through steps such as migration environment perception, migration decision-making, and migration synchronization, while meeting the requirements of system load balancing and application performance.

[0076] The present invention is a technical invention adapted to the current development status of cloud computing technology. The research results of the present invention will be directly applied to cloud data centers, which will help fill the domestic gap in such technology, promote the development of energy consumption management technology for cloud data centers, and form a reliable dynamic cross-region migration technology for computing tasks in terms of energy consumption optimization and job migration for cloud data centers. The economic benefits that may be generated by the present invention will depend on the scope of promotion and the degree of application. If it can be well utilized, it will play a very important role in the promotion and utilization of cloud computing technology, and will have a good driving effect on the development of the national economy and produce good economic benefits.

[0077] Those skilled in the art may make various modifications or additions to the specific implementation cases described, or replace them with similar methods, but they will not deviate from the spirit of the present invention or exceed the scope defined in the appended claims.

Claims

1. A method for migrating computing tasks across regions for cloud data centers, characterized by considering The cloud data center meets the data management and dynamic load balancing requirements, and aims to reduce system energy consumption by migrating computing tasks across regions, including the following steps: Step 1: Migrate environment awareness: Dynamically monitor the status of the cloud data center and obtain the characteristic attributes and operation information of the cloud data center load, power consumption, and computing tasks in real time; Step 2: Migration decision: Obtain the node information of the cloud data center through the migration environment perception step, determine the migration triggering time based on the adaptive dynamic threshold method of energy consumption level, comprehensively consider the optimization goals of low energy consumption, load balancing and QoS, determine the migration decision, and give the optimal migration task set and migration destination host set, including: Step B1, determining the migration timing: setting migration trigger thresholds for underload and overload, and performing migration when the load of the physical host reaches the migration trigger threshold, wherein the migration trigger threshold for overload is adaptively adjusted according to the energy consumption level of the physical host; specifically including: Set the migration trigger threshold and define the host's low load threshold as Thr low ={CPU low ,MEM low ,IO low ,BW low }, CPU low The component is the lower limit threshold of the host CPU utilization, MFM low The component is the lower limit threshold of memory utilization, IO low The component is the lower limit threshold of IO utilization, BW low The component is the lower limit threshold of network bandwidth utilization, and the host overload threshold is defined as Thr high ={CPU high ,MEM high ,IO high ,BW high }, CPU high The component is the upper threshold of the host CPU utilization, MEM high The component is the upper threshold of memory utilization, IO high The component is the upper threshold of IO utilization, BW high The component is the upper threshold of network bandwidth utilization, where CPU high Dynamically adjust according to energy consumption level, configure corresponding CPU for different energy consumption levels high ; Monitor the load of all physical hosts through the migration environment perception step. Collect m groups of load data of a physical host within the time interval T. When at least n groups of data among the m groups of data are less than Thr low All components of n≤m, the host triggers low-load migration; when at least n groups of data among m groups of data are greater than Thr high If any component of , the host triggers overload migration; Step B2, providing the optimal migration task set: when underload migration is triggered, all computing tasks on the underloaded host are migrated; when overload migration is triggered, the migrated computing tasks are arranged according to the load correlation between the virtual machine and the host, the container and the host, and the comprehensive load of the computing tasks. If the migration requires a specific reduction in the energy consumption of the host, then under the energy consumption reduction requirement, with the goal of minimizing the migration cost of the container and the virtual machine, the optimal migration task set is solved for all computing tasks on the overloaded host; specifically including: When overload migration is triggered, the following two optimal migration task methods are selected according to the needs: Method 1: Select computing tasks with high load correlation and small comprehensive load value for migration. First, the load correlation is calculated. The computing task is encapsulated in a virtual machine or container, that is, the load correlation between the virtual machine and the host, and the container and the host is calculated: Using the Pearson correlation coefficient method, the historical load records of the virtual machine are recorded as {a1, a2, …, a k }, the historical load record of the container is {b1,b2,…,b k }, the historical load record of the physical host is {c1,c2,…,c k }, the correlation coefficient calculation formula is as follows: Among them, r ac is the correlation coefficient between the virtual machine and the physical host, r bc is the correlation coefficient between the container and the physical host, are the average values ​​of historical load data of virtual machines, containers, and physical hosts, respectively. The correlation coefficient r is between [-1, 1]. The larger the absolute value of r, the higher the correlation. When 0.8<|r|≤1, the virtual machines, containers, and hosts are extremely strongly correlated. When 0.6<|r|≤0.8, the virtual machines, containers, and hosts are strongly correlated. Virtual machines and containers that meet the extremely strong correlation and strong correlation are added to the candidate migration set. Then calculate the comprehensive load value of the elements in the candidate migration set. The calculation formula is as follows: Load l =0.5×CPU l +0.3×MEM l +0.1×IO l +0.1×BW l Among them, Load l is the comprehensive load value of the lth element in the collection, CPU l is the CPU load value of element l, MEM l is the memory load value of element l, IO l is the IO load value of element l, BW l is the network bandwidth load value of element l, sort the virtual machines and containers in the candidate migration set in descending order according to the comprehensive load value, and migrate them in sequence until the overloaded host returns to the normal load level; Method 2: When migration requires a specific reduction in host energy consumption, dynamic programming is used to find the minimum set of migration jobs that meet the specific requirements for energy reduction. Assume that the specific reduction in energy consumption is E d , add all task objects on the overloaded host to the candidate migration set, and obtain the energy consumption measurement values ​​of the candidate migration set {e1,e2,e3,…,e N }, where e j is the energy consumption of task element j, and the selection problem of the outgoing task set is abstracted into the following optimization problem: That is, while meeting the energy consumption reduction requirement, the migration cost of containers and virtual machines is minimized, where f(E d ) is the set of candidate migration tasks, p j is the migration cost of task j, which is a positively correlated function of the amount of migrated data; Step B3, providing the optimal set of migration destination hosts: for hosts that meet the task migration conditions, comprehensively consider the host energy consumption, load balancing, resource utilization, and user service quality, and adopt a heuristic multi-objective optimization method to select the optimal set of migration hosts; specifically, including: firstly selecting hosts that meet the requirements in computing power, memory space, and network performance as the set of candidate migration hosts, and then comprehensively consider the host energy consumption, load balancing, resource utilization, and user service quality, and adopt a heuristic multi-objective optimization method to select the optimal set of migration hosts; wherein, when the cloud data center is at a low energy consumption level, internal migration is performed in the data center, that is, the host to be migrated out and the host to be migrated are in the same data center; when the cloud data center is at a high energy consumption level, cross-regional migration is performed, that is, the host to be migrated out and the host to be migrated are in different data centers; Step 3: Migration synchronization: Use incremental synchronization to synchronize data before and after virtual machine and container migration, and use task binding virtual IP addresses to enable applications to dynamically adapt to the network environment of the migration target host.

2. According to claim 1, a method for migrating computing tasks across regions for a cloud data center is characterized by: Migration environment awareness specifically includes: Step A1: Obtain the physical node load at set intervals and calculate the node resource utilization U = {U CPU ,U MEM ,U IO ,U BW },U CPU is the CPU utilization, U MEM is the memory utilization, U IO is the IO utilization, U BW is the network bandwidth utilization; Step A2: A model for measuring power consumption of virtual machines, containers, and physical nodes is established based on the cloud data center. The power consumption of virtual machines, containers, and physical nodes is obtained through the model and load data, and the energy consumption of the cloud data center is calculated. Among them, a decision tree, random forest, support vector machine SVM, association rule, or deep neural network intelligent learning method is combined with a multivariate polynomial regression method to perform a quantitative analysis of the energy consumption of physical hosts, virtual machines, and containers.

3. According to the method for cross-region migration of computing tasks for cloud data centers according to claim 2, it is characterized in that in step A1, the CPU load information is obtained by using the top command, and the CPU utilization is calculated using the following formula: in, The idle CPU percentage directly output by the top command; Use the free command to obtain memory load information and use the following formula to calculate memory utilization: Among them, M total is the total memory of the host, M free is the free memory size; Use the iostat command to obtain the number of disk I / Os per second U iops , use the following formula to calculate IO utilization: Among them, M iops It is the maximum number of IOs that the disk can achieve per unit time; Use the ifconfig command to obtain network load information and use the following formula to calculate network bandwidth utilization: where F in is the amount of data flowing into the local machine per unit time, F out is the amount of data flowing out of the local machine per unit time, B net The host network card bandwidth.

4. According to the method for cross-region migration of computing tasks for cloud data centers according to claim 2, it is characterized in that in step A2, the energy consumption calculation is specifically: 1) Physical host: The power consumption of the physical host is positively correlated with its CPU utilization, memory utilization, and bandwidth utilization. Taking the physical host as the modeling unit, the power consumption model generated by different load states under different hardware configurations is quantified; 2) Virtual Machine: Taking the virtual host as the modeling unit, a quantitative model of energy consumption of virtual machines with different configurations under various load conditions is established; 3) Container: Taking the container as the modeling unit, establish the energy consumption quantification model of containers with different configurations under different workload states; The energy consumption of a cloud data center is equal to the sum of the energy consumption of all physical nodes in the center, so the energy consumption E of the center in the time interval from t1 to t2 is calculated as follows: in, H is the set of all physical nodes in the cloud data center, P h (t) is the power consumption of physical node h at time t, which is given by the power consumption model of the above physical host, t∈[t1,t2].

5. The method for migrating computing tasks across regions for a cloud data center according to claim 1, characterized in that In step B1, based on the adaptive dynamic threshold method of energy consumption level, the migration trigger threshold is dynamically and automatically adjusted according to different energy consumption levels. The energy consumption level of the cloud data center is described using the three-color energy consumption representation method. The maximum tolerable energy consumption of the center is set as E max , when the central energy consumption is 0.5E max When the energy consumption is below 0.5E, the energy consumption state is blue. max and 0.75E max Between, the state is green, at 0.75E max The above status is red, the overload threshold Thr high MEM high ,IO high and BW high The component adopts a static threshold value, which is always 0.8 regardless of how the energy consumption level of the cloud data center changes. high CPU high The dynamic threshold is used for the component, and the specific calculation formula is: Among them, CPU max is the maximum CPU resource owned by the physical host, step is an adjustable parameter of (0, 0.1], α is an adjustable parameter of (0, 1), and when the energy consumption level of the cloud data center is in the blue state, the CPU overload threshold high Remains unchanged; when the energy consumption level is in the green state, the CPU high Enter the fixed step reduction process; when the energy consumption level is in the red state, the CPU high Enter the constant coefficient reduction process; the low load threshold remains unchanged, Thr low ={0.2, 0.2, 0.2, 0.2}.

6. The method for migrating computing tasks across regions for a cloud data center according to claim 1, characterized in that When the energy consumption level of the cloud data center is in the blue or green state, internal migration is performed within the data center, that is, the host to be migrated out and the host to be migrated in are in the same data center; when it is in the red state, cross-regional migration is performed, that is, the host to be migrated out and the host to be migrated in are in different data centers. First, a data center in the blue or green state is selected, and then the above-mentioned target host selection method is used in the data center to select the optimal migration destination host.

7. The method for migrating computing tasks across regions for a cloud data center according to claim 1, characterized in that Migration synchronization specifically includes: Step C1, incremental synchronization: After the migration starts, the virtual machines and containers of the source host are not terminated immediately, but continue to run. The memory data of the source node is sent to the target node in the transmission iteration. In the first iteration, all memory page data is sent. In the subsequent iterations, only the modified memory page data is sent until the memory page modification rate drops below the threshold or reaches the upper limit of the number of iterations. The iteration process ends, the source node virtual machine and container are shut down, and the last round of dirty page data is transmitted. Then, the virtual machine and container of the destination node are resumed and started, and the entire incremental synchronization process is completed. Step C2, network adaptation: By binding the computing task to the virtual IP address, through network address translation, the task can dynamically adapt to the resource environment and network environment of the migration target host, thereby shielding the heterogeneity and differences of network resources in different data centers, so that the migrated virtual machines and containers can seamlessly adapt to the network in the new host environment.

8. A cloud data center system, characterized in that The cloud data center system includes multiple data centers, and the cloud data center system is configured with a computer program, which, when executed, implements the method for cross-region migration of computing tasks described in any one of claims 1-7.

Images