Virtual machine real-time migration method based on credential cloud

Abstract

The invention relates to the technical field of virtual machine migration, and discloses a virtual machine real-time migration method based on credential cloud, which comprises the following steps of: normally operating a source host to be migrated, preparing resources of a target host, dividing priorities of applications of the source host, and distinguishing high-priority applications from low-priority applications; copying all memories in the source host to a destination host by using pre-copying; iteratively copying all dirty pages generated by the high-priority applications to a destination host; when a dirty page with a high priority in the virtual machine reaches a shutdown condition, pausing the operation of the source host; after all the remaining dirty pages with high priority in the source virtual machine are completely copied to the target host, the target host is started, and the application with high priority of the target host can run normally; copying dirty pages of the low-priority applications to a destination host; according to the method, the priorities of the applications are graded, a certain strategy is adopted, the overall migration time is shortened, and the downtime of the virtual machine is further shortened.

Description

Technical Field

[0001] This invention relates to the field of virtual machine migration technology, and more specifically, to a real-time virtual machine migration method based on domestically developed cloud computing. Background Technology

[0002] Live migration technology for virtual machines involves completely migrating and copying a virtual machine from its source physical host to another physical host, playing a crucial role in load balancing and disaster recovery. While pre-copy algorithms enable live migration, under high load scenarios, some memory pages are repeatedly transferred, severely impacting migration efficiency and extending the overall migration time. Summary of the Invention

[0003] This invention provides a real-time virtual machine migration method based on domestically developed cloud computing, which solves the technical problem of long real-time migration time for virtual machines in related technologies.

[0004] According to one aspect of the present invention, a method for real-time migration of virtual machines based on domestically developed cloud computing is provided, comprising the following steps: Step 101: The source host to be migrated is running normally. Prepare the resources of the destination host and prioritize the applications on the source host, distinguishing between high-priority and low-priority applications. Step 102: Use pre-copy to copy all memory from the source host to the destination host; Step 103: Iteratively copy all dirty pages generated by high-priority applications to the destination host; Step 104: When a high-priority dirty page in the virtual machine reaches the stop condition, suspend the operation of the source host; Step 105: After all remaining high-priority dirty pages in the source virtual machine have been completely copied to the destination host, start the destination host. High-priority applications on the destination host can run normally. Step 106: Copy the dirty pages of the low-priority application to the destination host. After copying all remaining dirty pages to the destination host, the low-priority application can be started.

[0005] Furthermore, methods for prioritizing applications on the source host include: Step 201: Extract application information from the source host; Step 202: Generate information entries for each application on the source host based on the application information of the source host. The information entries for an application include information such as the number of processes, the total number of threads, the average number of threads, the average running time of the process, the number of memory pages occupied by the process, and the amount of memory data occupied by the process. Step 203: Generate application attributes and attribute values ​​based on the application's information entries; Extract application information from historical data for historical hosts and generate the application attributes and attribute values ​​for those historical hosts. The original application set is obtained by combining the applications on the source host with the historical applications. Step 204: Create a square matrix containing N*N cells, randomly select N*N historical applications, and then assign the attributes and attribute values ​​of the selected historical applications to the cells of the square matrix. Step 205: Randomly select an application from the original application set as the first application, and then calculate the first distance between the first application and the unit of the square matrix; select the unit of the square matrix with the smallest first distance to the first application as the first unit; Step 206: Update the attribute values ​​of the first unit and its neighboring units. The update formula is as follows:

[0006] in This represents the value of the x-th attribute of a cell in the updated matrix. This represents the value of the x-th attribute of the element in the previous square matrix. This represents the value of the x-th attribute in the first application, and c represents the number of times the matrix cells are updated. Step 207: Iteratively execute steps 205 and 206 until the set number of iterations is reached; Step 208: Establish a cell domain for each cell of the square matrix, and divide the applications in the original application set into cell domains. The first distance between an application in a cell domain and the cell to which the cell domain belongs is less than the first distance between the application and other cells. Step 209: Calculate the priority parameters of the application on the source host. The formula for calculating the priority parameters is as follows:

[0007] in Let be the priority parameter for the y-th application, where Let be the average of the i-th attribute values ​​of all applications within the unit domain described in the y-th application. Let y be the total number of attributes of the y-th application; The weighted parameter for the y-th application is calculated using the following formula:

[0008] Where f is the number of memory pages occupied by all processes of the y-th application. Let be the average page number of the memory pages occupied by all processes of the y-th application. The page number of the z-th memory page occupied by all processes of the y-th application; Step 210: Prioritize the source host based on the priority parameters of the application on the source host; The application's priority is directly proportional to its priority parameter.

[0009] Furthermore, the method for iteratively copying all dirty pages generated by high-priority applications to the destination host includes: in the first round, copying the dirty page data generated by the current high-priority applications to the destination host; during the first round of copying, the source host continues to run instead of shutting down; after the first round of copying is completed, the dirty page data generated by the high-priority applications during this process is recalculated, and then a second round of copying is performed to copy this new dirty page data to the destination host; this process is repeated until the amount of newly generated dirty page data by high-priority applications is less than a set data amount threshold, at which point the system enters the shutdown copying phase, and the source host is suspended from running.

[0010] Furthermore, the application's attributes correspond to the number of processes, total number of threads, average number of threads, average process runtime, number of memory pages occupied by the process, and amount of memory data occupied by the process.

[0011] Furthermore, the first distance between the first application and the element of the square matrix is ​​calculated. The formula is as follows:

[0012] in, For the i-th attribute value of the first application, Let be the value of the i-th attribute of the element in the square matrix, and n be the total number of attributes applied in the first application.

[0013] Furthermore, the second distance between units The calculation formula is as follows:

[0014] in and These are the row numbers of the two units, respectively. and These are the column numbers of the two units, respectively.

[0015] Furthermore, the neighboring units of the first unit refer to units whose second distance from the first unit is less than 8.

[0016] Furthermore, the number of times steps 205 and 206 are executed iteratively = , where Z is the number of applications on the source host.

[0017] Furthermore, a first priority threshold is set. If the priority parameter of the application is less than or equal to the first priority threshold, it is classified as low priority; otherwise, it is classified as high priority.

[0018] According to one aspect of the present invention, a real-time virtual machine migration system based on domestically developed cloud computing is provided, comprising: The permission acquisition module is used to acquire management permissions for the source and destination hosts. The pre-verification module is used to verify whether the configurations of the destination host and the source host are consistent. The pre-copy module is used to copy all memory from the source host to the destination host; The iterative copy module is used to iteratively copy all dirty pages generated by high-priority applications to the destination host; The host startup module is used to start the destination host after iteratively copying all dirty pages generated by high-priority applications to the destination host and run the high-priority applications that have been copied. The general copy module is used to copy the dirty pages of low-priority applications to the destination host after the iterative copy module has finished executing. The full startup module starts low-priority applications on the destination host after the normal copy module has finished executing.

[0019] The beneficial effects of this invention are as follows: In a virtual machine, various applications are running. For users, some applications have higher priority, while others have little impact on users. The downtime requirements for these two types of applications are different. The original pre-copy algorithm treats all applications equally for overall migration, resulting in excessively long overall migration time. By prioritizing applications and adopting certain strategies, the overall migration time can be reduced, further reducing the downtime of the virtual machine. This invention uses historical data mining to determine application priority from the perspective of data migration, and makes independent judgments for different source hosts, making application priority determination more accurate. Attached Figure Description

[0020] Figure 1 This is a schematic diagram of a real-time virtual machine migration system based on the domestic IT cloud platform according to the present invention; Figure 2 This is a flowchart of a real-time virtual machine migration method based on the domestic IT cloud platform according to the present invention; Figure 3 This invention provides a method for prioritizing applications on a source host. Figure 1 ; Figure 4 This invention provides a method for prioritizing applications on a source host. Figure 2 .

[0021] The figure shows: a virtual machine real-time migration system 100 based on the domestic cloud platform, including a permission acquisition module 101, a pre-verification module 102, a pre-copy module 103, an iterative copy module 104, a host startup module 105, a general copy module 106, and a full startup module 107. Detailed Implementation

[0022] The subject matter described herein will now be discussed with reference to exemplary embodiments. It should be understood that these embodiments are discussed only to enable those skilled in the art to better understand and implement the subject matter described herein, and changes may be made to the function and arrangement of the discussed elements without departing from the scope of this specification. Various processes or components may be omitted, substituted, or added as needed in the examples. Furthermore, features described in some examples may be combined in other examples. Example

[0023] like Figures 2-4 As shown, a real-time virtual machine migration method based on domestically developed cloud computing includes the following steps: Step 101: Preparatory work before migration. Ensure the source host to be migrated is running normally. Prepare the resources for the destination host. Generally, the CPU, memory, disk, and other resources of the destination host should be identical to those of the source host. Alternatively, the destination and source hosts may share a disk. Establish a TCP connection between the source and destination hosts. Prioritize applications on the source host, distinguishing between high-priority and low-priority applications.

[0024] In one embodiment of the present invention, a method for prioritizing applications on a source host includes: Step 201: Extract application information from the source host; Step 202: Generate information entries for each application on the source host based on the application information of the source host. The information entries for an application include information such as the number of processes, the total number of threads, the average number of threads (total number of threads divided by the number of processes), the average running time of the process, the number of memory pages occupied by the process, and the amount of memory data occupied by the process. Step 203: Generate application attributes and attribute values ​​based on the application's information entries; In this step, the attributes corresponding to the number of processes, total number of threads, average number of threads (total number of threads divided by number of processes), average process runtime, number of memory pages used by a process, and amount of memory data used by a process are normalized, with the normalized value range being [missing value]. ; Extract application information from historical data for historical hosts and generate the application attributes and attribute values ​​for those historical hosts. The original application set is obtained by combining the applications on the source host with the historical applications. The historical host refers to the host where the virtual machine migration was performed before this virtual machine migration.

[0025] Step 204: Create a square matrix containing N*N cells, randomly select N*N historical applications, and then assign the attributes and attribute values ​​of the selected historical applications to the cells of the square matrix. In one embodiment of the present invention, N=4.

[0026] Step 205: Randomly select an application from the original application set as the first application, and then calculate the first distance between the first application and the unit of the square matrix; select the unit of the square matrix with the smallest first distance to the first application as the first unit; Calculate the first distance between the first application and the element of the square matrix. The formula is as follows:

[0027] in, For the i-th attribute value of the first application, Let be the value of the i-th attribute of the element in the square matrix, and n be the total number of attributes applied in the first application. Step 206: Update the attribute values ​​of the first unit and its neighboring units. The update formula is as follows:

[0028] in This represents the value of the x-th attribute of a cell in the updated matrix. This represents the value of the x-th attribute of the element in the previous square matrix. Let c represent the value of the x-th attribute of the first application, and c represent the number of times the matrix cells are updated. The value of c is incremented by 1 each time a cell of the matrix is ​​updated. Each update of a cell in the matrix means that all attributes of the cells in the matrix are updated once.

[0029] Step 207: Iteratively execute steps 205 and 206 until the set number of iterations is reached; In one embodiment of the present invention, the number of iterations = , where Z is the number of applications on the source host.

[0030] In one embodiment of the present invention, the neighboring units of the first unit refer to units that are less than 8 units away from the first unit.

[0031] Second distance between units The calculation formula is as follows:

[0032] in and These are the row numbers of the elements in the two square matrices, respectively. and These are the column numbers of the elements in the two square matrices, respectively.

[0033] Step 208: Establish a cell domain for each cell of the square matrix, and divide the applications in the original application set into cell domains. The first distance between an application in a cell domain and the cell to which the cell domain belongs is less than the first distance between the application and other cells. Step 209: Calculate the priority parameters of the application on the source host. The formula for calculating the priority parameters is as follows:

[0034] in Let be the priority parameter for the y-th application, where Let be the average of the i-th attribute values ​​of all applications within the unit domain described in the y-th application. The total number of attributes for the y-th application (equal to the total number of attributes for the first application, hence the same notation is used); The weighted parameter for the y-th application is calculated using the following formula:

[0035] Where f is the number of memory pages occupied by all processes of the y-th application. Let be the average page number of the memory pages occupied by all processes of the y-th application. The page number of the z-th memory page occupied by all processes of the y-th application; By setting weighting parameters, the continuous accumulation of memory pages occupied by frequently used applications can be used for weighted calculations, thereby improving the ability of priority parameters to express the priority of applications.

[0036] Step 210: Prioritize the source host based on the priority parameters of the application on the source host; The application's priority is directly proportional to its priority parameter. One method of classification is to set a first priority threshold. If the priority parameter is less than or equal to the first priority threshold, it is classified as low priority; otherwise, it is classified as high priority.

[0037] Step 102: Use pre-copy to copy all memory from the source host to the destination host.

[0038] Step 103: Iteratively copy all dirty pages generated by high-priority applications to the destination host. Low-priority applications are not copied for the time being.

[0039] Step 104: When a high-priority dirty page in the virtual machine reaches the shutdown condition, suspend the operation of the source host.

[0040] This embodiment provides an iterative copying method. After the migration begins, the first round copies the dirty page data generated by the current high-priority application to the destination host. During the first round of copying, the source host continues to run instead of shutting down. After the first round of copying ends, the dirty page data generated by the high-priority application during this process is recalculated, and then the second round of copying is performed to copy this new dirty page data to the destination host. This process is repeated until the amount of newly generated dirty page data by the high-priority application is less than a set data amount threshold, at which point the shutdown copying phase begins, and the source host operation is paused.

[0041] Step 105: After all remaining high-priority dirty pages in the source virtual machine have been completely copied to the destination host, start the destination host. High-priority applications on the destination host can run normally, while low-priority applications will not run for the time being.

[0042] Step 106: Copy the dirty pages of the low-priority application to the destination host. After copying all remaining dirty pages to the destination host, the low-priority application can be started.

[0043] like Figure 1 As shown, in one embodiment of the present invention, a virtual machine real-time migration system 100 based on domestically developed cloud computing is provided, comprising: The permission acquisition module 101 is used to acquire management permissions for the source host and the destination host. The pre-verification module 102 is used to verify whether the configurations of the destination host and the source host are consistent. The pre-copy module 103 is used to copy all memory from the source host to the destination host; Iterative copy module 104 is used to iteratively copy all dirty pages generated by high-priority applications to the destination host. The host startup module 105 is used to start the destination host and run the high-priority applications that have been copied after iteratively copying all the dirty pages generated by the high-priority applications to the destination host. General copy module 106 is used to copy dirty pages of low-priority applications to the destination host after the iterative copy module 104 has finished executing. The full startup module 107 starts a low-priority application on the destination host after the general copy module 106 has finished executing.

[0044] The embodiments of this example have been described above. However, this example is not limited to the specific implementation methods described above. The specific implementation methods described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms based on the guidance of this example, and all of them are within the protection scope of this example.

Claims

1. A method for real-time migration of virtual machines based on domestically developed cloud computing, characterized in that, Includes the following steps: Step 101: The source host to be migrated is running normally. Prepare the resources of the destination host and prioritize the applications on the source host, distinguishing between high-priority and low-priority applications. Step 102: Use pre-copy to copy all memory from the source host to the destination host; Step 103: Iteratively copy all dirty pages generated by high-priority applications to the destination host; Step 104: When a high-priority dirty page in the virtual machine reaches the stop condition, suspend the operation of the source host; Step 105: After all remaining high-priority dirty pages in the source virtual machine have been completely copied to the destination host, start the destination host. High-priority applications on the destination host can run normally. Step 106: Copy the dirty pages of the low-priority application to the destination host. After copying all remaining dirty pages to the destination host, the low-priority application can be started.

2. The method for real-time migration of virtual machines based on domestically developed cloud computing as described in claim 1, characterized in that, Methods for prioritizing applications on the source host include: Step 201: Extract application information from the source host; Step 202: Generate information entries for each application on the source host based on the application information of the source host. The information entries for an application include information such as the number of processes, the total number of threads, the average number of threads, the average running time of the process, the number of memory pages occupied by the process, and the amount of memory data occupied by the process. Step 203: Generate application attributes and attribute values ​​based on the application's information entries; Extract application information from historical data for historical hosts and generate the application attributes and attribute values ​​for those historical hosts. The original application set is obtained by combining the applications on the source host with the historical applications. Step 204: Create a square matrix containing N*N cells, randomly select N*N historical applications, and then assign the attributes and attribute values ​​of the selected historical applications to the cells of the square matrix. Step 205: Randomly select an application from the original application set as the first application, and then calculate the first distance between the first application and the unit of the square matrix; select the unit of the square matrix with the smallest first distance to the first application as the first unit; Step 206: Update the attribute values ​​of the first unit and its neighboring units. The update formula is as follows:

3. Among them This represents the value of the x-th attribute of a cell in the updated matrix. This represents the value of the x-th attribute of the element in the previous square matrix. This represents the value of the x-th attribute in the first application, and c represents the number of times the matrix cells are updated. Step 207: Iteratively execute steps 205 and 206 until the set number of iterations is reached; Step 208: Establish a cell domain for each cell of the square matrix, and divide the applications in the original application set into cell domains. The first distance between an application in a cell domain and the cell to which the cell domain belongs is less than the first distance between the application and other cells. Step 209: Calculate the priority parameters of the application on the source host. The formula for calculating the priority parameters is as follows:

4. Among them Let be the priority parameter for the y-th application, where Let be the average of the i-th attribute values ​​of all applications within the unit domain described in the y-th application. Let y be the total number of attributes of the y-th application; The weighted parameter for the y-th application is calculated using the following formula:

5. Among them, f is the number of memory pages occupied by all processes of the y-th application. Let be the average page number of the memory pages occupied by all processes of the y-th application. The page number of the z-th memory page occupied by all processes of the y-th application; Step 210: Prioritize the source host based on the priority parameters of the application on the source host; The application's priority is directly proportional to its priority parameter.

6. The method for real-time migration of virtual machines based on domestically developed cloud computing as described in claim 1, characterized in that, The method for iteratively copying all dirty pages generated by high-priority applications to the destination host includes: In the first round, the dirty page data generated by the current high-priority applications is copied to the destination host; during the first round of copying, the source host continues to run instead of shutting down; after the first round of copying is completed, the dirty page data generated by high-priority applications during this process is recalculated, and then a second round of copying is performed, copying this new dirty page data to the destination host; this process is repeated until the amount of newly generated dirty page data from high-priority applications is less than a set data threshold, at which point the system enters the shutdown copying phase, pausing the source host's operation.

7. The method for real-time migration of virtual machines based on domestically developed cloud computing as described in claim 2, characterized in that, The application's attributes correspond to the number of processes, total number of threads, average number of threads, average process runtime, number of memory pages used by a process, and amount of memory data used by a process.

8. A method for real-time migration of virtual machines based on domestically developed cloud computing, as described in claim 2, is characterized in that... Calculate the first distance between the first application and the element of the square matrix. The formula is as follows:

9. Among them, For the i-th attribute value of the first application, Let be the value of the i-th attribute of the element in the square matrix, and n be the total number of attributes applied in the first application.

10. A method for real-time migration of virtual machines based on a domestically developed cloud computing platform according to claim 2, characterized in that, Second distance between units The calculation formula is as follows:

11. Among them and These are the row numbers of the two units, respectively. and These are the column numbers of the two units, respectively.

12. The method for real-time migration of virtual machines based on domestically developed cloud computing according to claim 2, characterized in that, The neighboring cells of the first cell refer to cells whose second distance from the first cell is less than 8.

13. A method for real-time migration of virtual machines based on domestically developed cloud computing, as described in claim 2, is characterized in that... The number of times steps 205 and 206 are executed iteratively = , where Z is the number of applications on the source host.

14. A method for real-time migration of virtual machines based on domestically developed cloud computing, as described in claim 2, characterized in that, If the priority parameter of an application is less than or equal to the first priority threshold, it is classified as low priority; otherwise, it is classified as high priority.

15. A real-time virtual machine migration system based on domestically developed cloud computing, characterized in that, It is used to execute a real-time virtual machine migration method based on domestically developed cloud computing as described in any one of claims 1-8. A real-time virtual machine migration system based on domestically developed cloud computing includes: The permission acquisition module is used to acquire management permissions for the source and destination hosts. The pre-verification module is used to verify whether the configurations of the destination host and the source host are consistent. The pre-copy module is used to copy all memory from the source host to the destination host; The iterative copy module is used to iteratively copy all dirty pages generated by high-priority applications to the destination host; The host startup module is used to start the destination host after iteratively copying all dirty pages generated by high-priority applications to the destination host and run the high-priority applications that have been copied. The general copy module is used to copy the dirty pages of low-priority applications to the destination host after the iterative copy module has finished executing. The full startup module starts low-priority applications on the destination host after the normal copy module has finished executing.

Images