Performance optimization method and system for online cross-storage migration of virtual machine

Abstract

The application provides a performance optimization method and system for online cross-storage migration of a virtual machine, which comprises two parts of optimization for a raw format disk and optimization for a qcow2 format disk. After the execution of the seek data and hole once, the corresponding positions are recorded in the optimization for the raw format disk; in the optimization for the qcow2 format disk, the corresponding L2Table is checked only when the L2Table is loaded and flushed. Through the optimization of the migration logic of the disks in the raw and qcow2 formats, the application reduces the influence of the online cross-storage migration on the performance of the virtual machine. The application optimizes the functions that are particularly time-consuming in the online cross-storage migration, improves the efficiency, reduces the influence on the performance of the virtual machine, and improves the user experience.

Description

Technical Field

[0001] This invention relates to the field of computer technology, and in particular to a performance optimization method and system for online cross-storage migration of virtual machines. Background Technology

[0002] With its advantages such as reduced costs, improved operational efficiency, and accelerated enterprise innovation, virtualization technology is increasingly favored by users. Kernel Virtual Machine (KVM) virtualization has largely replaced Xen as the default open-source solution for creating and supporting virtual machines (VMs) on most Linux systems.

[0003] As business volume increases, the number of virtual machines also increases, leading to a growing demand for storage capacity. When storage needs maintenance, or when storage capacity is insufficient, or to prevent overloading of a particular storage pool, operations personnel often need to migrate virtual machines from one storage device to another. Especially as customer business data volume increases, cross-storage migration time continues to rise. Particularly when virtual machine disks contain numerous small files and are severely fragmented, some functions in online cross-storage migration are particularly time-consuming and can severely impact virtual machine performance.

[0004] Therefore, optimizing existing cross-storage migration processes so that online cross-storage migration will not affect virtual machine performance, even on large disks storing a large number of small files, has become a huge and urgent need to be addressed. Summary of the Invention

[0005] In view of this, the purpose of this invention is to propose a performance optimization method and system for online cross-storage migration of virtual machines. Through research, it has been found that when virtual machines migrate online across storage (including disks in both qcow2 and raw formats), online cross-storage migration of virtual machines can be supported by enabling virtualization products to host and run virtual machines. Two disk image formats are supported: raw image format and qcow2 format. By optimizing the disk migration logic for both raw and qcow2 formats, the impact of online cross-storage migration on virtual machine performance can be reduced.

[0006] To achieve the above objectives, on the one hand, this invention provides a performance optimization method for online cross-storage migration of virtual machines. This method includes two parts: optimization for raw format disks and optimization for qcow2 format disks. The optimization for raw format disks includes:

[0007] After performing a seek data and hole operation, record the corresponding position.

[0008] Before executing the next seek, check if the current offset is less than the position of the hole. If it is less, stop seeking, transfer all data between the data and the hole, and then proceed with the next seek.

[0009] If a hole is punched in the data area, the previously cached data and hole positions are discarded, and the seek will be executed again next time.

[0010] If the seek region is empty, the current hole and data positions are not retained;

[0011] The optimizations for qcow2 formatted disks include:

[0012] The corresponding L2 Table is validated only when the L2 Table is loaded and refreshed.

[0013] As a further aspect of the present invention, if the seek region is a hole, the current hole and data positions are not retained, and the hole region is still sought according to the size of 1M. At this time, the metadata corresponding to the hole has already been loaded into memory and does not need to be loaded again.

[0014] As a further aspect of the present invention, when verifying the corresponding L2 Table, the number of individual L2 Table entries verified each time is 256*1024 / 8, and the total number of verified entries is (1024 / 8)*(256*1024 / 8) = the total number of L2 Table entries. Therefore, it is only necessary to verify the L2 entry once.

[0015] As a further aspect of the present invention, the performance optimization method for online cross-storage migration of virtual machines also includes file simulation in QEMU, where file simulation in QEMU is divided into two layers:

[0016] The first layer is a raw / qcow2 simulation, which implements corresponding driver functions based on the characteristics of raw and qcow2 files; the qcow2 has L1 / L2 tables, and the total amount of data to be migrated is calculated before migration; the raw format disk has no metadata, and the total amount of data to be migrated is the size of the disk virtual machine;

[0017] The second layer is a file simulation, which abstracts the common functions of raw and qcow2 and implements them in bdrv_file.

[0018] As a further aspect of the present invention, QEMU online disk migration is achieved by calling the qmp_drive_mirror command. The migration is performed in the function mirror_run(block / mirror.c), where the migration process in mirror_run includes:

[0019] If bdrv_is_allocated_above-->bdrv_set_dirty, obtain the data to be migrated from the device. During migration, set the bit of that block in the bitmap to dirty. For qcow2 formatted disks, calculate the amount of data to be migrated.

[0020] The loop begins migrating data in the bitmap that is classified as dirty, until the virtual machine meets the switching conditions; among them, bdrv_get_dirty_count counts the number of data blocks to be migrated, and for raw format disks, holes need to be sought; mirror_iteration starts the data migration;

[0021] End migration.

[0022] As a further aspect of the present invention, the optimization for raw format disks involves using `bdrv_get_dirty_count` to count the number of data blocks that need to be migrated. After performing one seek of data and hole, the corresponding position is recorded. Before performing the next seek, it is determined whether the current offset is less than the position of hole. If it is less, the seek is stopped, all data between data and hole is transferred, and then the next seek is performed.

[0023] As a further aspect of the present invention, the optimization for qcow2 format disks is to optimize the Qcow2 L2 Entry verification logic for calculating the total amount of data migrated from qcow2 format disks. The current logic for optimizing qcow2 format disks is as follows: when calculating the position of each offset on the qcow2 disk or when flushing the L2 Table, all L2 entries of the L2 table in memory are verified. When calculating the total amount of migrated data, the total number of verification entries is: the total number of L2 table entries * [(256*1024 / 8)*8].

[0024] As a further aspect of the present invention, the optimization for qcow2 formatted disks also includes:

[0025] Cluster_size = 256K = 256 * 1024 bytes, which is the cluster size in bytes;

[0026] sizeof(uint64_t) = 64bit = 8 bytes, which is the size of the block device corresponding to the image, in bytes;

[0027] Disk L2_tables = disk size (in GB) / 8, that is, the file size corresponding to each L2 table is 8 GiB, and the total L2_tables corresponding to the disk is disk size (in GB) / 8;

[0028] L2_entries = (cluster_size / sizeof(uint64_t, i.e., the number of elements in each L2 table);

[0029] Total number of L2 table entries = L2_tables * L2_entries = (disk size / 8) * (cluster_size / sizeof(uint64_t)), where the disk size is in GB;

[0030] The corresponding L2 cache in memory is 2MiB, which means it caches 8 L2 tables. The number of L2 entries in the cache is ((256*1024 / 8)*8).

[0031] When calculating the location of each offset on the qcow2 disk or when flushing the L2 Table, all L2 entries in the L2 Table in memory are checked. The total number of check entries = the total number of L2 entries on disk * the number of L2 entries in the cache. That is, the total number of check entries = [(disk size / 8)*(cluster_size / sizeof(uint64_t))]*[((256*1024 / 8)*8)].

[0032] As a further aspect of the present invention, for the optimization of qcow2 format disks, the corresponding L2 Table is only verified during L2 Table loading and flushing.

[0033] Another aspect of the present invention provides a performance optimization system for online virtual machine cross-storage migration, comprising:

[0034] The raw disk optimization module is used to optimize raw format disks. During optimization, after performing a seek of data and hole, it records the corresponding positions. Before performing the next seek, it checks whether the current offset is less than the hole position. If it is less, it stops seeking, transfers all data between data and hole, and then performs the next seek. If a hole is created in the data area, the previously cached data and hole positions are discarded, and the seek will be performed again on the next attempt. If the seek area is empty, the current hole and data positions are not retained.

[0035] The raw disk optimization module is used to optimize qcow2 format disks, and only verifies the corresponding L2 Table when loading and flushing the L2 Table.

[0036] In another aspect, the present invention provides a computer device including a memory and a processor, wherein the memory stores a computer program that, when executed by the processor, performs any of the above-described performance optimization methods for online cross-memory migration of virtual machines according to the present invention.

[0037] In another aspect, the present invention provides a computer-readable storage medium storing computer program instructions that, when executed, implement any of the above-described performance optimization methods for online cross-storage migration of virtual machines according to the present invention.

[0038] The present invention has at least the following beneficial technical effects: The present invention proposes a performance optimization method and system for online cross-storage migration of virtual machines. By optimizing the disk migration logic of both raw and qcow2 formats, the impact of online cross-storage migration on virtual machine performance is reduced. The present invention optimizes functions that are particularly time-consuming in online cross-storage migration, thereby improving efficiency, reducing the impact on virtual machine performance, and enhancing user experience. Attached Figure Description

[0039] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other embodiments can be obtained based on these drawings without creative effort.

[0040] In the diagram:

[0041] Figure 1 This is a flowchart illustrating file simulation in QEMU within the performance optimization method for online cross-storage migration of virtual machines according to the present invention;

[0042] Figure 2 The flowchart below illustrates the virtual machine cross-storage migration process in the performance optimization method for online cross-storage migration of virtual machines according to the present invention.

[0043] Figure 3 The existing seek logic diagram in the performance optimization method for online cross-storage migration of virtual machines according to the present invention;

[0044] Figure 4 The logical diagram for optimizing seek in the performance optimization system for online cross-storage migration of virtual machines according to the present invention;

[0045] Figure 5This is a logical diagram of hole punching in the data region during Seek optimization in the performance optimization method for realizing online cross-storage migration of virtual machines according to the present invention;

[0046] Figure 6 This is a logical diagram of writing data in the hole region during Seek optimization in the performance optimization method for realizing online cross-storage migration of virtual machines according to the present invention;

[0047] Figure 7 The logical diagram of Qcow2 L2Entry verification in the performance optimization method for realizing online cross-storage migration of virtual machines according to the present invention;

[0048] Figure 8 A schematic diagram of the hardware structure of a computer device according to an embodiment of the performance optimization method for realizing online cross-storage migration of virtual machines according to the present invention;

[0049] Figure 9 This is a schematic diagram of the hardware structure of a storage medium according to an embodiment of the performance optimization method for realizing online cross-storage migration of virtual machines according to the present invention. Detailed Implementation

[0050] To make the objectives, technical solutions, and advantages of the present invention clearer, the embodiments of the present invention will be further described in detail below with reference to specific examples and the accompanying drawings.

[0051] It should be noted that all uses of "first" and "second" in the embodiments of the present invention are for the purpose of distinguishing two different entities or different parameters with the same name. Therefore, "first" and "second" are merely for convenience of expression and should not be construed as limiting the embodiments of the present invention. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion, such as other steps or units inherent in a process, method, system, product, or device that includes a series of steps or units.

[0052] Simply put, as the amount of customer business data increases, the time required for cross-storage migration continues to increase. In particular, when the virtual machine disk contains a large number of small files and is severely fragmented, some functions in online cross-storage migration are particularly time-consuming to execute and can seriously affect the performance of the virtual machine.

[0053] Therefore, optimizing the existing cross-storage migration process technology so that online cross-storage migration will not affect virtual machine performance, even on large-capacity disks storing a large number of small files, has become a huge need that our company urgently needs to address.

[0054] Research revealed that during online cross-storage migration of virtual machines (including disks in both qcow2 and raw formats), some functions are time-consuming to execute, affecting vCPU operation and causing virtual machine performance degradation or even stuttering. Therefore, it is necessary to optimize the online cross-storage migration logic of virtual machines.

[0055] For raw format disks, since raw data lacks metadata, the total amount of data migrated is equal to the size of the virtual machine disk. During storage migration, a seek operation is called to determine the data and hole locations (seek_data / seek_hole). A seek operation is performed every 1MB of data transferred, resulting in frequent seek calls. Seeking data is fast, but seeking holes is slow, especially when the raw disk file is large and the hole location is relatively far down the disk. Each seek hole attempt causes extents in memory to become invalid, requiring them to be reloaded from the disk. The long time required for each seek hole attempt can lead to competition for lock resources with the vCPU, impacting virtual machine performance.

[0056] For qcow2 formatted disks, because qcow2 has L1 / L2 tables, the total amount of data to be migrated can be calculated directly before migration. However, calculating the total amount of data before migration execution is time-consuming. When calculating the location of each offset on the qcow2 disk or when flushing the L2 tables, all L2 entries in the L2 tables in memory are verified, resulting in a large amount of computation and long processing time, impacting virtual machine performance.

[0057] Based on this, the present invention mainly provides a method and system device for optimizing the performance of online cross-storage migration of virtual machines. By optimizing the disk migration logic of both raw and qcow2 formats, the impact of online cross-storage migration on virtual machine performance is reduced.

[0058] Therefore, in a first aspect, the present invention provides a performance optimization method for online virtual machine cross-storage migration. The first part of this method focuses on optimization for raw format disks; the specific steps are as follows:

[0059] 1) After performing a seek operation on data and hole, record the corresponding positions. Before performing the next seek, check if the current offset is less than the hole position. If it is, stop seeking, transfer all data between data and hole, and then perform the next seek.

[0060] 2) If a hole is punched in the data area, the previously cached data and hole positions will be discarded, and the seek will be executed again next time;

[0061] 3) If the seek region is empty, the current hole and data positions are not retained. That is, the empty region is still sought according to the size of 1M, but the metadata corresponding to the hole has already been loaded into memory and does not need to be loaded again. The seek speed will be very fast and will not affect the performance of the virtual machine.

[0062] The second part focuses on optimizations for qcow2 formatted disks. L2 Table verification is performed only during L2 Table loading and flushing. The number of entries verified each time is 256 * 1024 / 8 (the number of entries in a single L2 Table), and the total number of verified entries is (1024 / 8) * (256 * 1024 / 8) = the total number of L2 Table entries. Therefore, L2 entries only need to be verified once.

[0063] To make the workflow of this system clearer and more intuitive, please refer to [link / reference]. Figure 1 and Figure 2 As shown,

[0064] In embodiments of the present invention, such as Figure 1 As shown, file simulation in QEMU is divided into two layers: The first layer is raw / qcow2 simulation, which implements corresponding driver functions based on the characteristics of raw and qcow2 files. qcow2 has L1 / L2 tables, allowing the total amount of data to be migrated to be calculated directly before migration. Raw format disks have no metadata, and the total amount of data to be migrated is the size of the disk virtual machine. The second layer is file simulation, which abstracts the common functions of raw and qcow2 and implements them in bdrv_file.

[0065] In embodiments of the present invention, such as Figure 2 As shown, the QEMU online disk migration process is implemented by calling the qmp_drive_mirror command. The optimized process is in the function mirror_run(block / mirror.c), and the detailed workflow of mirror_run is as follows:

[0066] 1) `bdrv_is_allocated_above` --> `bdrv_set_dirty`: This retrieves the data to be migrated from the device. If migration is required, the bit in the corresponding block of the bitmap is set to dirty. For qcow2 formatted disks, the amount of data to be migrated needs to be calculated; this process requires optimization.

[0067] 2) The loop begins migrating data in the bitmap that is currently in drity, until the virtual machine meets the switching condition (should_complete is assigned a value or block_job_is_cancelled):

[0068] a) `rv_is_allocated_above` --> `bdrv_set_dirty`: This retrieves the data to be migrated from the device. If migration is required, the bit in the corresponding block of the bitmap is set to dirty. For qcow2 formatted disks, the amount of data to be migrated needs to be calculated; this process requires optimization.

[0069] b) Start the loop to migrate data in the bitmap that is drity until the virtual machine meets the switching condition (should_complete is assigned a value or block_job_is_cancelled);

[0070] 3) End the migration.

[0071] In an embodiment of the present invention, the first part is an optimization for raw format disks.

[0072] Based on step a of section 2) of the above workflow, `bdrv_get_dirty_count`, when optimizing for raw format disks, the number of data blocks that need to be migrated is counted. For raw format disks, the seek logic here needs optimization. After performing one seek of data and hole, the corresponding position is recorded. Before performing the next seek, it is checked whether the current offset is less than the hole position. If it is less, the seek is stopped, all data between data and hole is transferred, and then the next seek is performed. Figure 3 This is the existing logic. Figure 4 It's the optimized logic.

[0073] In an embodiment of the present invention, if a hole is punched in the data area, the previously cached data and hole positions are discarded, and the seek operation will be performed again next time.

[0074] See Figure 3 As shown, Figure 3 Optimize Seek: Create holes in the data area.

[0075] If the seek region is empty, the current hole and the next data block location are not retained. That is, the empty region is still sought as a 1MB block, but the metadata corresponding to the hole has already been loaded into memory and does not need to be loaded again. The seek speed is very fast and will not affect the virtual machine. Figure 6 Optimization for Seek: A diagram illustrating the writing of data in the hole region.

[0076] In an embodiment of the present invention, the second part optimizes the Qcow2 L2Entry verification logic for calculating the total data volume of the qcow2 format disk migration.

[0077] In this embodiment, the current logic for calculating the migration of qcow2 format disks is as follows: when calculating the position of each offset on the qcow2 disk or when flushing the L2 Table, all L2 entries of the L2 table in memory are checked. When calculating the total amount of migration data, the total number of checked entries is: total number of L2 entries * [(256*1024 / 8)*8].

[0078] In an embodiment of the present invention, the optimization for qcow2 formatted disks further includes:

[0079] Cluster_size = 256K = 256 * 1024 bytes, which is the cluster size in bytes;

[0080] sizeof(uint64_t) = 64bit = 8 bytes, which is the size of the block device corresponding to the image, in bytes;

[0081] Disk L2_tables = disk size (in GB) / 8, that is, the file size corresponding to each L2 table is 8 GiB, and the total L2_tables corresponding to the disk is disk size (in GB) / 8;

[0082] L2_entries = (cluster_size / sizeof(uint64_t, i.e., the number of elements in each L2 table);

[0083] Total number of L2 table entries = L2_tables * L2_entries = (disk size / 8) * (cluster_size / sizeof(uint64_t)), where the disk size is in GB;

[0084] The corresponding L2 cache in memory is 2MiB, which means it caches 8 L2 tables. The number of L2 entries in the cache is ((256*1024 / 8)*8).

[0085] When calculating the location of each offset on the qcow2 disk or when flushing the L2 Table, all L2 entries in the L2 Table in memory are checked. The total number of check entries = the total number of L2 entries on disk * the number of L2 entries in the cache. That is, the total number of check entries = [(disk size / 8)*(cluster_size / sizeof(uint64_t))]*[((256*1024 / 8)*8)].

[0086] In an embodiment of the present invention, for the optimization of qcow2 format disks, the corresponding L2 Table is only verified during L2 Table loading and flushing.

[0087] For example, see Figure 7 As shown, the number of entries to be verified each time is 256 * 1024 / 8 (the number of individual L2 table entries). For a 1TB disk, the total number of verified entries = L2_tables * L2_entries = (1024 / 8) * (256 * 1024 / 8) = the total number of L2 table entries, meaning that only one L2 entry needs to be verified.

[0088] Therefore, virtual machine (VM) migration across storage is a frequent operation for data center operations personnel. Systems like databases, HIS (Hospital Information System), and PACS (Picture Archiving and Communication System) generate numerous small files and suffer from severe fragmentation as business operations continue. When storage needs maintenance, storage capacity is insufficient, or to prevent excessive load on a particular storage pool, VMs need to be migrated from one storage to another. When business operations require storage adjustments, online VM migration across storage can severely impact performance. This invention optimizes functions that are particularly time-consuming for online cross-storage migration, improving efficiency, reducing the impact on VM performance, and enhancing the user experience.

[0089] It should be noted that the seek logic in cross-storage migration of raw format disks in this invention is optimized. After performing a seek of data and hole, the corresponding position is recorded. Before performing the next seek, it is checked whether the current offset is less than the position of the hole. If it is less, the seek is stopped, all data between data and hole is transferred, and then the next seek is performed. This improves the efficiency of seek.

[0090] The Qcow2 L2 Entry verification logic for calculating the total data volume of qcow2 format disk migration has been optimized. The corresponding L2 Table is only verified during L2 Table loading and flushing, meaning the number of entries verified each time is 256*1024 / 8 (the number of entries in a single L2 table). This improves verification efficiency, as L2 entries only need to be verified once.

[0091] A second aspect of the present invention also provides a performance optimization system for online cross-storage migration of virtual machines, the system comprising:

[0092] The raw disk optimization module is configured to be installed in the chip architecture for optimizing raw format disks. During optimization, after performing a seek of data and hole, the corresponding positions are recorded. Before performing the next seek, it is determined whether the current offset is less than the hole position. If it is less, the seek is stopped, all data between data and hole is transferred, and the next seek is performed. If a hole is created in the data area, the previously cached data and hole positions are discarded, and the seek is performed again in the next iteration. If the seek area is empty, the current hole and data positions are not retained.

[0093] In the raw disk optimization module, when optimizing raw format disks, if bdrv_get_dirty_count is used to count the number of data blocks that need to be migrated, after performing a seek of data and hole, the corresponding position is recorded. Before performing the next seek, it is determined whether the current offset is less than the position of hole. If it is less, the seek is stopped, all data between data and hole is transferred, and then the next seek is performed.

[0094] The raw disk optimization module is used to optimize qcow2 format disks, and only verifies the corresponding L2 Table when loading and flushing the L2 Table.

[0095] When optimizing the raw disk optimization module for qcow2 format disks, specifically optimizing the Qcow2 L2 Entry verification logic for calculating the total amount of data migrated from qcow2 format disks, the current logic for optimizing qcow2 format disks is as follows: when calculating the position of each offset on the qcow2 disk or when flushing the L2 Table, all L2 entries in the L2 table in memory are verified. When calculating the total amount of migrated data, the total number of verification entries is: the total number of L2 table entries * [(256*1024 / 8)*8].

[0096] The third aspect of the embodiments of the present invention is described in [reference 1]. Figure 8 As shown, a computer device 400 is also provided, including a memory 420 and a processor 410. The memory stores a computer program, which, when executed by the processor, implements the method of any of the above embodiments. The method includes two parts: optimization for raw format disks and optimization for qcow2 format disks. The optimization for raw format disks includes:

[0097] After performing a seek data and hole operation, record the corresponding position.

[0098] Before executing the next seek, check if the current offset is less than the position of the hole. If it is less, stop seeking, transfer all data between the data and the hole, and then proceed with the next seek.

[0099] If a hole is punched in the data area, the previously cached data and hole positions are discarded, and the seek will be executed again next time.

[0100] If the seek region is empty, the current hole and data positions are not retained;

[0101] The optimizations for qcow2 formatted disks include:

[0102] The corresponding L2 Table is validated only when the L2 Table is loaded and refreshed.

[0103] A fourth aspect of the present invention also provides a computer-readable storage medium. Figure 9 This is a schematic diagram of a computer-readable storage medium for a performance optimization method of online cross-storage migration of virtual machines provided according to an embodiment of the present invention. Figure 9 As shown, the computer-readable storage medium 300 stores computer program instructions 310, which can be executed by a processor. When executed, the computer program instructions 310 implement the method of any of the above embodiments. This method includes two parts: optimization for a raw format disk and optimization for a qcow2 format disk. The optimization for the raw format disk includes:

[0104] After performing a seek data and hole operation, record the corresponding position.

[0105] Before executing the next seek, check if the current offset is less than the position of the hole. If it is less, stop seeking, transfer all data between the data and the hole, and then proceed with the next seek.

[0106] If a hole is punched in the data area, the previously cached data and hole positions are discarded, and the seek will be executed again next time.

[0107] If the seek region is empty, the current hole and data positions are not retained;

[0108] The optimizations for qcow2 formatted disks include:

[0109] The corresponding L2 Table is validated only when the L2 Table is loaded and refreshed.

[0110] It should be understood that, where there is no conflict, all the implementation methods, features, and advantages described above for the performance optimization method of virtual machine online cross-storage migration according to the present invention are equally applicable to the performance optimization system and storage medium for virtual machine online cross-storage migration according to the present invention.

[0111] like Figure 8 The diagram shown is a hardware structure schematic of an embodiment of a computer device for performing a performance optimization method for online cross-storage migration of virtual machines provided by the present invention. Figure 8 Taking the computer device 400 shown as an example, this computer device includes a processor 410 and a memory 420, and may also include an input device 430 and an output device 440. The processor 410, memory 420, input device 430, and output device 440 can be connected via a bus or other means. Figure 8 Taking a bus connection as an example, input device 430 can receive input digital or character information, and generate signal inputs related to performance optimization for online virtual machine cross-storage migration. Output device 440 may include display devices such as a display screen.

[0112] Memory 420, as a non-volatile computer-readable storage medium, can be used to store non-volatile software programs, non-volatile computer-executable programs, and modules, such as the program instructions / modules corresponding to the resource monitoring method in this embodiment. Memory 420 may include a program storage area and a data storage area, wherein the program storage area may store the operating system and application programs required for at least one function; the data storage area may store data created by the use of the resource monitoring method, etc. In addition, memory 420 may include high-speed random access memory and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other non-volatile solid-state storage device. In some embodiments, memory 420 may optionally include memory remotely located relative to processor 410, and these remote memories can be connected to the local module via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0113] The processor 410 executes various server functions and data processing by running non-volatile software programs, instructions, and modules stored in the memory 420, thereby implementing the resource monitoring method of the above method embodiment.

[0114] The various exemplary logic blocks, modules, and circuits described herein can be implemented or performed using the following components designed to perform the functions herein: general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs) or other programmable logic devices, discrete gate or transistor logic, discrete hardware components, or any combination of these components. A general-purpose processor may be a microprocessor, but alternatively, the processor may be any conventional processor, controller, microcontroller, or state machine. The processor may also be implemented as a combination of computing devices, such as a combination of a DSP and a microprocessor, multiple microprocessors, one or more microprocessors combined with a DSP, and / or any other such configuration.

[0115] Therefore, this invention proposes a performance optimization method and system for online cross-storage migration of virtual machines. By optimizing the disk migration logic for both raw and qcow2 formats, the impact of online cross-storage migration on virtual machine performance is reduced. This invention optimizes functions that are particularly time-consuming in online cross-storage migration, thereby improving efficiency, reducing the impact on virtual machine performance, and enhancing the user experience.

[0116] The above are exemplary embodiments disclosed in this invention. However, it should be noted that various changes and modifications can be made without departing from the scope of the embodiments of this invention as defined by the claims. The functions, steps, and / or actions of the methods according to the disclosed embodiments described herein do not need to be performed in any particular order. Furthermore, although the elements disclosed in the embodiments of this invention may be described or claimed individually, they may be understood as multiple unless explicitly limited to a singular number.

[0117] It should be understood that, as used herein, the singular form "a" is intended to include the plural form as well, unless the context clearly supports an exception. It should also be understood that, as used herein, "and / or" refers to any and all possible combinations of one or more of the associatedly listed items. The embodiment numbers disclosed above are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0118] Those skilled in the art should understand that the discussion of any of the above embodiments is merely exemplary and is not intended to imply that the scope of the invention (including the claims) is limited to these examples. Within the framework of the invention, technical features of the above embodiments or different embodiments can be combined, and many other variations of different aspects of the invention exist, which are not provided in the details for the sake of brevity. Therefore, any omissions, modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the invention should be included within the protection scope of the invention.

Claims

1. A performance optimization method for online cross-storage migration of virtual machines, characterized in that, This method comprises two parts: optimization for raw format disks and optimization for qcow2 format disks. The optimization for raw format disks includes: After performing a seek data and hole operation, record the corresponding position. Before executing the next seek, check if the current offset is less than the position of the hole. If it is, do not seek again. Then transfer all the data between the data and the hole, and then perform the next seek. If a hole is punched in the data area, the previously cached data and hole positions are discarded and the seek is re-executed; If the seek region is empty, the current hole and data positions are not retained; The optimizations for qcow2 formatted disks include: The corresponding L2 Table is validated only when the L2 Table is loaded and refreshed.

2. The performance optimization method for online cross-storage migration of virtual machines according to claim 1, characterized in that, If the seek region is a hole, the current hole and data positions are not retained, and the seek region of the hole is 1M in size.

3. The performance optimization method for online cross-storage migration of virtual machines according to claim 2, characterized in that, When validating the corresponding L2 Table, the number of L2 Table entries to be validated each time is 256 * 1024 / 8, and the total number of L2 Table entries to be validated is (1024 / 8) * (256 * 1024 / 8) = the total number of L2 Table entries. Each L2 entry is validated only once.

4. The performance optimization method for online cross-storage migration of virtual machines according to claim 3, characterized in that, The performance optimization method for online cross-storage migration of virtual machines also includes file simulation in QEMU, where file simulation is divided into two layers: The first layer is a raw / qcow2 simulation, which implements corresponding driver functions based on the characteristics of raw and qcow2 files; the qcow2 has L1 / L2 tables, and the total amount of data to be migrated is calculated before migration; the raw format disk has no metadata, and the total amount of data to be migrated is the size of the disk virtual machine; The second layer is a file simulation, which abstracts the common functions of raw and qcow2 and implements them in bdrv_file.

5. The performance optimization method for online cross-storage migration of virtual machines according to claim 4, characterized in that, QEMU online disk migration is achieved by calling the `qmp_drive_mirror` command. The migration process takes place in the `mirror_run` function, and the migration process in `mirror_run` includes: If bdrv_is_allocated_above --> bdrv_set_dirty, obtain the data to be migrated from the device. During migration, set the bit of the block in the bitmap to dirty. For qcow2 formatted disks, calculate the amount of data to be migrated. The process begins by migrating data in the bitmap that is classified as "drity" until the virtual machine meets the switching conditions. During this process, the number of data blocks to be migrated is counted, and then the data migration begins.

6. The performance optimization method for online cross-storage migration of virtual machines according to claim 5, characterized in that, The optimization for raw format disks involves using `bdrv_get_dirty_count` to count the number of data blocks that need to be migrated. After performing a seek of data and hole, the corresponding positions are recorded. Before performing the next seek, it is determined whether the current offset is less than the position of the hole. If it is, no further seek is performed, and all data between data and hole is transferred before the next seek is performed.

7. The performance optimization method for online cross-storage migration of virtual machines according to claim 5, characterized in that, The optimization for qcow2 format disks involves optimizing the Qcow2 L2 Entry verification logic for calculating the total amount of data migrated from qcow2 format disks. The current logic for optimizing qcow2 format disks is as follows: when calculating the location of each offset on the qcow2 disk or when flushing the L2 Table, all L2 entries of the L2 table in memory are verified. When calculating the total amount of migrated data, the total number of verification entries is: total number of L2 table entries * [(256 * 1024 / 8) *8].

8. The performance optimization method for online cross-storage migration of virtual machines according to claim 7, characterized in that, Optimizations for qcow2 formatted disks also include: Cluster_size=256K=256*1024 bytes, which is the cluster size in bytes; sizeof(uint64_t) = 64bit = 8 bytes, which is the size of the block device corresponding to the qcow2 format disk, in bytes; Disk L2_tables = disk size / 8, that is, the file size corresponding to each L2 table is 8GiB, and the total L2_tables corresponding to the disk is disk size / 8; L2_entries = (cluster_size / sizeof(uint64_t)), which is the number of elements in each L2 table; The total number of L2 table entries on disk = disk_tables * L2_entries = (disk size / 8) * (cluster_size / sizeof(uint64_t)), where the unit of disk size is GB; The corresponding L2 cache in memory is 2MiB, which means it caches 8 L2 tables. The number of L2 entries in the cache is ((256 * 1024 / 8) * 8). When calculating the location of each offset on the qcow2 disk or when flushing the L2 Table, all L2 entries in the L2 table in memory are checked. The total number of check entries = the total number of L2 entries on the disk * the number of L2 entries in the cache; that is, the total number of check entries = [(disk size / 8)*(cluster_size / sizeof(uint64_t))]*[((256 * 1024 / 8)*8)].

9. A performance optimization system for online cross-storage migration of virtual machines, characterized in that, include: The raw disk optimization module is used to optimize raw format disks. After performing a seek data and hole operation during optimization, it records the corresponding positions. Before executing the next seek, check if the current offset is less than the hole position. If it is, stop seeking, transfer all data between the data and the hole, and then perform the next seek. If a hole is created in the data area, discard the previously cached data and hole positions and re-execute the seek. If the seek area is empty, do not retain the current hole and data positions. The raw disk optimization module is used to optimize qcow2 format disks, and verifies the corresponding L2 Table only when the L2 Table is loaded and flushed.

Images