Virtual machine data migration method and equipment

By reading data slices to cache objects during virtual machine migration and using asynchronous write thread pool for writing, the problem of low data migration efficiency in cross-platform migration is solved, and efficient data migration is achieved.

CN119025218APending Publication Date: 2024-11-26XFUSION DIGITAL TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411045194.7
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-31
Publication Date
2024-11-26

AI Technical Summary

Technical Problem

During the cross-platform virtual machine migration process, data migration efficiency is low, resulting in low read and write efficiency and long time.

Method used

By reading the virtual machine data slices to the cache object and writing the data slices to the storage system using an asynchronous write thread pool, decoupling of read and write operations is achieved, making full use of the IO capabilities of the source and destinations, and optimizing the data migration process.

Benefits of technology

It improves the rate of virtual machine disk data migration, shortens the overall data migration time, and improves migration efficiency.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119025218A_ABST
    Figure CN119025218A_ABST
Patent Text Reader

Abstract

The embodiment of the invention provides a virtual machine data migration method and device, and relates to the technical field of servers. The method comprises the following steps: acquiring to-be-migrated data of a virtual machine at a source end; dividing the to-be-migrated data into a plurality of data slices; reading the first data slice to a first cache object of a destination end, wherein the first data slice is one of a plurality of data slices; after the first data slice is read to a first cache object of a destination end, the first data slice in the first cache object is written into a storage system of the destination end, a second data slice is read to a second cache object, and the second data slice is one of the multiple data slices. Therefore, the read operation and the write operation of the virtual machine data are decoupled through the data cache of the cache object, the two operations are executed in parallel, and the data migration efficiency is improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of server technology, and in particular to a method and device for migrating virtual machine data. Background Art

[0002] A virtual machine is a solution that provides hardware system virtualization, allowing multiple operating system instances to run in parallel on a single physical hardware. With the rapid development of cloud computing and data center technology, higher requirements are placed on the management and maintenance of virtual machines, including cross-platform migration of virtual machines.

[0003] Cross-platform migration usually involves transferring a virtual machine in operation or in a static state from one physical host to another, and the two physical hosts may use different virtualization platforms or hardware configurations. The cross-platform migration process involves the migration of virtual machine disk data. The principle of data migration is usually to divide the disk data of the virtual machine at the source end into several data slices, and then read and write them serially. Each serial read and write can only read one data slice first, and then write the data slice to the target storage system at the destination end, and then read the next data slice, ..., repeat the operation, and write these data slices one by one to the target storage system at the destination end. Therefore, during the entire data migration process, when reading data from the source end, the write bandwidth of the destination end is idle; when writing data to the destination end, the read bandwidth of the source end is idle, resulting in low read and write efficiency. To complete the reading and writing of all data slices, the entire migration process takes a very long time. Summary of the invention

[0004] The embodiments of the present application provide a virtual machine data migration method, apparatus, computing device, computer storage medium and computer program product, which can improve data migration efficiency and help shorten migration time.

[0005] In a first aspect, an embodiment of the present application provides a method for migrating virtual machine data, the method comprising: obtaining data to be migrated from a source virtual machine; dividing the data to be migrated into multiple data slices; reading a first data slice to a first cache object at a destination, the first data slice being one of multiple data slices; after reading the first data slice to the first cache object at the destination, writing the first data slice in the first cache object to a storage system at the destination, and reading a second data slice to a second cache object, the second data slice being one of multiple data slices.

[0006] In this way, the read and write data are buffered by the cache object. On the one hand, it can ensure the continuous reading of the source data and cope with the impact of changes in the write speed of the target end. On the other hand, it realizes the decoupling of the read operation and the write operation, so that the source end continues to read data outward and the target end continues to write data inward, so as to make full use of the IO capabilities of the source and target ends, and overlap the time consumption of reading data and writing data, thereby greatly improving the rate of virtual machine disk data migration and effectively shortening the overall data migration time.

[0007] Exemplarily, reading the first data slice to a first cache object at a destination end includes: determining, based on the granularity of the first data slice, a first cache object that matches the granularity of the first data slice from multiple cache objects at the destination end; and reading the first data slice to the first cache object.

[0008] In this way, after receiving each data slice, the management module 21 can match the cache object from the corresponding level cache pool according to the granularity of the data slice to speed up the efficiency of reading data.

[0009] Exemplarily, before reading the first data slice to the first cache object of the destination, the method includes: if multiple cache objects of the destination do not match the granularity of the first data slice, creating a first cache object that matches the granularity of the first data slice.

[0010] In this way, it can be ensured that all data slices have suitable cache objects, ensuring smooth migration.

[0011] Exemplarily, before reading the first data slice to the first cache object of the destination, the method includes: creating cache objects of multiple granularities in the cache pool module of the destination, wherein there are multiple cache objects of the same granularity; the multiple granularities include at least one granularity of 64K, 64M or 0-23M.

[0012] Exemplarily, after writing the first data slice in the first cache object to the storage system of the destination, the method includes: releasing the first cache object when the first data slice in the first cache object has been written to the storage system; reading the third data slice to the first cache object, the third data slice being a data slice with the same granularity as the first data slice.

[0013] In this way, when migrating large disks, frequent creation and deletion of cache objects can be effectively avoided by reusing cache objects, thereby reducing GC impact and reducing system resource consumption.

[0014] Exemplarily, after writing the first data slice in the first cache object to the storage system of the destination and reading the second data slice to the second cache object, the method includes: after multiple data slices are written to the storage system, destroying all cache objects at the destination to release corresponding memory resources.

[0015] Exemplarily, writing a first data slice in a first cache object to a storage system at a destination includes: utilizing an asynchronous write thread in a thread pool to execute writing of the first data slice in the first cache object to the storage system at the destination, and counting the total number of threads of the asynchronous write thread; if the total number of threads reaches the maximum number of threads in the thread pool, placing write tasks that are not assigned to corresponding threads into a task queue to wait for execution, wherein the write task is a task used to read the data slice to the corresponding cache object at the destination.

[0016] That is to say, the thread pool can schedule tasks based on factors such as the number of available threads. If the total number of threads reaches the maximum number of threads in the thread pool, the write tasks that are not assigned to the corresponding threads are placed in the write task queue to wait for execution.

[0017] Exemplarily, the method also includes: determining the resource occupancy rate of all threads currently in the thread pool; if all threads currently in the thread pool are in a non-idle state and the resource occupancy rate is lower than a first threshold, creating a new asynchronous write thread; if the number of threads currently in an idle state in the thread pool reaches an idle thread number threshold and the resource occupancy rate is lower than a second threshold, destroying the threads in the thread pool that are in an idle state, and the second threshold is less than the first threshold.

[0018] That is to say, the thread pool can also manage concurrently executed threads, dynamically expand and shrink the thread pool according to resource quotas, and allow multiple data migration tasks to be processed simultaneously, thereby improving the overall migration efficiency. For example, the thread pool can also determine the resource occupancy rate (including CPU, memory and other resources) of all current threads in the thread pool. If all current threads in the thread pool are in a non-idle state and the resource occupancy rate is lower than the preset first threshold, it means that the current resources are sufficient to support more IO overhead, so the thread pool can create a new asynchronous write thread to execute the write tasks queued in the task queue to speed up the migration progress. If the number of threads currently in the thread pool that are idle reaches the idle thread number threshold and the resource occupancy rate is lower than the preset second threshold (the second threshold is less than the first threshold), it means that many threads are currently idle and there are no more write tasks to be executed, so all or part of the threads in the thread pool that are idle can be destroyed to reduce consumption.

[0019] In a second aspect, an embodiment of the present application provides a virtual machine data migration device, the device comprising: an acquisition module, used to acquire the data to be migrated of the virtual machine at the source end; a processing module, used to divide the data to be migrated into multiple data slices; the processing module is also used to read the first data slice to the first cache object at the destination end, the first data slice is one of multiple data slices; the processing module is also used to write the first data slice in the first cache object to the storage system of the destination end, and read the second data slice to the second cache object, the second data slice is one of multiple data slices.

[0020] Exemplarily, the processing module is specifically used to: determine, according to the granularity of the first data slice, a first cache object matching the granularity of the first data slice from multiple cache objects at the destination end; and read the first data slice to the first cache object.

[0021] Exemplarily, the processing module is specifically used to: if multiple cache objects at the destination end do not match the first data slice granularity, create a first cache object that matches the first data slice granularity.

[0022] Exemplarily, exemplary, the processing module is specifically used to: create cache objects of multiple granularities in the cache pool module of the destination end, wherein there are multiple cache objects of the same granularity; the multiple granularities include at least one granularity of 64K, 64M or 0-23M.

[0023] Exemplarily, the processing module is specifically used to: release the first cache object when the first data slice in the first cache object has been written to the storage system; read the third data slice to the first cache object, and the third data slice is a data slice with the same granularity as the first data slice.

[0024] Exemplarily, the processing module is also used to: after multiple data slices are written to the storage system, destroy all cache objects at the destination end to release corresponding memory resources.

[0025] Exemplarily, the processing module is also used to: utilize the asynchronous write thread in the thread pool to execute writing of the first data slice in the first cache object to the storage system of the destination, and count the total number of threads of the asynchronous write thread; if the total number of threads reaches the maximum number of threads in the thread pool, the write task that is not assigned to the corresponding thread is placed in the task queue to wait for execution, and the write task is a task used to read the data slice to the corresponding cache object of the destination.

[0026] Exemplarily, the processing module is also used to: determine the resource occupancy rate of all threads currently in the thread pool; if all threads currently in the thread pool are in a non-idle state and the resource occupancy rate is lower than a first threshold, create a new asynchronous write thread; if the number of threads currently in an idle state in the thread pool reaches an idle thread number threshold and the resource occupancy rate is lower than a second threshold, destroy the threads in the thread pool that are in an idle state, and the second threshold is less than the first threshold.

[0027] In a third aspect, an embodiment of the present application provides a computing device, comprising: at least one memory for storing programs; and at least one processor for executing programs stored in the memory; wherein, when the program stored in the memory is executed, the processor is used to execute the method described in the first aspect or any possible implementation of the first aspect.

[0028] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, which stores a computer program. When the computer program runs on a processor, the processor executes the method described in the first aspect or any possible implementation of the first aspect.

[0029] In a fifth aspect, an embodiment of the present application provides a computer program product, characterized in that when the computer program product runs on a processor, the processor executes the method described in the first aspect or any possible implementation of the first aspect.

[0030] In the sixth aspect, an embodiment of the present application provides a chip, characterized in that it includes at least one processor and an interface; at least one processor obtains program instructions or data through the interface; at least one processor is used to execute program line instructions to implement the method described in the first aspect or any possible implementation method of the first aspect.

[0031] It can be understood that the beneficial effects of the second to sixth aspects mentioned above can be found in the relevant description of the first aspect mentioned above, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 It is a schematic diagram of a system architecture provided by an embodiment of the present application;

[0033] Figure 2 It is a schematic diagram of the process of cross-platform migration of virtual machine data in an embodiment of the present application;

[0034] Figure 3 It is a schematic diagram of the process of cross-platform migration of virtual machine data in an embodiment of the present application;

[0035] Figure 4 It is a flowchart of a virtual machine data migration method provided in an embodiment of the present application;

[0036] Figure 5A It is a flowchart of a virtual machine data migration method provided in an embodiment of the present application;

[0037] Figure 5B It is a flowchart of a virtual machine data migration method provided in an embodiment of the present application;

[0038] Figure 6 It is a structural diagram of a virtual machine data migration device provided in an embodiment of the present application;

[0039] Figure 7 It is a structural diagram of a computing device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0040] The term "and / or" in this article is a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. The symbol " / " in this article indicates that the associated objects are in an or relationship, for example, A / B means A or B.

[0041] The terms “first”, “second” and the like in the specification and claims herein are used to distinguish different objects rather than to describe a specific order of the objects.

[0042] In the description of the embodiments of the present application, unless otherwise specified, "multiple" means two or more than two. For example, multiple processing units refer to two or more processing units, etc.; multiple elements refer to two or more elements, etc.

[0043] To facilitate understanding of the technical solution of the present application, the technical terms involved in this document are explained below.

[0044] Virtual machine cross-platform migration: refers to migrating a virtual machine from one operating environment to another. The two environments may be different hardware platforms, operating systems, or virtualization technologies.

[0045] Disk data: refers to the data stored on the disk, including file systems, applications, user data, etc.

[0046] Source virtual machine: The source virtual machine refers to the virtual machine before migration, which runs on the source virtualization platform (source end). During the cross-platform migration of virtual machines, data will be read from the disk of the source virtual machine.

[0047] Destination virtual machine: The destination virtual machine refers to the virtual machine after migration, which runs on the destination virtualization platform (destination end). During the cross-platform migration of virtual machines, data will be written to the disk of the destination virtual machine.

[0048] Serial reading and writing: It means that only one data block can be read and written at a time. During the reading and writing process, the next operation can only be performed after the previous operation is completed.

[0049] Asynchronous writing: means that the next operation can be started without waiting for the current write operation to complete. This method can improve IO efficiency.

[0050] Thread pool module: It is a form of multi-threaded processing. During the processing, tasks are added to the queue and then automatically started after the thread is created.

[0051] IO capability: Input and output capability, usually refers to the ability of a computer system to read (input) and send (output) data to external devices.

[0052] GC (Garbage Collection): Garbage collection is an automatic memory management mechanism used to recycle memory space that is no longer used in the program.

[0053] In order to improve the data migration efficiency in the cross-platform migration scenario of virtual machines, a virtual machine data migration method is provided in an embodiment of the present application. The method mainly reads the data of the virtual machine at the source end serially into the corresponding cache object in the memory of the destination end, and then writes the data in these cache objects to the storage system of the destination end through asynchronous write operations. In this way, by decoupling the read and write operations, the efficiency of the write operation can be greatly accelerated and the overall time consumption of data migration can be reduced.

[0054] The following is an introduction to the system architecture provided by the embodiments of the present application.

[0055] For example, Figure 1 FIG. 1 is a schematic diagram of a system architecture for performing virtual machine migration provided by an embodiment of the present application. Figure 1 As shown, the figure shows a physical machine 1 and a physical machine 2 that are communicatively connected. Virtual machines (Virtua l Machine, VM) can be deployed on both physical machine 1 and physical machine 2, and a virtual machine VM running on one physical machine (source host) can be migrated to another physical machine (destination host) to run. The following takes the migration of the virtual machine VM from physical machine 1 to physical machine 2 as an example to illustrate the migration principle of the virtual machine VM.

[0056] The software layers of physical machine 1 and physical machine 2 may include: an operating system and a virtualization platform running on the operating system, wherein:

[0057] The operating system is used to control and manage the hardware and software resources on the physical machine, and reasonably organize and schedule the work of the physical machine and the allocation of resources to provide users and other software with a convenient interface and environment.

[0058] A virtualization platform is a software system that implements virtual machine (VM) management and resource allocation. It is responsible for creating, managing, monitoring, and destroying virtual machines, while providing functions such as resource scheduling, security control, and network configuration, so that the hardware resources on a physical machine can be shared by multiple virtual machines (VMs), improving resource utilization while ensuring isolation between different virtual machines. For example, but not limitation, a virtualization platform can be VMware, KVM, or a hyper-converged infrastructure (HCI).

[0059] In this embodiment, continue to refer to Figure 1 As shown, the virtualization platforms on physical machine 1 and physical machine 2 can be different. When the performance of physical machine 1 reaches a bottleneck, maintenance or decommissioning is required, the virtual machine VM running on physical machine 1 can be migrated from the current virtualization platform 1 (i.e., the source end) to the virtualization platform 2 of physical machine 2 (i.e., the destination end) to achieve the purpose of resource reallocation, hardware upgrade or disaster recovery, and ensure business continuity and stability.

[0060] In this regard, in this embodiment, in order to speed up the efficiency of cross-platform migration of virtual machines, a management system 20 can also be deployed on the destination end or destination host (i.e., the host that hosts the destination end) to optimize the management of disk data migration of the virtual machine VM during the cross-platform migration of the virtual machine VM.

[0061] Specifically, the management system 20 may include a management module 21, a thread pool module 22, and a cache pool module 23. The management module 21 may be used to obtain data slices from the disk of the virtual machine VM from the source end, and serially read these data slices into the cache object (buffer) of the destination end memory, and then write the data slices in the cache object into the storage system of the destination end (the physical storage space of the storage system is provided by the storage device of the destination host) through an asynchronous write operation. In this way, in the process of cross-platform data migration of virtual machines, by decoupling the read operation and the write operation, data can be written while reading data, thereby improving the efficiency of data migration. In addition, the thread pool module 22 may be used to provide and manage threads to execute the write tasks issued by the migration management module 21, and the cache pool module 23 may be used to provide the above-mentioned cache objects.

[0062] Next, based on Figure 1 The system architecture shown in FIG. 1 illustrates the principle of the management system 20 performing virtual machine disk data migration in a virtual machine cross-platform migration scenario. Figure 2 The diagram shows a flow chart of the management system 20 performing virtual machine disk data migration in a virtual machine cross-platform migration scenario. Figure 2As shown, the virtual machine VM on the physical machine 1 runs on the source end (VMware), and the physical machine 1 can send a migration request to the physical machine 2. The management system 20 responds to the migration request and starts the data migration process to migrate the data in the single disk 101 (the storage device belonging to the physical machine 1) of the virtual machine VM to the storage system of the destination end (HCI).

[0063] In this example, the data migration process is as follows: a single disk 101 includes multiple data blocks (also referred to as data to be migrated in this article, the same below), and the granularity of the block can be 0-2G. The physical machine 1 can determine the valid data in these blocks (such as operating system files, application files, user-generated data, etc. required for the operation of the virtual machine VM) through step 1 to perform segmented migration. When the management module 21 finishes reading the first data from the source end, it triggers the execution of optimized management of data migration. Specifically, the management module 21 instructs the physical machine 1 to execute step 2 to divide the valid data of each block on the single disk 101 into several data slice chunks with a granularity of 0-23M. Then the management module 21 obtains these chunks from the source end, and reads these chunks serially into the corresponding cache objects in the cache pool module (pool) 23 of the target end memory, wherein serial reading means that after reading a chunk into a cache object, it returns to read the next chunk without waiting for whether the chunk is written to the storage system of the target end. Next, the management module 21 executes step 4 for the cache object into which the chunk has been read, and asynchronously writes the chunk in the cache object into the storage system 201 of the destination. Asynchronous writing means that when a certain write operation is performed, the execution of the read task or other write tasks is not blocked. In this way, since the rate of reading data is often faster than the rate of writing data, in the data migration process, the management module 21 supports that in this scenario of fast reading and slow writing, each chunk is first read to the corresponding cache object in the cache pool module 23 at a faster speed, and synchronously executed is that the chunks in these cache objects are concurrently written to the storage system 201 of the destination through multiple asynchronous write tasks. In other words, the read and write data are buffered by the cache object, which can, on the one hand, ensure the continuous reading of the source data and cope with the influence of the change of the write speed of the target end. On the other hand, the decoupling of the read operation and the write operation is realized, so that the source end continuously reads data outward and the destination end continuously writes data inward, so as to make full use of the IO capabilities of the source end and the destination end, so that the time consumption of reading data and writing data overlaps, thereby greatly improving the rate of virtual machine disk data migration and effectively shortening the overall data migration time.

[0064] Next, based on the above description, a method for virtual machine cross-platform data migration provided by an embodiment of the present application is introduced. It can be understood that the method is proposed based on the above description, and part or all of the content of the method can refer to the above description.

[0065] See also Figure 3 , Figure 3 1 is a flowchart of a method for migrating data across virtual machines provided in an embodiment of the present application. It can be understood that the method can be executed by any device, equipment, platform, or device cluster with computing and processing capabilities. Figure 1 The execution on the physical machine 2 shown in the figure is taken as an example for explanation. Figure 3 As shown, the method may include:

[0066] In S301, the data to be migrated of the virtual machine on the source side is obtained.

[0067] In this embodiment, when the virtual machine VM is to be migrated from physical machine 1 to physical machine 2 for operation, physical machine 2, as the destination host, can obtain the data to be migrated in a single disk of the virtual machine VM from the source end of physical machine 1. The data to be migrated can mainly include operating system files, application files, user-generated data and other valid data required for the operation of the virtual machine VM.

[0068] Exemplarily, the data to be migrated may include multiple data blocks, and the granularity of a single data block is 0-2 GB. In this step, the data to be migrated may be sent from the source end to the physical machine 2 in blocks.

[0069] S302: Divide the data to be migrated into multiple data slices.

[0070] In this example, after receiving the data to be migrated, physical machine 2 may divide the data into multiple data slices with a granularity of 0-23M.

[0071] S303: Read a first data slice to a first cache object at a destination end, where the first data slice is one of a plurality of data slices.

[0072] In this embodiment, it can be combined with Figure 4As shown, on physical machine 2, cache pool module 23 can pre-create multiple levels of cache objects from the memory of the destination end, and the cache space size (i.e., granularity) of cache objects of the same level is the same, and these cache objects of different levels can be pre-created according to the normal situation of reading data. For example, the maximum one-time read block granularity supported by the source end is 64M, in other words, the granularity of most data slices is 64M, thus, a cache pool module 23a can be created to provide multiple cache objects with a cache space of 64M, and a cache pool module 23b is also created to provide multiple cache objects with a cache space of 64k, and a cache pool module 23c is created to provide multiple cache objects with a cache space of 0-23M, and so on.

[0073] In this way, after the physical machine 2 obtains the above-mentioned multiple data slices, when the management module 21 in the physical machine 2 reads one of the data slices (i.e., the first data slice), it can match the cache object (i.e., the first cache object) from the corresponding level cache pool module according to the granularity of the data slice to speed up the efficiency of reading data. Then, the management module 21 can read the first data slice into the first cache object.

[0074] Still Figure 4 As shown in the example, when the management module 21 reads the data slice 2, step S1 is executed to match a buffer 11 with a 64k cache space from the cache pool module 23b for the 64k data slice 2, and then step S2 is executed to read the data slice 2 into the buffer 11.

[0075] Exemplarily, a data slice may also be read into a cache object whose cache space is not less than the granularity of the data slice to ensure that the data slice can be completely placed in the cache object. As a specific example, the granularity of the data slice may be equal to the cache space size of the cache object to fully utilize memory resources.

[0076] In addition, if there is any data slice that has no matching cache object in the cache pool module 23, the management module 21 can notify the cache pool module 23 to create a corresponding cache object.

[0077] In this embodiment, the management module 21 reads the data slices to the cache object in a serial read operation, that is, after the management module 2 reads a data slice to the corresponding cache object, it returns to read the next data slice, thereby reading these data slices one by one to the cache object of the corresponding destination memory. It can be understood that the destination and the source can be different virtualization platforms.

[0078] S304: After reading the first data slice into the first cache object at the destination, write the first data slice in the first cache object into the storage system at the destination, and read the second data slice into the second cache object.

[0079] In this embodiment, when the management module 21 reads the first data slice into the first cache object, the thread pool module 22 can be notified to create an asynchronous write task associated with the first cache object, and the asynchronous write task is to write the first data slice in the first cache object to the storage system of the destination. Then, the thread pool module 22 assigns the asynchronous write task to an asynchronous write thread for execution, thereby writing the first data slice read from the first cache object to the storage system of the destination. Since writing the first data slice in the first cache object to the storage system of the destination is an asynchronous write task, that is, when executing this write task, the execution of other read tasks and write tasks is not blocked. Therefore, synchronously with this, the management module 21 reads the second data slice to the second cache object, and the second data slice is a slice read after the first data slice.

[0080] Still Figure 4 As shown in the example, the management module 21 reads the data slice 2 into the buffer 11, and then executes step S3 to notify the thread pool module 22 to create an asynchronous write task 2 associated with the buffer 11, and return to read the next data slice 3. Then, the thread pool module 22 can execute step S4 to assign the write task 2 to the asynchronous write thread 2 for execution, so as to write the data slice 2 in the buffer 11 to the storage system of the destination. After the write is completed, the buffer 11 is released back to the cache pool module 23b to support subsequent reuse. In this way, when migrating large disks, the frequent creation and deletion of cache objects can be effectively avoided by reusing cache objects, thereby reducing the impact of GC and reducing system resource consumption.

[0081] In this way, after being buffered by the cache object, the source data can continuously read data slices to the destination memory, and the destination can also write the data slices read from the memory in parallel to the storage system. The read and write operations are decoupled, so that the time spent reading and writing data overlaps, thereby greatly improving the rate of virtual machine disk data migration and effectively shortening the overall data migration time.

[0082] For example, Figure 5A The flowchart of a method for migrating data of a virtual machine across platforms provided by the present application is shown in FIG5A. The process shown in FIG5A can be executed on the destination host. Figure 5A The process shown is Figure 4 A further refinement of the process shown.

[0083] like Figure 5A As shown, the method may include:

[0084] S501, obtaining data to be migrated and starting the data migration process.

[0085] In this example, physical machine 1 may request physical machine 2 to migrate virtual machine VM and send data to be migrated. Then, in response to the request, physical machine 2 may receive the data to be migrated and start a cross-platform migration process for the virtual machine VM.

[0086] The data to be migrated is from a single disk of the virtual machine VM. Exemplarily, when data is migrated from a single disk of the virtual machine VM, it can be migrated in sections, such as by block.

[0087] In S502, the data to be migrated is divided into a plurality of data slices.

[0088] In this example, the execution principle of S502 is similar to the principle described for S302 in the above embodiment, and will not be repeated here.

[0089] Furthermore, the data slice may be valid data in the disk of the virtual machine VM, that is, the data slice may be valid data partitioned and cut out from all the data to be migrated in the disk, so as to perform subsequent migration operations.

[0090] In addition, during the entire process of obtaining data slices from the source, the management module 21 can also execute S5021 to continuously monitor whether all data slices have been successfully migrated to the destination storage system. If so, the current process is terminated; if not, the following S503 step is continued.

[0091] S503: Read the first data slice to the first cache object at the destination.

[0092] In this example, the execution principle of S503 is similar to the principle described in the above embodiment for S303, which will not be repeated here. And, as a specific example, S503 may include S5031 to S5033:

[0093] S5031, according to the granularity of the first data slice, determine a first cache object matching the granularity of the first data slice from multiple cache objects at the destination end;

[0094] In this step, based on the cache objects of multiple granularities pre-created by the cache pool module 23 before this step or before this process, the multiple granularities include at least one granularity of 64K, 64M or 0-23M. In this way, when the management module 21 reads the first data slice serially, it can determine the first cache object that matches the granularity of the first data slice from all cache objects at the destination end according to the granularity of the first data slice.

[0095] S5032: If the multiple cache objects at the destination end do not match the first data slice granularity, create the first cache object that matches the first data slice granularity.

[0096] In this step, if there is no cache object of a size matching the first data slice in the cache pool module 23, a first cache object matching the granularity of the first data slice can be created to ensure that each data slice can be read into the cache pool module normally.

[0097] S5033, read the first data slice to the first cache object.

[0098] In this step, after determining the first cache object, the management module 21 can read the first slice data into the first cache object.

[0099] In this embodiment, the management module 21 may perform a serial read operation to read data slices into a cache object, that is, one data slice must be read before the next data slice can be read, and each data slice is eventually read into a corresponding cache object.

[0100] In some possible implementations, if the physical machine 2 is a multi-core processor, multiple serial read tasks may be performed in parallel, and each core actually still executes an independent serial read task stream.

[0101] S504, after reading the first data slice into the first cache object of the destination, write the first data slice in the first cache object into the storage system of the destination, and read the second data slice into the second cache object.

[0102] In this example, the execution principle of S504 is similar to the principle described for S304 in the above embodiment, that is, after the management module 21 reads the first slice into the first cache object, on the one hand, it notifies the thread pool module 22 to create a corresponding asynchronous write task to write the first data slice in the first cache object to the storage system of the destination end, and on the other hand, it returns to execute the write task of reading the second data slice to the second cache object. In this way, the management system 20 performs the serial read task of all data slices in parallel with the write task of writing the data in each cache object into the storage system, which can greatly shorten the overall data migration time.

[0103] That is to say, the management system 20 decouples the serial read task of all data slices from the write task of writing the data in each cache object to the storage system, fully utilizes the IO capabilities of the source and destination ends, and overlaps the time consumption of reading data and writing data, thereby greatly improving the rate of virtual machine disk data migration and effectively shortening the overall data migration time.

[0104] In some possible implementations, in step S504, after the management module 21 reads the first data slice to the first cache object of the destination, it notifies the thread pool module 22 to generate an asynchronous write task associated with the first cache object, so as to write the first data slice in the first cache object to the storage system of the destination, and read the second data slice to the second cache object at the same time. Therefore, in order to improve the efficient management of IO operations during data migration, control concurrency, and reduce the consumption of system resources, optionally, in this example, after the thread pool module 22 creates the asynchronous write task associated with the first cache object, the write task can be assigned to an asynchronous write thread for execution, that is, when the asynchronous write thread executes the asynchronous write task, it does not block the execution of other write threads or read threads, so that the serial read task of the management module 21 and the asynchronous write tasks associated with other cache objects can be executed in parallel, thereby improving the migration efficiency.

[0105] Exemplarily, the management module 21 notifies the thread pool module 22 to create an asynchronous write task (which can also be understood as an asynchronous write task issued by the management module 21) which can be placed in a write task queue. The thread pool module 22 manages multiple parallel asynchronous write threads to take tasks from the task queue and execute them, thereby reducing the overhead of thread creation and destruction by reusing write threads, thereby reducing resource consumption and improving IO processing efficiency.

[0106] In addition, optionally, to improve system stability, you can refer to Figure 5B As shown, after S504, the method may further include:

[0107] S506, counting the total number of asynchronous writing threads;

[0108] S507, if the total number of threads reaches the maximum number of threads in the thread pool module, the write tasks that are not assigned to the corresponding threads are placed in the task queue to wait for execution. The write tasks are used to read the data slices to the corresponding cache objects at the destination.

[0109] In this example, because data is read quickly and written slowly, after multiple data slices have been serially written to the corresponding cache objects, the data slice in the cache object of the first data slice has not been completely written to the storage system. Therefore, the thread pool module 22 will generate corresponding asynchronous write tasks for these cache objects and put them into the write task queue, and manage multiple parallel asynchronous write threads to take tasks from the write task queue and execute them. During this period, the thread pool module 22 continues to count the total number of threads of all asynchronous write threads in the current pool. In order to prevent system crashes caused by resource exhaustion, the thread pool module 22 compares the total number of threads counted with the preset maximum number of threads. If the total number of threads reaches the maximum number of threads of the thread pool module, the newly created write task that has not been assigned to the corresponding thread can be put into the task queue, waiting for the asynchronous write thread in the current pool to complete the current task and then take the task from the task queue.

[0110] Optionally, in some possible implementations, after executing S504, the method may further include:

[0111] S508, determining the resource occupancy rate of all threads currently in the thread pool module.

[0112] In this step, the thread pool module 22 can also count the resource occupancy rates (such as CPU, memory and other resources) of all threads (mainly asynchronous write threads) in the current pool. It should be understood that the resource occupancy rates of threads can be implemented using some mature operating system tools or APIs, which will not be repeated here.

[0113] S509: If all threads in the thread pool module are currently in a non-idle state and the resource occupancy rate is lower than a first threshold, a new asynchronous write thread is created.

[0114] In this step, if all threads in the thread pool module 22 are currently in a non-idle state (for example, they are all executing write tasks) and the statistical resource occupancy rate is lower than the preset first threshold, it indicates that the current resources support more IO overhead and allow capacity expansion, that is, creating new asynchronous write threads to speed up the processing of backlogged write tasks in the task queue, thereby improving data migration efficiency while ensuring system stability.

[0115] S510: If the number of threads currently in an idle state in the thread pool module reaches an idle thread number threshold and the resource occupancy rate is lower than a second threshold, the threads in the idle state in the thread pool module are destroyed.

[0116] In this step, if the number of idle threads in the thread pool module 22 reaches the idle thread number threshold and the statistical resource occupancy rate is lower than the preset second threshold (the second threshold is less than the first threshold), it indicates that the current write tasks are relatively small and can be reduced in capacity, that is, some or all of the idle threads in the thread pool module 22 are destroyed to release the resources occupied by these threads.

[0117] In this embodiment, again refer to Figure 5A As shown, after executing S504, in addition to performing thread management through the above S505 to S510, the method further includes:

[0118] S512, when the data slices in the first cache object have been written into the storage system, the cache pool module of the first cache object is released to the memory.

[0119] Optionally, a third data slice may be further read into the first cache object, where the third data slice is a data slice having the same granularity as the first data slice.

[0120] In this example, after the data slice read into the first cache object has been written to the storage system, the management module 21 can release the first cache object to the cache pool module 23 of the memory, so that when a data slice (i.e., the third data slice) with a granularity matching the cache object is subsequently read, the third data slice can be read into the first cache object, thereby realizing the reuse of cache objects, so as to effectively avoid the frequent creation and deletion of cache objects, thereby reducing the impact of GC and reducing system resource consumption.

[0121] In addition, the management module 21 may also execute S513, and continuously monitor whether all data slices have been successfully migrated to the destination storage system. If so, the current process is terminated; if not, the process continues to return to execute step S503.

[0122] When all data slices have been successfully migrated to the destination storage system, the cross-platform migration of the virtual machine is completed, and the buffer objects in the buffer pool 23 are centrally destroyed to release memory resources.

[0123] Based on the method in the above embodiment, the present application embodiment provides a virtual machine data migration device. Figure 6 , Figure 6 It is a structural schematic diagram of a device for migrating data provided in an embodiment of the present application.

[0124] like Figure 6As shown, the device 600 may include: an acquisition module 601, used to acquire the data to be migrated of the virtual machine of the source end from the source end. A processing module 602, used to divide the data to be migrated into multiple data slices. The processing module 602 is also used to read the first data slice to the first cache object of the destination end, wherein the first data slice is one of multiple data slices, and the destination end and the source end are different virtualization platforms. The processing module 602 is also used to write the first data slice in the first cache object to the storage system of the destination end after reading the first data slice to the first cache object of the destination end, and read the second data slice to the second cache object, and the second data slice is one of multiple data slices.

[0125] It should be understood that the above-mentioned device 600 is used to execute the method for migrating data of a virtual machine across platforms in the above-mentioned embodiment. The implementation principle and technical effect of the corresponding program module in the device 600 are similar to those described in the above-mentioned method. The working process of the device can refer to the corresponding process in the above-mentioned method, and will not be repeated here.

[0126] Exemplarily, the device 600 may be implemented as Figure 1 The management system 20 shown in the figure, and the acquisition module 601 and the processing module 602 can respectively execute different steps in the virtual machine data migration method to realize all the functions of the management system 20.

[0127] Based on the method in the above embodiment, an embodiment of the present application provides an electronic device. The electronic device may include: at least one memory for storing programs; at least one processor for executing the programs stored in the memory; wherein, when the programs stored in the memory are executed, the processor is used to execute the method in the above embodiment.

[0128] Based on the method in the above embodiment, an embodiment of the present application provides a computer-readable storage medium, which stores a computer program. When the computer program runs on a processor, the processor executes the method in the above embodiment.

[0129] Based on the method in the above embodiment, an embodiment of the present application provides a computer program product, characterized in that when the computer program product runs on a processor, the processor executes the method in the above embodiment.

[0130] Based on the method in the above embodiment, the present application embodiment also provides a computing device. Figure 7 , Figure 7 This is a schematic diagram of the structure of a chip provided in an embodiment of the present application. Figure 7 As shown, the computing device 900 includes one or more processors 901 and a memory 902. Optionally, the computing device 900 may also include a bus 903. Among them:

[0131] Processor 901 is a central processing unit (CPU). CPU is the core component of a physical machine, responsible for executing calculations and instruction operations, and can specifically adopt a multi-core processor to provide higher processing power and concurrency to implement each step of the above method. In addition, the above-mentioned processor 901 can also be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, transistor logic devices, hardware components or any combination thereof. A general-purpose processor can be a microprocessor or any conventional processor.

[0132] The memory 902 may include a read-only memory and a random access memory, and provides operation instructions and data to the processor. A portion of the memory may also include a non-volatile random access memory (NVRAM).

[0133] Optionally, the memory stores executable software modules or data structures, and the processor can perform corresponding operations by calling operation instructions stored in the memory (the operation instructions can be stored in the operating system).

[0134] The memory 902 may be a random access memory (RAM), a flash memory, a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a register, a hard disk, a mobile hard disk, a CD-ROM, or any other form of storage medium known in the art. An exemplary storage medium is coupled to the processor so that the processor can read information from the storage medium and write information to the storage medium. Of course, the storage medium may also be a component of the processor. The processor and the storage medium may be located in an ASIC.

[0135] The interface circuit 902 can be used to send or receive data, instructions or information. The processor 901 can use the data, instructions or other information received by the interface circuit 902 to process, and can send the processing completion information through the interface circuit 902.

[0136] It should be noted that the functions corresponding to the processor 901 and the interface circuit 902 can be implemented through hardware design, software design, or a combination of hardware and software, which is not limited here.

[0137] It should be understood that each step of the above method embodiment can be completed by a hardware-based logic circuit or a software-based instruction in a processor.

[0138] It is understandable that the size of the sequence number of each step in the above embodiment does not mean the order of execution, and the execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the present application. In addition, in some possible implementations, each step in the above embodiment can be selectively executed according to actual conditions, and can be partially executed or fully executed, which is not limited here.

[0139] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium or transmitted through the computer-readable storage medium. The computer instructions may be transmitted from a website site, a computer, a server or a data center to another website site, a computer, a server or a data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium that a computer can access or a data storage device such as a server or a data center that includes one or more available media integrated. The available medium may be a magnetic medium, (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive (SSD)), etc.

[0140] It should be understood that the various numerical numbers involved in the embodiments of the present application are only used for the convenience of description and are not used to limit the scope of the embodiments of the present application.

Claims

1. A method for virtual machine data migration, characterized in that: The method comprises: Obtain the data of the virtual machine to be migrated on the source side; Dividing the data to be migrated into multiple data slices; Reading a first data slice to a first cache object at a destination end, where the first data slice is one of the multiple data slices; After reading the first data slice to the first cache object of the destination, writing the first data slice in the first cache object to the storage system of the destination, and reading the second data slice to the second cache object, the second data slice being one of the multiple data slices.

2. The method according to claim 1, characterized in that The step of reading the first data slice to the first cache object at the destination end includes: According to the granularity of the first data slice, determining the first cache object matching the granularity of the first data slice from the multiple cache objects at the destination end; The first data slice is read into the first cache object.

3. The method according to claim 2, characterized in that Before reading the first data slice to the first cache object at the destination end, the method includes: If the multiple cache objects at the destination end do not match the first data slice granularity, then create the first cache object that matches the first data slice granularity.

4. The method according to claim 1 or 2, characterized in that: Before reading the first data slice to the first cache object at the destination end, the method includes: Creating cache objects of multiple granularities in the cache pool module of the destination end, wherein there are multiple cache objects of the same granularity; The multiple granularities include at least one granularity of 64K, 64M, or 0-23M.

5. The method according to any one of claims 1 to 4, characterized in that: After writing the first data slice in the first cache object to the storage system of the destination end, the method includes: When the first data slice in the first cache object has been written into the storage system, releasing the first cache object; A third data slice is read into the first cache object, where the third data slice is a data slice having the same granularity as the first data slice.

6. The method according to any one of claims 1 to 5, characterized in that: After writing the first data slice in the first cache object to the storage system of the destination end and reading the second data slice to the second cache object, the method includes: After the multiple data slices are written into the storage system, all cache objects of the destination are destroyed to release corresponding memory resources.

7. The method according to any one of claims 1 to 6, characterized in that: The step of writing the first data slice in the first cache object to the storage system of the destination end includes: Using an asynchronous write thread in a thread pool, writing the first data slice in the first cache object to the storage system of the destination end is performed, Count the total number of threads of the asynchronous writing thread; If the total number of threads reaches the maximum number of threads in the thread pool, the write tasks that are not assigned to the corresponding threads are placed in the task queue to wait for execution. The write tasks are tasks for reading the data slices to the corresponding cache objects at the destination.

8. The method according to claim 7, characterized in that The method further comprises: Determine the resource occupancy rate of all threads currently in the thread pool; If all threads in the thread pool are currently in a non-idle state and the resource occupancy rate is lower than a first threshold, a new asynchronous write thread is created; If the number of threads currently in an idle state in the thread pool reaches an idle thread number threshold and the resource occupancy rate is lower than a second threshold, the threads in the idle state in the thread pool are destroyed, and the second threshold is lower than the first threshold.

9. A virtual machine data migration device, characterized in that: The device comprises: An acquisition module is used to acquire the data of the virtual machine to be migrated at the source end; A processing module, used for dividing the data to be migrated into a plurality of data slices; The processing module is further configured to read a first data slice to a first cache object at a destination end, wherein the first data slice is one of the multiple data slices; The processing module is further configured to write the first data slice in the first cache object into the storage system of the destination end, and read the second data slice into the second cache object, where the second data slice is one of the multiple data slices.

10. A computing device, characterized in that include: at least one memory for storing a program; at least one processor, configured to execute the program stored in the memory; Wherein, when the program stored in the memory is executed, the processor is used to execute the method according to any one of claims 1-9.