Virtual machine disk cross-cloud migration method, electronic device and storage medium

Abstract

The embodiment of the present application relates to the field of cloud computing technology, and discloses a virtual machine disk cross-cloud migration method, electronic device and storage medium. The method is applied to data migration service, and the data migration service is used to migrate the source virtual machine disk data on the source platform to the destination virtual machine disk file on the destination platform. The method includes the following steps: S1: obtaining at least one data block in the source virtual machine disk from the source platform; S2: determining the storage location of the at least one data block currently obtained in the destination virtual machine disk file according to the data storage structure of the destination virtual machine disk file on the destination platform; S3: writing the at least one data block currently obtained to the storage location; and repeatedly executing S1 to S3 until all data blocks in the source virtual disk are written to the destination virtual machine disk file. The disk migration is realized in the form of a stream, which saves temporary storage space and greatly saves storage overhead and time overhead.

Description

Technical Field

[0001] The present invention relates to the field of cloud computing technology, and in particular to a virtual machine disk cross-cloud migration method, electronic device and storage medium. Background Art

[0002] In the cross-cloud migration of virtual machines, the migration of virtual machine disks is the most important step. Since different cloud platforms use different virtualization technologies, the formats of disk files on which virtual machine disks are based are different. Therefore, the migration of virtual machine disks across clouds often involves data migration and format conversion.

[0003] For example, the vmware platform uses vmdk format disk files, and the openstack platform uses raw format disk files. In the prior art, in the scenario of migrating a virtual machine from the vmware platform to the openstack platform, the source vmdk file (source disk file) is first downloaded to a local temporary vmdk file, and then the vmdk file is converted into a temporary raw file using the qemu-img tool, and then the raw file is uploaded to the rbd of the ceph cluster used by the openstack platform for storage, and finally the temporary vmdk file and raw file are deleted to complete the disk migration.

[0004] This virtual machine disk solution used in the industry requires temporary storage to save temporary disk data. However, since the disk data is usually very large, there are problems of high storage overhead and time consumption. Summary of the invention

[0005] The purpose of the present invention is to solve the above problems and provide a method for cross-cloud migration of virtual machine disks, an electronic device and a storage medium, which solve the problems of data migration of virtual disk files, high storage overhead and high time consumption.

[0006] To solve the above problems, an embodiment of the present application provides a method for cross-cloud migration of a virtual machine disk, which is applied to a data migration service. The data migration service is used to migrate the source virtual machine disk data on a source platform to a destination virtual machine disk file on a destination platform. The method includes the following steps: S1: obtaining at least one data block in the source virtual machine disk from the source platform; S2: determining the storage location of the at least one data block currently obtained in the destination virtual machine disk file based on the data storage structure of the destination virtual machine disk file on the destination platform; S3: writing the at least one data block currently obtained into the storage location; and repeatedly executing S1 to S3 until all data blocks in the source virtual disk are written into the destination virtual machine disk file.

[0007] To solve the above problems, an embodiment of the present application also provides an electronic device, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the above-mentioned virtual machine disk cross-cloud migration method.

[0008] To solve the above problems, an embodiment of the present application further provides a computer-readable storage medium storing a computer program, which implements the above virtual machine disk cross-cloud migration method when executed by a processor.

[0009] In an embodiment of the present application, block data obtained from the source virtual disk file is written to the corresponding position in the destination virtual machine disk file based on the data composition structure of the destination virtual machine disk file, thereby realizing the migration of the virtual machine's disk data from one platform to another. The disk migration is realized in the form of a stream, eliminating temporary storage space, greatly saving storage overhead and time overhead, and improving user experience.

[0010] In addition, the source platform type includes: vmware platform; the target platform type includes: openstack platform. Taking the vmware platform as the source platform, the source platform data block can be quickly read through the vdkk tool, making the virtual machine disk cross-cloud migration process faster.

[0011] In addition, S2 includes: determining the block number of each currently acquired data block, and determining the storage position of each currently acquired data block in the destination virtual machine disk file according to the storage structure and block number of the raw format virtual machine disk file in the destination virtual machine disk. Numbering each acquired data block, and determining the position according to the storage structure of the disk file in the destination virtual machine disk file, makes the disk migration process smoother and more orderly, and the sequence number and storage position correspond one-to-one, avoiding disk storage errors that cause virtual machine disk cross-cloud migration to be blocked, thereby reducing the time consumed in the migration process.

[0012] In addition, the human-computer interaction interface can display the current progress of migrating the data of the source virtual machine disk to the target virtual machine disk file in real time; the progress is determined based on the total number of blocks in the source virtual machine disk and the number of blocks currently written to the target virtual machine disk file. The progress of data migration can be calculated in real time based on the total number of data blocks and the number of migrated data blocks. The human-computer interaction interface can more intuitively display the progress of current data migration.

[0013] In addition, before S1, the size of the source virtual machine disk is obtained; according to the size of the source virtual machine disk and the format of the target platform disk file, a storage area of ​​appropriate size is created on the target platform as the storage backend for the target virtual machine disk file. By creating an rbd block of the same size as the disk file on the target platform as the storage backend for the migrated disk data, the problem of using temporary storage files during the migration process, which consumes a lot of temporary storage space and storage time, is avoided. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] One or more embodiments are exemplarily described by pictures in the corresponding drawings, and these exemplified descriptions do not constitute limitations on the embodiments. Elements with the same reference numerals in the drawings represent similar elements, and unless otherwise stated, the figures in the drawings do not constitute proportional limitations.

[0015] Figure 1 This is a flow chart of a method for cross-cloud migration of virtual machine disks provided in one embodiment of the present application;

[0016] Figure 2 This is a flowchart of an implementation method of a virtual machine disk cross-cloud migration method provided by an embodiment of the present application;

[0017] Figure 3 A schematic diagram of the structure of an electronic device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0018] As can be seen from the background technology, in order to solve the problem of virtual machine disk migration, the disk data obtained from the source platform is usually placed in a temporary storage file, and then the disk data stored in the temporary file is converted into the file format of the target platform, and finally the temporary file of the converted format is uploaded to the target platform, so as to complete the disk migration. Although the disk migration can be achieved through the above-mentioned temporary storage and format conversion process, since the amount of disk data is usually large, it will take a lot of temporary storage space running time. Therefore, how to reduce the storage overhead and time overhead in the disk migration process is an urgent problem that needs to be solved.

[0019] In order to solve the above problems, an embodiment of the present application provides a method for cross-cloud migration of virtual machine disks. The method is applied to a data migration service, and the data migration service is used to migrate the source virtual machine disk data on a source platform to a destination virtual machine disk file on a destination platform. The method includes the following steps: S1: obtaining at least one data block in the source virtual machine disk from the source platform; S2: determining the storage location of the at least one data block currently obtained in the destination virtual machine disk file based on the data storage structure of the destination virtual machine disk file on the destination platform; S3: writing the at least one data block currently obtained into the storage location; and repeating S1 to S3 until all data blocks in the source virtual disk are written into the destination virtual machine disk file.

[0020] The virtual machine disk cross-cloud migration method provided in the embodiment of the present application, during the disk cloud migration process, after obtaining the data blocks in the virtual machine disk data from the source platform, determines the corresponding storage position of the data blocks obtained from the source platform in the disk file of the destination platform according to the data storage structure of the disk file of the destination platform, and writes the data blocks of the source platform into the disk file of the destination platform according to the above corresponding position. The virtual machine disk cross-cloud migration method provided in the embodiment of the present application is based on the data composition principle of the disk file of the destination platform, completes the format conversion of each disk data block in the memory, realizes the migration from the disk in the form of a stream, realizes the migration of the virtual machine disk from one platform to another, saves temporary storage space, and greatly saves storage overhead and time overhead.

[0021] In order to make the purpose, technical scheme and advantages of the embodiments of the present application clearer, each embodiment of the present application will be described in detail below in conjunction with the accompanying drawings. However, it will be appreciated by those skilled in the art that in each embodiment of the present application, many technical details are proposed in order to enable the reader to better understand the present application. However, even without these technical details and various changes and modifications based on the following embodiments, the technical scheme claimed in the present application can also be implemented.

[0022] The following is a detailed description of the implementation details of the virtual machine disk cross-cloud migration method in this embodiment. The following content is only for the convenience of understanding the implementation details of this solution and is not necessary for the implementation of this solution. Figure 1 As shown, the following steps may be included:

[0023] S1: Obtain at least one data block in a source virtual machine disk from a source platform; wherein the source platform type includes a vmware platform.

[0024] The virtual machine disk cross-cloud migration method provided in the embodiment of the present application is applied to a data migration service, and the data migration service is used to migrate the source virtual machine disk data on the source platform to the destination virtual machine disk file on the destination platform, wherein the data migration service can be deployed on the destination platform or on other independent servers, and the destination platform type includes an openstack platform.

[0025] In one example, the source platform is a VMware platform, the source virtual machine disk file is a vmdk file, and the data migration service uses the vddk tool to obtain at least one data block of the vmdk file. For example, the data migration service uses the vddk tool to obtain the first N data blocks of the source disk into the memory through network transmission. Since VMware's vdkk tool provides a method for directly reading disk blocks, when the VMware platform is used as the source platform, the vdkk tool can be used to quickly read the source platform data blocks, making the virtual machine disk cross-cloud migration process faster.

[0026] In one example, when the disk data migration performs the first acquisition of data blocks of the source virtual machine disk, the data migration service starts to acquire data blocks from the beginning of the source disk, that is, the first N data blocks of the source disk file; when the disk data migration is not the first acquisition of data blocks of the source virtual machine disk, the destination platform acquires N data blocks after the current position of the source disk. The number of data blocks of the source virtual machine disk acquired each time can be the same or different. For example, 5 data blocks can be acquired each time, or 10 data blocks can be acquired from the source virtual machine disk for the first time, 5 data blocks can be acquired from the source virtual machine disk for the second time, and 6 data blocks can be acquired from the source virtual machine disk for the third time.

[0027] In addition, before obtaining at least one data block in the source virtual machine disk from the source platform, the data migration service obtains the size of the source virtual machine disk; based on the size of the source virtual machine disk and the format of the target platform disk file, a storage area of ​​appropriate size is created on the target platform as the storage backend of the target virtual machine disk file.

[0028] In one example, the data migration service uses the vdkk tool to open the source vmdk file for subsequent operations on the source virtual machine disk. The vdkk is used to obtain the size of the source virtual machine disk. The openstack platform uses the rbd storage of the ceph cluster as the storage backend for the disk file. For example, after obtaining the size of the source virtual machine disk, librbd is used to create an rbd block with the same size as the source virtual machine disk as the storage backend for the raw file, which is used to store the generated raw file in the cluster. By creating an rbd block with the same size as the disk file on the target platform as the storage backend for migrating disk data, the problem of using temporary storage files during the migration process, which consumes a lot of temporary storage space and storage time, is avoided.

[0029] In addition, if the destination file is not in raw format, there may be other requirements for the size of the storage backend, and the specific requirements are determined according to the format of the destination file.

[0030] S2: Determine a storage position of the at least one data block currently obtained in the destination virtual machine disk file according to the data storage structure of the destination virtual machine disk file on the destination platform.

[0031] For a disk, no matter what type of internal physical structure the disk has, when LBA (Logical Block Addressing) is used to access the disk, logically, the disk is a sequence of blocks (512B), that is, the disk is composed of N data blocks; for a disk file, no matter what format the disk is in or what file system it is in, a file is a sequence of bytes (1B), that is, a file is composed of bytes 1 to L. Therefore, based on the above principles of disks and disk files, the principle of virtual machine disk files is to virtualize disk files into a disk.

[0032] In an example, the source virtual machine disk is a vmdk file; the target virtual machine disk file is a raw file. For a raw format disk file, the virtualization technology virtualizes every 512 bytes of the raw file into a disk block. The first 512 bytes of the raw file are virtualized into a disk block, the second 512 bytes are virtualized into the second disk block, and so on.

[0033] Specifically, the data migration service determines the block number of each currently acquired data block, and determines the storage location of each currently acquired data block in the destination virtual machine disk file based on the storage structure and block number of the raw format virtual machine disk file in the destination virtual machine disk. Each acquired data block is numbered, and the location is determined based on the storage structure of the disk file in the destination virtual machine disk file, making the disk migration process smoother and more orderly, with one-to-one correspondence between the sequence number and the storage location, avoiding disk storage errors that cause blockage in the cross-cloud migration of virtual machine disks, thereby reducing the time consumed in the migration process.

[0034] S3: Write at least one data block currently obtained into a storage location.

[0035] In one example, the data migration service uses librbd and writes the data blocks into the first N*512 bytes of the raw file according to the principle of raw format disk files, thereby completing a partial disk data migration.

[0036] In one example, when the disk data migration writes the source virtual machine disk data for the first time, the data migration service starts writing from the beginning of the destination virtual machine disk, that is, the first N*512 bytes of the destination virtual machine disk file; when the disk data migration is not the first time to write the source virtual machine disk data, the data migration service writes N source disk data blocks to the N*512 bytes after the current position of the destination virtual machine disk. Among them, the value of N can remain unchanged or change. If the number of data blocks obtained each time changes, then because the disk cross-cloud migration method of the embodiment of the present application is to write all the data blocks obtained each time at one time, therefore, when writing data blocks to the destination platform file, the number of writes each time will also be different.

[0037] S4: Determine whether all data blocks in the source virtual machine disk are written into the destination virtual machine disk file.

[0038] In one example, when all data blocks have not been written to the destination virtual machine disk file, the data migration service repeats S1 to S3 until all data blocks in the source virtual machine disk are written to the destination virtual machine disk file, and the data migration is considered completed. When all data blocks are written to the destination virtual machine disk file, the cross-cloud disk migration ends.

[0039] In one example, when the data migration service determines that all disk data has not been migrated, the current position of the source disk points to the next group of N data blocks, and the position of the destination disk file points to the next N*512 bytes.

[0040] Furthermore, the progress of disk cross-cloud migration is determined based on the total number of blocks in the source virtual machine disk and the number of blocks currently written to the target virtual machine disk file. Through the human-computer interaction interface, the current progress of migrating the data of the source virtual machine disk to the target virtual machine disk file can be displayed in real time during the data migration process.

[0041] For example, there are a total of 100 data blocks in the source virtual machine disk. At this point in the data migration process, 66 data blocks have been written to the destination platform disk file. Based on the total data blocks in the virtual machine disk and the data blocks written to the destination platform disk file, it can be concluded that 66% of the current data migration progress has been completed. Since in the relevant technical means, disk cross-cloud migration includes the process of storing disk data in a temporary file, data conversion, and uploading data to the destination platform disk file, it is difficult to calculate the real-time progress of the current disk cross-cloud migration in the above-mentioned related technical means, which makes it difficult to reflect the current disk cross-cloud migration progress to the user. In the disk cross-cloud migration method provided in the embodiment of the present application, since the disk migration is realized in the form of a stream, based on the total number of source virtual machine disk data blocks and the number of written data blocks, the real-time statistics of the virtual machine disk cross-cloud migration progress can be accurately realized, so the progress of the current data migration can be intuitively displayed through the progress bar of the human-computer interaction interface.

[0042] In one example, in order to make the implementation process of the virtual machine disk cross-cloud migration method of the embodiment of the present application clearer, refer to Figure 2 The method of cross-cloud disk migration is described in detail. The specific process is as follows:

[0043] In step 201, the vmdk file on the source side is opened using the vddk tool for use in operations on the source virtual machine disk in subsequent steps.

[0044] In step 202, the size of the source virtual machine disk is obtained using vddk.

[0045] In step 203, the OpenStack platform uses librbd to create an rbd block with the same size as the source virtual machine disk as the storage backend of the raw file.

[0046] In step 204, the vdkk tool is used to obtain N data blocks after the current position from the source virtual machine disk file, wherein the size of each data block is 512 bytes. When the data block is obtained for the first time, the current position is the beginning of the entire disk, and the first N data blocks of the disk are obtained.

[0047] In step 205, according to the data composition principle of the raw format disk file, the N data blocks obtained from the source virtual machine disk file are written to the N*512 bytes after the current position of the raw file, completing a migration of partial disk data (when writing for the first time, the current position is the beginning of the entire file, and the data is written to the first N*512 bytes of the entire file).

[0048] In step 206, the progress of the entire disk migration is calculated according to the amount of migrated data and the total amount of data.

[0049] In step 207 , it is determined whether the disk data migration is completed, if so, step 209 is executed, otherwise, step 208 is executed.

[0050] In step 208, the next position of the source disk points to the next group of N data blocks, and the position of the raw disk file points to the next N*512 bytes.

[0051] In step 209, the connection between the source disk file and the destination rbd storage is closed.

[0052] The virtual machine disk cross-cloud migration method provided in the embodiment of the present application is applicable to the cross-platform migration scenario where the source file is a vmdk file and the destination file is a raw file. In the above specific implementation process, based on the internal composition principle of raw format data, the vdkk tool is used, and the source virtual machine disk data is obtained, the data is converted, and the destination data is uploaded in a streaming manner to complete the migration of the disk from the source platform (vmware platform) to the end (OpenStack platform).

[0053] It should be noted that the above embodiments are only described with the VMware platform as the source platform and the OpenStack platform as the destination platform. The source platform and the destination platform mentioned in this application are not limited, nor are the file formats of the source files and the destination files in this application limited. The vddk tool and librbd used in the above application embodiments can be adjusted according to the actual platform technology, and the corresponding tools provided by the source platform and the destination platform can be selected. For example, when the destination platform is the Linux local file system, the system library can be used to complete step 205.

[0054] The virtual machine disk cross-cloud migration method provided by the embodiment of the present application is to number each acquired data block, and determine the position according to the storage structure of the disk file in the destination virtual machine disk file, so that the disk migration process is smoother and more orderly, and the sequence number and the storage position correspond one by one, avoiding the disk storage error causing the virtual machine disk cross-cloud migration to be blocked, thereby increasing the time consumed by the migration process. By creating an rbd block of the same size as the disk file on the destination platform as the storage backend for migrating disk data, the problem of using temporary storage files during the migration process, thereby consuming a large amount of temporary storage space and storage time, is avoided. According to the total number of data blocks in the source virtual machine disk and the number of data blocks written, the progress of the current disk migration can be obtained from the human-computer interaction interface in real time. Based on the data composition structure of the destination virtual machine disk file, the block data obtained from the source virtual disk file is written to the corresponding position in the destination virtual machine disk file, and the migration of the virtual machine disk data from one platform to another is realized, which saves temporary storage space, greatly saves storage overhead and time overhead, and can accurately monitor the migration progress of the disk data, improving the user experience.

[0055] The step division of the above methods is only for the purpose of clear description. When implemented, they can be combined into one step or some steps can be split and decomposed into multiple steps. As long as they include the same logical relationship, they are all within the scope of protection of this patent; adding insignificant modifications to the algorithm or process or introducing insignificant designs without changing the core design of the algorithm and process are all within the scope of protection of this patent.

[0056] The embodiment of the present application also provides an electronic device, such as Figure 3 As shown, it includes at least one processor 301; and a memory 302 that is communicatively connected to the at least one processor 301; wherein the memory 302 stores instructions that can be executed by the at least one processor 301, and the instructions are executed by the at least one processor 301 so that the at least one processor can execute the above-mentioned virtual machine disk cross-cloud migration method.

[0057] Among them, the memory and the processor are connected in a bus manner, and the bus may include any number of interconnected buses and bridges, and the bus connects various circuits of one or more processors and memories together. The bus can also connect various other circuits such as peripherals, voltage regulators, and power management circuits, which are well known in the art and are therefore not further described herein. The bus interface provides an interface between the bus and the transceiver. The transceiver can be one element or multiple elements, such as multiple receivers and transmitters, providing a unit for communicating with various other devices on a transmission medium. The data processed by the processor is transmitted on a wireless medium via an antenna, and further, the antenna also receives data and transmits the data to the processor.

[0058] The processor is responsible for managing the bus and general processing, and can also provide various functions, including timing, peripheral interfaces, voltage regulation, power management, and other control functions. Memory can be used to store data used by the processor when performing operations.

[0059] The above-mentioned product can execute the method provided in the embodiment of the present application, and has the functional modules and beneficial effects corresponding to the execution method. For technical details not fully described in this embodiment, please refer to the method provided in the embodiment of the present application.

[0060] The embodiment of the present application further provides a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, the above method embodiment is implemented.

[0061] Those skilled in the art can understand that all or part of the steps in the above-mentioned embodiment method can be completed by instructing the relevant hardware through a program, and the program is stored in a storage medium, including a number of instructions to enable a device (which can be a single-chip microcomputer, chip, etc.) or a processor to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

[0062] The above embodiments are provided for ordinary technicians in the field to implement and use the present application. Ordinary technicians in the field can make various modifications or changes to the above embodiments without departing from the inventive concept of the present application. Therefore, the protection scope of the present application is not limited to the above embodiments, but should conform to the maximum scope of the innovative features mentioned in the claims.

Claims

1. A method for migrating virtual machine disks across clouds, characterized in that: The method is applied to a data migration service, which is used to migrate source virtual machine disk data on a source platform to a destination virtual machine disk file on a destination platform. The method comprises the following steps: S1: Acquire at least one data block in the source virtual machine disk from the source platform, and number each acquired data block to obtain a data block sequence number; S2: determining a storage position of the at least one data block currently obtained in the destination virtual machine disk file according to the data storage structure of the destination virtual machine disk file on the destination platform, wherein the data block sequence number corresponds to the storage position one by one; S3: writing the at least one data block currently acquired into a storage location corresponding to the data block; S1 to S3 are repeatedly executed until all data blocks in the source virtual machine disk are written into the destination virtual machine disk file.

2. The method for cross-cloud migration of virtual machine disks according to claim 1, characterized in that: The source platform type includes: vmware platform; the destination platform type includes: openstack platform.

3. The method for cross-cloud migration of virtual machine disks according to claim 2, characterized in that: The S2 includes: Determine the block number of each of the data blocks currently obtained, and determine the storage position of each of the data blocks currently obtained in the destination virtual machine disk file according to the storage structure of the raw format virtual machine disk file in the destination virtual machine disk and the block number.

4. The method for cross-cloud migration of virtual machine disks according to claim 2, characterized in that: The S1 includes: Use the vddk tool to obtain at least one data block in the source virtual machine disk from the vmware platform.

5. The method for cross-cloud migration of virtual machine disks according to any one of claims 1 to 4, characterized in that: The method further comprises: The current progress of migrating the data of the source virtual machine disk to the destination virtual machine disk file is displayed in real time through the human-computer interaction interface.

6. The method for cross-cloud migration of virtual machine disks according to claim 5, characterized in that: The progress is obtained in the following ways: The progress is determined according to the total number of blocks in the source virtual machine disk and the number of blocks currently written to the destination virtual machine disk file.

7. The method for cross-cloud migration of virtual machine disks according to any one of claims 1 to 4, characterized in that: Before the S1, it also includes: Obtaining the size of the source virtual machine disk; According to the size of the source virtual machine disk and the format of the target platform disk file, a storage area of ​​appropriate size is created on the target platform as a storage backend for the target virtual machine disk file.

8. The method for cross-cloud migration of virtual machine disks according to any one of claims 1 to 4, characterized in that: The step of creating a storage area of ​​suitable size on the target platform as the storage backend of the target virtual machine disk file comprises: using librbd to create an rbd block of the same size as the source virtual machine disk as the storage backend of the target raw format virtual machine disk file.

9. An electronic device, characterized in that: include: at least one processor; as well as, a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the virtual machine disk cross-cloud migration method as described in any one of claims 1 to 8.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method for cross-cloud migration of virtual machine disks according to any one of claims 1 to 8 is implemented.

Images