Data migration method, data snapshot method and apparatus

Abstract

A data migration method, a data snapshot method, and an apparatus are disclosed. The method involves migrating data required for starting a virtual server instance from a first storage device to a second storage device, enabling a computing device to start the virtual server instance based on the data in the second storage device. After starting the virtual server instance, data read / write requests are executed on the second storage device. The virtual server instance-related data from the first storage device is then migrated to a third storage device. In response to a snapshot request, a snapshot of the virtual server instance-related data stored in the second storage device is performed to obtain a first snapshot. After the migration is complete, a snapshot of the virtual server instance-related data stored in the third storage device is performed to obtain a second snapshot. This reduces mutual exclusion during the migration process and allows the full data of the virtual server instance at the time of the snapshot request to be obtained based on the first and second snapshots.

Description

Technical Field

[0001] This disclosure relates to the field of virtual machine hot migration, and in particular to a data migration method, a data snapshot method, and an apparatus. Background Technology

[0002] Virtual machine (VM) hot migration technology mainly includes two types: PreCopy and PostCopy.

[0003] PreCopy is a method that performs data migration first and then switches I / O. When using PreCopy to migrate data from a virtual machine disk (cloud disk), if a user performs a snapshot operation on the cloud disk, the PreCopy migration process terminates because it's a background maintenance activity, prioritizing the user's snapshot request. Switching I / O after PreCopy migration is complete introduces the problem that cloud disk writes continue on the source cluster, preventing immediate data offloading. Furthermore, PreCopy often requires multiple rounds of copying, making the entire migration process time-consuming.

[0004] Therefore, the PostCopy migration method is preferred. PostCopy is a method that performs an IO stream switch before data migration. During PostCopy migration, since the source cluster only has the data before the IO stream switch, and the data after the IO stream switch is in the target cluster, neither cluster has the full data of the virtual machines. This makes it impossible to stop the migration and prioritize responding to snapshot requests during PostCopy migration.

[0005] Therefore, a solution is needed to support the generation of snapshots during the PostCopy migration process. Summary of the Invention

[0006] One technical problem this disclosure aims to solve is to provide a solution that supports the generation of snapshots during PostCopy migrations.

[0007] According to a first aspect of this disclosure, a data snapshot method is provided, comprising: migrating data required for starting a virtual server instance stored in a first storage device to a second storage device, so that a computing device can start the virtual server instance based on the data stored in the second storage device, wherein, after the computing device starts the virtual server instance based on the data stored in the second storage device, a data read / write request related to the virtual server instance is executed on the second storage device; migrating virtual server instance-related data associated with the operation of the virtual server instance stored in the first storage device to a third storage device; performing snapshot processing on the virtual server instance-related data stored in the second storage device in response to a snapshot request to obtain a first snapshot; and performing snapshot processing on the virtual server instance-related data stored in the third storage device after the migration is completed to obtain a second snapshot.

[0008] Optionally, the full data of the virtual server instance at the moment corresponding to the snapshot request can be obtained based on the first and second snapshots.

[0009] Optionally, the first snapshot includes virtual server instance-related data stored in the second storage device and a first data bitmap for representing logical addresses of the data. The second snapshot includes virtual server instance-related data stored in the third storage device and a second data bitmap for representing logical addresses of the data. The step of obtaining the full data of the virtual server instance at the time corresponding to the snapshot request based on the first and second snapshots includes: merging the first and second data bitmaps to obtain a third data bitmap that can represent the logical addresses of the full data of the virtual server instance at the time corresponding to the snapshot request; for each logical address in the third data bitmap, determining whether there is data corresponding to the logical address in the first snapshot; if so, selecting the data corresponding to the logical address from the first snapshot; if not, selecting the data corresponding to the logical address from the second snapshot.

[0010] Optionally, the method may further include: uploading the full data of the virtual server instance at the time corresponding to the snapshot request to the storage server for backup.

[0011] Optionally, the method may further include: importing the data from the second snapshot into a second storage device.

[0012] According to a second aspect of this disclosure, a data migration method is provided, comprising: migrating data required for starting a virtual server instance stored in a first storage device to a second storage device, so that a computing device can start the virtual server instance based on the data stored in the second storage device, wherein, after the computing device starts the virtual server instance based on the data stored in the second storage device, data read / write requests related to the virtual server instance are executed on the second storage device; migrating virtual server instance-related data associated with the operation of the virtual server instance stored in the first storage device to a third storage device; and importing the virtual server instance-related data stored in the third storage device into the second storage device after the migration is completed.

[0013] According to a third aspect of this disclosure, a data snapshot method is provided, comprising: migrating data required for starting a virtual server instance stored in a first storage device to a second storage device, so that a computing device can start the virtual server instance based on the data stored in the second storage device, wherein, after the computing device starts the virtual server instance based on the data stored in the second storage device, a data read / write request related to the virtual server instance is executed on the second storage device; performing snapshot processing on virtual server instance-related data stored in the first storage device that is associated with the operation of the virtual server instance to obtain a third snapshot; and performing snapshot processing on virtual server instance-related data stored in the second storage device in response to a snapshot request to obtain a first snapshot.

[0014] Optionally, the method may further include: obtaining the full data of the virtual server instance at the moment corresponding to the snapshot request based on the first snapshot and the third snapshot.

[0015] According to a fourth aspect of this disclosure, a data snapshot apparatus is provided, comprising: a first migration module, configured to migrate data required for starting a virtual server instance stored in a first storage device to a second storage device, so that a computing device can start the virtual server instance based on the data stored in the second storage device, wherein after the computing device starts the virtual server instance based on the data stored in the second storage device, a data read / write request related to the virtual server instance is executed on the second storage device; a second migration module, configured to migrate virtual server instance-related data associated with the operation of the virtual server instance stored in the first storage device to a third storage device; a first snapshot module, configured to perform snapshot processing on the virtual server instance-related data stored in the second storage device in response to a snapshot request, to obtain a first snapshot; and a second snapshot module, configured to perform snapshot processing on the virtual server instance-related data stored in the third storage device after the migration is completed, to obtain a second snapshot.

[0016] According to a fifth aspect of this disclosure, a data migration apparatus is provided, comprising: a first migration module, configured to migrate data required for starting a virtual server instance stored in a first storage device to a second storage device, so that a computing device can start the virtual server instance based on the data stored in the second storage device, wherein after the computing device starts the virtual server instance based on the data stored in the second storage device, data read / write requests related to the virtual server instance are executed on the second storage device; a second migration module, configured to migrate virtual server instance-related data associated with the operation of the virtual server instance stored in the first storage device to a third storage device; and an import module, configured to import the virtual server instance-related data stored in the third storage device into the second storage device after the migration is completed.

[0017] According to a sixth aspect of this disclosure, a data snapshot apparatus is provided, comprising: a migration module for migrating data required for starting a virtual server instance stored in a first storage device to a second storage device, so that a computing device can start the virtual server instance based on the data stored in the second storage device, wherein after the computing device starts the virtual server instance based on the data stored in the second storage device, a data read / write request related to the virtual server instance is executed on the second storage device; a third snapshot module for performing snapshot processing on virtual server instance-related data stored in the first storage device that is associated with the operation of the virtual server instance to obtain a third snapshot; and a first snapshot module for performing snapshot processing on virtual server instance-related data stored in the second storage device in response to a snapshot request during the migration process to obtain a first snapshot.

[0018] According to a seventh aspect of this disclosure, a computing device is provided, comprising: a processor; and a memory having executable code stored thereon, which, when executed by the processor, causes the processor to perform the method as described in any one of the first to third aspects above.

[0019] According to an eighth aspect of this disclosure, a computer program product is provided, including executable code that, when executed by a processor of an electronic device, causes the processor to perform the method described in any one of the first to third aspects described above.

[0020] According to a ninth aspect of this disclosure, a non-transitory machine-readable storage medium is provided, on which executable code is stored, which, when executed by a processor of an electronic device, causes the processor to perform the method described in any one of the first to third aspects described above.

[0021] Therefore, by setting up a third storage device as an intermediate migration target, the present disclosure can reduce the occurrence of the above-mentioned mutual exclusion situation to a certain extent by migrating the data in the first storage device to the third storage device. On this basis, in response to the snapshot request, the virtual server instance related data stored in the second storage device is snapshotted to obtain a first snapshot, and after the migration is completed, the virtual server instance related data stored in the second storage device is snapshotted to obtain a first snapshot, so that the full data of the virtual server instance at the time corresponding to the snapshot request can be obtained based on the first snapshot and the second snapshot. Attached Figure Description

[0022] The above and other objects, features and advantages of this disclosure will become more apparent from the more detailed description of exemplary embodiments thereof taken in conjunction with the accompanying drawings, wherein like reference numerals generally denote like parts.

[0023] Figure 1 A schematic diagram of the data migration process based on PostCopy is shown.

[0024] Figure 2 A schematic diagram of a data migration process based on an improved PostCopy according to an embodiment of the present disclosure is shown.

[0025] Figure 3 A schematic flowchart of a snapshot method in a data migration process according to an embodiment of the present disclosure is shown.

[0026] Figure 4 A schematic diagram of a snapshot apparatus according to an embodiment of the present disclosure is shown.

[0027] Figure 5 A schematic diagram of a data migration apparatus according to an embodiment of the present disclosure is shown.

[0028] Figure 6 A schematic diagram of the structure of a snapshot device in a data migration process according to another embodiment of the present disclosure is shown.

[0029] Figure 7 A schematic diagram of the structure of a computing device according to an embodiment of the present disclosure is shown.

[0030] Figure 8 A schematic diagram of the PostCopy migration process is shown.

[0031] Figure 9 This diagram illustrates the process of obtaining a complete snapshot during the PostCopy migration. Detailed Implementation

[0032] Preferred embodiments of the present disclosure will now be described in more detail with reference to the accompanying drawings. While preferred embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that the present disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.

[0033] Figure 1 A schematic diagram of the data migration process based on PostCopy is shown.

[0034] The first storage device is used as a storage device for a virtual server instance (such as a virtual machine), storing virtual server instance-related data associated with the operation of the virtual server instance. For example, the first storage device could refer to a virtual machine disk. A virtual machine disk, also known as a virtual machine block device, is often referred to as a cloud disk.

[0035] Virtual server instance related data can include data required for the virtual server instance to start, that is, data that the virtual server instance must rely on when starting, such as the system disk data of the virtual machine.

[0036] The virtual server instance-related data may also include data written to the first storage device during the operation of the virtual server instance, which may be accessed during the operation of the virtual server instance.

[0037] The first storage device can be considered the source of the data migration, and the second storage device can be considered the destination of the data migration. When performing a data migration operation (that is, migrating the virtual server instance-related data from the first storage device to the second storage device), the data required to start the virtual server instance can first be migrated to the second storage device, so that the computing device can start the virtual server instance based on the data stored in the second storage device.

[0038] The computing device provides the runtime environment for the virtual server instance. The second storage device can refer to a storage device that is physically independent of the computing device. The computing device and the second storage device can form a computing-storage separation architecture. That is, the computing device can refer to the computing server in this architecture that provides computing resources, and the second storage device can refer to the storage server in this architecture that provides storage resources. Before the data migration operation begins, the computing device can start and run the virtual server instance based on the data stored in the first storage device.

[0039] After the computing device starts a virtual server instance based on the data stored in the second storage device, it executes data read and write requests related to the virtual server instance on the second storage device. That is, after starting the virtual server instance based on the data stored in the second storage device, the IO requests (i.e., IO streams) related to the virtual server instance will switch from the first storage device to the second storage device. As a result, new data during the operation of the virtual server instance will be written to the second storage device.

[0040] After launching a virtual server instance based on the data stored in the second storage device and switching the IO stream from the first storage device to the second storage device, the data migration process can continue to be executed, continuously migrating the data related to other virtual server instances stored in the first storage device to the second storage device.

[0041] After the IO stream switches from the first storage device to the second storage device, for data write requests in the IO request, the new data can be directly written to the second storage device; for data read requests in the IO request, the second storage device can be searched to see if the data corresponding to the data read request exists. If it exists, it can be read directly; if it does not exist, the data can be obtained from the first storage device.

[0042] exist Figure 1 In the PostCopy-based data migration process shown, there are two types of data write operations on the second storage device: data write operations during the migration of data from the first storage device to the second storage device; and data write operations performed on the second storage device in response to data write requests during the operation of the virtual server instance. When the above two write operations target the same data object, write mutual exclusion will occur.

[0043] To avoid the aforementioned mutual exclusion, this disclosure proposes an improved PostCopy data migration process.

[0044] Figure 2 A schematic diagram of a data migration process based on an improved PostCopy according to an embodiment of the present disclosure is shown.

[0045] and Figure 1The difference in the data migration process shown is that this embodiment sets up a third storage device as an intermediate migration target. After starting the virtual server instance based on the data stored in the second storage device and switching the IO stream from the first storage device to the second storage device, the other virtual server instance-related data in the first storage device is not directly migrated to the second storage device. Instead, the virtual server instance-related data stored in the first storage device is migrated to the third storage device. After the migration is completed, the virtual server instance-related data stored in the third storage device is imported into the second storage device, so that the second storage device has all the data of the virtual server instance, thus completing the data migration.

[0046] The virtual server instance-related data migrated to the third storage device can refer to all virtual server instance-related data associated with the operation of the virtual server instance stored in the first storage device, or it can refer to the data after removing the data required for virtual machine startup from all virtual server instance-related data.

[0047] In real-world applications, the first and second storage devices can belong to different Elastic Block Service (EBS) clusters, while the third and second storage devices can belong to the same cluster. Cross-cluster data migration is time-consuming and more prone to write-exclusive scenarios.

[0048] This disclosure reduces the occurrence of the aforementioned mutual exclusion by setting up a third storage device that belongs to the same device storage cluster as the second storage device as an intermediate migration target, migrating data from the first storage device to the third storage device, and then importing data from the third storage device to the second storage device.

[0049] exist Figure 2 Based on the improved PostCopy data migration process shown, this disclosure proposes a snapshot scheme in the data migration process.

[0050] Figure 3 A schematic flowchart of a snapshot method in a data migration process according to an embodiment of the present disclosure is shown. The data migration process may refer to... Figure 2 The data migration process based on the improved PostCopy is shown.

[0051] like Figure 3 As shown, in step S310, in response to the snapshot request, snapshot processing is performed on the virtual server instance related data stored in the second storage device to obtain the first snapshot.

[0052] In step S320, after the migration is completed, a snapshot of the virtual server instance data stored in the third storage device is performed to obtain a second snapshot.

[0053] A snapshot request can refer to a snapshot request made during the process of migrating virtual server instance-related data stored in a first storage device to a third storage device.

[0054] The first snapshot obtained by snapshotting the second storage device mainly includes the data portion of the virtual server instance after the IO stream switch; the second snapshot obtained by snapshotting the third storage device after the migration is completed mainly includes the data portion of the virtual server instance before the IO stream switch. Therefore, the first and second snapshots cover the full data (i.e., complete data) of the virtual server instance at the time corresponding to the snapshot request, making it possible to obtain the full data of the virtual server instance at the time corresponding to the snapshot request based on the first and second snapshots.

[0055] This disclosure allows uploading the full data of a virtual server instance at the moment of the snapshot request to a storage server for backup. Compared to the data in the second snapshot, the data in the first snapshot can be considered as new data for the virtual server instance. Considering scenarios where data may be overwritten, when uploading the full data, data can first be read from the first snapshot and uploaded. If data is missing in the first snapshot (i.e., there is no data at a certain logical address), then data can be read from the second snapshot (data at that logical address) and uploaded.

[0056] Specifically, the first snapshot represents the virtual server instance-related data and its logical addresses stored in the second storage device, and the second snapshot represents the virtual server instance-related data and its logical addresses stored in the third storage device. The virtual server instance-related data stored in the second storage device represented by the first snapshot may include data required for the virtual server instance to start, as well as data written to the second storage device after switching I / O streams. The virtual server instance-related data stored in the third storage device represented by the second snapshot refers to all data related to the virtual server instance that has been migrated to the third storage device, such as the full data of the virtual server instance before switching I / O streams. The logical address may refer to the Logical Block Address (LBA).

[0057] The logical addresses of the first and second snapshots can be merged to obtain a total set of logical addresses. For each logical address in the set, if the first snapshot contains data corresponding to that logical address, the data is retrieved from the first snapshot and uploaded. If the first snapshot does not contain data corresponding to that logical address, the data corresponding to that logical address is retrieved from the second snapshot and uploaded.

[0058] As an example, the first snapshot may include virtual server instance related data stored in the second storage device and a first data bitmap for representing the logical address of the data, and the second snapshot may include virtual server instance related data stored in the third storage device and a second data bitmap for representing the logical address of the data.

[0059] The first and second data bitmaps can be merged to obtain a third data bitmap that represents the logical addresses of the full data of the virtual server instance at the time of the snapshot request. Then, for each logical address in the third data bitmap, it is determined whether data corresponding to that logical address exists in the first snapshot. If it exists, the data corresponding to that logical address is selected from the first snapshot; otherwise, the data corresponding to that logical address is selected from the second snapshot. Thus, the data corresponding to each logical address in the third data bitmap, i.e., the full data of the virtual server instance at the time of the snapshot request, can be obtained.

[0060] After taking a snapshot of the third storage device to obtain a second snapshot, the data from the second snapshot can be imported into the second storage device, so that the second storage device has all the current data of the virtual server instance. In other words, migration can be achieved by importing the data from the second snapshot into the second storage device.

[0061] When generating a snapshot of the virtual server instance after the migration is complete, since the second storage device already has all the data of the virtual server instance, only the second storage device needs to be snapshotted.

[0062] In summary, this disclosure presents a method for piecing together the full data of a virtual server instance from multiple snapshots (first snapshot and second snapshot) during cloud disk PostCopy migration. This addresses the drawback of needing to roll back during snapshot generation operations in PreCopy scenarios, and enables support for snapshot generation operations during PostCopy migration. The snapshot method of this disclosure does not require immediate data completion upon responding to a snapshot request, nor does it require adding a separate process to supplement data from the source after the snapshot generation operation is completed, thus avoiding increased system complexity.

[0063] As an optional embodiment, for Figure 1 or Figure 2In addition to the data migration process shown, this disclosure also proposes another snapshot method in the data migration process. Figure 3 The difference in the snapshot method shown is that, in this embodiment, instead of generating a snapshot on the third storage device, a snapshot can be generated on the virtual server instance-related data stored in the first storage device that is associated with the operation of the virtual server instance to obtain a third snapshot; based on the third snapshot and the first snapshot, the full data of the virtual server instance at the time corresponding to the snapshot request is obtained.

[0064] Specifically, the first snapshot represents the virtual server instance-related data and its logical address stored in the second storage device, and the third snapshot represents the virtual server instance-related data and its logical address stored in the first storage device. The logical addresses of the first and third snapshots can be merged to obtain a total set of logical addresses. For each logical address in the set, if data corresponding to that logical address exists in the first snapshot, the data is retrieved from the first snapshot and uploaded. If data corresponding to that logical address does not exist in the first snapshot, the data corresponding to that logical address is retrieved from the third snapshot and uploaded.

[0065] As an example, the first snapshot may include virtual server instance related data stored in the second storage device and a first data bitmap for representing the logical address of the data, and the third snapshot may include virtual server instance related data stored in the first storage device and a fourth data bitmap for representing the logical address of the data.

[0066] The first and fourth data bitmaps can be merged to obtain a fifth data bitmap that represents the logical addresses of the virtual server instance's full data at the time of the snapshot request. Then, for each logical address in the fifth data bitmap, it is determined whether data corresponding to that logical address exists in the first snapshot. If it exists, the data corresponding to that logical address is selected from the first snapshot; otherwise, the data corresponding to that logical address is selected from the third snapshot. Thus, the data corresponding to each logical address in the fifth data bitmap, i.e., the full data of the virtual server instance at the time of the snapshot request, can be obtained.

[0067] The snapshot method in the data migration process disclosed herein can also be implemented as a snapshot device in the data migration process. Figure 4 A schematic diagram of a snapshot device according to an embodiment of the present disclosure is shown. The functional units of the snapshot device can be implemented by hardware, software, or a combination of hardware and software that implement the principles of the present disclosure. Those skilled in the art will understand that... Figure 4The described functional units can be combined or divided into sub-units to achieve the principles of the invention described above. Therefore, the description herein can support any possible combination, division, or further limitation of the functional units described herein.

[0068] The following is a brief description of the functional units that a snapshot device can have and the operations that each functional unit can perform. For details, please refer to the relevant descriptions above, which will not be repeated here.

[0069] See Figure 4 The snapshot device 400 includes a first migration module 410, a second migration module 420, a first snapshot module 430, and a second snapshot module 440.

[0070] The first migration module 410 and the second migration module 420 can be used to perform the above-mentioned combination Figure 2 The data migration process based on the improved PostCopy is shown.

[0071] Specifically, the first migration module 410 is used to migrate the data required for starting the virtual server instance stored in the first storage device to the second storage device, so that the computing device can start the virtual server instance based on the data stored in the second storage device. After the computing device starts the virtual server instance based on the data stored in the second storage device, it executes data read / write requests related to the virtual server instance on the second storage device. The second migration module 420 is used to migrate virtual server instance-related data associated with the operation of the virtual server instance stored in the first storage device to the third storage device.

[0072] The first snapshot module 430 is used to perform snapshot processing on the virtual server instance-related data stored in the second storage device in response to a snapshot request, thereby obtaining a first snapshot. The second snapshot module 440 is used to perform snapshot processing on the virtual server instance-related data stored in the third storage device after the migration is completed, thereby obtaining a second snapshot.

[0073] The snapshot device 400 may further include a full data acquisition module, used to obtain the full data of the virtual server instance at the time corresponding to the snapshot request based on the first snapshot and the second snapshot. For details on the full data acquisition process, please refer to the relevant description above.

[0074] The snapshot device 400 may also include an upload module for uploading the full data of the virtual server instance at the time corresponding to the snapshot request to the storage server for backup.

[0075] The snapshot device 400 may also include an import module for importing data from the second snapshot into the second storage device.

[0076] Figure 5 A schematic diagram of a data migration apparatus according to an embodiment of the present disclosure is shown. The functional units of the data migration apparatus can be implemented by hardware, software, or a combination of hardware and software that implement the principles of the present disclosure. Those skilled in the art will understand that... Figure 5 The described functional units can be combined or divided into sub-units to achieve the principles of the invention described above. Therefore, the description herein can support any possible combination, division, or further limitation of the functional units described herein.

[0077] The data migration device 500 can be used to perform... Figure 2 The data migration process based on the improved PostCopy is shown below. The functional units that the data migration device 500 can have and the operations that each functional unit can perform are briefly described below. For details, please refer to the relevant descriptions above; they will not be repeated here.

[0078] See Figure 5 The data migration device 500 includes a first migration module 510, a second migration module 520, and an import module 530.

[0079] The first migration module 510 is used to migrate the data required for starting the virtual server instance stored in the first storage device to the second storage device, so that the computing device can start the virtual server instance according to the data stored in the second storage device. After the computing device starts the virtual server instance according to the data stored in the second storage device, it executes data read and write requests related to the virtual server instance on the second storage device.

[0080] The second migration module 520 is used to migrate virtual server instance-related data associated with the operation of the virtual server instance stored in the first storage device to the third storage device.

[0081] Import module 530 is used to import virtual server instance related data stored in the third storage device into the second storage device after the migration is completed. As an example, data migration apparatus 500 may also include a snapshot module for taking a snapshot of the virtual server instance related data stored in the third storage device after the migration is completed, obtaining a second snapshot. Import module 530 can then import the data from the second snapshot into the second storage device.

[0082] Figure 6 A schematic diagram of a snapshot apparatus during a data migration process according to another embodiment of the present disclosure is shown. In this embodiment, the data migration process can refer to... Figure 1 The data migration process based on PostCopy shown can also refer to... Figure 2The data migration process based on the improved PostCopy is shown.

[0083] See Figure 6 The snapshot device 600 includes a migration module 610, a third snapshot module 620, and a first snapshot module 630.

[0084] The migration module 610 is used to migrate the data required for starting the virtual server instance stored in the first storage device to the second storage device, so that the computing device can start the virtual server instance based on the data stored in the second storage device. After the computing device starts the virtual server instance based on the data stored in the second storage device, it executes data read / write requests related to the virtual server instance on the second storage device. Specifically, the migration module 610 can perform... Figure 1 The PostCopy-based data migration process shown can also be executed. Figure 2 The data migration process based on the improved PostCopy is shown.

[0085] The third snapshot module 620 is used to perform snapshot processing on the virtual server instance-related data stored in the first storage device that is associated with the operation of the virtual server instance, to obtain a third snapshot;

[0086] The first snapshot module 630 is used to perform snapshot processing on the virtual server instance related data stored in the second storage device in response to a snapshot request during the migration process, and obtain a first snapshot.

[0087] The snapshot device 600 may further include a full data acquisition module, used to obtain the full data of the virtual server instance at the time corresponding to the snapshot request based on the first snapshot and the third snapshot. For details on the full data acquisition process, please refer to the relevant description above.

[0088] The snapshot device 600 may also include an upload module for uploading the full data of the virtual server instance at the time corresponding to the snapshot request to the storage server for backup.

[0089] Figure 7 A schematic diagram of a computing device that can be used to implement the snapshot method or data migration method described above, according to an embodiment of the present disclosure, is shown.

[0090] See Figure 7 The computing device 700 includes a memory 710 and a processor 720.

[0091] Processor 720 may be a multi-core processor or may contain multiple processors. In some embodiments, processor 720 may include a general-purpose main processor and one or more special-purpose coprocessors, such as a graphics processing unit (GPU), a digital signal processor (DSP), etc. In some embodiments, processor 720 may be implemented using custom circuitry, such as an application-specific integrated circuit (ASIC) or a field-programmable gate array (FPGA).

[0092] Memory 710 may include various types of storage units, such as system memory, read-only memory (ROM), and permanent storage devices. ROM may store static data or instructions required by the processor 720 or other modules of the computer. Permanent storage devices may be read-write storage devices. Permanent storage devices may be non-volatile storage devices that retain stored instructions and data even when the computer is powered off. In some embodiments, permanent storage devices use mass storage devices (e.g., magnetic or optical disks, flash memory) as permanent storage devices. In other embodiments, permanent storage devices may be removable storage devices (e.g., floppy disks, optical drives). System memory may be a read-write storage device or a volatile read-write storage device, such as dynamic random access memory. System memory may store some or all of the instructions and data required by the processor during operation. Furthermore, memory 710 may include any combination of computer-readable storage media, including various types of semiconductor memory chips (DRAM, SRAM, SDRAM, flash memory, programmable read-only memory), and disks and / or optical disks may also be used. In some embodiments, memory 710 may include a removable storage device that is readable and / or writable, such as a laser disc (CD), a read-only digital multifunction optical disc (e.g., DVD-ROM, dual-layer DVD-ROM), a read-only Blu-ray disc, an ultra-high density optical disc, a flash memory card (e.g., SD card, mini SD card, Micro-SD card, etc.), a magnetic floppy disk, etc. Computer-readable storage media do not contain carrier waves or transient electronic signals transmitted wirelessly or via wired connections.

[0093] The memory 710 stores executable code, which, when processed by the processor 720, enables the processor 720 to execute the snapshot method or data migration method described above.

[0094] Application examples

[0095] When migrating cloud disks from one version (e.g., version 1.0) of an EBS architecture cluster to another version (e.g., version 2.0) of an EBS architecture cluster via PostCopy, the snapshot scheme described above can be executed to support users taking snapshots of the cloud disks during the PostCopy migration process.

[0096] After the cloud disk PostCopy migration is initiated, the new write I / O stream will switch from the source cluster (such as EBS 1.0) to the target cluster (such as EBS 2.0), resulting in both the source and target clusters having a portion of the cloud disk's data, neither of which is complete. Therefore, when responding to user snapshot requests, the most important task is to piece together a complete set of data from the data at the moment of snapshot creation.

[0097] Figure 8 A schematic diagram of the PostCopy migration process is shown.

[0098] clustrer1 is the source cluster, and clustrer2 is the target cluster. device corresponds to the first storage device mentioned above, which is the cloud disk to be migrated in the source cluster. device2 corresponds to the second storage device mentioned above, which is the target disk in the target cluster. device1 corresponds to the third storage device mentioned above, which is the intermediate disk (tempsevice) in the target cluster.

[0099] device1 acts as a temporary disk, receiving migration data from the device in the background. After device1 finishes receiving the migration data, it performs a snapshot process to obtain snapshot1, which is then used to import the data into device2. Snapshot1 corresponds to the second snapshot mentioned above.

[0100] Before the cloud disk PostCopy migration started, all IO streams (data reads and writes) from the data center (TDC) were executed on the device. After the cloud disk PostCopy migration started, the new IO stream switched to device2. Data that was not found on device2 after the IO stream switch can now be read from the device.

[0101] Therefore, during the cloud disk PostCopy migration process, the foreground IO stream is switched to device2, and the data of device is migrated to device1 in the background. The foreground and background do not interfere with each other during the migration process.

[0102] Figure 9 This diagram illustrates the process of obtaining a complete snapshot during the PostCopy migration.

[0103] 1. User takes a snapshot

[0104] During the PostCopy migration process (which can be managed by the target cluster), when a snapshot request is received from the user, a snapshot of the existing data on the target cluster can be taken, resulting in a user snapshot. This snapshot corresponds to the first snapshot mentioned above. Each snapshot has its own data and a verMap representing the write location. (Note that the snapshot only contains a portion of the data; the other portion resides on the source cluster being migrated.)

[0105] 2. Take a snapshot on the intermediate disk

[0106] After the migration of the target cluster's intermediate disk device1 is complete, an internal snapshot is taken and converted into an external snapshot, snapshot1. Snapshot1 contains its own data and a data bitmap verMap1 indicating the write location. In some cases (such as the EBS 2.0 architecture), only the external snapshot can be used for data import.

[0107] 3. Import snapshot1

[0108] The data from snapshot1 can be imported into the target disk device2. At this point, device2 will have all the data, which can be used to create a full snapshot later.

[0109] 4. Upload the snapshot to OSS to complete the backup.

[0110] At this point, the complete data can be pieced together using snapshot and snapshot1. The specific stitching process is as follows:

[0111] 1) By merging the verMaps of snapshot and snapshot1, a complete data bitmap is obtained;

[0112] 2) Using the complete data bitmap obtained in the previous step, retrieve the corresponding data blocks for uploading to OSS (Object Storage Service). The data blocks have two sources: data written after the PostCopy IO stream switch is on snapshot, and data written before the PostCopy IO stream switch is on snapshot1. Considering scenarios where data blocks might be overwritten, during upload, data retrieval only needs to ensure that snapshot is retrieved first; if not found there, it is then retrieved from snapshot1.

[0113] In summary, this embodiment provides a method for taking snapshots of cloud disks during PostCopy migration. This method does not require immediate snapshot data completion during snapshot taking, nor does it require adding a separate process to supplement data from the source after snapshot taking, thereby increasing system complexity.

[0114] As an optional embodiment, after taking a snapshot, the snapshot contains a portion of the data, and the remaining data can be obtained from the source disk device, but this implementation is more complex than the one in this embodiment.

[0115] The snapshot method, data migration method, apparatus, and computing device according to this disclosure have been described in detail above with reference to the accompanying drawings.

[0116] Furthermore, the method according to this disclosure can also be implemented as a computer program or computer program product, which includes computer program code instructions for performing the steps defined in the above-described method of this disclosure.

[0117] Alternatively, this disclosure may be implemented as a non-transitory machine-readable storage medium (or computer-readable storage medium, or machine-readable storage medium) storing executable code (or computer program, or computer instruction code) that, when executed by a processor of an electronic device (or computing device, server, etc.), causes the processor to perform the steps of the method described above according to this disclosure.

[0118] Those skilled in the art will also understand that the various exemplary logic blocks, modules, circuits, and algorithm steps described in connection with the disclosure herein can be implemented as electronic hardware, computer software, or a combination of both.

[0119] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems and methods according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0120] The various embodiments of this disclosure have been described above. These descriptions are exemplary and not exhaustive, nor are they limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen to best explain the principles, practical application, or improvement of the technology in the market, or to enable others skilled in the art to understand the embodiments disclosed herein.

Claims

1. A data snapshot method, comprising: migrating data required for starting a virtual server instance stored in a first storage device to a second storage device, so as to enable a computing device to start the virtual server instance according to data stored in the second storage device, wherein, after the computing device starts the virtual server instance according to data stored in the second storage device, data read-write requests related to the virtual server instance are performed on the second storage device; migrating virtual server instance related data associated with running of the virtual server instance stored in the first storage device to a third storage device; and importing the virtual server instance related data stored in the third storage device into the second storage device after the migration is completed, so that the second storage device has full data of the virtual server instance; in response to a snapshot request, performing snapshot processing on the virtual server instance related data stored in the second storage device, to obtain a first snapshot; wherein the first snapshot comprises a data part of the virtual server instance after IO stream switching; performing snapshot processing on the virtual server instance related data stored in the third storage device after the migration is completed, to obtain a second snapshot; wherein the second snapshot comprises a data part of the virtual server instance before IO stream switching. 2.The method of claim 1, further comprising: based on the first snapshot and the second snapshot, obtaining full data of the virtual server instance at a time corresponding to the snapshot request.

3. The method of claim 2, wherein, The first snapshot comprises the virtual server instance related data stored in the second storage device and a first data bitmap used to represent logical addresses of data, the second snapshot comprises the virtual server instance related data stored in the third storage device and a second data bitmap used to represent logical addresses of data, and the step of obtaining full data of the virtual server instance at a time corresponding to the snapshot request based on the first snapshot and the second snapshot comprises: merging the first data bitmap and the second data bitmap to obtain a third data bitmap capable of representing logical addresses of full data of the virtual server instance at a time corresponding to the snapshot request; for each logical address in the third data bitmap, determining whether there is data corresponding to the logical address in the first snapshot, if there is, selecting data corresponding to the logical address from the first snapshot, and if there is not, selecting data corresponding to the logical address from the second snapshot. 4.The method of claim 2, further comprising: uploading the full data of the virtual server instance at a time corresponding to the snapshot request to a storage server for backup. 5.The method of claim 1, further comprising: importing data of the second snapshot into the second storage device. 6.A data migration method, comprising: migrate data required for starting the virtual server instance stored in the first storage device to the second storage device, so as to enable the computing device to start the virtual server instance according to data stored in the second storage device, wherein, after the computing device starts the virtual server instance according to data stored in the second storage device, data read-write requests related to the virtual server instance are executed for the second storage device; migrate virtual server instance related data associated with running of the virtual server instance stored in the first storage device to the third storage device; after the migration is completed, import virtual server instance related data stored in the third storage device into the second storage device, so that the second storage device has full amount of data of the virtual server instance.

7. A data snapshot method, comprising: migrate data required for starting the virtual server instance stored in the first storage device to the second storage device, so as to enable the computing device to start the virtual server instance according to data stored in the second storage device, wherein, after the computing device starts the virtual server instance according to data stored in the second storage device, data read-write requests related to the virtual server instance are executed for the second storage device; snapshot virtual server instance related data associated with running of the virtual server instance stored in the first storage device, to obtain a third snapshot; in response to a snapshot request, snapshot virtual server instance related data stored in the second storage device, to obtain a first snapshot, wherein the first snapshot comprises a data part of the virtual server instance after IO stream switching; based on the first snapshot and the third snapshot, obtain full amount of data of the virtual server instance at a time corresponding to the snapshot request.

8. A data snapshot device, comprising: a first migration module, configured to migrate data required for starting the virtual server instance stored in the first storage device to the second storage device, so as to enable the computing device to start the virtual server instance according to data stored in the second storage device, wherein, after the computing device starts the virtual server instance according to data stored in the second storage device, data read-write requests related to the virtual server instance are executed for the second storage device; a second migration module, configured to migrate virtual server instance related data associated with running of the virtual server instance stored in the first storage device to the third storage device, and after the migration is completed, import virtual server instance related data stored in the third storage device into the second storage device, so that the second storage device has full amount of data of the virtual server instance; a first snapshot module, configured to, in response to a snapshot request, snapshot virtual server instance related data stored in the second storage device, to obtain a first snapshot; wherein the first snapshot comprises a data part of the virtual server instance after IO stream switching. a second snapshot module configured to take a snapshot of the virtual server instance related data stored in the third storage device after the migration is completed, to obtain a second snapshot, the second snapshot comprising a data portion of the virtual server instance before the IO stream is switched.

9. A data migration apparatus comprising: a first migration module configured to migrate data required for starting a virtual server instance stored in a first storage device to a second storage device, to enable a computing device to start the virtual server instance according to data stored in the second storage device, wherein after the computing device starts the virtual server instance according to data stored in the second storage device, data read and write requests related to the virtual server instance are performed on the second storage device; a second migration module configured to migrate virtual server instance related data associated with running of the virtual server instance stored in the first storage device to a third storage device; an import module configured to import the virtual server instance related data stored in the third storage device to the second storage device after the migration is completed, to enable the second storage device to have full data of the virtual server instance.

10. A data snapshot apparatus comprising: a migration module configured to migrate data required for starting a virtual server instance stored in a first storage device to a second storage device, to enable a computing device to start the virtual server instance according to data stored in the second storage device, wherein after the computing device starts the virtual server instance according to data stored in the second storage device, data read and write requests related to the virtual server instance are performed on the second storage device; a third snapshot module configured to take a snapshot of virtual server instance related data associated with running of the virtual server instance stored in the first storage device, to obtain a third snapshot; a first snapshot module configured to take a snapshot of virtual server instance related data stored in the second storage device in response to a snapshot request in a migration process, to obtain a first snapshot, wherein the first snapshot comprises a data portion of the virtual server instance after an IO stream is switched; full data of the virtual server instance at a time corresponding to the snapshot request is obtained based on the first snapshot and the third snapshot.

11. A computing device comprising: a processor; and a memory having stored thereon executable code that, when executed by the processor, causes the processor to perform the method of any of claims 1-7.

12. A computer program product comprising executable code that, when executed by a processor of an electronic device, causes the processor to perform the method of any of claims 1-7.

13. A non-transitory machine-readable storage medium having stored thereon executable code that, when executed by a processor of an electronic device, causes the processor to perform the method of any of claims 1-7.

Images