A data migration method, system and storage medium based on storage cluster

Abstract

This application discloses a data migration method, system, and storage medium based on a storage cluster. Data migration is essentially user-imperceptible, and the impact of IO request latency is in the microsecond range. The method includes: responding to a data migration instruction, triggering a dual-write mode of a block storage access client, and synchronizing incremental data to a source storage cluster and a target storage cluster by executing the dual-write mode; wherein the data migration instruction is used to instruct the migration of data from the source storage cluster to the target storage cluster; and triggering the storage nodes in the source storage cluster to respond to the data migration instruction, generate corresponding snapshot data for the existing data, and migrate the snapshot data to the target storage cluster.

Description

Technical Field

[0001] The present application relates to the field of cloud storage technology, and in particular to a data migration method, system, and storage medium based on a storage cluster. Background Art

[0002] Currently, cloud computing vendors typically offer EBS (Elastic Block Storage) services, providing users with the ability to dynamically allocate and adjust storage resources. This EBS service allows users to quickly expand or contract storage resources based on changing business needs. This places high demands on cloud computing vendors' operational and maintenance capabilities, requiring them to dynamically schedule storage resources by comprehensively considering storage capacity and performance. In practice, EBS data migration allows for secondary scheduling of already created cloud disks, enabling dynamic scheduling of storage resources.

[0003] In existing technologies, if data needs to be migrated from one storage cluster to another, a data migration agent module is set up to continuously compare and synchronize data between the cloud disks of the two storage clusters. However, to ensure data consistency, this method prevents the cloud disks of the storage cluster from processing new IO (Input / Output) requests during the last data synchronization, resulting in a longer IO request interruption time that users may perceive. Summary of the Invention

[0004] In view of this, embodiments of the present application provide a data migration method, system, and storage medium based on a storage cluster. Data migration is basically imperceptible to users, and the impact of IO request delay is in the microsecond range.

[0005] To solve the above problems, the technical solutions provided in the embodiments of the present application are as follows:

[0006] In a first aspect, an embodiment of the present application provides a data migration method based on a storage cluster, the method comprising:

[0007] In response to a data migration instruction, triggering a dual-write mode of the block storage access client, and synchronizing incremental data to the source storage cluster and the target storage cluster by executing the dual-write mode; wherein the data migration instruction is used to instruct to migrate data from the source storage cluster to the target storage cluster; and

[0008] The storage nodes in the source storage cluster are triggered to respond to the data migration instruction, generate corresponding snapshot data for the existing data, and migrate the snapshot data to the target storage cluster.

[0009] In a second aspect, an embodiment of the present application provides a data migration system based on a storage cluster, the system comprising:

[0010] Multiple storage clusters and a management and control server for managing and controlling the multiple storage clusters;

[0011] The management and control server is configured to execute a response data migration instruction, trigger the dual-write mode of the block storage access client, and synchronize incremental data to the source storage cluster and the target storage cluster by executing the dual-write mode; wherein, the data migration instruction is used to instruct the migration of data from the source storage cluster to the target storage cluster; and trigger the storage nodes in the source storage cluster to respond to the data migration instruction, generate corresponding snapshot data for the existing data, and migrate the snapshot data to the target storage cluster.

[0012] In a third aspect, an embodiment of the present application provides a computer-readable storage medium, wherein instructions are stored in the computer-readable storage medium. When the instructions are executed on a terminal device, the terminal device executes the data migration method based on the storage cluster as described above.

[0013] It can be seen that the embodiments of the present application have the following beneficial effects:

[0014] In the embodiment of the present application, the block storage availability zone management server can respond to the data migration instruction to migrate the data of the source storage cluster to the target storage cluster. During the data migration process, the dual-write mode of the block storage access client is triggered, and the dual-write mode is executed by the block storage access client to synchronize the incremental data to the source storage cluster and the target storage cluster. It also triggers the storage nodes in the source storage cluster to generate corresponding snapshot data for the existing data, and migrates the snapshot data to the target storage cluster. The existing data is migrated to the target storage cluster through the snapshot data, and the incremental data is directly sent to the source storage cluster and the target storage cluster. Only when the dual-write mode of the block storage access client is triggered, the IO request is suspended in microseconds. The IO request interruption time during the data migration process is extremely short, and is basically imperceptible to the user. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 A schematic diagram of a data migration scenario based on a storage cluster in an embodiment of the present application;

[0016] Figure 2 A schematic diagram of a data migration method based on a storage cluster in the prior art;

[0017] Figure 3 A schematic diagram of an exemplary application scenario provided in an embodiment of the present application;

[0018] Figure 4 A flowchart of a data migration method based on a storage cluster provided in an embodiment of the present application;

[0019] Figure 5 A schematic diagram of a data migration method based on a storage cluster in an embodiment of the present application;

[0020] Figure 6 This is a schematic diagram of the data migration process after the EBS client turns on the dual write mode in an embodiment of the present application;

[0021] Figure 7 A schematic diagram of sending a write request in a dual write mode in the prior art;

[0022] Figure 8 This is a schematic diagram of sending a write request in the dual write mode in an embodiment of the present application;

[0023] Figure 9 A schematic diagram of a two-stage write mode switching mechanism provided in an embodiment of the present application;

[0024] Figure 10 A schematic diagram of a data migration system based on a storage cluster provided in an embodiment of the present application. DETAILED DESCRIPTION

[0025] In order to make the above-mentioned purposes, features and advantages of the embodiments of the present application more obvious and easy to understand, the embodiments of the present application are further described in detail below with reference to the accompanying drawings and specific implementation methods.

[0026] In order to facilitate understanding and explanation of the technical solutions provided by the embodiments of the present application, the technical terms that may be involved in the embodiments of the present application will be explained below.

[0027] Elastic Computing Service (ECS): Elastic computing service provided by cloud computing vendors.

[0028] Elastic Block Store (EBS): an elastic block storage service provided by cloud computing vendors.

[0029] Node: A server in ECS and EBS.

[0030] Virtual Machine (VM): A virtual machine provided by a cloud computing vendor. For example, a VM can be provided in an ECS node.

[0031] Cluster: In ECS and EBS, a management unit consisting of multiple nodes.

[0032] Availability Zone (AZ): In cloud computing scenarios, an administrative unit with independent power and network resources that can contain multiple clusters.

[0033] Cloud disk (Block Device): A virtual block device provided by EBS.

[0034] Device Segment: A cloud disk is divided into segments of a fixed size. For example, a cloud disk is divided into segments of 32GB.

[0035] Storage node: A process that provides read and write services at the device segment granularity and is deployed on each storage node.

[0036] EBS Client: An EBS access module deployed on an ECS node, responsible for forwarding IO requests to the backend EBS cluster.

[0037] Snapshot: Saves the data of the cloud disk at a certain moment.

[0038] IO Hang: The IO request fails during the retry process.

[0039] Snapshot Rebase: Use snapshot data as the earliest data in a cloud disk time series.

[0040] DMA (Direct Memory Access): A memory access technology that allows hardware to directly access memory without the intervention of the CPU (Central Processing Unit). Hardware often uses this technology to accelerate I / O requests.

[0041] In order to facilitate understanding and explanation of the technical solutions provided by the embodiments of the present application, the background technology of the embodiments of the present application will be further described below.

[0042] The EBS service allows users to quickly expand or shrink storage resources based on changing business needs, eliminating the need to plan and purchase a fixed number of storage devices. This places high demands on cloud computing vendors' operational and maintenance capabilities, requiring them to comprehensively consider storage capacity and performance, evenly distributing user storage resources across multiple clusters within the same AZ (Availability Zone) to fully utilize storage resources. Furthermore, as storage technology evolves and iterates, higher-performance, lower-cost storage media and servers will gradually replace older servers. As a storage provider, you must ensure the durability and reliability of user data, minimizing disruption to user experience and impacting their business. EBS data migration is crucial to meeting these requirements. Data migration allows for rescheduling of existing cloud disks, balancing the capacity and performance of existing clusters. Data migration can also be transparently completed during server upgrades without disrupting user business.

[0043] See also Figure 1 The figure shows a schematic diagram of a data migration scenario. A virtual machine (VM) in an ECS cluster can access cloud disks in different clusters through an EBS client. Specifically, the VM can send I / O requests, including read and write requests, to the EBS cluster through the EBS client. When data from cloud disk A in EBS cluster A is migrated to cloud disk B in EBS cluster B, the data remains consistent even though different EBS clusters are accessed before and after the migration. At the same time, the access link switch needs to be as transparent to the user as possible, and the switch must be completed automatically.

[0044] See also Figure 2 As shown, a schematic diagram of a data migration method in the prior art is shown. In the prior art, data migration is achieved by setting a data migration Agent module to continuously compare the data between the cloud disks of the two EBS clusters and synchronize the data. Specifically, the process of implementing data migration includes: the Agent module compares the data differences, and reads the difference data of cloud disk A and writes it to cloud disk B, and continuously synchronizes the data; when the Agent module finds that the data difference is small enough, it notifies the EBS client to suspend writing and suspends the newly received IO request; the data synchronization action is performed for the last time to achieve a data consistent synchronization state; the Agent module notifies the EBS client to switch the access link to cloud disk B, and at the same time resumes the foreground write, and re-sends the previously suspended IO request to cloud disk B.

[0045] However, the existing technology has the following technical problems: when the write traffic is large, especially in high-performance scenarios, the IO request interruption time will be relatively long, and the data migration process will produce significant delay jitter. In order to ensure data consistency, the existing technical solution cannot continue to write data during the last data synchronization. Otherwise, after the last synchronization is completed, some of the newly written data of Cloud Disk A will be missing on Cloud Disk B. The IO request interruption time perceived by the user is the overall time taken for the last data synchronization. If the user's write throughput is relatively large, the amount of data synchronized in the last synchronization may be very large, affecting the estimated time by more than seconds. In addition, the introduction of the Agent module increases the complexity of the architecture and consumes more resources. The existing technical solution specifically sets up a data migration Agent module for data migration, which is responsible for the synchronization of migrated data. In addition, the data needs to go through an extra hop of the network through the Agent module. The concurrency capability of data migration is limited by the number of Agent modules deployed, and the resource consumption is also higher.

[0046] Based on this, the embodiments of the present application provide a data migration method, system and storage medium based on a storage cluster, and the block storage availability zone management server responds to the data migration instruction to migrate the data of the source storage cluster to the target storage cluster. During the data migration process, the dual-write mode of the block storage access client is triggered, and the dual-write mode is executed by the block storage access client to synchronize the incremental data to the source storage cluster and the target storage cluster. It also triggers the storage nodes in the source storage cluster to generate corresponding snapshot data for the existing data, and migrates the snapshot data to the target storage cluster. The existing data is migrated to the target storage cluster through the snapshot data, and the incremental data is directly sent to the source storage cluster and the target storage cluster. Only when the dual-write mode of the block storage access client is triggered, the IO request is suspended in microseconds. The IO request interruption time during the data migration process is extremely short, and is basically imperceptible to the user.

[0047] In order to facilitate understanding of the data migration method based on the storage cluster provided in the embodiment of the present application, Figure 3 See the example scenario shown. Figure 3 As shown in the figure, this figure is a schematic diagram of an exemplary application scenario provided by an embodiment of the present application.

[0048] In actual applications, the virtual machines VM in the ECS cluster access different storage clusters through the block storage access client, that is, the VM can send IO requests to different storage clusters through the block storage access client, and IO requests include read requests, write requests, etc. In some scenarios, it is necessary to migrate the data in the source storage cluster to the target storage cluster. In an embodiment of the present application, online data migration is achieved by reusing the original capabilities of existing components. Among them, the block storage availability zone management server can manage all storage clusters and send the user's common API (Application Programming Interface) requests for the storage cluster to the specific storage cluster, such as creating a cloud disk, deleting a cloud disk, modifying a cloud disk, querying a cloud disk, creating a cloud disk snapshot, and other related operation requests. The embodiment of the present application reuses the atomic capabilities of the storage cluster snapshot to complete data replication, and then uses the write mode switching of the block storage access client to ensure the synchronization of incremental data during data migration.

[0049] Those skilled in the art will understand that Figure 3 The framework diagram shown is only an example in which the embodiments of the present application can be implemented. The scope of application of the embodiments of the present application is not limited by any aspect of the framework.

[0050] To facilitate understanding of the embodiments of the present application, a data migration method based on a storage cluster provided in an embodiment of the present application is described below with reference to the accompanying drawings.

[0051] See also Figure 4 As shown in FIG, this figure is a flow chart of a data migration method based on a storage cluster provided in an embodiment of the present application, as shown in FIG. Figure 4 As shown, the method may include S401-S402:

[0052] S401: In response to a data migration instruction, triggering a dual write mode of a block storage access client to synchronize incremental data to a source storage cluster and a target storage cluster by executing the dual write mode. The data migration instruction is used to instruct data from the source storage cluster to be migrated to the target storage cluster.

[0053] When data migration is required, the block storage availability zone control server obtains data migration instructions and, based on the data migration instructions, determines the source and target storage clusters before and after the data migration. Data migration instructions instruct the migration of data from the source storage cluster to the target storage cluster. Specifically, they involve migrating data from the source cloud disk of the source storage cluster to the target storage cluster. In actual applications, the block storage availability zone control server sends a create a blank cloud disk instruction to the target storage cluster. Based on this create a blank cloud disk instruction, the target storage cluster creates the target cloud disk and feeds back relevant information about the target cloud disk to the block storage availability zone control server. The target cloud disk is used to store data migrated from the source storage cluster.

[0054] In response to a data migration instruction, the block storage availability zone management server can trigger the dual-write mode of the block storage access client. The block storage access client executes the dual-write mode to synchronize incremental data to the source storage cluster and the target storage cluster.

[0055] Specifically, a dual-write mode enable instruction may be sent to the block storage access client.

[0056] In one possible implementation, triggering the dual-write mode of the block storage access client may include:

[0057] In response to the dual write mode enable instruction, a first synchronization point for executing the dual write mode is determined. The first synchronization point is used to indicate that all preceding IOs issued before the first synchronization point are first flushed to disk, and then subsequent IOs issued after the first synchronization point are processed in the dual write mode.

[0058] When switching the write mode, a time point needs to be determined. All IO requests issued before the time point are successfully written to the disk, and all IO requests after the time point are processed in the new write mode. This ensures that the synchronization of existing data does not miss some in-flight IOs. Therefore, the key to switching the write mode is to determine a synchronization point.

[0059] The block storage access client determines the first synchronization point of the IO request in response to the instruction to turn on the dual-write mode. The block storage access client determines the first synchronization point of the IO request by temporarily suspending the IO request, and can distinguish between the preceding IO request before receiving the instruction to turn on the dual-write mode and the subsequent IO request after receiving the instruction to turn on the dual-write mode. The preceding IO request before the first synchronization point continues to be processed according to the original processing method and continues to be sent to the source storage cluster, for example, to the source cloud disk of the source storage cluster. The subsequent IO request after the first synchronization point performs dual-write processing according to the new processing method, and the write data generated by the subsequent IO request is incremental data, so that the incremental data is sent to the source storage cluster and the target storage cluster at the same time, for example, the incremental data is sent to the source cloud disk of the source storage cluster and the target cloud disk of the target storage cluster at the same time.

[0060] After the block storage access client completes processing the preceding IO requests before the first synchronization point, it will feedback the write mode switching completion information to the block storage availability zone management server. At this time, the preceding IO requests before the first synchronization point are all written to the source storage cluster, and the newly generated subsequent IO requests after the first synchronization point have been migrated to the target storage cluster and are also retained in the source storage cluster.

[0061] Regarding the specific implementation process of the block storage access client executing the dual write mode, please refer to the subsequent embodiments and will not be described in detail here.

[0062] S402: triggering the storage nodes in the source storage cluster to respond to the data migration instruction, generate corresponding snapshot data for the existing data, and migrate the snapshot data to the target storage cluster.

[0063] In response to the write mode switch completion information, the block storage availability zone management server sends a snapshot creation instruction for the existing data to the source storage cluster. In response to the data migration instruction, the storage nodes in the source storage cluster create snapshot data corresponding to the existing data, such as the source cloud disk snapshot of the source cloud disk. The snapshot data has snapshot information, which records the creation path and creation time of the snapshot data. Since the source storage cluster responded to the IO request before the first synchronization point, the snapshot is created after receiving the write mode switch completion information. The previous IO request before the first synchronization point has been processed, ensuring the integrity of the data in the snapshot data.

[0064] The block storage availability zone control server sends the snapshot information to the target storage cluster. The target storage cluster can then read the snapshot data from the source storage cluster and copy it to the target cloud disk. While the snapshot data is being copied to the target cloud disk, the target cloud disk can also receive new I / O requests. In particular, when the I / O request is a write request, incremental data is also written to the target cloud disk while the existing data is copied to the target cloud disk. This ensures data consistency without pausing I / O requests.

[0065] In addition, when copying snapshot data, the target storage cluster can divide the snapshot data into multiple segments and copy the segments of the snapshot data in parallel to increase the speed of copying the snapshot data.

[0066] In this way, in the embodiment of the present application, the block storage availability zone management server can respond to the data migration instruction to migrate the data of the source storage cluster to the target storage cluster. During the data migration process, the dual-write mode of the block storage access client is triggered, and the dual-write mode is executed by the block storage access client to synchronize the incremental data to the source storage cluster and the target storage cluster. It also triggers the storage nodes in the source storage cluster to generate corresponding snapshot data for the existing data, and migrates the snapshot data to the target storage cluster. The existing data is migrated to the target storage cluster through the snapshot data, and the incremental data is directly sent to the source storage cluster and the target storage cluster. Only when the dual-write mode of the block storage access client is triggered, the IO request is suspended in microseconds. The IO request interruption time during the data migration process is extremely short, and is basically imperceptible to the user.

[0067] In a possible implementation, the following may also be included:

[0068] In response to the instruction to turn off the dual write mode, a second synchronization point for turning off the dual write mode is determined. The second synchronization point is used to indicate that all IOs issued before the second synchronization point are first written to the disk in the dual write mode, and then the IOs issued after the second synchronization point are processed in the single write mode.

[0069] When the block storage availability zone control server copies snapshot data to the target storage cluster, it needs to poll the status of the copied snapshot data. When it determines that the snapshot data copy is complete, it can send a command to the block storage access client to disable dual write mode.

[0070] The block storage access client determines the second synchronization point of the IO request in response to the instruction to turn off the dual-write mode. The block storage access client determines the second synchronization point of the IO request by temporarily suspending the IO request, and can distinguish the IO request before receiving the instruction to turn off the dual-write mode from the IO request after receiving the instruction to turn off the dual-write mode. The IO request before receiving the instruction to turn off the dual-write mode is the preceding IO request before the second synchronization point, and continues to be processed according to the original processing method and continues to be sent to the source storage cluster and the target storage cluster. The IO request after receiving the instruction to turn off the dual-write mode is the subsequent IO request after the second synchronization point, and is processed according to the new processing method to execute the single-write mode and is only sent to the target storage cluster.

[0071] In actual applications, the block storage availability zone management server can be the available block storage management module. The available block storage management module is an AZ-level management component responsible for managing all storage clusters under the AZ. The block storage access client can be an EBS client. Virtual machines (VMs) in an ECS cluster can access cloud disks in different storage clusters through the EBS client. That is, VMs can send I / O requests to the EBS cluster through the EBS client. I / O requests include read requests and write requests. In some scenarios, data from the source cloud disk in the source storage cluster needs to be migrated to the target cloud disk in the target storage cluster. For example, the source storage cluster is EBS cluster A, the source cloud disk is cloud disk A, the target storage cluster is EBS cluster B, and the target cloud disk is cloud disk B. Migrate the data from cloud disk A in EBS cluster A to cloud disk B in EBS cluster B.

[0072] See also Figure 5 , which is a schematic diagram showing an actual application scenario of a data migration method based on a storage cluster in an embodiment of the present application.

[0073] In this embodiment of the application, it is necessary to migrate data from a source cloud disk in a source storage cluster to a target cloud disk in a target storage cluster. This example uses EBS cluster A as the source storage cluster, Cloud Disk A as the source cloud disk, EBS cluster B as the target storage cluster, and Cloud Disk B as the target cloud disk. The following description uses the example of migrating data from Cloud Disk A in EBS cluster A to Cloud Disk B in EBS cluster B.

[0074] The Available Block Storage Management Module triggers EBS Cluster B to create Cloud Disk B and sends a command to the EBS client to enable dual write mode. The EBS client determines the first synchronization point for I / O requests and, after processing all preceding I / O requests up to the first synchronization point, sends a write mode switch completion message to the Available Block Storage Management Module. Upon receiving the write mode switch completion message, the Available Block Storage Management Module triggers EBS Cluster A to create a Cloud Disk A snapshot of Cloud Disk A. Subsequent I / O requests after the first synchronization point are sent by the EBS client to both Cloud Disk A and Cloud Disk B.

[0075] It's understandable that when a snapshot of the source cloud disk is created after the EBS client processes IO requests before the first synchronization point, the snapshot will contain all the existing data, ensuring consistency after data migration. However, due to the time difference between when IO requests are sent to the source cloud disk and when the snapshot data is created, the snapshot data may also include some data written by IO requests after the first synchronization point. This data also exists on the target cloud disk. The generation of some redundant data does not affect data consistency.

[0076] The Available Block Storage Management Module sends the snapshot information of the source cloud disk snapshot to EBS Cluster B, which then copies the snapshot of Cloud Disk A to Cloud Disk B. When the Available Block Storage Management Module detects that the copy is complete, it sends a command to the EBS client to disable dual-write mode. The EBS client determines the second synchronization point for I / O requests. After processing I / O requests before the second synchronization point, it stops sending I / O requests after the second synchronization point to Cloud Disk A and sends only I / O requests after the second synchronization point to Cloud Disk B, completing the data migration process.

[0077] See also Figure 6 As shown, a schematic diagram of the data migration process after the EBS client turns on the dual-write mode in an embodiment of the present application is shown. In stage 1, data migration begins, the source cloud disk stores the original stock data, and a target cloud disk is created. In stage 2, the EBS client receives an instruction to turn on the dual-write mode, and after processing the IO requests before the first synchronization point, it starts to send IO requests to the source cloud disk and the target cloud disk at the same time, and incremental data 1 is generated in the source cloud disk and the target cloud disk. Then a snapshot of the source cloud disk is created, and the snapshot data includes the stock data and incremental data 1. The snapshot data serves as the basis for data migration to the target cloud disk. In stage 3, the target cloud disk copies the snapshot data, and the EBS client sends IO requests to the source cloud disk and the target cloud disk at the same time, and incremental data 2 is generated in the source cloud disk and the target cloud disk. In stage 4, the copy is completed, and the EBS client receives an instruction to turn off the dual-write mode. After processing the IO requests before the second synchronization point, it only sends IO requests to the target cloud disk. Then, during the data migration process, the stock data and incremental data are completely consistent. There is some redundant incremental data 1 in the target cloud disk, but it does not affect the consistency of data migration. Existing data is migrated in parallel on the cluster side, while incremental data is synchronized by the dual-write mode. The dual-write mode is switched on and off smoothly to ensure that users are unaware of the entire process.

[0078] In order to ensure the performance of the dual-write mode of the block storage access client, the embodiment of the present application also optimizes the writing method of the block storage access client.

[0079] In one possible implementation, synchronizing incremental data to the source storage cluster and the target storage cluster by executing the dual-write mode may include:

[0080] Before writing incremental data, first apply for DMA memory and store the incremental data in the DMA memory, and then use the DMA memory to send the incremental data to the source storage cluster and the target storage cluster.

[0081] When using user-mode transmission protocols, such as user-mode TCP (Transmission Control Protocol) and RDMA (Remote Direct Memory Access), application layer data usually needs to be copied into DMA memory DMA_MEM before being sent by the network card. A write request requires a copy from the user write buffer to the DMA memory. Figure 7 FIG. 1 is a schematic diagram showing a prior art method for sending a write request in a dual write mode. In the dual write mode, the prior art method requires two copies of the user buffer to the DMA memory.

[0082] See also Figure 8 As shown, a schematic diagram of the write request sent in the dual write mode in an embodiment of the present application is shown. Because the write data in the dual write mode is unchanged, in an embodiment of the present application, DMA memory is applied for in advance at the block storage access client application layer, and the user buffer data (i.e., the write data of the write request) is copied to the DMA memory. If the write data is directly stored in the DMA memory, the network card can send it directly. Then, the DMA memory can be used directly to send the data twice, reducing the copy overhead once and optimizing the throughput and latency of the dual write mode.

[0083] In an embodiment of the present application, DMA memory is pre-allocated to reduce the data copying overhead from 2 times to 1 time, thereby greatly optimizing the performance overhead of the dual-write mode, so that the performance of the dual-write mode is basically the same as that of the single-write mode.

[0084] In order to ensure smooth switching of the write mode of the EBS client, the embodiment of the present application provides a two-stage write mode switching mechanism.

[0085] In a possible implementation, in response to an instruction to enable the dual write mode, determining a first synchronization point for executing the dual write mode may include:

[0086] A1: In response to an instruction to enable dual-write mode, the controller notifies each IO queue to suspend foreground IO and determines the maximum IO sequence number among all IO queues as the first synchronization point for executing dual-write mode. The IO sequence numbers in each IO queue are assigned in ascending order based on the order of IO reception time.

[0087] A2: The controller sends the first synchronization point to each I / O queue and notifies each I / O queue to resume foreground I / O. After all I / O less than or equal to the first synchronization point is completed, the controller returns a successful dual-write switch and controls all I / O greater than the first synchronization point to be processed in dual-write mode.

[0088] Traditional HDD (Hard Disk Drive) devices usually have only one queue, and the latency itself is relatively high, so the simplest IO Fencing can be used to implement it. Suspend the foreground IO request, wait for the in-flight IO request to be written to the disk, and re-issue the suspended IO to meet the above requirements. However, today's storage media are changing with each passing day, and performance is constantly improving. In order to achieve higher throughput capabilities, multiple queues are usually supported. For example, the NVMe protocol supports a maximum of 64K queues, which greatly improves the concurrency of IO requests. In high-performance storage application scenarios, if the number of concurrent IO requests is large, then all foreground suspended IO requests need to be written to the disk before the slowest IO request before the synchronization point. Ordinary IO Fencing is likely to bring more obvious IO long-tail delays.

[0089] In order to solve the above problems, the embodiment of the present application designs a two-stage write mode switching mechanism that supports multiple queues to achieve smooth write mode switching. First, it is necessary to implement an IO sequence number allocator to assign a sequence number to each IO request when the IO request is received, to ensure that the IO request that arrives first has a smaller sequence number and the IO that arrives later has a larger sequence number. In a multi-queue multi-threaded scenario, a timestamp can be used as the sequence number, or a variable can be used in combination with a CAS instruction to complete the sequence number allocation. The IO sequence number allocator ensures the semantics of the synchronization point in the embodiment of the present application.

[0090] In a possible implementation, the method further includes: when receiving an IO, using the timestamp of the IO reception time as the serial number of the IO; or using a variable in combination with a CAS instruction to assign the serial number of the IO.

[0091] Then, in the first stage of the two-stage write mode switching mechanism, the block storage access client receives the instruction to open the bidirectional mode, notifies each IO queue to suspend the foreground IO request, and returns the current maximum IO sequence number of each IO queue. After all IO queues respond, the maximum IO sequence number is taken, and the obtained value is the first synchronization point.

[0092] In the second phase of the two-phase write mode switching mechanism, the first synchronization point is sent to each IO queue. Each IO queue immediately resumes foreground IO, and all IO requests greater than the first synchronization point are processed in the new manner and sent to both the source storage cluster and the target source storage cluster. Each IO queue waits for all IO requests less than or equal to the first synchronization point to be processed, and then returns the first switching completion feedback information. After receiving the first switching completion feedback information from all IO queues, the block storage access client can send a write mode switching completion information to the block storage availability zone management server.

[0093] See also Figure 9 As shown, a schematic diagram of the two-stage write mode switching mechanism provided by an embodiment of the present application is shown. A controller may be included in the block storage access client. In the first stage of the two-stage write mode switching mechanism, after the controller receives the instruction to turn on the dual write mode, it notifies each IO queue to suspend the foreground IO request and returns the current maximum IO sequence number of each IO queue. After all IO queues respond, the controller takes the maximum value of the maximum IO sequence number, and the obtained value is the sequence number Tsync of the first synchronization point, for example, Tsync is T4.

[0094] In the second phase of the two-phase write mode switching mechanism, the controller sends the first synchronization point's serial number, Tsync, to each I / O queue. Each I / O queue immediately resumes foreground I / O, and all I / O requests greater than Tsync are processed in dual-write mode. For example, the IO requests marked with a shadow in the figure are all processed in dual-write mode. Each I / O queue waits for all I / O requests less than or equal to Tsync to complete before returning to the controller to indicate the switch is complete, i.e., returning the first switch completion feedback message. After all I / O queues have completed the switch, the dual-write mode switch is enabled.

[0095] In an embodiment of the present application, the communication between the controller and the IO queue is implemented by configuring a polling mode with two lock-free queues, one lock-free queue stores requests from the controller to the IO queue, and the other lock-free queue stores responses from the IO queue to the controller.

[0096] This embodiment ensures that the impact of an I / O request only involves the notification time between the controller and the I / O queues, including the controller notifying each I / O queue to suspend the foreground I / O request, each I / O queue returning the current maximum I / O sequence number, and the controller sending the first synchronization point to each I / O queue. Theoretically, this impact time is only microseconds, avoiding long I / O request blocking caused by waiting for I / O requests before the synchronization point to complete.

[0097] To ensure that the notification delay between the controller and the I / O queue is sufficiently small, the communication mechanism between the controller and the I / O queue is implemented using two lock-free queues configured in polling mode. The two lock-free queues store notifications from the controller to the I / O queue and responses from the I / O queue to the controller. The lock-free queues and polling mechanism ensure that notifications in the two-stage write mode switching mechanism can be processed as quickly as possible with extremely small delays, ensuring that the delay does not fluctuate.

[0098] In addition, the switchover is completed only after waiting for I / O requests before the first synchronization point to complete, ensuring data consistency during data migration. Splitting I / O Fencing into two phases allows for smooth write mode switching in multi-queue, high-throughput scenarios, ensuring uncompromised user performance during data migration.

[0099] Similarly, when the dual write mode is disabled, the above two-stage write mode switching mechanism can also be used.

[0100] In one possible implementation, in response to an instruction to disable the dual write mode, determining a second synchronization point for disabling the dual write mode includes:

[0101] B1: In response to the dual-write mode disable instruction, the controller notifies each IO queue to suspend foreground IO and determines the maximum IO sequence number among all IO queues as the second synchronization point for executing the dual-write mode. The IO sequence numbers in each IO queue are assigned in ascending order based on the IO reception time.

[0102] B2: The controller sends the second synchronization point to each IO queue and notifies each IO queue to resume foreground IO. After all IOs less than or equal to the second synchronization point are completed, the controller returns a single-write switch success message and controls all IOs greater than the second synchronization point to be processed in single-write mode.

[0103] In the first phase of the two-phase write mode switching mechanism, the block storage access client receives an instruction to disable dual-write mode, notifies each IO queue to suspend foreground IO requests, and returns the current maximum IO sequence number of each IO queue. After all IO queues respond, the maximum IO sequence number is taken, and the obtained value is the second synchronization point.

[0104] In the second phase of the two-phase write mode switching mechanism, the second synchronization point is sent to each IO queue. Each IO queue immediately resumes foreground IO, and all IO requests greater than the second synchronization point are processed in the new manner and sent only to the target storage cluster. Each IO queue waits for all IO requests less than or equal to the second synchronization point to be processed, and then returns the second switching completion feedback information. After receiving the second switching completion feedback information from all IO queues, the block storage access client can determine that the switching is complete. At this time, it no longer sends IO requests to the source storage cluster, and only sends IO requests to the target storage cluster to execute the single write mode.

[0105] In an embodiment of the present application, a two-stage write mode switching mechanism for multiple queues combines multiple technologies, including an IO sequence number allocator, a two-stage timing synchronization mechanism, message notification based on a lock-free queue, etc., which can achieve smooth switching of write modes in multi-queue high-throughput scenarios, greatly optimizing the user IO interruption time during data migration and ensuring lossless user performance during data migration.

[0106] Furthermore, during the data migration process, since the block storage access client has dual-write mode enabled, I / O requests are still being written to the source storage cluster, allowing the data migration process to be rolled back at any time. Specifically, in response to a data migration rollback instruction, the block storage availability zone control server can trigger the block storage access client to stop sending I / O requests to the target storage cluster, effectively switching from dual-write mode back to single-write mode, where I / O requests are only sent to the source storage cluster. The data migration can then be rolled back by deleting the target cloud disk from the target storage cluster.

[0107] Based on the description of the above embodiments, in the embodiment of the present application, in a high-throughput usage scenario, data migration is basically imperceptible to users, and the storage cluster switching of data migration does not generate IO request jitter.

[0108] In the existing technology, it is assumed that the upper limit of the cloud disk throughput used by the user is 350MB / s and the data copy speed is 500MB / s. Assuming that the last data synchronization without pausing the foreground IO request takes 20 seconds, and the user continues to write at the maximum throughput during the data synchronization period, the time taken for the last data synchronization after pausing the IO request is 350*20 / 500=14 seconds. The user can perceive the 14-second increase in IO request latency. Similarly, even if the time taken for the last data synchronization without pausing the foreground IO request is optimized to 3 seconds, the user can still perceive a maximum increase in IO latency of 2.1 seconds.

[0109] In the embodiment of the present application, regardless of the user's cloud disk throughput limit or data copy speed, the IO jitter time depends only on the delay of the IO fencing message notification in the block storage access client. Each queue includes two lock-free queue message communications in the first phase and one lock-free queue message communication in the second phase. Assume that the message communication delay of a lock-free queue is L (usually within 10us, Ln represents the delay of queue n). IO impact time

[0110] =max(2*L1,2*L2,...2*Ln)+max(L1,L2,...Ln)≈30us. The IO jitter is expected to be reduced to about 30us. Compared with existing technical solutions, the impact of data migration delay is expected to be smoother and more controllable, and the delay impact is in the microsecond range.

[0111] Based on the data migration method based on a storage cluster provided in the above method embodiment, the embodiment of the present application also provides a data migration system based on a storage cluster, which will be described below with reference to the accompanying drawings.

[0112] See also Figure 10 As shown in FIG, this figure is a structural diagram of a data migration system based on a storage cluster provided by an embodiment of the present application. Figure 10 As shown, the data migration system includes:

[0113] Multiple storage clusters 1001 and a management server 1002 for managing the multiple storage clusters;

[0114] The management and control server 1002 is configured to execute a response data migration instruction, trigger the dual-write mode of the block storage access client, and synchronize incremental data to the source storage cluster and the target storage cluster by executing the dual-write mode; wherein, the data migration instruction is used to instruct the migration of data from the source storage cluster to the target storage cluster; and trigger the storage nodes in the source storage cluster to respond to the data migration instruction, generate corresponding snapshot data for the existing data, and migrate the snapshot data to the target storage cluster.

[0115] In one possible implementation, the system further includes a cloud computing cluster; the cloud computing cluster is provided with a block storage access client; and the cloud computing cluster is configured to:

[0116] Before writing incremental data, a direct memory access (DMA) memory is requested and the incremental data is stored in the DMA memory. The DMA memory is then used to send the incremental data to the source storage cluster and the target storage cluster.

[0117] In a possible implementation, the cloud computing cluster is further configured to:

[0118] In response to the dual write mode enable instruction, a first synchronization point for executing the dual write mode is determined. The first synchronization point is used to indicate that all preceding IOs issued before the first synchronization point are first written to disk and then subsequent IOs issued after the first synchronization point are processed in the dual write mode.

[0119] In a possible implementation, the cloud computing cluster is further configured to:

[0120] In response to the dual-write mode enable instruction, the controller notifies each IO queue to suspend the foreground IO and determines the maximum IO sequence number among all IO queues as the first synchronization point for executing the dual-write mode; wherein the IO sequence numbers in each IO queue are assigned in ascending order according to the order in which the IOs are received;

[0121] The controller sends the first synchronization point to each IO queue and notifies each IO queue to resume the foreground IO. After waiting for all IOs less than or equal to the first synchronization point to complete, the controller returns a successful dual-write switch and controls all IOs greater than the first synchronization point to be processed in dual-write mode.

[0122] In a possible implementation, the cloud computing cluster is further configured to:

[0123] In response to the instruction to turn off the dual write mode, a second synchronization point for turning off the dual write mode is determined. The second synchronization point is used to indicate that all IOs issued before the second synchronization point are first written to the disk in the dual write mode, and then the IOs issued after the second synchronization point are processed in the single write mode.

[0124] In a possible implementation, the cloud computing cluster is further configured to:

[0125] In response to the dual-write mode disable instruction, the controller notifies each IO queue to suspend the foreground IO and determines the maximum IO sequence number among all IO queues as the second synchronization point for executing the dual-write mode; wherein the IO sequence numbers in each IO queue are assigned in ascending order according to the order in which the IOs are received;

[0126] The controller sends the second synchronization point to each IO queue and notifies each IO queue to resume the foreground IO. After waiting for all IOs less than or equal to the second synchronization point to complete, the controller returns a single-write switch success and controls all IOs greater than the second synchronization point to be processed in single-write mode.

[0127] In a possible implementation, the cloud computing cluster is further configured to:

[0128] When receiving an IO, the timestamp of the IO reception time is used as the IO sequence number; or, a variable is used in combination with a CAS instruction to assign the IO sequence number.

[0129] In a possible implementation, the communication between the controller and the IO queue is implemented using two lock-free queues configured in a polling mode, one lock-free queue stores requests from the controller to the IO queue, and the other lock-free queue stores responses from the IO queue to the controller.

[0130] In addition, an embodiment of the present application further provides a computer-readable storage medium, characterized in that the computer-readable storage medium stores instructions, which, when executed on a terminal device, enable the terminal device to execute any of the storage cluster-based data migration methods described above.

[0131] An embodiment of the present application further provides a computer program product, including computer program instructions. When the computer program instructions are executed on a computer, the computer is caused to execute any of the above-described storage cluster-based data migration methods.

[0132] In the embodiment of the present application, the block storage availability zone management server can respond to the data migration instruction to migrate the data of the source storage cluster to the target storage cluster. During the data migration process, the dual-write mode of the block storage access client is triggered, and the dual-write mode is executed by the block storage access client to synchronize the incremental data to the source storage cluster and the target storage cluster. It also triggers the storage nodes in the source storage cluster to generate corresponding snapshot data for the existing data, and migrates the snapshot data to the target storage cluster. The existing data is migrated to the target storage cluster through the snapshot data, and the incremental data is directly sent to the source storage cluster and the target storage cluster. Only when the dual-write mode of the block storage access client is triggered, the IO request is suspended in microseconds. The IO request interruption time during the data migration process is extremely short, and is basically imperceptible to the user.

[0133] It should be noted that the various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. Reference can be made to the common and similar parts between the various embodiments. For the systems or devices disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the description is relatively simple, and the relevant parts can be referred to the description of the methods.

[0134] It should be understood that in this application, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship. "At least one of the following items" or similar expressions refers to any combination of these items, including any combination of single items or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0135] It should also be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of additional identical elements in the process, method, article, or device comprising the element.

[0136] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein may be implemented directly using hardware, a software module executed by a processor, or a combination of the two. The software module may be placed in a random access memory (RAM), internal memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

[0137] The above description of the disclosed embodiments is intended to enable one skilled in the art to implement or use the present application. Various modifications to these embodiments will be readily apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application is not limited to the embodiments shown herein, but is intended to conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A data migration method based on a storage cluster, characterized in that: The method comprises: In response to the data migration instruction, the block storage access client is triggered to respond to the dual-write mode opening instruction, first suspend the foreground input and output IO and determine the maximum IO sequence number as the first synchronization point for executing the dual-write mode and then resume the foreground IO, the first synchronization point is used to distinguish between the preceding IO before receiving the dual-write mode opening instruction and the subsequent IO after receiving the dual-write mode opening instruction, the first synchronization point is used to indicate that all the preceding IOs issued before the first synchronization point are first written to the disk and then the subsequent IOs issued after the first synchronization point are processed in the dual-write mode, and the incremental data is sent to the source storage cluster and the target storage cluster at the same time by executing the dual-write mode; wherein, the data migration instruction is used to indicate that the data of the source storage cluster is migrated to the target storage cluster; and The storage nodes in the source storage cluster are triggered to respond to the data migration instruction, generate corresponding snapshot data for the existing data, and migrate the snapshot data to the target storage cluster.

2. The method according to claim 1, characterized in that The method of synchronizing incremental data to the source storage cluster and the target storage cluster by executing the dual-write mode includes: Before writing incremental data, a direct memory access (DMA) memory is requested and the incremental data is stored in the DMA memory. The DMA memory is then used to send the incremental data to the source storage cluster and the target storage cluster.

3. The method according to claim 1, characterized in that In response to the dual-write mode opening instruction, the foreground input and output IO is suspended first, a maximum IO sequence number is determined as a first synchronization point for executing the dual-write mode, and then the foreground IO is resumed, including: In response to the dual-write mode enable instruction, the controller notifies each IO queue to suspend the foreground IO and determines the largest IO sequence number among all IO queues as the first synchronization point for executing the dual-write mode; wherein the sequence numbers of the IOs in each IO queue are assigned in ascending order according to the order in which the IOs were received; The controller sends the first synchronization point to each IO queue and notifies each IO queue to resume the foreground IO. After waiting for all IOs less than or equal to the first synchronization point to complete, the controller returns a successful dual-write switch and controls all IOs greater than the first synchronization point to be processed in dual-write mode.

4. The method according to claim 1, characterized in that The method further comprises: In response to the instruction to turn off the dual write mode, a second synchronization point for turning off the dual write mode is determined. The second synchronization point is used to indicate that all IOs issued before the second synchronization point are first written to the disk in the dual write mode, and then the IOs issued after the second synchronization point are processed in the single write mode.

5. The method according to claim 4, characterized in that: The step of determining a second synchronization point for closing the dual write mode in response to the dual write mode closing instruction includes: In response to the dual-write mode disable instruction, the controller notifies each IO queue to suspend the foreground IO and determines the maximum IO sequence number among all IO queues as the second synchronization point for executing the dual-write mode; wherein the IO sequence numbers in each IO queue are assigned in ascending order according to the order in which the IOs are received; The controller sends the second synchronization point to each IO queue and notifies each IO queue to resume the foreground IO. After waiting for all IOs less than or equal to the second synchronization point to complete, the controller returns a single-write switch success and controls all IOs greater than the second synchronization point to be processed in single-write mode.

6. The method according to claim 3 or 5, characterized in that: The method further comprises: When receiving an IO, the timestamp of the IO reception time is used as the IO sequence number; or, a variable is used in combination with a CAS instruction to assign the IO sequence number.

7. The method according to claim 3 or 5, characterized in that The communication between the controller and the IO queue is implemented by configuring a polling mode with two lock-free queues, one lock-free queue stores requests from the controller to the IO queue, and the other lock-free queue stores responses from the IO queue to the controller.

8. A data migration system based on a storage cluster, characterized in that: The system comprises: Multiple storage clusters and a management and control server for managing and controlling the multiple storage clusters; The management and control server is configured to execute a response data migration instruction, trigger the block storage access client to respond to the dual-write mode opening instruction, first suspend the foreground input and output IO and determine the maximum IO sequence number as the first synchronization point for executing the dual-write mode and then resume the foreground IO, the first synchronization point is used to distinguish between the preceding IO before receiving the dual-write mode opening instruction and the subsequent IO after receiving the dual-write mode opening instruction, the first synchronization point is used to indicate that all the preceding IOs issued before the first synchronization point are first written to the disk and then the subsequent IOs issued after the first synchronization point are processed in the dual-write mode, and the incremental data is sent to the source storage cluster and the target storage cluster at the same time by executing the dual-write mode; wherein, the data migration instruction is used to indicate that the data of the source storage cluster is migrated to the target storage cluster; and, trigger the storage nodes in the source storage cluster to respond to the data migration instruction, generate corresponding snapshot data for the existing data, and migrate the snapshot data to the target storage cluster.

9. A computer-readable storage medium, characterized in that The computer-readable storage medium stores instructions, and when the instructions are executed on a terminal device, the terminal device executes the data migration method based on a storage cluster according to any one of claims 1 to 7.

Images