Data online migration method and device, electronic equipment and storage medium

Abstract

The invention discloses a data online migration method and device, electronic equipment and a storage medium, and relates to the technical field of data storage.The method comprises the steps that block equipment to be migrated in a first storage system (old storage system) and target block equipment in a second storage system (new storage system) are determined; the method comprises the following steps: pre-connecting host equipment with to-be-migrated block equipment, disconnecting a data transmission channel between the host equipment and the to-be-migrated block equipment after the host equipment is connected with target block equipment, only keeping connection between the host equipment and the to-be-migrated block equipment, and migrating equipment data of the to-be-migrated block equipment to a data storage space corresponding to the target block equipment. Therefore, the second storage system responds to and completes a service request (such as a write-in request or a read request) from the host equipment, the problem of service interruption during data migration of the heterogeneous storage system in the related technology is solved, and the technical effect of improving service continuity and data migration reliability is achieved.

Description

Technical Field

[0001] The present application relates to the technical field of data storage, and in particular to a method, device, electronic device, and storage medium for online data migration. Background Art

[0002] With explosive data growth, it's necessary to expand storage system capacity or replace older storage systems with newer ones with larger capacities to meet data storage needs. Heterogeneous storage systems are also called new and older storage systems. These systems are from different vendors, different versions of the same vendor, or different architectures of the same vendor. Heterogeneous storage systems generally don't support online data synchronization.

[0003] Currently, data migration between heterogeneous storage systems is performed offline. This involves first connecting the new storage system to the service host, then stopping service access. The service host then copies the data from the old storage system to the new one. The service host's metadata is then updated, and the data and service requests are re-linked to the new storage system. Finally, service access is restored. This offline migration requires shutting down the service host's equipment, and the time-consuming recovery process after restarting the equipment can result in lengthy service interruptions for the client.

[0004] Therefore, in the related art, there is a problem of business interruption when migrating data in heterogeneous storage systems.

[0005] Regarding the problem of business interruption during data migration of heterogeneous storage systems in related technologies, no effective solution has been proposed so far.

[0006] Therefore, it is necessary to improve the related technology to overcome the above-mentioned defects in the related technology. Summary of the Invention

[0007] The present application provides a method, device, electronic device and storage medium for online data migration, so as to at least solve the problem of service interruption during data migration in heterogeneous storage systems in related technologies.

[0008] The present application provides a method for online data migration, comprising: determining, based on a block device mapping relationship, a target block device after a block device to be migrated in a first storage system is mapped to a second storage system, wherein the block device mapping relationship represents a path mapping relationship between the block device to be migrated and the target block device, and the block device information of the block device to be migrated is the same as the block device information of the target block device; after a host device is connected to the target block device, disconnecting a data transmission channel between the host device and the block device to be migrated; and migrating device data of the block device to be migrated to data storage space corresponding to the target block device, wherein the host device has been pre-connected to the block device to be migrated.

[0009] In an exemplary embodiment, before determining the target block device after the block device to be migrated in the first storage system is mapped to the second storage system based on the block device mapping relationship, the method further includes: adding an access control list corresponding to the system node of the second storage system to the device data of the block device to be migrated, and mounting the device node of the block device to be migrated to the system node according to the access control list; creating a blank block device in the second storage system based on the block device information of the block device to be migrated, setting the block device information of the block device to be migrated to the block device information of the blank block device, and obtaining the target block device; setting a migration mark for the target block device, binding the migration mark and the access path of the block device to be migrated, and obtaining the block device mapping relationship.

[0010] In an exemplary embodiment, the host device is connected to the target block device in the following manner: determining a first device node path of the block device to be migrated from block device information of the block device to be migrated, and determining a second device node path of the target block device from block device information of the target block device, the first device node path and the second device node path having the same device number; storing the first device node path and the second device node path in the host device to obtain aggregate path information; and mounting the device node of the target block device to the device node of the host device according to the aggregate path information to connect the host device to the target block device.

[0011] In an exemplary embodiment, migrating the device data of the block device to be migrated to the data storage space corresponding to the target block device includes: establishing a data migration thread between the first storage system and the second storage system; using the data migration thread to migrate all data shards corresponding to the device data of the block device to be migrated to the target block device; and generating a migration result after the migration of all data shards is completed, sending the migration result to the first storage system and the second storage system, the migration result including at least one of the following: migration completed, migration interrupted, migration failed, and migration in progress.

[0012] In an exemplary embodiment, the method further includes: in the process of migrating the device data of the block device to be migrated to the data storage space corresponding to the target block device, determining the data shard to be written corresponding to the write request on the target block device according to the first offset value and the first data length of the write request of the host device; and when it is determined that the data shard to be written already exists in the second storage system, writing the data to be written in the write request to the data shard to be written.

[0013] In an exemplary embodiment, the method further includes: in a case where it is determined that the data shard to be written does not exist in the second storage system, determining the original data shard corresponding to the first offset value from all data shards of the block device to be migrated, wherein the original data shard has a starting offset value and an ending offset value, the first offset value is greater than the starting offset value, and the first offset value is less than the ending offset value; determining the difference between the first offset value and the starting offset value as the relative offset value of the data to be written; determining a data sub-shard to be updated in the original data shard, the starting offset value of the data sub-shard to be updated is the same as the relative offset value, and the length between the starting offset value of the data sub-shard to be updated and the ending offset value of the data sub-shard to be updated is the same as the first data length; using the data to be written to update the data in the data sub-shard to be updated to obtain a target data shard; and writing the data of the target data shard to the target block device.

[0014] In an exemplary embodiment, the method further includes: in the process of migrating the device data of the block device to be migrated to the data storage space corresponding to the target block device, determining the first data slice to be read corresponding to the read request on the target block device according to the second offset value and second data length of the read request of the host device; and when it is determined that the first data slice to be read already exists in the second storage system, obtaining data read from the first data slice to be read.

[0015] The present application also provides an online data migration device, comprising: a determination module, configured to determine, based on a block device mapping relationship, a target block device after a block device to be migrated in a first storage system is mapped to a second storage system, wherein the block device mapping relationship represents a path mapping relationship between the block device to be migrated and the target block device, and the block device information of the block device to be migrated is the same as the block device information of the target block device; a disconnection module, configured to disconnect a data transmission channel between the host device and the block device to be migrated after the host device is connected to the target block device; and a migration module, configured to migrate device data of the block device to be migrated to a data storage space corresponding to the target block device, wherein the host device is pre-connected to the block device to be migrated.

[0016] The present application also provides an electronic device, comprising: a memory for storing a computer program; and a processor for implementing the steps of any of the above-mentioned methods for online data migration when executing the computer program.

[0017] The present application also provides a computer-readable storage medium, in which a computer program is stored. When the computer program is executed by a processor, the steps of any of the above-mentioned methods for online data migration are implemented.

[0018] The present application also provides a computer program product, including a computer program, which implements the steps of any of the above-mentioned data online migration methods when executed by a processor.

[0019] Through the present application, by determining the block device to be migrated in the first storage system (old storage system) and the target block device in the second storage system (new storage system), the host device is pre-connected to the block device to be migrated. After the host device is connected to the target block device, the data transmission channel between the host device and the block device to be migrated is disconnected, and only the connection between the host device and the block device to be migrated is retained. The device data of the block device to be migrated is migrated to the data storage space corresponding to the target block device, so that the second storage system responds to and completes the service request (such as a write request or a read request) from the host device, thereby solving the problem of service interruption during data migration of heterogeneous storage systems in the related art, reducing the data lost due to interruption of the data migration process, realizing seamless data migration between storage systems, supporting online takeover of online data migration between the host system and the heterogeneous storage system, realizing host-side imperceptible service switching, improving the continuity of the host-side service, and further enhancing the reliability of data migration. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the embodiments of the present application, the following is a brief introduction to the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0021] Figure 1 A hardware structure block diagram of a computer terminal for an online data migration method provided in an embodiment of the present application;

[0022] Figure 2 A flowchart of a method for online data migration provided in an embodiment of the present application;

[0023] Figure 3 A schematic diagram of the principle of online data migration provided in an embodiment of the present application;

[0024] Figure 4 A schematic diagram of a write request provided in an embodiment of the present application;

[0025] Figure 5 A schematic diagram of an online data migration device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0026] The following will be combined with the accompanying drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0027] It should be noted that, in the description of this application, the terms "comprises," "includes," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. The terms "first," "second," etc., in this application are used to distinguish similar objects, and are not used to describe a particular order or sequence.

[0028] The method embodiments provided in the embodiments of the present application can be executed in a computer terminal, a cloud server corresponding to the computer terminal, or a similar computing device. Taking running on a computer terminal as an example, Figure 1 This is a hardware structure block diagram of a computer terminal of a data online migration method according to an embodiment of the present application. Figure 1 As shown, the computer terminal may include one or more ( Figure 1 Only one is shown) a processor 102 (the processor 102 may include but is not limited to a microprocessor or a programmable logic device) and a memory 104 for storing data. The computer terminal may also include a transmission device 106 and an input / output device 108 for communication functions. It will be understood by those skilled in the art that Figure 1 The structure shown is only for illustration and does not limit the structure of the above-mentioned computer terminal. For example, the computer terminal may also include Figure 1 More or fewer components than shown, or with Figure 1 Different configurations shown.

[0029] The memory 104 can be used to store computer programs, for example, software programs and modules of application software, such as the computer program corresponding to the online data migration method in the embodiment of the present application. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, that is, realizes the above-mentioned online data migration method. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include a memory remotely located relative to the processor 102, and these remote memories may be connected to a card in the visible area via a network. Examples of the above-mentioned network include but are not limited to the Internet, an intranet, a local area network, a mobile communication network, and a combination thereof.

[0030] The transmission device 106 is used to receive or transmit data via a network. A specific example of the aforementioned network may include a wireless network provided by a communication provider of a computer terminal. In one embodiment, the transmission device 106 includes a network interface controller (NIC), which can be connected to other network devices via a base station to enable communication with the Internet. In another embodiment, the transmission device 106 may be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0031] In this embodiment, a method for online data migration is provided, which is applied to the above-mentioned computer terminal and is described in detail in conjunction with the following execution process. Figure 2 is a flow chart of a method for online data migration according to an embodiment of the present application. Figure 2 As shown, the process includes the following steps:

[0032] Step S202: determining a target block device after the block device to be migrated in the first storage system is mapped to the second storage system based on a block device mapping relationship, wherein the block device mapping relationship represents a path mapping relationship between the block device to be migrated and the target block device, and the block device information of the block device to be migrated is the same as the block device information of the target block device;

[0033] Step S204: After the host device is connected to the target block device, disconnect the data transmission channel between the host device and the block device to be migrated;

[0034] Step S206 , migrating the device data of the block device to be migrated to the data storage space corresponding to the target block device, wherein the host device is pre-connected to the block device to be migrated.

[0035] Through the above steps, by determining the block device to be migrated in the first storage system (old storage system) and the target block device in the second storage system (new storage system), the host device is pre-connected to the block device to be migrated, and after the host device is connected to the target block device, the data transmission channel between the host device and the block device to be migrated is disconnected, only the connection between the host device and the block device to be migrated is retained, and the device data of the block device to be migrated is migrated to the data storage space corresponding to the target block device, so that the second storage system responds to and completes the service request (such as a write request or a read request) from the host device, solving the problem of service interruption during data migration of heterogeneous storage systems in the related art, reducing data loss due to interruption in the data migration process, realizing seamless data migration between storage systems, supporting online takeover of online data migration between the host system and the heterogeneous storage system, realizing seamless service switching on the host side, improving the continuity of the host side service, and further enhancing the reliability of data migration.

[0036] It should be noted that the above-mentioned block device can represent a physical storage device such as a hard disk drive, a solid-state drive, or a logical storage device in a virtualized environment. A block device is represented in the operating system as a specific file system node, such as / dev / sda. Block device information includes attributes such as the device type, size, and identifier, which are crucial for the correct reading and writing of data. In this embodiment, by creating a target block device in the second storage system with the same block device information as the block device to be migrated in the first storage system, it is possible to ensure that the attributes of the block device before and after data migration are consistent, thereby avoiding data incompatibility or read and write errors caused by differences in device attributes.

[0037] In an exemplary embodiment, before determining the target block device after the block device to be migrated in the first storage system is mapped to the second storage system based on the block device mapping relationship, an access control list corresponding to the system node of the second storage system can be added to the device data of the block device to be migrated, and the device node of the block device to be migrated can be mounted to the system node based on the access control list; a blank block device is created in the second storage system based on the block device information of the block device to be migrated, and the block device information of the block device to be migrated is set to the block device information of the blank block device to obtain the target block device; a migration mark is set for the target block device, and the migration mark is bound to the access path of the block device to be migrated to obtain the block device mapping relationship. By pre-creating a target block device with the same block device information as that in the first storage system on the second storage system and performing path mapping on the host device, normal access of the host device during the data migration process is achieved, thereby improving the user experience.

[0038] ACL (Access Control List) is used in storage systems to control which hosts can access specific storage resources, such as LUN (Logical Unit Number).

[0039] The ACL information for a storage system's system node contains a series of rules that determine whether a host is allowed to read or write storage resources based on the host's IQN (Initiator Qualified Name, a unique identifier for iSCSI) or WWN (World Wide Name, a unique identifier for FC protocols).

[0040] In heterogeneous storage migration scenarios, by adding the ACL information of the new storage system's node to the block device to be migrated in the old storage system, the new storage system can legally access the data on the old storage system, thereby enabling subsequent data migration. Therefore, by adding the ACL corresponding to the system node of the second storage system to the device data of the block device to be migrated, access rights to the target block device can be controlled, ensuring that only authorized host devices can access it, thereby increasing the security of data migration. Furthermore, by copying the block device information of the block device to be migrated to a blank block device in the second storage system, the target block device created has the same properties as the block device to be migrated, ensuring the consistency of data migration.

[0041] Furthermore, setting a migration flag and binding it to the access path identifies the data migration status, helping subsequent operations (such as reading and writing data) determine whether special processing is required, such as prioritizing the use of migrated data. This embodiment addresses the preparatory work before data migration, ensuring that access conflicts do not occur during the data migration process and improving data migration security.

[0042] You can query the block device information of the block device to be migrated by using query commands such as "REPORT LUN, INQUIRY".

[0043] In an exemplary embodiment, the host device is connected to the target block device in the following manner: determining a first device node path of the block device to be migrated from block device information of the block device to be migrated, and determining a second device node path of the target block device from block device information of the target block device, the first device node path and the second device node path having the same device number; storing the first device node path and the second device node path in the host device to obtain aggregate path information; and mounting the device node of the target block device to the device node of the host device according to the aggregate path information to connect the host device to the target block device.

[0044] The device number represents a number that uniquely identifies each block device. By ensuring that the first device node path and the second device node path have the same device number, the host device can access the block device through the same path before and after data migration. This avoids access interruption caused by path changes and ensures a smooth transition of data migration.

[0045] The above-mentioned aggregate path information represents the merged information of the two device node paths, so that the host device can intelligently switch the access path during data migration, giving priority to accessing the target block device. When the target block device is unavailable or the data migration is not completed, it can fall back to the block device to be migrated, which not only improves the reliability of data migration but also reduces system downtime.

[0046] In an exemplary embodiment, the implementation process of migrating the device data of the block device to be migrated to the data storage space corresponding to the target block device includes: establishing a data migration thread between the first storage system and the second storage system; using the data migration thread to migrate all data shards corresponding to the device data of the block device to be migrated to the target block device; and generating a migration result after the migration of all data shards is completed, and sending the migration result to the first storage system and the second storage system, the migration result including at least one of the following: migration completed, migration interrupted, migration failed, and migration in progress.

[0047] It should be noted that the data migration thread is a specific execution unit responsible for data replication during the data migration process, which can be a separate process or thread, used to migrate data slices of the block device to be migrated to the target block device one by one.

[0048] Data sharding is the process of dividing large amounts of data into smaller chunks. This facilitates parallel processing and increases migration speed. Once all shards have been migrated, the generated migration results are promptly fed back to both storage systems, allowing them to monitor the data migration.

[0049] This embodiment significantly improves the speed of data migration through thread-level concurrent processing. At the same time, the real-time feedback mechanism of migration results makes the entire migration process more transparent and controllable, thereby improving the efficiency and security of data migration.

[0050] Optionally, after all data shards are migrated, the connection between the block device to be migrated and the second storage system is disconnected, and the thread resources of the data migration thread are destroyed.

[0051] In an exemplary embodiment, further, in the process of migrating the device data of the block device to be migrated to the data storage space corresponding to the target block device, the data shard to be written corresponding to the write request on the target block device is determined based on the first offset value and the first data length of the write request of the host device; and when it is determined that the data shard to be written already exists in the second storage system, the data to be written in the write request is written into the data shard to be written.

[0052] The offset value refers to the data position relative to the start position of the block device, and the data length refers to the size of the written data. During the data migration process, in this embodiment, when a host device requests a write request to a target block device and the corresponding data shard to be written already exists on the target block device, the data is directly written, thereby simplifying unnecessary data copying operations and improving the efficiency of data migration.

[0053] In this way, the write operation of the host device during the data migration process can be processed, thereby improving the data consistency of the data migration process.

[0054] Optionally, after the data to be written in the write request is written into the data slice to be written, a prompt message indicating that the write is successful may be sent to the host device.

[0055] In an exemplary embodiment, optionally, when it is determined that the data shard to be written does not exist in the second storage system, the original data shard corresponding to the first offset value is determined from all data shards of the block device to be migrated, wherein the original data shard has a starting offset value and an ending offset value, the first offset value is greater than the starting offset value, and the first offset value is less than the ending offset value; the difference between the first offset value and the starting offset value is determined as the relative offset value of the data to be written; a data sub-shard to be updated in the original data shard is determined, the starting offset value of the data sub-shard to be updated is the same as the relative offset value, and the length between the starting offset value of the data sub-shard to be updated and the ending offset value of the data sub-shard to be updated is the same as the first data length; the data in the data sub-shard to be updated is updated using the data to be written to obtain a target data shard; and the data of the target data shard is written to the target block device. During data migration, this embodiment responds to a host device's write request. If the corresponding data shard to be written does not exist on the target block device, the original data shard containing the write location is found on the target block device. A relative offset is then calculated to locate the specific sub-shard to be updated. The data is then updated before being written to the target block device. This ensures that even if data migration is incomplete, the host device's write operation will not affect the progress of the data migration, while also ensuring data consistency and integrity.

[0056] Optionally, in this embodiment, writing the data of the target data shard into the target block device means writing the data into a data shard corresponding to the target data shard in the target block device.

[0057] In an exemplary embodiment, during the process of migrating device data from the block device to be migrated to the data storage space corresponding to the target block device, the first data shard to be read corresponding to the read request on the target block device is determined based on the second offset value and second data length of the read request from the host device; upon determining that the first data shard to be read already exists in the second storage system, data read from the first data shard to be read is obtained. During the data migration process, in this embodiment, for a read request from the host device to the second storage system, if a corresponding data shard to be read exists on the target block device, data is directly read from it, thereby avoiding repeated reading from the block device to be migrated and improving the efficiency of data migration.

[0058] Optionally, when it is determined that the first data shard to be read does not exist in the second storage system, a second data shard to be read is determined from all data shards of the block device to be migrated, wherein the starting offset value of the second data shard to be read is the same as the second offset value, and the length between the starting offset value of the second data shard to be read and the ending offset value of the second data shard to be read is the same as the second data length; and data read from the second data shard to be read is obtained. In this embodiment, during the data migration process, for a read request from the host device, if the corresponding data shard to be read does not exist on the target block device, the corresponding data shard to be read is determined from the first storage system and read. During the data migration process, accurate data is obtained based on the read operation of the host device, thereby enhancing the reliability of data migration while ensuring business continuity on the host side.

[0059] In order to enable those skilled in the art to better understand the present application, the present application is further described in detail below with reference to the accompanying drawings and specific implementation methods.

[0060] In this embodiment, combined with Figure 3 A specific application environment on which the execution of the online data migration method depends is provided, and the architecture of the specific application environment is described herein.

[0061] like Figure 3 As shown, the system includes an old storage system (the first storage system), a new storage system (the second storage system), and a client host (corresponding to the host device). The functional modules within the new storage system include a heterogeneous storage volume management module, a data migration module, and a service request processing module. The functional module within the client host is the host-side path control module. All of these functional modules are developed in C++. The following implementation steps illustrate the data migration process.

[0062] In step 1, the heterogeneous storage volume management module adds the ACL information for the new storage system node to the old storage system's volume (represented as LUN1, corresponding to the block device to be migrated). LUN1 from the old storage system is mounted to the new storage system's node, allowing the new storage system to access, control, and store LUN1's data.

[0063] Among them, the block device information such as LUN_ID (Logical Unit Number), capacity, and WWN (World Wide Name) of the storage system LUN1 can be queried through commands such as "REPORT LUN, INQUIRY, READ_CAPACITY".

[0064] The LUN_ID is an identifier in the storage system that uniquely identifies a storage device or a logical partition. Each LUN is assigned a LUN_ID to help the host determine which storage device to communicate with.

[0065] Capacity refers to the storage capacity of LUN1, that is, the size that can be used to store data, in bytes, and can be expressed in GB (gigabyte), TB (terabyte), etc.

[0066] A WWN, a globally unique identifier for a storage device, is used in network storage environments such as Fibre Channel (FC). During heterogeneous storage system migration, the WWN helps ensure that the newly created LUN2 and the original LUN1 are identified on the host as different paths to the same device, thus enabling multipath management.

[0067] In step 2, the heterogeneous storage volume management module creates a new volume (represented as LUN2, corresponding to the target block device) in the new storage system based on the block device information obtained in step 1. The WWN, capacity, LUN_ID and other information of LUN2 are consistent with those of LUN1. A migration target mark is added to LUN2 in the new storage system, indicating that the volume is a target volume for heterogeneous storage migration.

[0068] Get the file descriptor (fd_src) of LUN1 mounted in step 1. This is used to access the new storage system and read data from LUN1 in the old storage system. Pass fd_src to LUN2 as its data source.

[0069] Step 3: The host-side path control module completes the service redirection process.

[0070] Mount LUN2 of the new storage system to the host. Disconnect the host from the old storage system, that is, disconnect LUN1 from the host. Notify the data migration module to start the data migration process.

[0071] The host side only retains the path to LUN2, and service requests from the host side are automatically sent to the new storage system through the link to LUN2.

[0072] In this step, LUN2 and LUN1 share the same block device information, such as LUN_ID. The host uses the block device ID to identify LUN2 and LUN1 as multiple paths to the same block device, thereby deriving an aggregated path for LUN1 and LUN2. When LUN1 is disconnected and the path to LUN2 is switched, service requests are sent to the new storage system via the path to LUN2. This allows both the block device to be migrated and the target block device to be connected to the host simultaneously. The host aggregates the two paths to the same block device, and subsequently disconnects the path to achieve seamless service redirection at the service level.

[0073] In step 4, after disconnecting LUN1 from the host in step 3, the data migration module performs data migration. LUN1 is linked to the new storage system based on LUN2's target volume tag and fd_src, and the data pointed to by fd_src is migrated to LUN2. For example, the data migration thread reads LUN1 data one slice at a time and writes it to the new storage system, LUN2. When all slices have completed the migration process, the data migration is reported as complete. LUN1 is disconnected from the new storage system, and the thread resources for the data migration thread are destroyed.

[0074] Step 5: The service request processing module processes the read and write requests from the host side.

[0075] For the write request W1 sent from the host to LUN2, according to the offset of W1 (i.e. Figure 4 off1) and data length (i.e. Figure 4 Calculate the corresponding data shards. Check whether the corresponding data shards exist in the new storage system.

[0076] If the data slice already exists (generated after writing the LUN1 data read by the data migration module to LUN2 or generated by step ③), the data of W1 is written into the corresponding data slice.

[0077] like Figure 4 As shown in steps ① to ④, if it is determined that the data slice does not exist, the corresponding data slice in LUN1 is read through fd_src to obtain data DATA1, and then W1 and DATA1 are merged according to the offset to be written by W1 to form new data DATA2.

[0078] Specifically, the data's relative offset on the slice is calculated using the offset off2 of the corresponding data slice in LUN1 and the offset off1 of W1, where off = off1 - off2. Then, based on len, the data to be written by W1 is overwritten with the data in DATA1 to form data DATA2. This data is then written to the new storage system. A successful write request for W1 is then returned to the host.

[0079] For read request R1 sent from the host to LUN2, the corresponding data slice is calculated based on the offset and length of R1. The corresponding data slice is then checked in the new storage system to see if it exists.

[0080] If the data slice already exists, the new storage system data is directly read. If the data slice does not exist, the offset and length of the read request R1 are used to read the data corresponding to the offset and length in LUN1 through fd_src. Finally, a success message is returned to the host side, indicating that the read request R1 is successful, and the read data is returned.

[0081] In this embodiment, taking the iSCSI application scenario as an example, the volumes of the old storage system are linked to the new storage system through the collaborative work of four modules. The new storage system generates new volumes based on the original block devices of the old storage system, and then maps the new volumes to the host side through iSCSI and other methods. After that, the host-side path control module, data migration module and business request processing module jointly perform data migration and business request processing between heterogeneous storage systems, thereby completing online data migration between different storage systems.

[0082] Through the description of the above implementation methods, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by means of software plus the necessary general hardware platform, and of course it can also be implemented by hardware, but in many cases the former is a better implementation method.

[0083] The embodiment of the present application also provides a data online migration device, such as Figure 5 Shown, including:

[0084] a determining module 52 configured to determine, based on a block device mapping relationship, a target block device after the block device to be migrated in the first storage system is mapped to the second storage system, wherein the block device mapping relationship represents a path mapping relationship between the block device to be migrated and the target block device, and the block device information of the block device to be migrated is the same as the block device information of the target block device;

[0085] A disconnection module 54 is configured to disconnect the data transmission channel between the host device and the block device to be migrated after the host device is connected to the target block device;

[0086] The migration module 56 is configured to migrate the device data of the block device to be migrated to the data storage space corresponding to the target block device, wherein the host device is pre-connected to the block device to be migrated.

[0087] Through the above-mentioned device, by determining the block device to be migrated in the first storage system (old storage system) and the target block device in the second storage system (new storage system), the host device is pre-connected to the block device to be migrated. After the host device is connected to the target block device, the data transmission channel between the host device and the block device to be migrated is disconnected, and only the connection between the host device and the block device to be migrated is retained. The device data of the block device to be migrated is migrated to the data storage space corresponding to the target block device, so that the second storage system responds to and completes the service request (such as a write request or a read request) from the host device, solving the problem of service interruption during data migration of heterogeneous storage systems in the related art, reducing data loss due to interruption of the data migration process, realizing seamless data migration between storage systems, supporting online takeover of online data migration between the host system and the heterogeneous storage system, realizing host-side senseless service switching, improving the continuity of host-side services, and further enhancing the reliability of data migration.

[0088] In an exemplary embodiment, before determining the target block device after the block device to be migrated in the first storage system is mapped to the second storage system based on the block device mapping relationship, the determination module is further used to: add an access control list corresponding to the system node of the second storage system to the device data of the block device to be migrated, and mount the device node of the block device to be migrated to the system node according to the access control list; create a new blank block device in the second storage system based on the block device information of the block device to be migrated, set the block device information of the block device to be migrated to the block device information of the blank block device, and obtain the target block device; set a migration mark for the target block device, bind the migration mark and the access path of the block device to be migrated, and obtain the block device mapping relationship.

[0089] In an exemplary embodiment, the disconnection module is further configured to connect the host device to the target block device in the following manner: determining a first device node path of the block device to be migrated from the block device information of the block device to be migrated, and determining a second device node path of the target block device from the block device information of the target block device, wherein the first device node path and the second device node path have the same device number; storing the first device node path and the second device node path in the host device to obtain aggregate path information; and mounting the device node of the target block device to the device node of the host device according to the aggregate path information to connect the host device to the target block device.

[0090] In an exemplary embodiment, the migration module is further used to: establish a data migration thread between the first storage system and the second storage system; use the data migration thread to migrate all data shards corresponding to the device data of the block device to be migrated to the target block device; and generate a migration result after the migration of all data shards is completed, and send the migration result to the first storage system and the second storage system, wherein the migration result includes at least one of the following: migration completed, migration interrupted, migration failed, and migration in progress.

[0091] In an exemplary embodiment, the migration module is further used to determine, during the process of migrating the device data of the block device to be migrated to the data storage space corresponding to the target block device, the data shard to be written corresponding to the write request on the target block device according to the first offset value and the first data length of the write request of the host device; and when it is determined that the data shard to be written already exists in the second storage system, write the data to be written in the write request to the data shard to be written.

[0092] In an exemplary embodiment, the migration module is further used to: when it is determined that the data shard to be written does not exist in the second storage system, determine the original data shard corresponding to the first offset value from all data shards of the block device to be migrated, wherein the original data shard has a starting offset value and an ending offset value, the first offset value is greater than the starting offset value, and the first offset value is less than the ending offset value; determine the difference between the first offset value and the starting offset value as the relative offset value of the data to be written; determine a data sub-shard to be updated in the original data shard, the starting offset value of the data sub-shard to be updated is the same as the relative offset value, and the length between the starting offset value of the data sub-shard to be updated and the ending offset value of the data sub-shard to be updated is the same as the first data length; use the data to be written to update the data in the data sub-shard to be updated to obtain a target data shard; and write the data of the target data shard to the target block device.

[0093] In an exemplary embodiment, the migration module is further used to determine, during the process of migrating the device data of the block device to be migrated to the data storage space corresponding to the target block device, the first data shard to be read corresponding to the read request on the target block device according to the second offset value and second data length of the read request of the host device; and when it is determined that the first data shard to be read already exists in the second storage system, obtain data read from the first data shard to be read.

[0094] For the description of the features in the embodiment corresponding to the above-mentioned online data migration device, reference can be made to the relevant description of the embodiment corresponding to the above-mentioned online data migration method, and no further details will be given here.

[0095] An embodiment of the present application further provides an electronic device, comprising a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to execute the steps in any of the above-mentioned embodiments of the online data migration method.

[0096] An embodiment of the present application further provides a computer-readable storage medium, in which a computer program is stored. The computer program is configured to execute the steps of any of the above-mentioned embodiments of the online data migration method when running.

[0097] In an exemplary embodiment, the computer-readable storage medium may include, but is not limited to, various media that can store computer programs, such as a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk, or an optical disk.

[0098] An embodiment of the present application further provides a computer program product, which includes a computer program. When the computer program is executed by a processor, the steps of any of the above-mentioned embodiments of the online data migration method are implemented.

[0099] An embodiment of the present application also provides another computer program product, including a non-volatile computer-readable storage medium, wherein the non-volatile computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of any of the above-mentioned data online migration method embodiments are implemented.

[0100] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the above description has generally described the components and steps of each example according to their functions. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0101] The above is a detailed introduction to a method, device, electronic device and storage medium for online data migration provided by the present application. This article uses specific examples to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea. It should be pointed out that for ordinary technicians in this technical field, without departing from the principles of the present application, several improvements and modifications can be made to the present application, and these improvements and modifications also fall within the scope of protection of the claims of the present application.

Claims

1. A method for online data migration, characterized in that: include: Determine, based on a block device mapping relationship, a target block device after the block device to be migrated in the first storage system is mapped to the second storage system, wherein the block device mapping relationship represents a path mapping relationship between the block device to be migrated and the target block device, and the block device information of the block device to be migrated is the same as the block device information of the target block device; After the host device is connected to the target block device, disconnecting the data transmission channel between the host device and the block device to be migrated; The device data of the block device to be migrated is migrated to the data storage space corresponding to the target block device, and the host device is pre-connected to the block device to be migrated.

2. The online data migration method according to claim 1, characterized in that: Before determining, according to the block device mapping relationship, the target block device after the block device to be migrated in the first storage system is mapped to the second storage system, the method further includes: Adding an access control list corresponding to the system node of the second storage system to the device data of the block device to be migrated, and mounting the device node of the block device to be migrated to the system node according to the access control list; Creating a blank block device in the second storage system according to the block device information of the block device to be migrated, setting the block device information of the block device to be migrated as the block device information of the blank block device, and obtaining the target block device; A migration mark is set for the target block device, and the migration mark is bound to the access path of the block device to be migrated to obtain the block device mapping relationship.

3. The online data migration method according to claim 1, characterized in that: Connect the host device to the target block device in the following manner: Determining a first device node path of the block device to be migrated from the block device information of the block device to be migrated, and determining a second device node path of the target block device from the block device information of the target block device, wherein the first device node path and the second device node path have the same device number; Storing the first device node path and the second device node path in the host device to obtain aggregated path information; The device node of the target block device is mounted to the device node of the host device according to the aggregate path information, so as to connect the host device to the target block device.

4. The online data migration method according to claim 1, wherein: Migrating the device data of the block device to be migrated to the data storage space corresponding to the target block device includes: Establishing a data migration thread between the first storage system and the second storage system; Migrating all data slices corresponding to the device data of the block device to be migrated to the target block device using the data migration thread; A migration result is generated after all data shards are migrated, and the migration result is sent to the first storage system and the second storage system. The migration result includes at least one of the following: migration completed, migration interrupted, migration failed, and migration in progress.

5. The online data migration method according to claim 1, wherein: The method further comprises: In the process of migrating the device data of the block device to be migrated to the data storage space corresponding to the target block device, determining the data shard to be written corresponding to the write request on the target block device according to the first offset value and the first data length of the write request of the host device; If it is determined that the data slice to be written already exists in the second storage system, the data to be written in the write request is written into the data slice to be written.

6. The method for online data migration according to claim 5, characterized in that: The method further comprises: If it is determined that the data shard to be written does not exist in the second storage system, determine, from all data shards of the block device to be migrated, an original data shard corresponding to the first offset value, wherein the original data shard has a starting offset value and an ending offset value, and the first offset value is greater than the starting offset value, and the first offset value is less than the ending offset value; Determine the difference between the first offset value and the starting offset value as the relative offset value of the data to be written; Determine a data sub-shard to be updated in the original data shard, where a starting offset value of the data sub-shard to be updated is the same as the relative offset value, and a length between the starting offset value of the data sub-shard to be updated and the ending offset value of the data sub-shard to be updated is the same as the first data length; Using the data to be written to update the data in the to-be-updated data sub-shard to obtain a target data shard; Writing the data of the target data slice into the target block device.

7. The online data migration method according to claim 1, characterized in that: The method further comprises: In the process of migrating the device data of the block device to be migrated to the data storage space corresponding to the target block device, determining, according to the second offset value and the second data length of the read request of the host device, a first data fragment to be read corresponding to the read request on the target block device; When it is determined that the first data slice to be read already exists in the second storage system, data read from the first data slice to be read is acquired.

8. A data online migration device, characterized in that: include: a determining module, configured to determine, based on a block device mapping relationship, a target block device after the block device to be migrated in the first storage system is mapped to the second storage system, wherein the block device mapping relationship represents a path mapping relationship between the block device to be migrated and the target block device, and the block device information of the block device to be migrated is the same as the block device information of the target block device; a disconnection module, configured to disconnect a data transmission channel between the host device and the block device to be migrated after the host device is connected to the target block device; The migration module is configured to migrate device data of the block device to be migrated to a data storage space corresponding to the target block device, wherein the host device is pre-connected to the block device to be migrated.

9. An electronic device, characterized in that: include: Memory for storing computer programs; A processor, configured to implement the steps of the method for online data migration as claimed in any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, wherein when the computer program is executed by a processor, the steps of the online data migration method according to any one of claims 1 to 7 are implemented.

Images