Data migration system and method, electronic equipment and storage medium

Abstract

The invention discloses a data migration system and method, electronic equipment and a storage medium, and relates to the technical field of computers.The method comprises the steps that under the condition that a target data disk meets a data migration session establishment request sent by the source data disk, a corresponding target namespace of to-be-migrated data of the source data disk is established, the information of successful establishment of the data migration session is sent to the corresponding source data disk; the source data disk writes to-be-migrated data into a target namespace of a corresponding target data disk according to a corresponding source logic address of the to-be-migrated data in the source data disk based on the data migration session, and the storage cluster management module stores the to-be-migrated data in the target namespace of the corresponding target data disk under the condition that the data migration task is completed. And recording a corresponding relationship between the source logic address of the to-be-migrated data and the target namespace of the target data disk corresponding to the to-be-migrated data in a system layer. The technical problem that the performance of the storage system is reduced due to the fact that data migration needs to modify a large amount of metadata in the related technology is solved, and the technical effect of improving the performance of the storage system is achieved.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to data migration systems, methods, electronic devices and storage media. Background Technology

[0002] With the rapid development of data storage technology, solid-state storage devices are increasingly widely used in storage systems, such as solid-state drives (SSDs) using the Non-Volatile Memory Express (NVMe) protocol. Storage clusters are increasingly demanding high availability and data redundancy to address issues such as storage node failures, data reconstruction, and load balancing. Data migration, as the core operation for coordinating these issues, has become a crucial step in ensuring the stable operation of storage clusters.

[0003] In related technologies, storage systems employ an append-only write model, using distributed metadata to record the mapping relationship between user addresses and logical block addresses (LBAs) of solid-state storage devices. During data migration, the logical address of the solid-state storage device corresponding to the data changes, while the user address remains unchanged. Therefore, a large amount of metadata updates are required to ensure correct data access. However, this method of modifying a large amount of metadata during data migration reduces the performance of the storage system. Summary of the Invention

[0004] This application provides a data migration system, method, electronic device, and storage medium to at least address the problem in the related art where data migration requires modification of a large amount of metadata, which reduces the performance of the storage system.

[0005] This application provides a data migration system, including:

[0006] The source data disk is used to determine the corresponding target data disk based on the data migration task and send a data migration session establishment request to the corresponding target data disk.

[0007] The target data disk is used to establish the target namespace of the data to be migrated from the corresponding source data disk when the data migration session establishment request is met, and to send the information that the data migration session has been successfully established to the corresponding source data disk.

[0008] The source data disk is also used to write the data to be migrated into the target namespace of the corresponding target data disk according to its corresponding source logical address in the source data disk, based on the data migration session when the data migration session is successfully established.

[0009] The storage cluster management module is used to record the correspondence between the source logical address of the data to be migrated and the target namespace of the target data disk corresponding to the data to be migrated at the system level, after the source data disk has completed writing the data to be migrated into the target namespace of the corresponding target data disk.

[0010] This application also provides a data migration method, applied to any of the above-mentioned data migration systems, including:

[0011] Using the source data disk, based on the data migration task, determine the corresponding target data disk and send a data migration session establishment request to the corresponding target data disk;

[0012] Based on the target data disk, if the data migration session establishment request is met, the target namespace of the data to be migrated on the corresponding source data disk is established, and the information that the data migration session has been successfully established is sent to the corresponding source data disk.

[0013] Based on the source data disk, if the data migration session is successfully established, the data to be migrated is written to the target namespace of the corresponding target data disk according to its corresponding source logical address in the source data disk.

[0014] Based on the storage cluster management module, after the source data disk has completed writing the data to be migrated into the target namespace of the corresponding target data disk, the system layer records the correspondence between the source logical address of the data to be migrated and the target namespace of the target data disk corresponding to the data to be migrated.

[0015] This application also provides an electronic device, including: a memory for storing a computer program; and a processor for implementing the above-described data migration method when executing the computer program.

[0016] This application also provides a computer-readable storage medium storing a computer program, wherein the computer program, when executed by a processor, implements the steps of the above-described data migration method.

[0017] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the above-described data migration method.

[0018] This application achieves the following: The source data disk determines the corresponding target data disk based on the data migration task and sends a data migration session establishment request to the target data disk. If the target data disk satisfies the data migration session establishment request, it establishes the target namespace for the data to be migrated from the source data disk and sends a successful data migration session establishment message to the source data disk. Upon successful data migration session establishment, the source data disk writes the data to be migrated to the target namespace of the target data disk according to its corresponding source logical address in the source data disk. After the source data disk completes writing the data to be migrated to the target namespace of the target data disk, the storage cluster management module records the correspondence between the source logical address of the data to be migrated and the target namespace of the target data disk at the system layer. By writing the data to be migrated to the target namespace of the target data disk according to its corresponding source logical address in the source data disk, the logical address of the data remains unchanged after migration. Therefore, there is no need to modify the mapping relationship between user addresses and logical addresses in the metadata. This solves the technical problem in related technologies where data migration requires modifying a large amount of metadata, which reduces the performance of the storage system. This achieves the technical effect of avoiding metadata modification during data migration and improving the performance of the storage system. Attached Figure Description

[0019] To more clearly illustrate the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 This is a schematic diagram of the structure of a data migration system provided in an embodiment of this application;

[0021] Figure 2 A flowchart illustrating a data migration method provided in an embodiment of this application;

[0022] Figure 3 A timing diagram illustrating yet another data migration method provided in an embodiment of this application;

[0023] Figure 4 A schematic diagram illustrating data migration provided in an embodiment of this application;

[0024] Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0025] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the protection scope of this application.

[0026] It should be noted that, in the description of this application, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. The terms "first," "second," etc., in this application are used to distinguish similar objects and are not used to describe a specific order or sequence.

[0027] To enable those skilled in the art to better understand the present application, the present application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0028] With the rapid development of data storage technology, solid-state storage devices, such as NVMe disks, are being used more and more widely in storage systems. Storage clusters are facing increasingly stringent requirements for high availability and data redundancy to address issues such as storage node failures, data reconstruction, and load balancing. Data migration, as the core operation for coordinating these issues, has become a crucial step in ensuring the stable operation of storage clusters.

[0029] In related technologies, storage systems employ an append-only write model, using distributed metadata to record the mapping between user addresses and logical block addresses (LBAs) of solid-state storage devices. During data migration, the logical address of the corresponding solid-state storage device changes, while the user address remains the same. Therefore, a large amount of metadata updates are required to ensure correct data access. This data migration method, which involves rebuilding the source data and modifying a large amount of metadata, consumes significant resources and is inefficient, reducing storage system performance and increasing maintenance difficulty and cost.

[0030] To address the aforementioned issues, embodiments of this application provide a data migration system, method, electronic device, and storage medium. The system includes: a source data disk, used to determine a corresponding target data disk based on a data migration task and send a data migration session establishment request to the corresponding target data disk; a target data disk, used to establish a target namespace for the data to be migrated on the corresponding source data disk when the data migration session establishment request is satisfied, and to send a successful data migration session establishment message to the corresponding source data disk; the source data disk is also used to, upon successful data migration session establishment, write the data to be migrated according to its corresponding source logical address in the source data disk into the target namespace of the corresponding target data disk; and a storage cluster management module, used to record, at the system layer, the correspondence between the source logical address of the data to be migrated and the target namespace of the target data disk corresponding to the data to be migrated, after the source data disk has completed writing the data to be migrated into the target namespace of the corresponding target data disk. The system provided by the above solution writes the data to be migrated to the target namespace of the corresponding target data disk according to its source logical address in the source data disk. This ensures that the logical address of the data remains unchanged after migration. Therefore, there is no need to modify the mapping relationship between user address and logical address in the metadata. This solves the technical problem that data migration requires modifying a large amount of metadata, which reduces the performance of the storage system. It achieves the technical effect of avoiding modification of metadata during data migration, reducing the number of metadata updates, and improving the performance and reliability of the storage system.

[0031] Embodiments of this application provide a data migration system. Figure 1 This is a schematic diagram of the structure of the data migration system provided in the embodiments of this application, such as... Figure 1 As shown, the data migration system 10 includes:

[0032] Source data disk 101 is used to determine the corresponding target data disk based on the data migration task and send a data migration session establishment request to the corresponding target data disk.

[0033] Specifically, when the storage cluster experiences a failure, expansion, or scaling down, the storage cluster management module triggers data migration, sending a data migration task to the corresponding source data disk. For example, if a storage node fails and data reconstruction is required, the data migration task could migrate data from data disk 1 of storage node 1 to data disk 1 of storage node 2. The data migration task describes the migration of data from the source data disk to the corresponding target data disk. A mapping relationship exists between the source and target data disks.

[0034] Because different namespaces of NVMe disks have independent LBA spaces, this characteristic of NVMe disks allows for direct disk-to-disk data migration, avoiding updates to logical addresses and reducing the need for metadata updates. It is understood that the data disk in this application is an NVMe disk, including both a source data disk and a target data disk.

[0035] After receiving the data migration task, the source data disk determines the data to be migrated and the corresponding target data disk based on the data migration task, and sends a data migration session establishment request to the corresponding target data disk. Specifically, based on the data migration task, the source logical address corresponding to the data to be migrated in the source data disk is determined: the metadata list of the data to be migrated is obtained from the storage cluster management module, and based on the source data list of the data to be migrated, the source logical address corresponding to the data to be migrated in the source data disk is determined. The metadata list of the data to be migrated includes multiple metadata entries, each of which records the mapping relationship between the user address of the data to be migrated and the source logical address of the source data disk. The user address is the address information used by the user to identify and access data when operating the storage system (such as accessing files, reading and writing data). It establishes a connection between the user and the actual data storage location on the storage device. For example, the user address can be a file path.

[0036] The target data disk 102 is used to establish the target namespace of the data to be migrated on the corresponding source data disk when the data migration session establishment request is met, and to send the information that the data migration session has been successfully established to the corresponding source data disk.

[0037] The source data disk 101 is also used to write the data to be migrated into the target namespace of the corresponding target data disk according to its corresponding source logical address in the source data disk, based on the data migration session when the data migration session is successfully established.

[0038] The source data disk determines the source logical address and data length of the data to be migrated based on the metadata list of the data to be migrated. Based on the source logical address and data length, the source data disk reads the data to be migrated and, according to the data migration session, writes the data to be migrated to the target namespace of the corresponding target data disk according to its source logical address in the source data disk.

[0039] In other words, the source data disk writes the data to be migrated directly to the target namespace of the corresponding target data disk, specifying that the logical address of the data to be migrated in the target namespace of the corresponding target data disk is the same as its source logical address in the source data disk.

[0040] The storage cluster management module 103 is used to record the correspondence between the source logical address of the data to be migrated and the target namespace of the target data disk corresponding to the data to be migrated at the system layer after the source data disk has completed writing the data to be migrated into the target namespace of the corresponding target data disk.

[0041] After the source data disk completes the writing of the data to be migrated into the target namespace of the corresponding target data disk, thus completing the data migration task, the storage cluster management module records the target data disk and target namespace in the disk space management information of the storage system. At the system level, it records the correspondence between the source logical address of the data to be migrated and the target namespace of the target data disk corresponding to the data to be migrated, so that when a user accesses the data to be migrated based on the user address of the data to be migrated, the data to be migrated can be obtained from the target namespace of the target data disk corresponding to the data to be migrated.

[0042] The data migration system provided in this application utilizes the characteristic that different namespaces of NVMe disks have independent LBA address spaces. It writes the data to be migrated into the target namespace of the corresponding target data disk according to its source logical address in the source data disk. This ensures that the logical address of the data remains unchanged after migration. Therefore, there is no need to modify the mapping relationship between user address and logical address in the metadata. This solves the technical problem in related technologies where data migration requires modification of a large amount of metadata, which reduces the performance of the storage system. It achieves the technical effect of avoiding modification of metadata during data migration, improving the performance of the storage system, and also improving the security and reliability of the data.

[0043] In some alternative implementations, the target data disk 102 is specifically used for:

[0044] Obtain the remaining space of the target data disk and the space required to migrate the data from the source data disk corresponding to the target data disk in the data migration session establishment request.

[0045] Understandably, the source data disk determines the data to be migrated based on the data migration task, then determines the space required for the data to be migrated, and generates a data migration session establishment request based on the space required for the data to be migrated.

[0046] If the remaining space on the target data disk is not less than the space required for the data to be migrated, the data migration session establishment request is satisfied, and the target namespace for the data to be migrated is established based on the space required for the data to be migrated.

[0047] Understandably, if the remaining space on the target data disk is less than the space required for the data to be migrated, it is determined that the data migration session establishment request cannot be met, and a data migration session establishment failure message is sent to the source data disk. Upon receiving the data migration session establishment failure message, the source data disk sends the message to the storage cluster management module as an alarm.

[0048] The data migration system provided in this application avoids unnecessary migration attempts, reduces the waste of system resources, and improves the overall stability of the system by establishing a data migration session only when it is determined that the target data disk has enough space to accommodate the data to be migrated.

[0049] In some alternative implementations, the source data disk 101 described above is specifically used for:

[0050] If the data migration session is successfully established, at least one data transmission channel is established between the data migration session and the corresponding target data disk.

[0051] In the event that the data migration session is successfully established, an NVMe data transmission channel is established between the source data disk and the target data disk based on the data migration session. Specifically, the target data disk starts the NVMe target to receive data service, and the source data disk starts the NVMe initator to send data service. The parameters of the NVMe data transmission channel are negotiated through the data migration session to establish the NVMe data transmission channel.

[0052] Based on the number of data transmission channels, the data to be migrated is divided into at least one data block.

[0053] The number of data blocks can be the same as the number of data transmission channels.

[0054] Based on the data transmission channel, data blocks are written to the target namespace of the corresponding target data disk according to their source logical address in the source data disk.

[0055] It is understandable that when the number of data transmission channels is greater than one, data blocks are written in parallel to the target namespace of the corresponding target data disk according to their corresponding source logical addresses in the source data disk, based on the data transmission channels, in order to improve writing efficiency.

[0056] The data migration system provided in this application embodiment achieves data migration while maintaining the logical address by specifying the NVMe data transmission channel from the source namespace of the source data disk to the target namespace of the target data disk. Utilizing the characteristic that different namespaces of NVMe disks have independent LBA address spaces, the appended data (i.e., the data to be migrated) is migrated directly from disk to disk, avoiding LBA updates and achieving data migration without logical address updates. This reduces the need for metadata updates, increases the speed of data migration, reduces data migration time, improves the performance of the storage system, and reduces maintenance difficulty and cost.

[0057] In some optional implementations, the storage cluster management module 103 is further configured to:

[0058] In the event of a state change in the storage cluster, determine the source data disk and the corresponding target data disk.

[0059] Understandably, the storage cluster management module can also determine the data to be migrated from the source data disk.

[0060] A data migration task is generated based on the source data disk and the corresponding target data disk.

[0061] A data migration task is generated based on the source data disk, the target data disk corresponding to the source data disk, and the data to be migrated on the source data disk.

[0062] Send the data migration task to the corresponding source data disk.

[0063] The data migration system provided in this application improves system stability by automatically identifying the source data disk and its corresponding target data disk when the cluster state changes, thereby quickly generating and issuing data migration tasks.

[0064] In some optional implementations, the storage cluster management module 103 described above is specifically used for:

[0065] If the state of the storage cluster changes to the presence of at least one faulty storage node, then all data disks of the redundant storage node corresponding to the faulty storage node will be used as the source data disks, provided that the redundant storage node corresponding to the faulty storage node has not failed.

[0066] In this embodiment, the data in all data disks of the redundant storage node is the data to be migrated.

[0067] For any source data disk, obtain the status information of the data disks in the storage nodes of the storage cluster, excluding the failed storage node and the redundant storage node corresponding to the failed storage node.

[0068] Based on the status information of the data disks of other storage nodes, the target data disk corresponding to the source data disk is selected from the data disks of other storage nodes.

[0069] It is understandable that once the target data disk corresponding to a source data disk is determined, the status information of the target data disk is updated.

[0070] The data migration system provided in this application embodiment can quickly identify all data disks on the redundant nodes of the faulty node as source data disks when a faulty node occurs in the storage cluster, and use their data as data to be migrated to migrate the data to the target data disk, thereby ensuring the high availability and business continuity of the system.

[0071] In some optional implementations, the storage cluster management module 103 described above is specifically used for:

[0072] For any source data disk, based on the remaining space size represented by the status information of the data disks of other storage nodes, the data disk with the largest remaining space size is selected from the data disks of other storage nodes, and the data disk with the largest remaining space size is taken as the target data disk corresponding to the source data disk.

[0073] The data migration system provided in this application embodiment can effectively utilize data disks with more free capacity in the cluster by prioritizing the data disk with the largest remaining space as the target data disk, thus avoiding data migration failure due to insufficient space.

[0074] In some optional implementations, the storage cluster management module 103 described above is specifically used for:

[0075] If the storage cluster status changes to include the addition of multiple first storage nodes, the status information of the other storage nodes in the cluster, excluding the first storage nodes, is obtained. If at least one of the other storage nodes' status information indicates that its remaining space is not greater than a first remaining space threshold, then based on the remaining space of each data disk in that storage node's data disks, the data disks in that storage node whose remaining space is not greater than a second remaining space threshold are identified as source data disks. For any source data disk, a target first storage node is randomly selected from the multiple first storage nodes, and a target first data disk is randomly selected from the target first storage node. These target first data disks are then used as the target data disks corresponding to the source data disk. The first and second remaining space thresholds are determined by technical personnel and are not specifically limited here.

[0076] The status change of the storage cluster to indicate that multiple first storage nodes have been added to the storage cluster indicates that the storage cluster has been expanded.

[0077] If the storage cluster status changes to a reduction of multiple second storage nodes, then the data disk of the second storage node is used as the source data disk. For any source data disk, the status information of each storage node in the storage cluster is obtained. Based on the status information of each storage node, the storage node with the largest remaining space in the storage cluster is determined. From the storage node with the largest remaining space in the storage cluster, the data disk with the largest remaining space is selected as the target data disk corresponding to the source data disk.

[0078] The change in the storage cluster's status to indicate the reduction of several second storage nodes suggests that the storage cluster has undergone a downsizing process. It's understandable that after downsizing, the status information of each storage node in the storage cluster does not include the status information of the second storage nodes.

[0079] The data migration system provided in this application uses a data disk with less remaining space in a storage node with insufficient remaining space as the source data disk, and migrates the data to be migrated from the source data disk to the newly added first storage node. This helps to balance the space utilization of each storage node, avoid storage node overload, and improve the success rate and efficiency of data migration.

[0080] When multiple storage nodes are deleted from the storage cluster, all data disks on those nodes are marked as source disks, and a migration process is initiated. This ensures that data integrity and high availability are maintained even when storage nodes are scaled down, preventing service interruptions or data loss. For each source disk, the system assesses the remaining space on other storage nodes in the entire storage cluster and prioritizes the storage node with the largest remaining space as the target storage node. Then, it selects the data disk with the largest remaining space on that target storage node as the target data disk. This approach significantly improves the success rate of migration tasks and reduces failures due to insufficient space.

[0081] In some optional implementations, the storage cluster management module 103 described above is specifically used for:

[0082] If the state of the storage cluster changes to at least one faulty data disk in at least one storage node of the storage cluster, then the redundant data disk corresponding to the faulty data disk will be used as the source data disk, provided that the redundant data disk corresponding to the faulty data disk has not failed.

[0083] For any source data disk, obtain the status information of other data disks in the storage cluster, excluding the faulty data disk and the redundant data disk corresponding to the faulty data disk.

[0084] Based on the remaining space size represented by the status information of other data disks, the data disk with the largest remaining space size is selected from the other data disks, and the data disk with the largest remaining space size is used as the target data disk corresponding to the source data disk.

[0085] The data migration system provided in this application improves the system's fault tolerance and the continuity of data services by quickly locating the corresponding redundant data disk as the source data disk when a data disk in the storage cluster fails, using its data as the data to be migrated, and migrating the data to be migrated to the target data disk.

[0086] In some optional implementations, the storage cluster management module 103 is further configured to:

[0087] If the source data disk fails to successfully write the data to be migrated to the target namespace of the corresponding target data disk, restore the source data disk and the target data disk to the state before the data migration task was executed, and reissue the corresponding data migration task to the source data disk.

[0088] An alarm will be triggered if the number of times the step of resending the corresponding data migration task to the source data disk is exceeded by a preset threshold, and the source data disk still fails to successfully write the data to be migrated to the target namespace of the corresponding target data disk.

[0089] The data migration system provided in this application, by introducing failure rollback, automatic retry, and alarm mechanisms, constructs a highly reliable, fault-tolerant, and easily maintainable data migration assurance system. It not only enhances the system's self-healing capability in the face of migration failures but also strengthens the visibility and controllability of operations and maintenance through intelligent alarms.

[0090] Embodiments of this application provide a data migration method applicable to any of the aforementioned data migration systems. Figure 2 This is a flowchart illustrating the data migration method provided in an embodiment of this application, as shown below. Figure 2 As shown, the process includes the following steps:

[0091] Step S201: Using the source data disk, based on the data migration task, determine the corresponding target data disk and send a data migration session establishment request to the corresponding target data disk.

[0092] Step S202: Based on the target data disk, if the data migration session establishment request is satisfied, establish the target namespace of the data to be migrated on the corresponding source data disk, and send the information that the data migration session has been successfully established to the corresponding source data disk.

[0093] Step S203: Based on the source data disk, if the data migration session is successfully established, the data to be migrated is written to the target namespace of the corresponding target data disk according to its corresponding source logical address in the source data disk.

[0094] Step S204: Based on the storage cluster management module, after the source data disk has completed writing the data to be migrated into the target namespace of the corresponding target data disk, the system layer records the correspondence between the source logical address of the data to be migrated and the target namespace of the target data disk corresponding to the data to be migrated.

[0095] For a detailed description of the method steps, please refer to the relevant description in the foregoing system embodiments, which will not be repeated here.

[0096] The data migration method provided in this application writes the data to be migrated to the target namespace of the corresponding target data disk according to its source logical address in the source data disk. This ensures that the logical address of the data remains unchanged after migration. Therefore, there is no need to modify the mapping relationship between user address and logical address in the metadata. This solves the technical problem in related technologies where data migration requires modification of a large amount of metadata, which reduces the performance of the storage system. It achieves the technical effect of avoiding modification of metadata during data migration and improving the performance of the storage system.

[0097] Embodiments of this application provide a data migration method. Figure 3 A timing diagram of the data migration method provided in the embodiments of this application, as shown below. Figure 3 As shown, this data migration method includes the following steps:

[0098] Step 301: When the storage cluster undergoes a state change, the storage cluster management module sends the data migration task to the source data disk.

[0099] Step 302: The source data disk receives the data migration task and determines the corresponding target data disk and the data to be migrated.

[0100] Step 303: The source data disk sends a data migration session establishment request to the target data disk to establish a data migration session with the target data disk.

[0101] Step 304: After successfully establishing a data migration session with the target data disk, the source data disk establishes a data transfer channel with the target data disk.

[0102] Step 305: The source data disk transfers the data to be migrated based on the data transfer channel. Specifically, based on the data transfer channel, the data to be migrated is written to the target namespace of the corresponding target data disk according to its source logical address in the source data disk.

[0103] Step 306: After the source data disk completes the data migration task, the information on the completion of the cluster status change is fed back to the storage cluster management module so that the storage cluster management module records the correspondence between the source logical address of the data to be migrated and the target namespace of the target data disk corresponding to the data to be migrated at the system level.

[0104] For example, Figure 4 This is a schematic diagram illustrating data migration as provided in an embodiment of this application. Figure 4 Therefore, the source data disk and the target data disk are NVMe disks on different storage nodes. Upon receiving the data migration task, the source data disk determines the corresponding target data disk and the data to be migrated, and sends a data migration session establishment request to the target data disk, including the required space size for the data to be migrated. The target data disk checks its remaining space. If its remaining space meets the data migration session establishment request, the target data disk creates a new namespace (target namespace) and returns a success message indicating successful data migration session establishment. Based on the data migration session, the source and target data disks establish an NVMe data transfer channel. The source data disk determines the LBA and data length of the data to be migrated according to the metadata list managed by the storage cluster management module, and reads the data to be migrated from the source data disk based on the LBA and data length. The source data disk sends the data to be migrated directly to the NVMe target of the target data disk through the NVMe initator, specifying the same logical address and a new namespace, thus completing the data migration from the source data disk to the target data disk.

[0105] The data migration method provided in this application utilizes the characteristic that different namespaces of NVMe disks have independent LBA address spaces. The data to be migrated is written to the target namespace of the corresponding target data disk according to its source logical address in the source data disk. This ensures that the logical address of the data after migration does not change, thereby avoiding modification of metadata during data migration, improving the performance of the storage system, and also improving the security and reliability of the data.

[0106] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods according to the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method.

[0107] Embodiments of this application also provide an electronic device, such as... Figure 5 As shown, it includes a processor 501 and a memory 502, in which a computer program is stored. The processor 501 is configured to run the computer program to perform the steps in any of the above-described data migration method embodiments.

[0108] Embodiments of this application also provide a computer-readable storage medium storing a computer program configured to execute the steps in any of the above-described data migration method embodiments at runtime.

[0109] In one exemplary embodiment, the aforementioned computer-readable storage medium may include, but is not limited to, various media capable of storing computer programs, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard disk, magnetic disk, or optical disk.

[0110] Embodiments of this application also provide a computer program product, which includes a computer program that, when executed by a processor, implements the steps in any of the above-described data migration method embodiments.

[0111] Embodiments of this application also provide another computer program product, including a non-volatile computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps in any of the above-described data migration method embodiments.

[0112] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0113] The data migration system, method, electronic device, and storage medium provided in this application have been described in detail above. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the embodiments above are only for the purpose of helping to understand the method and core ideas of this application. It should be noted that those skilled in the art can make several improvements and modifications to this application without departing from the principles of this application, and these improvements and modifications also fall within the protection scope of the claims of this application.

Claims

1. A data migration system, characterized in that, The system includes: The source data disk is used to determine the corresponding target data disk based on the data migration task and send a data migration session establishment request to the corresponding target data disk. The target data disk is used to establish the target namespace of the data to be migrated from the corresponding source data disk when the data migration session establishment request is met, and to send the information that the data migration session has been successfully established to the corresponding source data disk. The source data disk is also used to write the data to be migrated into the target namespace of the corresponding target data disk according to its corresponding source logical address in the source data disk, based on the data migration session when the data migration session is successfully established. The storage cluster management module is used to record the correspondence between the source logical address of the data to be migrated and the target namespace of the target data disk corresponding to the data to be migrated at the system layer after the source data disk has completed writing the data to be migrated into the target namespace of the corresponding target data disk.

2. The system according to claim 1, characterized in that, The target data disk is specifically used for: Obtain the remaining space size of the target data disk and the space required for the data to be migrated from the source data disk corresponding to the target data disk in the data migration session establishment request; If the remaining space on the target data disk is not less than the space required by the data to be migrated, the data migration session establishment request is satisfied, and a target namespace for the data to be migrated is established based on the space required by the data to be migrated.

3. The system according to claim 1, characterized in that, The source data disk is specifically used for: If the data migration session is successfully established, at least one data transmission channel is established between the data migration session and the corresponding target data disk. Based on the number of data transmission channels, the data to be migrated is divided into at least one data block; Based on the data transmission channel, the data block is written into the target namespace of the corresponding target data disk according to its corresponding source logical address in the source data disk.

4. The system according to claim 1, characterized in that, The storage cluster management module is also used for: In the event of a state change in the storage cluster, the source data disk and the corresponding target data disk must be identified. Generate a data migration task based on the source data disk and the target data disk corresponding to the source data disk; Send the data migration task to the corresponding source data disk.

5. The system according to claim 4, characterized in that, The storage cluster management module is specifically used for: If the state of the storage cluster changes to the presence of at least one faulty storage node, then, assuming that the redundant storage node corresponding to the faulty storage node has not failed, all data disks of the redundant storage node corresponding to the faulty storage node will be used as the source data disks. For any source data disk, obtain the status information of the data disks in the other storage nodes of the storage cluster, excluding the faulty storage node and the redundant storage node corresponding to the faulty storage node. Based on the status information of the data disks of the other storage nodes, the target data disk corresponding to the source data disk is selected from the data disks of the other storage nodes.

6. The system according to claim 5, characterized in that, The storage cluster management module is specifically used for: For any source data disk, based on the remaining space size represented by the status information of the data disks of the other storage nodes, the data disk with the largest remaining space size is selected from the data disks of the other storage nodes, and the data disk with the largest remaining space size is taken as the target data disk corresponding to the source data disk.

7. The system according to claim 4, characterized in that, The storage cluster management module is specifically used for: If the state of the storage cluster changes to the addition of multiple first storage nodes, then the state information of other storage nodes in the storage cluster besides the first storage nodes is obtained. If the state information of at least one storage node indicates that the remaining space of the storage node is not greater than the first remaining space threshold, then based on the remaining space of each data disk in all data disks of the storage node, the data disks in the storage node whose remaining space is not greater than the second remaining space threshold are determined as source data disks. For any source data disk, a target first storage node is randomly selected from multiple first storage nodes, and a target first data disk is randomly selected from the target first storage node. The target first data disk is used as the target data disk corresponding to the source data disk. If the status of the storage cluster changes to the point that the storage cluster has reduced multiple second storage nodes, then the data disk of the second storage node is used as the source data disk. For any source data disk, the status information of each storage node in the storage cluster is obtained. Based on the status information of each storage node, the storage node with the largest remaining space in the storage cluster is determined. The data disk with the largest remaining space from the storage node with the largest remaining space in the storage cluster is selected as the target data disk corresponding to the source data disk.

8. A data migration method, characterized in that, The method, applied to the data migration system according to any one of claims 1 to 7, comprises: Using the source data disk, based on the data migration task, determine the corresponding target data disk and send a data migration session establishment request to the corresponding target data disk; Based on the target data disk, if the data migration session establishment request is met, the target namespace of the data to be migrated on the corresponding source data disk is established, and the information that the data migration session has been successfully established is sent to the corresponding source data disk. Based on the source data disk, if the data migration session is successfully established, the data to be migrated is written to the target namespace of the corresponding target data disk according to its source logical address in the source data disk. Based on the storage cluster management module, after the source data disk completes the writing of the data to be migrated into the target namespace of the corresponding target data disk, the system layer records the correspondence between the source logical address of the data to be migrated and the target namespace of the target data disk corresponding to the data to be migrated.

9. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor for implementing the steps of the data migration method as described in claim 8 when executing the computer program.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, wherein the computer program, when executed by a processor, implements the steps of the data migration method as described in claim 8.

Images