Data migration method and device, electronic equipment, storage medium and program product

Abstract

The application discloses a data migration method and device, electronic equipment, a storage medium and a program product, and relates to the technical field of storage. The target storage type can indicate a storage mode of data, the target strip size can indicate the length of reconstructed data, and the total quantity can indicate a position allocation mode of data in a storage array. In order to reduce the data migration amount, a first starting address and a first space length of an effective storage space in an original storage array can be acquired, so as to determine the position of effective data. Further, based on the target strip size, the first starting address and the first space length can be adjusted to obtain target position information, so that the space length of a target task space indicated by the target position information is an integer multiple of the target strip size. Finally, based on the target strip size, the target storage type and the total quantity, data of the target task space can be completely migrated to a target storage array, and the migration efficiency can be greatly improved.

Description

Technical Field

[0001] This application relates to the field of storage technology, and in particular to data migration methods, apparatus, electronic devices, storage media, and program products. Background Technology

[0002] In the field of storage technology, storage systems are generally used to store data. A storage system can include a storage controller and a storage array. The storage controller can be used to read and write data, perform tier migrations and expansion operations on the storage array, etc. The storage array can include multiple storage devices, which can store data. For example, a storage device can be a hard disk.

[0003] Current storage systems typically migrate data across all storage spaces in the storage array during tier migration and expansion tasks, resulting in low migration efficiency. Summary of the Invention

[0004] This application provides data migration methods, apparatus, electronic devices, storage media, and program products to solve the problem of low migration operation efficiency.

[0005] This application provides a data migration method, including:

[0006] The configuration information of the target storage array and the initial location information of the effective storage space for storing valid data in the original storage array are obtained. The configuration information includes the target storage type, the total number of storage devices in the target storage array, and the target stripe size corresponding to the target storage type. The initial location information includes the first starting address and the first space length.

[0007] The target location information is determined based on the first starting address, the first spatial length, and the target strip size;

[0008] Based on the target location information, target storage type, total quantity, and target stripe size, the data in the target task space corresponding to the target location information in the original storage array is migrated to the target storage array. The length of the target task space is an integer multiple of the target stripe size.

[0009] This application also provides a data migration apparatus, including:

[0010] The acquisition module is used to obtain the configuration information of the target storage array and the initial location information of the effective storage space in the original storage array that stores valid data. The configuration information includes the target storage type, the total number of storage devices in the target storage array, and the target stripe size corresponding to the target storage type. The initial location information includes the first starting address and the first space length.

[0011] The determination module is used to determine the target location information based on the first starting address, the first spatial length, and the target strip size;

[0012] The migration module is used to migrate data from the target task space corresponding to the target location information in the original storage array to the target storage array based on the target location information, target storage type, total quantity, and target stripe size. The length of the target task space is an integer multiple of the target stripe size.

[0013] This application also provides an electronic device, including: a memory for storing a computer program; and a processor for implementing the steps of any of the above-described data migration methods when executing the computer program.

[0014] This application also provides a computer-readable storage medium storing a computer program, wherein the computer program, when executed by a processor, implements the steps of any of the above-described data migration methods.

[0015] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of any of the above-described data migration methods.

[0016] This application, firstly, identifies the target storage type as indicating the data storage method, the target stripe size as indicating the length of the reconstructed data, and the total quantity as indicating the data's location allocation within the storage array. Furthermore, to reduce data migration, the first starting address and first space length of the effective storage space in the original storage array can be obtained to determine the location of the effective data. Further, since the length of the effective data cannot be guaranteed to be an integer multiple of the target stripe size, the first starting address and first space length can be adjusted based on the target stripe size to obtain target location information, ensuring that the space length of the target task space indicated by the target location information is an integer multiple of the target stripe size. Finally, based on the target stripe size, target storage type, and total quantity, all data in the target task space is migrated to the target storage array. This data migration process significantly reduces the amount of data migrated, and migrating data in a full-strip manner greatly improves migration efficiency and reduces resource waste. Additionally, since data reconstruction may involve both level migration and capacity adjustment tasks, typically one task is completed before the other, resulting in longer task execution times and higher resource consumption. Therefore, this solution only requires knowledge of the target storage array's structure (target stripe size, total number, and target storage type) to combine two types of tasks into a single task. Only one data reconstruction operation is needed, significantly reducing task execution time and resource consumption. Furthermore, a similar approach can be used to execute data migration tasks for both single-level migration tasks and single-capacity adjustment tasks. Attached Figure Description

[0017] To more clearly illustrate the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 A schematic diagram of the architecture of a storage system provided in an embodiment of this application;

[0019] Figure 2 A schematic diagram of a strip structure provided in an embodiment of this application;

[0020] Figure 3 A flowchart illustrating a data migration method provided in an embodiment of this application;

[0021] Figure 4 A flowchart illustrating another data migration method provided in an embodiment of this application;

[0022] Figure 5A flowchart illustrating yet another data migration method provided in an embodiment of this application;

[0023] Figure 6 A schematic diagram of the structure of the storage array provided in the embodiments of this application. Figure 1 ;

[0024] Figure 7 A schematic diagram of the structure of the storage array provided in the embodiments of this application. Figure 2 ;

[0025] Figure 8 A schematic diagram of the structure of the storage array provided in the embodiments of this application. Figure 3 ;

[0026] Figure 9 A schematic diagram of the structure of the storage array provided in the embodiments of this application. Figure 4 ;

[0027] Figure 10 A schematic diagram of the structure of the storage array provided in the embodiments of this application. Figure 5 ;

[0028] Figure 11 A schematic diagram of the structure of the storage array provided in the embodiments of this application. Figure 6 ;

[0029] Figure 12 A schematic diagram of the structure of the storage array provided in the embodiments of this application. Figure 7 ;

[0030] Figure 13 This is a schematic flowchart of a data migration apparatus provided in an embodiment of this application;

[0031] Figure 14 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0032] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the protection scope of this application.

[0033] It should be noted that, in the description of this application, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. The terms "first," "second," etc., in this application are used to distinguish similar objects and are not used to describe a specific order or sequence.

[0034] To enable those skilled in the art to better understand the present application, the present application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0035] The data migration method provided in this application can be implemented by a storage system, such as... Figure 1 As shown, a storage system may include a storage controller and a storage array. The storage array may include multiple storage devices, and data may be stored in a stripe-like distribution. For example, a storage array may be a Redundant Array of Independent Disks (RAID), which is a large-capacity disk group composed of multiple independent disks. The stripe structure in a storage array can be as follows... Figure 2 As shown, each of the four blocks in stripe 0 is stored on a storage device.

[0036] Embodiments of this application provide a data migration method, which can be executed by the aforementioned storage controller, such as... Figure 3 As shown, the specific processing steps of the data migration method may include:

[0037] Step S301: Obtain the configuration information of the target storage array and the initial location information of the effective storage space storing valid data in the original storage array.

[0038] The configuration information may include the target storage type of the target storage array, the total number of storage devices in the target storage array, and the target stripe size corresponding to the target storage type. The target storage type can be a RAID level. The initial location information includes the first starting address and the first space length.

[0039] Specifically, in storage systems, reconstruction tasks typically include RAID level migration tasks and capacity adjustment tasks. RAID level migration tasks can be upward or downward migration tasks. Capacity adjustment tasks can be expansion or reduction tasks. For example, an upward migration task might involve migrating from RAID 0 to RAID 5 or RAID 6 to improve data redundancy and fault tolerance, or migrating from RAID 0 to RAID 10 to improve data read / write speed. A downward migration task might involve migrating to a lower RAID level configuration when a storage device fails, freeing up the failed storage device, and then replacing it; specifically, it could be migrating from RAID 6 to RAID 5.

[0040] Since different refactoring tasks mainly involve changes in one or more parameters such as the number of storage devices, storage type, and stripe size, the storage controller can obtain the target storage type, target stripe size, and the total number of storage devices in the target storage array when executing different refactoring tasks. The target storage type indicates the data storage method; for example, in RAID 5, one parity data is generated for each stripe of data, while in RAID 6, two parity data are generated for each stripe of data.

[0041] Furthermore, storage space in a storage array is typically divided into valid storage space for storing user data (also known as effective data) and unused invalid storage space (which may contain obsolete data). During the refactoring process, to reduce the amount of data migration, only the valid data in the valid storage space can be refactored and migrated. Therefore, before executing the refactoring task, the storage controller can identify the first starting address and first space length of the valid storage space storing valid data in the original storage array.

[0042] Step S302: Determine the target location information based on the first starting address, the first spatial length, and the target strip size.

[0043] Specifically, while migrating only valid data can significantly reduce the amount of data to be migrated, the length of valid data is often not an integer multiple of the target stripe size. This means that valid data cannot be migrated entirely in full stripe form, leading to stripe misalignment and other issues. Since full-strip migration operations consume fewer I / O resources, the storage controller can adjust the first starting address and the first space length based on the target stripe size to obtain the target location information. This ensures that the data corresponding to the target location information is an integer multiple of the target stripe size, facilitating subsequent data reconstruction and migration operations using full stripes and improving the efficiency of reconstruction tasks.

[0044] Step S303: Based on the target location information, target storage type, total quantity, and target stripe size, migrate the data in the target task space corresponding to the target location information in the original storage array to the target storage array.

[0045] The spatial length of the target task space is an integer multiple of the target strip size.

[0046] Specifically, the storage controller can determine the target task space corresponding to the target location information within the original storage array. Since the data corresponding to the target location information is an integer multiple of the target stripe size, the length of the target task space is also an integer multiple of the target stripe size. Then, based on the target stripe size, the storage controller can read at least one set of data of the same size as the target stripe from the target task space in the original storage array. Then, based on the target storage type and the total quantity, it performs a reconstruction operation corresponding to the target storage type on each set of data, obtaining reconstructed data for each set. Finally, the storage controller can store the reconstructed data in the target storage array to complete the data migration operation of the target task space.

[0047] The data migration method of this application, according to embodiments, firstly, indicates the data storage method by specifying the target storage type, the target stripe size by specifying the length of the reconstructed data, and the total quantity by specifying the data allocation method within the storage array. Furthermore, to reduce the amount of data migration, the first starting address and first space length of the effective storage space in the original storage array can be obtained to determine the location of the effective data. Further, since the length of the effective data cannot be guaranteed to be an integer multiple of the target stripe size, the first starting address and first space length can be adjusted based on the target stripe size to obtain target location information, ensuring that the space length of the target task space indicated by the target location information is an integer multiple of the target stripe size. Finally, based on the target stripe size, target storage type, and total quantity, all data in the target task space is migrated to the target storage array. In the above data migration process, the amount of data migrated is significantly reduced, and migrating in a full-strip manner greatly improves migration efficiency and reduces resource waste. Additionally, since data reconstruction may involve both level migration and capacity adjustment tasks, typically one task is completed before the other, resulting in longer task execution times and higher resource consumption. Therefore, this solution only requires knowledge of the target storage array's structure (target stripe size, total number, and target storage type) to combine two types of tasks into a single task. Only one data reconstruction operation is needed, significantly reducing task execution time and resource consumption. Furthermore, a similar approach can be used to execute data migration tasks for both single-level migration tasks and single-capacity adjustment tasks.

[0048] In some optional implementations, in step S302 above, the storage controller may specifically determine the target location information based on the first starting address, the first space length, and the target stripe size using the following steps:

[0049] Step 1: Determine the second starting address based on the first starting address and the target stripe size.

[0050] Step 2: Determine the second space length based on the first starting address, the second starting address, the first space length, and the target stripe size.

[0051] Step 3: Determine the target location information based on the second starting address and the second spatial length.

[0052] Specifically, the first starting address is the starting address of the effective storage space, and the target stripe size is the stripe size of the target storage array. Therefore, the storage controller can adjust the first starting address using the target stripe size to obtain a new starting address, which is the second starting address. Furthermore, the storage controller can adjust the first space length using the first starting address, the second starting address, and the target stripe size to determine the second space length. Finally, the storage controller can use the second starting address and the second space length together to determine the target location information.

[0053] In this way, by combining the original first starting address and first space length of the effective storage space with the target stripe size to redetermine the target location information of the migrated data, the determined target location information can both cover the original effective storage space and ensure that the target location information meets the conditions for full stripe writing, thereby reducing the amount of migrated data and increasing efficiency.

[0054] For example, step one above can be expressed as follows:

[0055] (1)

[0056] Where slba' is the second starting address, slab is the first starting address, and x is the target strip size.

[0057] In this way, by rounding down the ratio of the first starting address to the target stripe size, and then multiplying the rounded value by the target stripe size, the determined second starting address can be aligned with the stripe in the original storage array. This facilitates subsequent reading of data at the full target stripe size and reduces the occupation of various resources.

[0058] Step two can be performed using the following expression:

[0059] (2)

[0060] Where nlb' is the second space length, slab is the first starting address, slba' is the second starting address, nlb is the first space length, and x is the target strip size.

[0061] In this way, the formula ensures that the final determined second space length is an integer multiple of the target strip size, and that the final determined second space length is greater than the first space length. This guarantees that the effective data can be completely migrated, while also reducing the consumption of various resources.

[0062] In some alternative implementations, in step one above, the storage controller may specifically determine the second starting address based on the first starting address and the target stripe size using the following steps:

[0063] Step 1: Determine the stripe index based on the first starting address and the target stripe size.

[0064] Step 2: Determine the second starting address based on the stripe index and the target stripe size.

[0065] Specifically, the storage controller can determine the stripe index of the stripe containing the first starting address based on the first starting address and the target stripe size, and then determine the second starting address based on the stripe index and the target stripe size, so that the determined second starting address can contain the first starting address.

[0066] In some optional implementations, in step two above, the storage controller may specifically determine the second space length based on the first starting address, the second starting address, the first space length, and the target stripe size using the following steps:

[0067] Step 1: Determine the number of stripes based on the first starting address, the second starting address, the first space length, and the target stripe size.

[0068] Step 2: Determine the second space length based on the number of strips and the target strip size.

[0069] Specifically, the storage controller can calculate the number of stripes by considering the first starting address, the second starting address, the first space length, and the target stripe size. Finally, the storage controller can determine the second space length corresponding to the same number of stripes as the target stripe size, based on the number of stripes and the target stripe size.

[0070] In some optional implementations, in step 1 above, the storage controller may specifically determine the number of stripes based on the first starting address, the second starting address, the first space length, and the target stripe size, including:

[0071] Step a: Determine the third space length based on the first starting address, the second starting address, and the first space length.

[0072] Step b: Determine the number of stripes based on the length of the third space and the size of the target stripe.

[0073] Specifically, the storage controller can perform calculations on the first starting address, the second starting address, and the first space length to obtain the third space length, and then determine the number of stripes of data to be migrated based on the third space length and the target stripe size.

[0074] In some alternative implementations, in step S303 above, the storage controller can complete the data migration operation in multiple rounds, correspondingly, such as Figure 4As shown, the specific steps may include:

[0075] Step 1: In the current round, based on the target strip size, obtain the target sub-position information corresponding to the current round.

[0076] Step 2: Based on the target sub-location information, read the initial data corresponding to the current round from the target sub-task space corresponding to the target sub-location information in the original storage array.

[0077] Step 3: Based on the target storage type and total quantity, reconstruct the initial data corresponding to the current round to obtain the final data corresponding to the current round.

[0078] Step four: Store the final data corresponding to the current round into the target storage array.

[0079] Step 5: Based on the target location information and target sub-location information, determine whether the current round is the last round. If it is determined that the current round is the last round, migrate all data in the target task space to the target storage array and stop processing. Alternatively, if it is determined that the current round is not the last round, proceed to the next round of processing until all data in the target task space has been migrated to the target storage array, and then stop processing.

[0080] Specifically, the storage controller can execute the data migration process in multiple rounds. In each round, the storage controller can read initial data of the same size as the target stripe from the original storage array, and perform a reconstruction operation on the initial data according to the target storage type to obtain the final data for that round. The storage controller can then store the final data of that round into the target storage array.

[0081] In step one, for each round, the storage controller can first determine the sub-location information of the data to be migrated in that round. The rounds are divided into the first round and non-first rounds. The way the sub-location information is determined differs between the two rounds, but both use the target stripe size to ensure that data of the target stripe size can be read.

[0082] For example, if the current round is the first round, the first end address is determined based on the second start address and the target strip size. The second start address and the first end address are then used as the target sub-location information. Alternatively, if the current round is not the first round, the end address included in the sub-location information corresponding to the previous round is determined as the third start address. The second end address is determined based on the third start address and the target strip size. The third start address and the second end address are then used as the target sub-location information.

[0083] Specifically, in the first round, the storage controller can determine the first end address by summing the second start address and the target stripe size. Then, the second start address and the first end address are used to determine the target sub-location information for the current round. In rounds other than the first, the storage controller can use the end address determined in the previous round as the third start address for the current round, and then determine the second end address by summing the third start address and the target stripe size. Finally, the third start address and the second end address are used to determine the target sub-location information for the current round. The start address of each round can be called the Low Transform Logical Block Address Mark (LTLM) of the target sub-task space, and the start address of each round can be called the High Transform Logical Block Address Mark (HTLM) of the target sub-task space.

[0084] In step two, the storage controller can determine the target sub-task space corresponding to the target sub-location information in the target task space based on the target sub-location information, and read the initial data to be migrated in the current round from the target sub-task space.

[0085] In step three, the target storage type represents the storage type that the reconstruction task aims to achieve. Furthermore, different storage types require different reconstruction methods. Therefore, the storage controller can obtain the reconstruction method corresponding to the target storage type (e.g., how to generate verification data and how to divide the data into blocks) according to the target storage type and the total number of data. Based on the obtained reconstruction method, the controller performs reconstruction operations on the initial data of the current round to obtain the final data of the current round.

[0086] In step four, the storage controller can determine the target storage location for the current round (e.g., a storage space of the same size as the target stripe after the final data stored in the previous round) and store the final data of the current round in the target storage location.

[0087] In step five, the storage controller can compare the target location information with the target sub-location information of the current round to determine whether the current round is the last round (for example, it can determine the third end address based on the second start address and second space length in the target location information, and compare whether the end address in the target sub-location information is consistent with the third end address; if they are consistent, the current round is determined to be the last round; if they are inconsistent, the current round is determined not to be the last round). In other words, it determines whether all data in the target task space has been migrated to the target storage array. If yes, processing can stop; if not, it can proceed to the next round, and so on, until all data in the target task space has been migrated to the target storage array, at which point processing stops.

[0088] For example, for an expansion task, if the target storage type is RAID0 and the total number of storage devices in the original storage array is 2, the blocks of reconstructed data 1 can be stored on storage device 1 and storage device 2 respectively, and the blocks of reconstructed data 2 can be stored on storage device 1 and storage device 2 respectively. If the total number of storage devices in the target storage array is 4, the blocks of reconstructed data 1 can be stored on storage device 1 and storage device 2 respectively, and the blocks of reconstructed data 2 can be stored on storage device 3 and storage device 4 respectively.

[0089] In this way, the amount of data read and written in each round is exactly equal to the target stripe size. This means that each write operation is for a complete stripe of the target storage array, requiring only the writing of new data and the calculation of new checksum data, resulting in high efficiency. Secondly, breaking down the massive migration task into multiple rounds of sub-tasks ensures that each round only consumes a small portion of I / O and computational resources. Furthermore, after completing one round, resources can be briefly returned to requests handling foreground business logic before proceeding to the next round. This avoids the performance impact of long-running, high-intensity background tasks on the foreground application, ensuring business continuity and responsiveness.

[0090] In some alternative implementations, the storage controller may also perform the following steps:

[0091] Step 1: Obtain the backup location information of the backup area in the original storage array.

[0092] Step 2: Based on the backup location information, back up the initial data corresponding to the current round to the backup area in the original storage array.

[0093] Step 3: Write the target sub-location information into non-volatile memory, and set the backup marker corresponding to the target sub-location information in the non-volatile memory as the first preset marker.

[0094] The first preset marker can be used to indicate that the backup operation of the initial data corresponding to the target sub-location information has been completed.

[0095] Specifically, during the migration process, abnormal events may occur. Therefore, to avoid the impact of such abnormal events on the accuracy and efficiency of data migration, the storage controller can directly obtain the backup location information of the backup area in the original storage array. In this way, during each round of data processing, the storage controller can determine the backup area in the original storage array based on the backup location information and back up the initial data of the current round to the original storage array, avoiding data loss due to failures during the migration process. Furthermore, the storage controller can write the target location sub-location information of the current round into non-volatile memory and set the backup flag corresponding to the target sub-location information as a first preset flag. After resolving any faults, the storage controller can determine the progress of the data migration process based on the sub-location information recorded in the non-volatile memory and the corresponding backup flag. Then, it can continue the data migration operation based on the original progress, without restarting the data migration process, resulting in higher efficiency and resource conservation.

[0096] In some alternative implementations, the storage controller may perform the following specific steps:

[0097] Step 1: After detecting the abnormal event and completing the restart, read the backup marker from non-volatile memory.

[0098] Step 2: Based on the backup marker, continue to perform the data migration operation corresponding to the backup marker.

[0099] Specifically, during the current processing round, the storage system may experience an anomaly (e.g., power outage, storage device failure, etc.). This anomaly may cause write holes, meaning that some data is written successfully while others fail to be written. After an anomaly occurs, the storage system can be restarted. After the storage controller restarts, it can first read the backup markers from non-volatile memory, and based on the different statuses of the backup markers, continue the data migration operation.

[0100] This approach eliminates the need to start the data migration process from scratch. Instead, the migration progress prior to the anomaly can be used as a foundation to continue the migration, saving resources and improving efficiency. Alternatively, in storage technology, a common method for handling anomalies involves using supercapacitors in the storage system. When an anomaly occurs, the supercapacitor is activated to provide power, allowing the storage controller to save cached data to non-volatile memory. After the storage system restarts, the data migration can continue using the data in non-volatile memory. However, this approach consumes significant hardware resources, increasing hardware design complexity and cost.

[0101] In some optional implementations, during the data migration process, the storage controller can back up the initial data of the current round to a backup area, and then read the initial data of the current round from the backup area to perform the data migration operation. Accordingly, such as... Figure 5 As shown, the storage controller can perform the following specific steps:

[0102] In scenario one, once the backup marker is determined to be the first preset marker, the initial data corresponding to the current round is directly read from the backup area. Then, based on the target storage type and total quantity, the initial data corresponding to the current round is used to obtain the final data corresponding to the current round, and the final data corresponding to the current round is migrated to the target storage array.

[0103] Specifically, once the target backup marker is determined to be the first preset marker, it indicates that the initial data of the target subtask space has been backed up to the backup area. The abnormal event may have occurred during the migration of the initial data of the target subtask space. Therefore, the storage controller can directly read the initial data of the current round from the backup area. Then, based on the target storage type and total quantity, the backup data is reconstructed to obtain the final data of the current round, which is then stored in the target storage array. Further, step five above can be followed to determine whether to proceed with the next round of processing, until all data in the target task space has been migrated to the target storage array, at which point processing stops.

[0104] In scenario two, once the backup marker is determined to be the second preset marker, the target sub-location information is read from non-volatile memory. The second preset marker indicates that the backup operation is incomplete. Based on the target sub-location information, the initial data corresponding to the current round is read from the target sub-task space and backed up to the backup area. Furthermore, based on the target storage type and total quantity, the initial data corresponding to the current round is reconstructed to obtain the final data corresponding to the current round, which is then migrated to the target storage array.

[0105] Specifically, once the backup marker is determined to be the second preset marker, it indicates that the initial data for the current round has not yet been backed up to the backup area. Therefore, the storage controller can read the target sub-location information from non-volatile memory, determine the target sub-task space based on the target sub-location information, and back up the initial data for the current round stored in the target sub-task space to the backup area. Additionally, the storage controller can use the target sub-location information as a starting point to continue the data migration operation according to steps one through five above. That is, it first continues to reconstruct the initial data corresponding to the current round based on the target storage type and total quantity to obtain the final data corresponding to the current round, and stores it in the target storage array. Then, following the processing in step five, it determines whether to continue processing for the next round until all data in the target task space has been migrated to the target storage array, at which point processing stops.

[0106] In some optional implementations, in step one above, the storage controller may specifically obtain the backup location information of the backup area in the original storage array using the following steps:

[0107] Step 1: Obtain the target end address of the original storage array.

[0108] Step 2: Determine the backup location information based on the target end address and the target stripe size.

[0109] Specifically, the length of the backup area can be equal to the target stripe size, and to minimize the impact on other storage spaces, a segment of storage space can be reserved at the end of the original storage array as a backup area. Accordingly, the storage controller can first determine the target end address of the original storage array, and then use the target end address and the target stripe size to determine the backup location information. Alternatively, the difference between the target end address and the target stripe size can be used to determine the target start address, and then the target start address and target end address can be used to determine the backup location information.

[0110] In some alternative implementations, prior to step S201, the storage controller may also perform the following steps:

[0111] Step 1: Obtain the refactoring task type.

[0112] Step 2: If the reconstruction task type is determined to include storage type change task type and storage capacity adjustment task type, obtain the number of first storage devices corresponding to the storage type change task type and the number of second storage devices corresponding to the storage capacity adjustment task type.

[0113] Step 3: Based on the number of the first storage devices and the number of the second storage devices, determine the target number of storage devices to be adjusted, which is used to indicate the adjustment of the number of storage devices in the original storage array to obtain the target storage array.

[0114] Specifically, the storage controller can first obtain the reconstruction task type, which may include one or more of the following: storage type change task type (which may be the aforementioned upward or downward migration) and storage capacity adjustment task type (which may be the aforementioned expansion or reduction). If the reconstruction task type includes both storage type and storage capacity adjustment task type, the controller can obtain the first number of storage devices corresponding to the storage type change task type and the second number of storage devices corresponding to the storage capacity adjustment task type. Then, based on the first and second storage device numbers, the target number of storage devices to be adjusted is determined. This allows technicians to adjust the number of storage devices in the original storage array based on the target number of storage devices to obtain the target storage array.

[0115] For example, the number of storage devices in the original storage array can be n, the number of first storage devices corresponding to the storage type change task type can be a, and the number of second storage devices corresponding to the storage capacity adjustment task type can be b. When the storage type change task type is upward migration and the storage capacity adjustment task type is expansion, the number of target adjustment storage devices to be added is a+b, and the total number is n+a+b. When the storage type change task type is downward migration and the storage capacity adjustment task type is expansion, if a is greater than b, then no new storage devices are needed, and the number of surplus storage devices is ab, that is, the target adjustment storage device number is determined to be ab, and the total number is n-(ab). If a equals b, then no storage devices need to be adjusted, and the total number remains n. If a is less than b, then the number of new storage devices to be added is ba, and the total number is n+ba.

[0116] refer to Figure 6 The original number of storage devices was 3, and the level was RAID0. The reconstruction task required expansion to 6 disks, and the level was RAID6. This is an upward migration and expansion. The number of new storage devices is 3, which means that there can be 2 storage devices in each stripe for additional storage of parity data P and Q, that is, a=2, b=1.

[0117] refer to Figure 7 The original storage array has 5 storage devices and is RAID5. Each stripe has one disk for storing parity data P. The required migration level is RAID0. RAID0 does not require additional storage for parity data. Each stripe has one storage device that can be expanded. This is a downward migration and expansion, i.e., a=1, b=1, and no new storage devices are needed.

[0118] In the above embodiments, the relationship between the effective storage space of the original storage array, the target task space, the backup area, and the available storage space of the target storage array can be as follows: Figure 8 As shown. Available space can contain the target task space, which in turn contains effective storage space. To avoid the backup area affecting data in other areas, the backup area can be placed at the end of the storage array.

[0119] refer to Figure 9 The top image shows the original storage array, and the bottom image shows the target storage array. The black box represents the target task space, and the light gray box represents the backup area. The initial state of both HTLM and LTLM is the starting address of the target task space, i.e., the second starting address.

[0120] refer to Figure 10 The top image shows the original storage array, and the bottom image shows the target storage array. The dark gray box represents the target subtask space corresponding to the first round, the black box represents the task space within the target task space where data has not yet been migrated, and the light gray box of the original storage array represents the backup area. The storage controller first reads the initial data of the first round from the target subtask space and backs it up to the backup area. Then, it reads the initial data of the first round from the backup area into the storage controller's own cache. Based on the target storage type and total quantity, the initial data of the first round is reconstructed (for example, when the reconstruction process involves generating verification data, verification data is calculated through XOR operation). After this reconstruction, the final data of the first round is obtained and migrated to the light gray box of the target storage array.

[0121] refer to Figure 11 The top image shows the original storage array, and the bottom image shows the target storage array. The double-slash-filled boxes represent sub-task spaces that have completed migration. The dark gray boxes represent target sub-task spaces that are not in the first round, and the black boxes represent task spaces within the target task space that have not yet had their data migrated. The light gray box representing the original storage array represents the backup area. The storage controller first reads the initial data from the target sub-task space for the non-first round and backs it up to the backup area. Then, it reads the initial data from the backup area for the non-first round into its own cache. Based on the target storage type and total quantity, the initial data for the non-first round is reconstructed to obtain the final data for the non-first round, which is then migrated to the light gray box of the target storage array.

[0122] like Figure 12As shown, the top is the original storage array, a 2-disk RAID 0 array with a total of m stripes. The right side is the target storage array, a 6-disk RAID 5 array. The data in the gray box is valid data, i.e., the effective storage space occupied by data 1 to data 32. After completing the upward migration and expansion, data 1, originally distributed on storage device 1 of stripe 2 in the original storage array, will be migrated to storage device 5 of stripe 0 in the target storage array. Data 32, originally distributed on storage device 2 of stripe 17 in the original storage array, will be migrated to storage device 1 of stripe 7 in the target storage array. The data in the white box is invalid data. Among them, 1'-4' and 5'-8' are invalid data added to meet the requirements of full stripe migration, i.e., the storage space occupied by stripes 0 to 19 is the storage space corresponding to the target location information.

[0123] In some alternative implementations, the aforementioned start and end addresses can be represented by logical block addresses (LBAs), and the space length can be represented by the number of logical blocks (NLBs).

[0124] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods according to the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method.

[0125] Embodiments of this application also provide a data migration apparatus, such as... Figure 13 As shown, it includes:

[0126] The acquisition module 1310 is used to acquire the configuration information of the target storage array and the initial location information of the effective storage space storing valid data in the original storage array. The configuration information includes the target storage type, the total number of storage devices in the target storage array, and the target stripe size corresponding to the target storage type. The initial location information includes the first starting address and the first space length.

[0127] The determination module 1320 is used to determine the target location information based on the first starting address, the first spatial length, and the target strip size;

[0128] The migration module 1330 is used to migrate data in the target task space corresponding to the target location information in the original storage array to the target storage array based on the target location information, the target stripe size, and the target storage type. The length of the target task space is an integer multiple of the target stripe size.

[0129] In some alternative implementations, the determining module 1320 is specifically used for:

[0130] Determine the second starting address based on the first starting address and the target strip size;

[0131] The second space length is determined based on the first starting address, the second starting address, the first space length, and the target stripe size;

[0132] The target location information is determined based on the second starting address and the second spatial length.

[0133] In some alternative implementations, the determining module 1320 is specifically used for:

[0134] Determine the stripe index based on the first starting address and the target stripe size;

[0135] Determine the second starting address based on the stripe index and the target stripe size.

[0136] In some alternative implementations, the determining module 1320 is specifically used for:

[0137] The number of stripes is determined based on the first starting address, the second starting address, the first space length, and the target strip size;

[0138] The length of the second space is determined based on the number of stripes and the size of the target stripe.

[0139] In some alternative implementations, the determining module 1320 is specifically used for:

[0140] The length of the third space is determined based on the first starting address, the second starting address, and the first space length;

[0141] The number of stripes is determined based on the length of the third space and the size of the target strip.

[0142] In some alternative implementations, the second starting address is determined based on the first starting address and the target stripe size, using the following expression:

[0143] Where slba' is the second starting address, slab is the first starting address, and x is the target strip size.

[0144] In some optional implementations, the second space length is determined based on the first starting address, the second starting address, the first space length, and the target stripe size, using the following expression:

[0145]

[0146] Where nlb' is the second space length, slab is the first starting address, slba' is the second starting address, nlb is the first space length, and x is the target strip size.

[0147] In some alternative implementations, the migration module 1330 is specifically used for:

[0148] In the current round, based on the target strip size, obtain the target sub-position information corresponding to the current round;

[0149] Based on the target sub-location information, read the initial data corresponding to the current round from the target sub-task space corresponding to the target sub-location information in the original storage array;

[0150] Based on the target storage type and total quantity, the initial data corresponding to the current round is reconstructed to obtain the final data corresponding to the current round;

[0151] The final data corresponding to the current round is stored in the target storage array;

[0152] Based on the target location information and target sub-location information, determine whether the current round is the last round;

[0153] If it is determined that the current round is the last round, then all data in the target task space will be migrated to the target storage array, and processing will be stopped.

[0154] Alternatively, if it is determined that the current round is not the last round, proceed to the next round of processing until all data in the target task space has been migrated to the target storage array, and then stop processing.

[0155] In some alternative implementations, the migration module 1330 is specifically used for:

[0156] If the current round is the first round, determine the first end address based on the second starting address and the target strip size;

[0157] The second starting address and the first ending address are determined as the target sub-location information.

[0158] In some alternative implementations, the migration module 1330 is specifically used for:

[0159] If the current round is not the first round, the end address included in the sub-position information corresponding to the previous round corresponding to the current round will be determined as the third starting address;

[0160] The second end address is determined based on the third starting address and the target strip size;

[0161] The third starting address and the second ending address are determined as the target sub-location information.

[0162] In some alternative embodiments, the device further includes a backup module 1340 for:

[0163] Obtain backup location information for the backup area in the original storage array;

[0164] Based on the backup location information, the initial data corresponding to the current round is backed up to the backup area;

[0165] Write the target sub-location information into non-volatile memory;

[0166] Furthermore, after completing the backup operation of the initial data corresponding to the current round, the backup flag corresponding to the target sub-location information in the non-volatile memory is set to the first preset flag, wherein the first preset flag is used to indicate that the backup operation of the initial data corresponding to the target sub-location information is completed.

[0167] In some alternative implementations, the backup module 1340 is further configured to:

[0168] After detecting an abnormal event and completing the restart, the backup marker is read from non-volatile memory;

[0169] Based on the backup marker, continue to perform the data migration operation corresponding to the backup marker.

[0170] In some alternative implementations, the backup module 1340 is specifically used for:

[0171] Once the backup marker is determined to be the first preset marker, the initial data corresponding to the current round is directly read from the backup area. Then, based on the target storage type and total quantity, the initial data corresponding to the current round is used to obtain the final data corresponding to the current round. Finally, the final data corresponding to the current round is migrated to the target storage array.

[0172] In some alternative implementations, the backup module 1340 is specifically used for:

[0173] Once the backup marker is determined to be the second preset marker, the target sub-location information is read from non-volatile memory, wherein the second preset marker is used to indicate that the backup operation is not completed;

[0174] Based on the target sub-location information, the initial data corresponding to the current round is read from the target sub-task space and backed up to the backup area; and, based on the target storage type and total quantity, the initial data corresponding to the current round is reconstructed to obtain the final data corresponding to the current round and migrated to the target storage array.

[0175] In some alternative implementations, the backup module 1340 is specifically used for:

[0176] Obtain the target end address of the original storage array;

[0177] Determine the backup location information based on the target end address and the target stripe size.

[0178] In some alternative implementations, the acquisition module 1310 is further configured to:

[0179] Get the refactoring task type;

[0180] If the refactoring task type is determined to include storage type change task type and storage capacity adjustment task type, obtain the number of first storage devices corresponding to storage type change task type and the number of second storage devices corresponding to storage capacity adjustment task type.

[0181] Based on the first number of storage devices and the second number of storage devices, a target number of storage devices is determined to indicate the adjustment of the number of storage devices in the original storage array, thereby obtaining the target storage array.

[0182] For a description of the features in the embodiments corresponding to the data migration device, please refer to the relevant descriptions in the embodiments corresponding to the data migration method, which will not be repeated here.

[0183] Embodiments of this application also provide an electronic device, such as... Figure 14 As shown, it includes a memory 10 and a processor 20. The memory 10 stores a computer program, and the processor 20 is configured to run the computer program to perform the steps in any of the above-described data migration method embodiments.

[0184] Embodiments of this application also provide a computer-readable storage medium storing a computer program configured to execute the steps in any of the above-described data migration method embodiments at runtime.

[0185] In one exemplary embodiment, the aforementioned computer-readable storage medium may include, but is not limited to, various media capable of storing computer programs, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard disk, magnetic disk, or optical disk.

[0186] Embodiments of this application also provide a computer program product, which includes a computer program that, when executed by a processor, implements the steps in any of the above-described data migration method embodiments.

[0187] Embodiments of this application also provide another computer program product, including a non-volatile computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps in any of the above-described data migration method embodiments.

[0188] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0189] The data migration method, apparatus, electronic device, storage medium, and program product provided in this application have been described in detail above. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the embodiments above are merely for the purpose of helping to understand the method and its core ideas. It should be noted that those skilled in the art can make various improvements and modifications to this application without departing from its principles, and these improvements and modifications also fall within the protection scope of the claims of this application.

Claims

1. A data migration method, characterized in that, include: Obtain the configuration information of the target storage array corresponding to the reconstruction task type and the initial location information of the effective storage space storing valid data in the original storage array. The configuration information includes the target storage type, the total number of storage devices in the target storage array, and the target stripe size corresponding to the target storage type. The initial location information includes a first starting address and a first space length. The reconstruction task type includes storage type change task type and storage capacity adjustment task type. Based on the first starting address and the target stripe size, a second starting address is determined, wherein the second starting address is calculated using the following expression: Wherein, slba' is the second starting address, slab is the first starting address, and x is the target stripe size; The second space length is determined based on the first starting address, the second starting address, the first space length, and the target stripe size, wherein the second space length is calculated using the following expression: Where nlb' is the second space length and nlb is the first space length; The target location information is determined based on the second starting address and the second spatial length; Based on the target location information, the target storage type, the total quantity, and the target stripe size, the data in the target task space corresponding to the target location information in the original storage array is migrated to the target storage array, wherein the space length of the target task space is an integer multiple of the target stripe size.

2. The data migration method according to claim 1, characterized in that, The step of migrating data from the target task space corresponding to the target location information in the original storage array to the target storage array based on the target location information, the target storage type, the total quantity, and the target stripe size includes: In the current round, based on the target strip size, obtain the target sub-position information corresponding to the current round; Based on the target sub-location information, read the initial data corresponding to the current round from the target sub-task space corresponding to the target sub-location information in the original storage array; Based on the target storage type and the total quantity, the initial data corresponding to the current round is reconstructed to obtain the final data corresponding to the current round; The final data corresponding to the current round is stored in the target storage array; Based on the target location information and the target sub-location information, determine whether the current round is the last round; If it is determined that the current round is the last round, then it is determined that all data in the target task space will be migrated to the target storage array, and processing will be stopped. Alternatively, if it is determined that the current round is not the last round, the next round of processing can begin, and processing can continue until all data in the target task space has been migrated to the target storage array, at which point processing can stop.

3. The data migration method according to claim 2, characterized in that, The step of obtaining target sub-position information corresponding to the current round based on the target strip size includes: If the current round is the first round, the first end address is determined based on the second start address and the target strip size; The second starting address and the first ending address are determined as the target sub-location information.

4. The data migration method according to claim 3, characterized in that, The step of obtaining target sub-position information corresponding to the current round based on the target strip size includes: If the current round is not the first round, the end address included in the sub-position information corresponding to the previous round corresponding to the current round will be determined as the third starting address; The second end address is determined based on the third starting address and the target strip size; The third starting address and the second ending address are determined as the target sub-location information.

5. The data migration method according to claim 2, characterized in that, The method further includes: Obtain the backup location information of the backup area in the original storage array; Based on the backup location information, the initial data corresponding to the current round is backed up to the backup area; Write the target sub-location information into non-volatile memory; Furthermore, after completing the backup operation of the initial data corresponding to the current round, the backup flag in the non-volatile memory corresponding to the target sub-location information is set as a first preset flag, wherein the first preset flag is used to indicate that the backup operation of the initial data corresponding to the target sub-location information is completed.

6. The data migration method according to claim 5, characterized in that, The method further includes: After detecting an abnormal event and completing a restart, the backup marker is read from the non-volatile memory; Based on the backup marker, continue to perform the data migration operation corresponding to the backup marker.

7. The data migration method according to claim 6, characterized in that, The step of continuing to perform the data migration operation corresponding to the backup marker based on the backup marker includes: Once the backup marker is determined to be the first preset marker, the initial data corresponding to the current round is directly read from the backup area. Then, based on the target storage type and the total quantity, the final data corresponding to the current round is obtained from the initial data corresponding to the current round, and the final data corresponding to the current round is migrated to the target storage array.

8. The data migration method according to claim 7, characterized in that, The method further includes: Once the backup marker is determined to be the second preset marker, the target sub-location information is read from the non-volatile memory, wherein the second preset marker is used to indicate that the backup operation is not completed; Based on the target sub-location information, the initial data corresponding to the current round is read from the target sub-task space and backed up to the backup area; and, based on the target storage type and the total quantity, the initial data corresponding to the current round is reconstructed to obtain the final data corresponding to the current round and migrated to the target storage array.

9. The data migration method according to any one of claims 5 to 8, characterized in that, The step of obtaining the backup location information of the backup area in the original storage array includes: Obtain the target end address of the original storage array; The backup location information is determined based on the target end address and the target stripe size.

10. The data migration method according to claim 1, characterized in that, Before obtaining the configuration information of the target storage array corresponding to the reconstruction task type and the initial location information of the effective storage space storing valid data in the original storage array, the method further includes: Get the refactoring task type; If the reconstruction task type is determined to include storage type change task type and storage capacity adjustment task type, the number of first storage devices corresponding to the storage type change task type and the number of second storage devices corresponding to the storage capacity adjustment task type are obtained. Based on the number of the first storage devices and the number of the second storage devices, a target number of storage devices is determined to indicate the adjustment of the number of storage devices in the original storage array, thereby obtaining the target storage array.

11. A data migration device, characterized in that, include: The acquisition module is used to obtain the configuration information of the target storage array corresponding to the reconstruction task type and the initial location information of the effective storage space storing valid data in the original storage array. The configuration information includes the target storage type, the total number of storage devices in the target storage array, and the target stripe size corresponding to the target storage type. The initial location information includes a first starting address and a first space length. The reconstruction task type includes storage type change task type and storage capacity adjustment task type. The determining module is configured to determine a second starting address based on the first starting address and the target stripe size, wherein the second starting address is calculated using the following expression: Wherein, slba' is the second starting address, slab is the first starting address, and x is the target stripe size; The second space length is determined based on the first starting address, the second starting address, the first space length, and the target stripe size, wherein the second space length is calculated using the following expression: Where nlb' is the second space length and nlb is the first space length; The target location information is determined based on the second starting address and the second spatial length; The migration module is used to migrate data in the target task space corresponding to the target location information in the original storage array to the target storage array based on the target location information, the target storage type, the total quantity, and the target stripe size, wherein the space length of the target task space is an integer multiple of the target stripe size.

12. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor for executing the computer program to implement the steps of the data migration method as described in any one of claims 1 to 10.

13. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, wherein the computer program, when executed by a processor, implements the steps of the data migration method as described in any one of claims 1 to 10.

14. A computer program product, characterized in that, The computer program product includes a computer program, wherein the computer program, when executed by a processor, implements the steps of the data migration method as described in any one of claims 1 to 10.

Images