Data reconstruction method and device fusing data migration and electronic equipment

Abstract

The invention discloses a data reconstruction method and device fusing data migration and electronic equipment. If the disk reconstruction task is detected, cancelling the PDM task; a disk indicated by the disk reconstruction task is a first target disk, a disk indicated by the PDM task is a second target disk, and the first target disk and the second target disk are one of N in-service disks; according to the disk description information of the first hot spare disk and the disk description information of the second hot spare disk, the first data strip set is updated, and a second data strip set is obtained; the first data stripe set is generated according to respective disk description information of the N in-service disks; aiming at each data strip in the second data strip set, determining an operation type of each data unit indicated by the data strip; and according to the operation type of each data unit indicated by the data strip, processing the data stored in the corresponding data unit to complete the reconstruction of the first target disk and the migration of the second target disk. The reconstruction and migration synchronization is realized, and the task execution efficiency is improved.

Description

Technical Field

[0001] The present application relates to the technical field of data reconstruction, and in particular to a data reconstruction method, device and electronic device integrating data migration. Background Art

[0002] As the characteristics of storage systems gradually develop, existing storage systems not only need to deal with data imbalance, but also must cope with possible data loss events. In order to avoid data migration tasks and data reconstruction tasks operating on the same data stripe (dseg) at the same time, thereby wasting read and write bandwidth, it is necessary to ensure that they can be executed mutually exclusive.

[0003] In the related art, a mutually exclusive scheme of serial execution is usually used to determine that data reconstruction and data migration are mutually exclusive. Specifically: before creating a data migration task, the system will first check whether there is a reconstruction task in progress. If so, the data migration task will not be created to prevent unnecessary waste of resources. When the data migration task is being executed, if a reconstruction event is detected, the system will immediately suspend the current migration task and start the reconstruction task. After the reconstruction task is completed, the system will resume and continue to execute the previously suspended migration task to ensure the integrity of the data and the stability of the storage system.

[0004] Although this serial execution mutual exclusion strategy can effectively avoid competition, optimize the utilization of system resources, and ensure the effectiveness and efficiency of storage operations, if there are disk-based data reconstruction and data migration tasks at the same time, the serial execution mutual exclusion solution has a cumbersome processing flow. When the scenarios to be considered are more complex, the execution efficiency of the tasks is significantly reduced. Summary of the invention

[0005] In an exemplary embodiment of the present application, a data reconstruction method, device and electronic device integrating data migration are provided to realize simultaneous execution of data migration tasks and data reconstruction tasks, simplify the processing flow and improve the execution efficiency of tasks.

[0006] According to a first aspect of an exemplary embodiment, a data reconstruction method for fusion data migration is provided, comprising:

[0007] In a redundant disk array consisting of N in-service disks, if a disk reconstruction task is detected, the currently existing disk pre-failure data migration PDM task is canceled; wherein the disk indicated by the disk reconstruction task is a first target disk; the disk indicated by the PDM task is a second target disk, and the first target disk and the second target disk are one of the N in-service disks; N is greater than 2;

[0008] According to the disk description information of the first hot spare disk and the second hot spare disk, the first data stripe set is updated to obtain the second data stripe set; wherein the first data stripe set is generated according to the disk description information of each of the N active disks;

[0009] For each data stripe in the second data stripe set, determining an operation type for each data unit indicated by the data stripe;

[0010] According to the operation type of each data unit indicated by the data stripe, the data stored in the corresponding data unit is processed to complete the data reconstruction of the first target disk and the data migration of the second target disk.

[0011] The embodiment of the present application utilizes the idea of ​​mutual exclusion between PDM tasks and data reconstruction tasks in the prior art, and cancels the currently existing PDM tasks when executing the reconstruction tasks. In addition, the data stripe set is updated through the disk description information of the hot spare disk, and then the data stored in the corresponding data unit is processed according to the operation type of each data unit indicated by each data stripe in the updated data stripe set. Data processing is performed at the data unit granularity, and the canceled PDM tasks are integrated into the data reconstruction process, processing data migration and data reconstruction tasks at one time. The task execution efficiency is improved.

[0012] In an optional embodiment, the method further includes:

[0013] Randomly select a data unit from each of the N active disks, and determine that description information corresponding to each of the selected data units constitutes a data stripe information;

[0014] Among them, a data stripe information includes:

[0015] The identifier of the data stripe, the identifier of the disk associated with the data stripe, the identifier of the data units included in the data stripe, and the storage location indicated by the data units included in the data stripe.

[0016] In the above embodiment, by randomly selecting the data stripe information composed of the descriptive information corresponding to the data units of each disk among N active disks, the identifier of the data stripe, the disk identifier associated with the data stripe, the identifier of the data units included in the data stripe, and the storage location indicated by the data units included in the data stripe can be determined, and then this information can be used to accurately perform data migration and data reconstruction.

[0017] In an optional implementation, updating the first data stripe set according to the disk description information of the first hot spare disk and the second hot spare disk to obtain the second data stripe set includes:

[0018] For each first data stripe in the first data stripe set, a data unit of a first hot spare disk and a data unit of a second hot spare disk selected randomly are applied to update the first data stripe to obtain a corresponding second data stripe;

[0019] The updated second data stripes constitute a second data stripe set.

[0020] In the above embodiment, when the data stripe set is updated, the predetermined data unit of the hot spare disk is applied to ensure that the target disk corresponding to the data migration and data reconstruction and the data unit used for data migration or data reconstruction in the target disk can be accurately determined through the updated data stripe set.

[0021] In an optional implementation, applying a data unit of a first hot spare disk and a data unit of a second hot spare disk selected randomly to update the first data stripe includes:

[0022] The storage position of the data unit of the first target disk in the first data stripe information is replaced with the storage position of the data unit of the first hot spare disk; the storage position of the data unit of the second target disk in the first data stripe information is replaced with the storage position of the data unit of the second hot spare disk.

[0023] In the above embodiment, through the replacement process, the data to be processed can be accurately determined through the storage position of the data unit of the corresponding hot spare disk.

[0024] In an optional implementation, the operation type includes part or all of copy, EC encoding, inheritance, and EC decoding; for each data stripe in the second data stripe set, determining the operation type for each data unit indicated by the data stripe, the operation for each data unit includes:

[0025] If the storage content indicated by the data unit is not data, determining that the operation type of the data unit is EC coding;

[0026] If the storage content indicated by the data unit is data, and the flag bit value of the data unit is not the first set value, determining that the operation type of the data unit is inheritance;

[0027] If the storage content indicated by the data unit is data, and the flag bit value of the data unit is the first set value, and the indicated disk state is not PDM, then determining that the operation type of the data unit is EC decoding;

[0028] If the storage content indicated by the data unit is data, the flag value of the data unit is the first set value, and the indicated disk state is the PDM state, then the operation type of the data unit is determined to be copy.

[0029] In the above embodiment, EC encoding, inheritance and EC decoding correspond to operations in the data reconstruction process, and copying corresponds to operations in the data migration process. Therefore, for different data units, the operation types are determined according to the above logic. With the data unit as the granularity, data migration is integrated into the data reconstruction process to achieve the synchronization of data migration and data reconstruction.

[0030] In an optional embodiment, the method further includes:

[0031] For each active disk, determine the disk status by querying the information on the dashboard:

[0032] If the state of the disk is normal, determining that the flag bit value corresponding to the data unit of the disk in the data stripe information is a second set value; wherein the normal state indicates that the disk is in an inserted state and is operating normally;

[0033] If the state of the disk is an isolated state, determining that the flag value corresponding to the data unit of the disk in the data stripe information is a first set value; wherein the isolated state indicates that the disk is in an unplugged state;

[0034] If the disk is in a sub-healthy state, it is determined that the flag value corresponding to the data unit of the disk in the data stripe information is a first set value; wherein the sub-healthy state indicates that the disk is about to be unplugged.

[0035] In the above embodiment, generally, disk removal is a common operation that triggers a data reconstruction task. Therefore, for each in-service disk, the state of the disk can be determined by the above method, and then the operation type of the corresponding data unit can be determined.

[0036] According to a second aspect of an exemplary embodiment, a data reconstruction device for integrating data migration is provided, comprising:

[0037] The processing unit is used to: in a redundant disk array composed of N in-service disks, if a disk reconstruction task is detected, cancel the currently existing disk pre-failure data migration PDM task; wherein the disk indicated by the disk reconstruction task is a first target disk, the disk indicated by the PDM task is a second target disk, and the first target disk and the second target disk are one of the N in-service disks; N is greater than 2;

[0038] An updating unit, configured to update the first data stripe set according to the disk description information of the first hot spare disk and the second hot spare disk, to obtain the second data stripe set; wherein the first data stripe set is generated according to the disk description information of each of the N active disks;

[0039] The processing unit is further configured to: determine, for each data stripe in the second data stripe set, an operation type for each data unit indicated by the data stripe;

[0040] The processing unit is further used to: process the data stored in the corresponding data unit according to the operation type of each data unit indicated by the data stripe, and complete the data reconstruction of the first target disk and the data migration of the second target disk.

[0041] According to a third aspect of an exemplary implementation, an electronic device is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of any of the above methods when executing the computer program.

[0042] According to a fourth aspect of the exemplary implementation, a computer-readable storage medium is provided, on which computer program instructions are stored, and when the computer program instructions are executed by a processor, the steps of any of the above methods are implemented.

[0043] According to a fifth aspect of the exemplary implementation, a computer program product is provided, comprising a computer program, wherein when the computer program is executed by a processor, the steps of any method provided in the first aspect of the present application are implemented. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0045] Figure 1 An exemplary timing diagram of a mutually exclusive data reconstruction task and a data migration task provided in an embodiment of the present application is shown;

[0046] Figure 2 A flowchart of a data reconstruction method for fusion data migration provided by an embodiment of the present application is exemplarily shown;

[0047] Figure 3 A flowchart of a data reconstruction task integrating a data migration task provided in an embodiment of the present application is exemplarily shown;

[0048] Figure 4 The following is a schematic diagram showing the structure of a data reconstruction device for integrating data migration provided in an embodiment of the present application;

[0049] Figure 5 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application is exemplarily shown. DETAILED DESCRIPTION

[0050] In order to make the purpose, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application.

[0051] For ease of understanding, the terms involved in the embodiments of the present application are explained below:

[0052] (1) Data migration: Migrate data from one disk to another.

[0053] (2) Data reconstruction: When a disk is unplugged and its data is lost, the data on the disk is restored through the data stored in other disks. That is, the disk unplugging operation can trigger data reconstruction, and other operations can also trigger data reconstruction. In the embodiment of the present application, the disk unplugging operation is used as an example to trigger data reconstruction.

[0054] (3) Disk pre-failure data migration (PDM): When a hard disk is about to fail, the data in the failed disk is migrated out in advance.

[0055] (3) Redundant Arrays of Independent Disks (RAID): An array with redundancy composed of several independent disks. A disk array is composed of many independent disks, which are combined into a disk group with huge capacity. The performance of the entire disk system is improved by the additive effect of data provided by individual disks. Using this technology, data is divided into many segments and stored on each hard disk. The disk array can also use the concept of parity check. When any hard disk in the array fails, the data can still be read. When reconstructing data, the data can be recalculated and placed in the new hard disk. RAID redundant disk arrays are divided into different levels to meet the needs of different data applications.

[0056] (5)EC (Encoding): A coding and decoding algorithm used for data recovery.

[0057] (6) dObj: The non-volatile memory express (NVMe) disk storage space is divided into fixed sizes, and each divided space is a hard disk object, abbreviated as dObj.

[0058] (7) vObj: The logical data unit corresponding to dobj, with independent metadata and access control policies, providing a higher level of abstraction to simplify data management and access.

[0059] (8) dseg: A logical RAID data stripe with data redundancy capability formed in units of dObj.

[0060] (9)dseg width: the number of columns in the RAID data stripe.

[0061] (10)dsegInfo: dseg information, including dsegid, associated vObjId, disk id, associated dObj, etc.

[0062] (11) Kanban: A public structure used to query the disk status.

[0063] (12) Data reconstruction: After unplugging the disk, the system will use the data and redundant information on other disks to reconstruct the lost data.

[0064] PDM refers to the automatic migration of data to other healthy storage devices when a hard disk or storage device is detected to be about to fail. This migration is usually performed when the hard disk shows signs of abnormality to avoid data loss. This technology is a migration task based on the disk dimension, and more complex scenarios need to be considered. If the PDM disk is the same as the rebuilt disk, the current PDM task needs to be canceled; if multiple disks are undergoing PDM at the same time, the system needs to suspend all PDM tasks and cancel the tasks related to pulling out the disk. After the reconstruction task is completed, all suspended PDM tasks will continue to execute.

[0065] Therefore, when there are reconstruction and migration tasks based on the disk dimension at the same time, the mutually exclusive solution of serial execution is adopted, and the processing flow is more cumbersome. When the scenarios to be considered are more complex, the execution efficiency of the tasks is significantly reduced.

[0066] To this end, an embodiment of the present application provides a data reconstruction method that integrates data migration. The method is based on the characteristics of disk tasks and adopts data reconstruction and data merging methods to move the data of pre-failed disks while executing tasks, thereby achieving mutual exclusivity of disk-dimensional data migration and data reconstruction tasks, improving task execution efficiency, and simplifying the processing flow.

[0067] After introducing the design ideas of the embodiments of the present application, the following briefly introduces the application scenarios to which the technical solutions of the embodiments of the present application can be applied. It should be noted that the application scenarios introduced below are only used to illustrate the embodiments of the present application and are not limited. In specific implementation, the technical solutions provided by the embodiments of the present application can be flexibly applied according to actual needs.

[0068] refer to Figure 1, shows a sequence diagram of mutually exclusive data reconstruction tasks and data migration tasks. The mutually exclusive process of reconstruction tasks and PDM is as follows: when the reconstruction task is ready to be created, check whether there is a PDM task. If so, cancel the PDM task; when the PDM task is ready to be created, check whether there is a disk unplug event. If so, do not create the PDM task. After the reconstruction task is successfully created, it is executed normally until the reconstruction task is completed.

[0069] To further illustrate the technical solution provided by the embodiment of the present application, this is described in detail below in conjunction with the accompanying drawings and specific implementation methods. Although the embodiment of the present application provides the method operation steps shown in the following embodiments or drawings, more or fewer operation steps may be included in the method based on routine or no creative labor. In the steps where there is no necessary causal relationship logically, the execution order of these steps is not limited to the execution order provided in the embodiment of the present application.

[0070] Combine the following Figure 1 The application scenario shown is shown in Figure 2 A flow chart of a data reconstruction method for integrating data migration is shown, and the technical solution provided in the embodiment of the present application is explained by taking the reconstruction task triggered by disk removal as an example.

[0071] S201: In a redundant disk array composed of N in-service disks, if a disk reconstruction task is detected, the currently existing disk pre-failure data migration PDM task is canceled.

[0072] S202: Update the first data stripe set according to the disk description information of the first hot spare disk and the second hot spare disk to obtain the second data stripe set.

[0073] The first data stripe set is generated according to the disk description information of each of the N in-service disks.

[0074] S203: For each data stripe in the second data stripe set, determine an operation type for each data unit indicated by the data stripe.

[0075] S204: Process the data stored in the corresponding data unit according to the operation type of each data unit indicated by the data stripe, and complete the data reconstruction of the first target disk and the data migration of the second target disk.

[0076] The embodiment of the present application utilizes the idea of ​​mutual exclusion between PDM tasks and data reconstruction tasks in the prior art, and cancels the currently existing PDM tasks when executing the reconstruction tasks. In addition, the data stripe set is updated through the disk description information of the hot spare disk, and then the data stored in the corresponding data unit is processed according to the operation type of each data unit indicated by each data stripe in the updated data stripe set. Data processing is performed at the data unit granularity, and the canceled PDM tasks are integrated into the data reconstruction process, processing data migration and data reconstruction tasks at one time. The task execution efficiency is improved.

[0077] In relation to S201, an active disk refers to a disk in use, and N active disks constitute a redundant disk array. The first target disk and the second target disk are one of the N active disks, and the disk indicated by the PDM task is the second target disk, that is, the second target disk is a disk that is about to fail and needs to perform a data migration task. In a redundant disk array, if it is detected that the first target disk is unplugged, it means that the first target disk needs to perform a data reconstruction task, and the current PDM task is canceled at this time.

[0078] Usually, N is greater than 2. In a specific example, N can be 7, which are respectively called disk No. 1, disk No. 2, disk No. 3, disk No. 4, disk No. 5, disk No. 6 and disk No. 7. The first target disk is disk No. 1, and the second target disk is disk No. 2.

[0079] In a specific example, disk No. 1 has just been unplugged and a data reconstruction task needs to be performed, so the PDM task of disk No. 2 is canceled at this time.

[0080] In S202, the hot spare disk is a disk in a hot spare state, which can be used as a destination disk for data migration tasks and data reconstruction tasks. For example, the first hot spare disk can be disk No. 8, and the second hot spare disk can be disk No. 9.

[0081] In the embodiment of the present application, a first data stripe set may be generated according to the disk description information of each of the N active disks, and then the first data stripe set may be updated according to the disk description information of each of the first hot spare disk and the second hot spare disk to obtain a second data stripe set.

[0082] The first data stripe set includes multiple first data stripes, and each first data stripe information is generated in the following manner: randomly selecting a data unit from each of N active disks, and determining that the description information corresponding to each of the selected data units constitutes a data stripe information.

[0083] Optionally, each first data stripe information includes an identifier of the data stripe, an identifier of a disk associated with the data stripe, an identifier of a data unit included in the data stripe, and a storage location indicated by the data unit included in the data stripe.

[0084] For example, the data units of disk No. 1 are a1, b1, c1, e1, e1, and f1 respectively; the data units of disk No. 2 are a2, b2, c2, e2, e2, and f2 respectively; the data units of disk No. 3 are a3, b3, c3, e3, e3, and f3 respectively; the data units of disk No. 4 are a4, b4, c4, e4, e4, and f4 respectively; the data units of disk No. 5 are a5, b5, c5, e5, e5, and f5 respectively; the data units of disk No. 6 are a6, b6, c6, e6, e6, and f6 respectively; the data units of disk No. 7 are a7, b7, c7, e7, e7, and f7 respectively. a1, a2, a3, a4, a5, a6, and a7 all perform the first processing on the data, and b1, b2, b3, b4, b5, b6, and b7 all perform the second processing on the data, and so on. That is, for each disk, the function of each data unit thereof is the same as that of the data units at the corresponding positions of other disks.

[0085] The seven first data stripes included in the first data stripe set may be desg1, desg2, desg3, desg4, desg5, desg6, and desg7, respectively. For example, desg1 is composed of a1, a2, a3, a4, a5, a6, and a7, desg2 is composed of b1, b2, b3, b4, b5, b6, and b7, desg3 is composed of c1, c2, c3, c4, c5, c6, and c7, desg4 is composed of e1, e2, e3, e4, e5, e6, and e7, desg5 is composed of e1, e2, e3, e4, e5, e6, and e7, desg6 is composed of f1, f2, f3, f4, f5, f6, and f7, and desg7 is composed of g1, g2, g3, g4, g5, g6, and g7. In actual application, data units of each disk may be randomly selected to form a data stripe. This is only an example and does not constitute a specific limitation.

[0086] In this way, for example, in the data stripe information of desg1, the data stripe is identified as desg1; the associated disks are identified as disk 1, disk 2, disk 3, disk 4, disk 5, disk 6, and disk 7; the data units included in the data stripe are identified as a1, a2, a3, a4, a5, a6, and a7; and the indicated storage locations of the data units included in the data stripe correspond one-to-one to the identifiers of the data units.

[0087] In addition, according to the disk description information of the first hot spare disk and the second hot spare disk, the process of updating the first data stripe set to obtain the second data stripe set can be implemented through steps A1-A2:

[0088] A1: For each first data stripe in the first data stripe set, a data unit of a first hot spare disk and a data unit of a second hot spare disk selected randomly are applied to update the first data stripe to obtain a corresponding second data stripe.

[0089] Specifically, the storage position of the data unit of the first target disk in the first data stripe information is replaced with the storage position of the data unit of the first hot spare disk; the storage position of the data unit of the second target disk in the first data stripe information is replaced with the storage position of the data unit of the second hot spare disk.

[0090] For example, disk 1 has just been unplugged, and a PDM task exists for disk 2. desg1' is composed of a8, a9, a3, a4, a5, a6, and a7; desg2' is composed of b8, b9, b3, b4, b5, b6, and b7; desg3' is composed of c8, c9, c3, c4, c5, c6, and c7; desg4' is composed of e8, e9, e3, e4, e5, e6, and e7; desg5' is composed of e8, e9, e3, e4, e5, e6, and e7; desg6' is composed of f8, f9, f3, f4, f5, f6, and f7; and desg7' is composed of g8, g9, g3, g4, g5, g6, and g7.

[0091] A2: Each updated second data stripe constitutes a second data stripe set

[0092] Among them, desg1', desg2', desg3', desg4', desg5', desg6', and desg7' constitute a second data stripe set.

[0093] In S203, for each data stripe in the second data stripe set, an operation type for each data unit indicated by the data stripe is determined, wherein the operation type includes part or all of copy, EC encoding, inheritance, and EC decoding. The process includes at least the following four cases:

[0094] In a data stripe, each data unit corresponds to one bit, and the number of bits is the width of the data stripe.

[0095] Case 1: If the storage content indicated by the data unit is not data, it is determined that the operation type of the data unit is EC encoding.

[0096] In this case, the storage content indicated by the data unit is not data, which means that the processing performed on the data unit at this time is data reconstruction. Therefore, it is determined that the operation type of the data unit is EC encoding.

[0097] Case 2: If the storage content indicated by the data unit is data, and the flag value of the data unit is not the first set value (for example, it may be 1), then the operation type of the data unit is determined to be inheritance.

[0098] In this case, the first setting value indicates that the current disk state is abnormal, that is, isolated (disk has been unplugged) or sub-healthy (disk is about to be unplugged). At this time, it is explained that the processing performed on the data unit is data reconstruction, so the operation type of the data unit is determined to be inheritance.

[0099] Case 3: If the storage content indicated by the data unit is data, and the flag bit of the data unit is a first set value (eg, 1), and the indicated disk state is not PDM, then the operation type of the data unit is determined to be EC decoding.

[0100] In this case, the disk status is PDM, which indicates that the processing performed on the data unit is data reconstruction. Therefore, the operation type of the data unit is determined to be EC decoding.

[0101] Case 4: If the storage content indicated by the data unit is data, and the flag bit value of the data unit is the first set value, and the indicated disk state is the PDM state, then the operation type of the data unit is determined to be copy.

[0102] In this case, it can be determined that the operation type for the data unit is copy.

[0103] In the above example, the task corresponding to copying is the data migration task, and the tasks corresponding to EC encoding, inheritance, and EC decoding are the data reconstruction tasks.

[0104] In S204, according to the operation type of each data unit indicated by the data stripe, the data stored in the corresponding data unit is processed to complete the data reconstruction of the first target disk and the data migration of the second target disk.

[0105] In the above four cases, one bit (one data unit) in each data stripe in the second data stripe set is processed. When corresponding data processing is performed on each bit in each data stripe, data migration and data reconstruction can be completed.

[0106] In addition, in the embodiment of the present application, for each in-service disk, the status of the disk can also be determined by querying the information on the dashboard, for example:

[0107] If the disk is in a normal state, it is determined that the flag bit corresponding to the disk data unit in the data stripe information is a second set value (eg, 0); wherein the normal state indicates that the disk is inserted and operating normally.

[0108] If the state of the disk is an isolated state, determining that the flag bit value corresponding to the data unit of the disk in the data stripe information is a first set value (for example, it may be 1); wherein the isolated state indicates that the disk is in an unplugged state;

[0109] If the disk is in a sub-healthy state, the flag bit value corresponding to the disk data unit in the data stripe information is determined to be a first set value (eg, 1); wherein the sub-healthy state indicates that the disk is about to be unplugged.

[0110] Figure 3 A flowchart of a data reconstruction task that integrates a data migration task is provided in an embodiment of the present application, which includes periodically querying the disk status, detecting whether a disk is unplugged, creating a reconstruction task, executing the reconstruction task (simultaneously enabling the data migration task), completing the reconstruction task, deleting the reconstruction task, etc.

[0111] Periodically check the disk status. If the disk status is unplugged, cancel the current PDM task, create a rebuild task, execute the rebuild task until the task is completed, and delete the rebuild task.

[0112] The specific process of executing the reconstruction task is as follows:

[0113] Query all related dsegIds according to the disk id. Referring to the above example, all desgIds related to disk 1 are desg1, desg2, desg3, desg4, desg5, desg6, and desg7.

[0114] Look up dsegInfo according to dsegId. Referring to the above example, each desg corresponds to the corresponding data stripe information dsegInfo, so each dsegInfo can be queried. The content of each dsegInfo can be referred to the above example, which will not be repeated here.

[0115] For each dseg, traverse by width, process bit by bit, and calculate the bitmap. In the above example, the width of desg is 7, and each data unit is called a bit. The bitmap here can be information represented by 0 or 1 to describe the status of the disk associated with desg.

[0116] Each bit corresponds to a disk id. The dashboard is queried based on the disk id to determine the current disk status. If the disk is not in normal status, the bitmap at the corresponding position is set to 1, otherwise it is set to 0. Determine whether the last data bit is traversed. If so, jump to the dseginfo update step. Otherwise, repeat the process until the last data bit is traversed.

[0117] Update dseginfo to obtain dseginfo'. Referring to the above example, dseginfo corresponds to the data stripe information of the first data stripe set, and dseginfo' corresponds to the data stripe information of the second data stripe set.

[0118] For each dseg in the second data stripe, traverse by width, process bit by bit, and obtain the operation type.

[0119] Determine whether the current data unit is a data column, if so, execute S304-10, otherwise set the operation type to EC coding.

[0120] Determine whether the data bit corresponding to the current data unit in the bitmap is 1, otherwise set the operation type to inheritance, if yes, query the dashboard according to the disk ID, determine whether the disk status is PDM, if yes, set the operation type to copy, otherwise set the operation type to EC decoding. Determine whether the last bit is traversed, if yes, jump to the data processing process (process the data of the corresponding data unit according to the operation type), otherwise repeat the execution.

[0121] Modify the mapping relationship between dobj and vobj. Wherein, dobj is the identifier of the data unit in the above embodiment, and vobj is the identifier of the data storage space corresponding to the data unit in the above embodiment. Delete the first data stripe set, and perform the reconstruction task to end.

[0122] Although in the related art, mutual exclusion between reconstruction and migration tasks can be achieved by serializing the two tasks, which can reduce the waste of read and write bandwidth, and this mutual exclusion is considerable for general migration tasks, but for PDM tasks that also use disks as the dimension, there are many scenarios that need to be considered and the processing is more complicated. Therefore, for data migration tasks in the disk dimension, this application pays attention to the disk status during data reconstruction, distinguishes the two according to the disk status and bitmap, determines the operation type of the data block, and then integrates the migration and reconstruction tasks together, avoiding complex data processing processes, improving the efficiency of task execution, and improving the utilization of system resources.

[0123] like Figure 4 As shown, based on the same inventive concept, an embodiment of the present application provides a data reconstruction device integrating data migration, including a processing unit 41 and an updating unit 42.

[0124] The processing unit 41 is used to: in a redundant disk array composed of N in-service disks, if a disk reconstruction task is detected, cancel the currently existing disk pre-failure data migration PDM task; wherein the disk indicated by the disk reconstruction task is the first target disk; the disk indicated by the PDM task is the second target disk, and the first target disk and the second target disk are one of the N in-service disks; N is greater than 2;

[0125] The updating unit 42 is used to update the first data stripe set according to the disk description information of the first hot spare disk and the second hot spare disk, so as to obtain the second data stripe set; wherein the first data stripe set is generated according to the disk description information of each of the N active disks;

[0126] The processing unit 41 is further configured to: determine, for each data stripe in the second data stripe set, an operation type for each data unit indicated by the data stripe;

[0127] The processing unit 41 is further used to process the data stored in the corresponding data unit according to the operation type of each data unit indicated by the data stripe, and complete the data reconstruction of the first target disk and the data migration of the second target disk.

[0128] In an optional implementation, the processing unit 41 is further configured to:

[0129] Randomly select a data unit from each of the N active disks, and determine that description information corresponding to each of the selected data units constitutes a data stripe information;

[0130] Among them, a data stripe information includes:

[0131] The identifier of the data stripe, the identifier of the disk associated with the data stripe, the identifier of the data units included in the data stripe, and the storage location indicated by the data units included in the data stripe.

[0132] In an optional implementation, the processing unit 41 is specifically configured to: for each first data stripe in the first data stripe set, apply a data unit of the randomly selected first hot spare disk and a data unit of the second hot spare disk to update the first data stripe to obtain a corresponding second data stripe;

[0133] The updated second data stripes constitute a second data stripe set.

[0134] In an optional implementation manner, the processing unit 41 is specifically configured to:

[0135] The storage position of the data unit of the first target disk in the first data stripe information is replaced with the storage position of the data unit of the first hot spare disk; the storage position of the data unit of the second target disk in the first data stripe information is replaced with the storage position of the data unit of the second hot spare disk.

[0136] In an optional implementation manner, the operation type includes part or all of copy, EC encoding, inheritance, and EC decoding; for each data unit, the processing unit 41 is specifically configured to:

[0137] If the storage content indicated by the data unit is not data, determining that the operation type of the data unit is EC coding;

[0138] If the storage content indicated by the data unit is data, and the flag bit value of the data unit is not the first set value, determining that the operation type of the data unit is inheritance;

[0139] If the storage content indicated by the data unit is data, and the flag bit value of the data unit is the first set value, and the indicated disk state is not PDM, then determining that the operation type of the data unit is EC decoding;

[0140] If the storage content indicated by the data unit is data, the flag value of the data unit is the first set value, and the indicated disk state is the PDM state, then the operation type of the data unit is determined to be copy.

[0141] In an optional implementation, the processing unit 41 is further configured to:

[0142] For each active disk, determine the disk status by querying the information on the dashboard:

[0143] If the state of the disk is normal, determining that the flag bit value corresponding to the data unit of the disk in the data stripe information is a second set value; wherein the normal state indicates that the disk is in an inserted state and is operating normally;

[0144] If the state of the disk is an isolated state, determining that the flag value corresponding to the data unit of the disk in the data stripe information is a first set value; wherein the isolated state indicates that the disk is in an unplugged state;

[0145] If the disk is in a sub-healthy state, it is determined that the flag value corresponding to the data unit of the disk in the data stripe information is a first set value; wherein the sub-healthy state indicates that the disk is about to be unplugged.

[0146] Since the device is the device in the method in the embodiment of the present application, and the principle of solving the problem by the device is similar to that of the method, the implementation of the device can refer to the implementation of the method, and the repeated parts will not be repeated.

[0147] Based on the same inventive concept as the data reconstruction method of the above-mentioned fusion data migration, the embodiment of the present application also provides an electronic device, which can be a desktop computer, a portable computer, a smart phone, a tablet computer, a personal digital assistant (PDA), a server, etc. For example, if the electronic device is a desktop computer, the client can be deployed on the desktop computer. Figure 5As shown, the electronic device may include a processor 51 and a memory 52 .

[0148] The processor 51 may be a general-purpose processor, such as a central processing unit (CPU), a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, and may implement or execute the methods, steps and logic block diagrams disclosed in the embodiments of the present application. A general-purpose processor may be a microprocessor or any conventional processor, etc. The steps of the method disclosed in conjunction with the embodiments of the present application may be directly embodied as being executed by a hardware processor, or may be executed by a combination of hardware and software modules in the processor.

[0149] The memory 52 is a non-volatile computer-readable storage medium that can be used to store non-volatile software programs, non-volatile computer executable programs and modules. The memory may include at least one type of storage medium, such as a flash memory, a hard disk, a multimedia card, a card-type memory, a random access memory (Random Access Memory, RAM), a static random access memory (Static Random Access Memory, SRAM), a programmable read-only memory (Programmable Read Only Memory, PROM), a read-only memory (Read Only Memory, ROM), an electrically erasable programmable read-only memory (Electrically Erasable Programmable Read-Only Memory, EEPROM), a magnetic memory, a disk, an optical disk, etc. The memory is any other medium that can be used to carry or store a desired program code in the form of an instruction or data structure and can be accessed by a computer, but is not limited thereto. The memory 52 in the embodiment of the present application can also be a circuit or any other device that can realize a storage function, for storing program instructions and / or data.

[0150] A person of ordinary skill in the art can understand that: all or part of the steps of implementing the above method embodiments can be completed by hardware related to program instructions, and the aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it executes the steps of the above method embodiments; the above computer storage medium can be any available medium or data storage device that can be accessed by a computer, including but not limited to: mobile storage devices, random access memory (RAM, Random Access Memory), magnetic storage (such as floppy disks, hard disks, magnetic tapes, magneto-optical disks (MO)), optical storage (such as CD, DVD, BD, HVD, etc.), and semiconductor storage (such as ROM, EPROM, EEPROM, non-volatile memory (NAND FLASH), solid-state drive (SSD)) and other media that can store program codes.

[0151] Alternatively, if the above-mentioned integrated unit of the present application is implemented in the form of a software function module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the embodiment of the present application is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the methods of each embodiment of the present application. The aforementioned storage medium includes: a mobile storage device, a random access memory (RAM, Random Access Memory), a magnetic storage device (such as a floppy disk, a hard disk, a magnetic tape, a magneto-optical disk (MO), etc.), an optical storage device (such as CD, DVD, BD, HVD, etc.), and a semiconductor memory (such as ROM, EPROM, EEPROM, non-volatile memory (NAND FLASH), a solid-state drive (SSD)) and other various media that can store program codes.

[0152] Based on the same inventive concept, the embodiment of the present application also provides a computer program product, which includes: computer program code, when the computer program code is run on a computer, the computer executes any of the data reconstruction methods for fusion data migration discussed above. Since the principle of solving the problem by the above computer program product is similar to that of the data reconstruction method for fusion data migration, the implementation of the above computer program product can refer to the implementation of the method, and the repeated parts will not be repeated.

[0153] The above embodiments are only used to introduce the technical solutions of the present application in detail, but the description of the above embodiments is only used to help understand the methods of the embodiments of the present application and should not be understood as limiting the embodiments of the present application. Any changes or substitutions that can be easily thought of by technicians in this technical field should be included in the protection scope of the embodiments of the present application.

Claims

1. A data reconstruction method for fusion data migration, characterized in that: include: In a redundant disk array composed of N in-service disks, if a disk reconstruction task is detected, the currently existing disk pre-failure data migration PDM task is canceled; wherein the disk indicated by the disk reconstruction task is a first target disk, the disk indicated by the PDM task is a second target disk, and the first target disk and the second target disk are one of the N in-service disks; N is greater than 2; According to the disk description information of the first hot spare disk and the second hot spare disk, the first data stripe set is updated to obtain the second data stripe set; wherein the first data stripe set is generated according to the disk description information of each of the N active disks; For each data stripe in the second data stripe set, determining an operation type for each data unit indicated by the data stripe; According to the operation type of each data unit indicated by the data stripe, the data stored in the corresponding data unit is processed to complete the data reconstruction of the first target disk and the data migration of the second target disk.

2. The method according to claim 1, characterized in that The method further comprises: Randomly select a data unit from each of the N active disks, and determine that description information corresponding to each of the selected data units constitutes a data stripe information; The data stripe information includes: The identifier of a data stripe, the identifier of a disk associated with the data stripe, the identifier of the data units included in the data stripe, and the storage location indicated by the data units included in the data stripe.

3. The method according to claim 2, characterized in that The updating of the first data stripe set according to the disk description information of the first hot spare disk and the second hot spare disk to obtain the second data stripe set includes: For each first data stripe in the first data stripe set, apply a data unit of the first hot spare disk and a data unit of the second hot spare disk selected randomly to update the first data stripe to obtain a corresponding second data stripe; The updated second data stripes constitute a second data stripe set.

4. The method according to claim 3, characterized in that The applying randomly selected one data unit of the first hot spare disk and one data unit of the second hot spare disk to update the first data stripe includes: The storage position of the data unit of the first target disk in the first data stripe information is replaced with the storage position of the data unit of the first hot spare disk; the storage position of the data unit of the second target disk in the first data stripe information is replaced with the storage position of the data unit of the second hot spare disk.

5. The method according to claim 1, characterized in that The operation type includes part or all of copy, EC encoding, inheritance, and EC decoding; in the determining, for each data stripe in the second data stripe set, the operation type for each data unit indicated by the data stripe, the operation for each data unit includes: If the storage content indicated by the data unit is not data, determining that the operation type of the data unit is EC coding; If the storage content indicated by the data unit is data, and the flag bit value of the data unit is not the first set value, determining that the operation type of the data unit is inheritance; If the storage content indicated by the data unit is data, and the flag bit value of the data unit is a first set value, and the indicated disk state is not PDM, then determining that the operation type of the data unit is EC decoding; If the storage content indicated by the data unit is data, the flag bit value of the data unit is a first set value, and the indicated disk state is a PDM state, then the operation type of the data unit is determined to be copy.

6. The method according to claim 5, characterized in that The method further comprises: For each active disk, determine the status of the disk by querying the information on the dashboard: If the state of the disk is a normal state, determining that the flag bit value corresponding to the data unit of the disk in the data stripe information is a second set value; wherein the normal state indicates that the disk is in an inserted state and is operating normally; If the state of the disk is an isolated state, determining that the flag value corresponding to the data unit of the disk in the data stripe information is the first set value; wherein the isolated state indicates that the disk is in an unplugged state; If the state of the disk is a sub-healthy state, it is determined that the flag value corresponding to the data unit of the disk in the data stripe information is the first set value; wherein the sub-healthy state indicates that the disk is about to be pulled out.

7. A data reconstruction device integrating data migration, characterized in that: include: A processing unit, configured to: in a redundant disk array composed of N in-service disks, if a disk reconstruction task is detected, cancel a currently existing disk pre-failure data migration PDM task; wherein the disk indicated by the disk reconstruction task is a first target disk; the disk indicated by the PDM task is a second target disk, and the first target disk and the second target disk are one of the N in-service disks; and N is greater than 2; An updating unit, configured to update a first data stripe set according to the disk description information of the first hot spare disk and the second hot spare disk, to obtain a second data stripe set; wherein the first data stripe set is generated according to the disk description information of each of the N active disks; The processing unit is further configured to: determine, for each data stripe in the second data stripe set, an operation type for each data unit indicated by the data stripe; The processing unit is further used to: process the data stored in the corresponding data unit according to the operation type of each data unit indicated by the data stripe, and complete the data reconstruction of the first target disk and the data migration of the second target disk.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium having computer program instructions stored thereon, characterized in that: When the computer program instructions are executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images