Data reconstruction method and device, computer equipment, storage medium and program product

Abstract

The invention relates to a data reconstruction method and device, computer equipment, a storage medium and a program product. The method comprises the following steps: performing data reconstruction on each fault disk in a disk array in sequence to obtain reconstruction data of each fault disk, and performing data reconstruction on the disk array based on the reconstruction data of each fault disk. Wherein in the data reconstruction process, each time data reconstruction of one fault disk is carried out, reconstruction data of the corresponding fault disk is stored in the hot spare disk inserted into the slot position of the previous fault disk, and the slot position of the corresponding fault disk is released for inserting a new hot spare disk. By adopting the method, multi-disk data reconstruction of the disk array can be completed under the condition of insufficient hot standby space.

Description

Technical Field

[0001] This application relates to the field of data processing technology, and in particular to a data reconstruction method, apparatus, computer equipment, storage medium, and program product. Background Technology

[0002] In the storage field, a Redundant Array of Independent Disks (RAID) is a logical disk group composed of multiple physical disks.

[0003] In related technologies, when there is a faulty disk in a logical disk group with an abnormal state, it is necessary to use the data of the normal disk to recover the data of the faulty disk, and write the recovered data to the hot spare disk. Then, the data in the hot spare disk and the data of the normal disk are combined as the data reconstruction of the faulty disk.

[0004] However, in the data reconstruction process of related technologies, there is a risk of insufficient hot backup space, which may prevent the data reconstruction from being completed. Summary of the Invention

[0005] Therefore, it is necessary to provide a data reconstruction method, apparatus, computer equipment, storage medium, and program product to address the above-mentioned technical problems, which can complete data reconstruction when hot spare space is insufficient.

[0006] Firstly, this application provides a data reconstruction method, which includes:

[0007] Data reconstruction is performed on each failed disk in the disk array in sequence to obtain the reconstruction data of each failed disk. During the data reconstruction process, the reconstruction data of each failed disk is stored in the hot spare disk inserted in the slot of the previous failed disk, and the slot of the corresponding failed disk is released for the insertion of a new hot spare disk.

[0008] Based on the reconstruction data of each failed disk, the disk array is reconstructed.

[0009] In the technical solution provided in this application embodiment, data reconstruction is performed sequentially on each failed disk in the disk array to obtain reconstruction data for each failed disk. Based on the reconstruction data of each failed disk, the disk array is reconstructed. Specifically, during the data reconstruction process, for each failed disk, the reconstructed data of that failed disk is stored in a hot spare disk inserted into the slot of the previous failed disk, and the corresponding slot is released for inserting a new hot spare disk. This is equivalent to releasing the slot of each failed disk after its reconstruction is completed during the disk-by-disk data reconstruction process. This ensures that while hot spare disks are dynamically consumed, they are continuously replenished during the entire disk array data reconstruction process, providing hot spare space for the reconstruction data of each failed disk. This guarantees the smooth completion of the data reconstruction for each failed disk. Even after the data reconstruction of the last failed disk is completed, a new hot spare disk is inserted to provide hot spare space for the reorganization of data from non-failed disks, facilitating the storage of reconstruction data from non-failed disks and ensuring the smooth completion of the entire disk array data reconstruction.

[0010] In one embodiment, data reconstruction is performed sequentially on each failed disk in the disk array to obtain reconstructed data for each failed disk, including:

[0011] For any faulty disk, obtain the data stripe corresponding to each faulty data block in the faulty disk;

[0012] Based on the position of each faulty data block in the data stripe, identify the faulty user data blocks in the faulty disk;

[0013] The data in the faulty user data block is recovered to obtain the user's repaired data;

[0014] Based on user-repaired data, reconstruction data for the faulty disk is obtained.

[0015] In the technical solution provided in this application, for any faulty disk, based on the position of each faulty data block in the data stripe, the user data and checksum data in the faulty disk are determined, and the user data is recovered first to obtain user repair data. Based on the user repair data, the reconstructed data of the faulty disk is obtained. This is equivalent to prioritizing the reconstruction of user data during the data reconstruction process, considering the risk that insufficient hot spare space may lead to incomplete reconstruction. This ensures the data reading efficiency of the front-end IO of the data storage system during the data reconstruction phase, and then reconstructs the non-user data, i.e., checksum data, to avoid redundant encoding of user data and reduce the EC encoding / decoding computation load of a single disk reconstruction.

[0016] In one embodiment, obtaining the data stripes corresponding to each faulty data block in the faulty disk includes:

[0017] Obtain the original stripe corresponding to each faulty data block in each faulty disk;

[0018] The faulty data blocks in each original stripe are set as invalid data blocks to obtain the data stripe corresponding to each faulty data block.

[0019] In the technical solution provided in this application embodiment, for each faulty disk, the faulty data blocks in the original stripe corresponding to each faulty data block are set as invalid data blocks, which enhances the effectiveness of the readable data in the data stripe, and at the same time eliminates the interference of faulty data in the faulty data blocks in the original stripe on the single disk reconstruction calculation process, avoids premature stripe EC encoding and writing, and makes the single disk reconstruction more efficient.

[0020] In one embodiment, data in the user data fault block is recovered to obtain user repair data, including:

[0021] For any user data fault block, obtain the normal data blocks other than the user data fault block in the data stripe to which the user data fault block belongs;

[0022] The data in each normal data block is decoded to recover the user repair data corresponding to the user data fault block.

[0023] In the technical solution provided in this application embodiment, for any user data fault block, the normal data blocks other than the fault block in the data stripe to which the user data fault block belongs are obtained as the basis for user data fault block data recovery. Then, the data in each normal data block is decoded to recover the data of the user data fault block and obtain the user repair data. This data recovery method makes full use of the correlation between data in a single data stripe to ensure the accuracy of user repair data. Furthermore, this embodiment adopts a disk-by-disk, user data fault block recovery method to perform single-disk data reconstruction without needing to recover data from other faulty disks in the same stripe. This can reduce the hot spare space for single-disk reconstruction to a certain extent, improve the utilization rate of hot spare space, ensure the smooth execution of single-disk data reconstruction tasks, and improve the efficiency of single-disk data reconstruction.

[0024] In one embodiment, the user repair data includes multiple components; based on the user repair data, reconstruction data for the faulty disk is obtained, including:

[0025] Each user's repair data is written into a data block in the hot spare disk inserted into the slot of the previous failed disk, resulting in multiple user data columns;

[0026] Based on the data column length threshold, each user data column is combined to obtain at least one reconstructed strip;

[0027] The user repair data in each reconstruction strip is encoded to obtain the verification data of each reconstruction strip;

[0028] The user repair data and the verification data of each reconstruction stripe are identified as the reconstruction data of the faulty disk.

[0029] In the technical solution provided in this application embodiment, each user's repair data is written into a data block in a hot spare disk inserted into the slot of the previous faulty disk, resulting in multiple user data columns. Based on the data column length threshold, each user data column is combined to obtain at least one reconstruction stripe, ensuring that the reconstruction stripe meets the data read / write requirements of the data storage system and guarantees the readability of the data in the reconstruction stripe. Next, the user repair data in each reconstruction stripe is encoded to obtain the verification data of each reconstruction stripe, and the content of the verification data column in the reconstruction stripe is supplemented. Finally, the user repair data in each reconstruction stripe and the verification data of each reconstruction stripe are summarized, thereby determining that the reconstruction data of the faulty disk has both integrity and reliability.

[0030] In one embodiment, data reconstruction of the disk array is performed based on the reconstruction data of each failed disk, including:

[0031] Obtain the data stripe corresponding to each faulty data block in each faulty disk;

[0032] Faulty data blocks are removed from each data stripe to obtain multiple stripes to be reassembled;

[0033] The user data in each stripe to be reassembled is reassembled to obtain the reassembled data of the disk array;

[0034] Based on the reconstruction data of each failed disk and the reorganization data of the disk array, the reconstruction data of the disk array is determined.

[0035] In the technical solution provided in this application embodiment, based on obtaining the data stripes corresponding to each faulty data block in each faulty disk, the faulty data blocks in each data stripe are removed to obtain multiple stripes to be reconstructed, and the user data in each stripe to be reconstructed is further determined. Next, the user data in each stripe to be reconstructed is integrated to obtain the user data of the entire disk array. Finally, combined with the reconstruction data of each faulty disk, the reconstruction data of the disk array is obtained. In the data reconstruction process, this application embodiment treats the user data in each data stripe as a whole, performs a reconstruction task once, and obtains the reconstruction data of the entire disk array, improving data reconstruction efficiency and shortening the disk array data reconstruction time to a certain extent.

[0036] In one embodiment, user data in each stripe to be reassembled is reassembled to obtain reassembled data for the disk array, including:

[0037] Obtain the number of user data columns in each strip to be reorganized;

[0038] If the number of user data columns is less than or equal to the data column length threshold, then the user data in each strip to be reassembled will be determined as the reassembled user data.

[0039] The recombined user data is encoded to obtain recombined verification data;

[0040] The reorganization data of the disk array is obtained based on the reorganized user data and the reorganization verification data.

[0041] In the technical solution provided in this application embodiment, when the number of user data columns is less than or equal to the data column length threshold, each stripe to be reassembled is combined into a reassembled stripe, and the user data in each stripe to be reassembled is encoded to obtain the reassembled verification data of the reassembled stripe, thereby realizing the efficient integration of normal user data in each data stripe in the disk array.

[0042] In one embodiment, user data in each stripe to be reassembled is reassembled to obtain reassembled data for the disk array, including:

[0043] If the number of user data columns in each strip to be reassembled is greater than the data column length threshold, the user data in each strip to be reassembled is divided into a first reassembled user data set and a second reassembled user data set; both the first reassembled user data set and the second reassembled user data set match the data column length threshold.

[0044] The user data in the first recombined user data set is encoded to obtain the first recombined verification data, and the user data in the second recombined user data set is encoded to obtain the second recombined verification data;

[0045] User data in the first recombined user data set and user data in the second recombined user data set are identified as recombined user data, and the first recombined verification data and the second recombined verification data are identified as recombined verification data.

[0046] The reorganization data of the disk array is obtained based on the reorganized user data and the reorganization verification data.

[0047] In the technical solution provided in this application embodiment, when the number of user data columns exceeds the data column length threshold, the user data in each stripe to be reassembled is re-divided into a first reassembled user data set and a second reassembled user data set to achieve grouping of user data; then, the user data in the first reassembled user data set is encoded to obtain the first reassembled verification data, and the user data in the second reassembled user data set is encoded to obtain the second reassembled verification data, setting valid verification data for the reassembled user data. The reassembly process is divided into two stages: user data reassembly and verification data calculation, thereby achieving standardized processing of non-faulty user data of the disk array.

[0048] Secondly, this application also provides a data reconstruction apparatus, comprising:

[0049] The data acquisition module is used to sequentially reconstruct the data of each failed disk in the disk array to obtain the reconstructed data of each failed disk. During the data reconstruction process, each time the data of a failed disk is reconstructed, the reconstructed data of the corresponding failed disk is stored in the hot spare disk inserted in the slot of the previous failed disk, and the slot of the corresponding failed disk is released for inserting a new hot spare disk.

[0050] The data reconstruction module is used to reconstruct the data of the disk array based on the reconstruction data of each failed disk.

[0051] Thirdly, this application also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of the method in any of the embodiments of the first aspect described above.

[0052] Fourthly, this application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method in any of the embodiments of the first aspect described above.

[0053] Fifthly, this application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the method in any of the embodiments of the first aspect described above. Attached Figure Description

[0054] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the drawings used in the description of the embodiments of this application or related technologies will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0055] Figure 1This is a diagram illustrating the application environment of the data reconstruction method in one embodiment;

[0056] Figure 2 This is a flowchart illustrating a data reconstruction method in one embodiment;

[0057] Figure 3 This is a flowchart illustrating the data acquisition steps in one embodiment;

[0058] Figure 4 This is a schematic diagram of the disk array structure in one embodiment;

[0059] Figure 5 This is a flowchart illustrating the data stripe acquisition steps in one embodiment;

[0060] Figure 6 This is a flowchart illustrating the user data recovery steps in one embodiment;

[0061] Figure 7 This is a flowchart illustrating the data acquisition steps in another embodiment;

[0062] Figure 8 This is a flowchart illustrating the data acquisition steps in another embodiment;

[0063] Figure 9 This is a flowchart illustrating the data acquisition steps in one embodiment;

[0064] Figure 10 This is a flowchart illustrating the data acquisition step in another embodiment;

[0065] Figure 11 This is a schematic diagram illustrating different stages of data reconstruction in one embodiment;

[0066] Figure 12 This is a structural block diagram of a data reconstruction apparatus in one embodiment;

[0067] Figure 13 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0068] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0069] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of the embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0070] The technical background of this application will be explained below.

[0071] In the storage field, Redundant Array of Independent Disks (RAID) is a logical disk group composed of multiple physical disks, which uses a redundant information storage method to ensure data security.

[0072] When a faulty disk exists in a logical disk group, its contents can be recovered using the contents of a healthy disk. The recovered contents are then written to a hot spare disk, completing the data reconstruction of the independent redundant disk array. It should be noted that the hot spare disk is used for disk data reconstruction in cases of disk malfunction; its space will be reclaimed when a user adds a new disk or the faulty disk is resolved.

[0073] Taking a multi-disk failure in an independent redundant disk array as an example, related technologies involve reconstructing multiple failed disks simultaneously. Before the disk data reconstruction is complete, the slot numbers occupied by the disks cannot be released, and new disks cannot be inserted into the slots to supplement the storage pool. This means that disk data reconstruction requires sufficient hot spare space; insufficient hot spare space will prevent the data reconstruction from completing successfully.

[0074] Therefore, in scenarios involving multiple disk failures, successfully completing data reconstruction when hot spare disk space is insufficient is a pressing technical problem that needs to be solved. To address this, embodiments of this application provide a data reconstruction method, apparatus, computer equipment, storage medium, and program product to solve the aforementioned problem. Of course, the technical solutions provided in these embodiments are not limited to solving only the above-mentioned problems and also possess other technical effects, which can be found in the following embodiments.

[0075] Before describing the embodiments of this application, the application scenarios of the embodiments of this application will be explained first.

[0076] The data reconstruction method provided in this application embodiment can be applied to, for example... Figure 1In the application environment shown, the data storage system communicates with the server via a network. The data storage system can store data that the server needs to process. In this embodiment, the storage system includes multiple disks and can be an independent data storage system, a distributed storage system, etc. The data storage system can be integrated onto a server, or it can be located on a cloud or other network server. The server can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing cloud computing services.

[0077] The technical solution of this application and how it solves the above-mentioned technical problems will be described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of this application will be described below with reference to the accompanying drawings.

[0078] In one exemplary embodiment, such as Figure 2 As shown, a data reconstruction method is provided. This method is illustrated using a server as an example. It is understood that this method can also be applied to a terminal, and further to a system including both a terminal and a server, and is implemented through interaction between the terminal and the server. This application embodiment includes the following steps:

[0079] S201, sequentially reconstruct the data of each failed disk in the disk array to obtain the reconstructed data of each failed disk; wherein, during the data reconstruction process, each time the data of a failed disk is reconstructed, the reconstructed data of the corresponding failed disk is stored in the hot spare disk inserted in the slot of the previous failed disk, and the slot of the corresponding failed disk is released for inserting a new hot spare disk.

[0080] In a storage system, a disk array comprises multiple disks used to store user-uploaded data. In this embodiment, a data striping strategy is employed to store user data: the user data is divided into multiple small blocks, and these blocks are stored on different disks to improve the read / write performance of the data storage system.

[0081] In real-world applications, disks in a disk array may fail due to various factors, such as disk component aging, external magnetic field interference, moisture intrusion, and abnormal disk power supply. Therefore, when a disk fails in the array, it is necessary to promptly recover the data from the failed disk to ensure the integrity of the stored data. In this embodiment, the disk array may contain one or more failed disks.

[0082] In the case of a disk array containing multiple failed disks, a single-disk reconstruction strategy is adopted to reconstruct the data of each failed disk one by one, and write the reconstructed data of the failed disk to a hot spare disk. Then, the slots of the corresponding failed disks are released, and the released slots are used to insert new hot spare disks, providing hot spare space for the reconstruction data of subsequent failed disks.

[0083] In an optional embodiment, the single-disk reconstruction strategy may be to recover faulty data in the faulty disk stripe by stripe, and based on the correlation between non-faulty data and faulty data in a single stripe, perform erasure code (EC) decoding on the non-faulty data to obtain the faulty data about the faulty disk in that stripe.

[0084] In an optional embodiment, the release of a slot can be achieved by generating a slot release instruction for the corresponding failed disk after the data reconstruction and storage of a single failed disk is completed. This instruction is used to indicate that the failed disk is removed from the slot and a new disk is inserted as a hot spare.

[0085] It should be noted that data storage systems typically have pre-configured free memory as hot spare disks. In this embodiment, when reconstructing the first failed disk, the reconstructed data of the first failed disk is written to the pre-prepared hot spare disk in the data storage system, and the slot of the first failed disk is released for inserting a new hot spare disk. Next, data reconstruction is performed on the second failed disk. The data obtained from the reconstruction of the second failed disk is stored in the new hot spare disk inserted in the slot of the first failed disk, and the second failed disk is released. This process continues to provide hot spare space for the reconstruction data of the third failed disk, and so on, until the data reconstruction of all failed disks is completed, releasing the slot of the last failed disk to allow for the insertion of a new hot spare disk and obtain new hot spare space.

[0086] S202, based on the reconstruction data of each failed disk, performs data reconstruction on the disk array.

[0087] The reconstruction process involves removing the corresponding faulty data from the original faulty disks from the disk array. Based on data attributes, user data is then extracted from the removed disk data. Next, this user data is reconstructed to obtain reconstructed user data. This reconstructed user data is then encoded to obtain corresponding checksum data. This reconstructed user data and checksum data are used as the reconstruction data for the non-faulty disks. Finally, the reconstruction data from each faulty disk and the non-faulty disks are combined to form the overall reconstruction data for the disk array.

[0088] In this embodiment, data reconstruction is performed sequentially on each failed disk in the disk array to obtain reconstruction data for each failed disk. Based on the reconstruction data of each failed disk, the disk array is then reconstructed. Specifically, during the data reconstruction process, after each failed disk is reconstructed, the corresponding reconstructed data is stored in a hot spare disk inserted into the slot of the previous failed disk, and the corresponding slot is released for inserting a new hot spare disk. This is equivalent to releasing the slot of each failed disk after its reconstruction is completed during the disk-by-disk data reconstruction process. This ensures that while hot spare disks are dynamically consumed, they are continuously replenished during the entire disk array data reconstruction process, providing hot spare space for the reconstruction data of each failed disk. This guarantees the smooth completion of the data reconstruction for each failed disk. Even after the data reconstruction of the last failed disk is completed, a new hot spare disk is inserted to provide hot spare space for the reorganization of data from non-failed disks, facilitating the storage of reconstruction data from non-failed disks and ensuring the smooth completion of the entire disk array data reconstruction.

[0089] During data reconstruction, the data reconstruction task of a disk array can be broken down into a disk-by-disk reconstruction task targeting multiple failed disks and a final data reconstruction task targeting the non-failed disks. The disk-by-disk reconstruction task targeting failed disks will be explained below.

[0090] In one exemplary embodiment, such as Figure 3 As shown, data reconstruction is performed sequentially on each failed disk in the disk array to obtain the reconstructed data for each failed disk, including the following steps:

[0091] S301, For any faulty disk, obtain the data stripe corresponding to each faulty data block in the faulty disk.

[0092] Here, a stripe is a data read / write unit in the storage system; please refer to [link / reference]. Figure 4 , Figure 4 This is a schematic diagram of stripes in a disk array. Figure 4 In a disk array, each column represents a disk, and each row represents a stripe. In other words, individual data blocks from each disk in a disk array together form a stripe, and a disk array can include multiple stripes.

[0093] When a faulty disk is identified, the data blocks storing data on that disk can be considered faulty data blocks, and there is a one-to-one correspondence between the number of faulty data blocks and the number of data stripes; that is, each faulty data block corresponds to one data stripe. For each data stripe to which a faulty data block belongs, the data stripe includes both faulty data blocks from the faulty disk and normal data blocks from both faulty disks.

[0094] S302, based on the position of each faulty data block in the data stripe, determine the user data faulty block in the faulty disk.

[0095] A data stripe comprises multiple columns of data blocks. Based on the attributes of the data blocks, each column in the data stripe can be divided into a data column and at least one check column. The data column corresponds to the user data fault block, storing the user-uploaded data; the check column corresponds to the check data block, storing the check data obtained based on the user data encoded in the data stripe.

[0096] It should be noted that for each data strip, the attributes of each data column within that strip are pre-defined, for example... Figure 4 In the data stripes, the first four columns from left to right are the data columns, and the last two columns are the check data columns. The attributes of the data columns in different data stripes can be the same or different.

[0097] In this embodiment, based on the attribute information of different data columns in the data strip, the data columns and check columns in the data strip are determined. If the faulty data block is located in the data column position in the data strip, the faulty data block is determined as a user data faulty block. If the faulty data block is located in the check column position in the data strip, the faulty data block is determined as a check data faulty block.

[0098] For the verification data in the verification data fault block, since the verification data is calculated after the valid user data of the data stripe is determined, and in actual scenarios, a data stripe may include multiple invalid user data with user data faults, and there may be a situation of data stripe shrinkage, based on this, in the embodiment of this application, the verification data fault block is first set as an invalid data block, and the server is not allowed to read it during the reconstruction process.

[0099] S303, recover the data in the user data fault block to obtain the user repair data.

[0100] For the data in the user data fault block, that is, the user data, the data in the user data fault block in the data stripe, that is, the user repair data, can be determined based on the non-faulty data in the data stripe where the user data fault block is located.

[0101] Optionally, the verification data of the data fault block is determined from the non-faulty data in the data stripe where the user data fault block is located. Then, according to the decoding strategy of the data stripe, erasure code (EC) decoding is performed on the verification data to obtain all user data in the data stripe and the position of each user data in the data stripe. Then, the user data corresponding to the position of the user data fault block in the data stripe is determined as the user repair data of the user data fault block.

[0102] S304: Based on user-repaired data, reconstructed data for the faulty disk is obtained.

[0103] All user repair data for a single failed disk are combined to construct a new data stripe, which has a reserved column for checksum data. Then, based on the user repair data in the new data stripe, the checksum data of the data stripe is calculated and filled into the checksum data column of the data stripe. The user repair data and checksum data in the entire data stripe are used as the reconstruction data for the failed disk.

[0104] In this embodiment, for any faulty disk, based on the position of each faulty data block in the data stripe, the user data and checksum data in the faulty disk are determined, and the user data is recovered first to obtain user repair data. Based on the user repair data, the reconstructed data of the faulty disk is obtained. This is equivalent to prioritizing the reconstruction of user data during the data reconstruction process, considering the risk of incomplete reconstruction due to insufficient hot spare space. This ensures the data reading efficiency of the front-end I / O of the data storage system during the data reconstruction phase, and then reconstructs the non-user data, i.e., the checksum data, to avoid redundant encoding of user data and reduce the EC encoding / decoding computation load of a single disk reconstruction.

[0105] In an exemplary embodiment, one possible implementation of the aforementioned step S301, "obtaining the data stripes corresponding to each faulty data block in the faulty disk," will be described, such as... Figure 5 As shown, it includes the following steps:

[0106] S501, obtain the original stripe corresponding to each faulty data block in each faulty disk.

[0107] The original stripe refers to a stripe in a disk array that includes faulty data blocks. In this embodiment, each data block in the original stripe is readable. By reading according to the position of the data blocks in the stripe, the user data and checksum data in the original stripe can be determined. By reading according to whether the data blocks in the stripe are faulty, the faulty data blocks and non-faulty data blocks in the original stripe can be determined.

[0108] S502, set the faulty data blocks in each original stripe as invalid data blocks to obtain the data stripe corresponding to each faulty data block.

[0109] It is important to emphasize that the number of columns in the data stripe is the same as that in the original stripe. The difference between the two is that faulty data blocks in the data stripe are invalid data blocks, that is, unreadable, in order to enhance the effectiveness of readable data blocks in the data stripe.

[0110] In this embodiment of the application, for each faulty disk, the faulty data blocks in the original stripe corresponding to each faulty data block are set as invalid data blocks, which enhances the effectiveness of the readable data in the data stripe, and at the same time eliminates the interference of faulty data in the faulty data blocks in the original stripe on the single disk reconstruction calculation process, avoids premature stripe EC encoding and writing, and makes the single disk reconstruction more efficient.

[0111] In an exemplary embodiment, one possible implementation of the aforementioned step S303, "recovering the data in the user data fault block to obtain user repair data," will be described, such as... Figure 6 As shown, it includes the following steps:

[0112] S601, For any user data fault block, obtain the normal data blocks other than the user data fault block in the data stripe to which the user data fault block belongs.

[0113] For a single user data fault block to which a data stripe belongs, a normal data block refers to a data block in the data stripe other than the user data fault block. It can be a normal user data fault block, a normal check data block, or both.

[0114] S602 decodes the data in each normal data block to recover the user repair data corresponding to the user data fault block.

[0115] For any user fault data block, the data in each normal data block and the position of the user data fault block in the data stripe are taken as input and fed into the stripe decoding module. The stripe decoding module performs EC decoding on the input data to recover the data of the user fault data block, which is the user repair data corresponding to the user fault data block.

[0116] It should be noted that when repairing the data of a user faulty data block in a single faulty disk, a data block-by-block data recovery method is adopted. That is, each decoding process only recovers the user data in a single user data faulty block in a single faulty disk, and does not recover the data in data faulty blocks in other faulty disks in the stripe, so as to minimize the hot spare space occupied by single disk data reconstruction.

[0117] In this embodiment, for any user data fault block, normal data blocks other than the faulty user data block are obtained from the data stripe to which the faulty user data block belongs. These normal data blocks are used as the basis for recovering the faulty user data block. Then, the data in each normal data block is decoded to recover the data of the faulty user data block, thus obtaining the user's repaired data. This data recovery method fully utilizes the correlation between data in a single data stripe to ensure the accuracy of the user's repaired data. Furthermore, this embodiment adopts a disk-by-disk, block-by-block user data fault recovery method to perform single-disk data reconstruction without needing to recover data from other faulty disks in the same stripe. This can reduce the hot spare space required for single-disk reconstruction to a certain extent, improve the utilization rate of hot spare space, ensure the smooth execution of the single-disk data reconstruction task, and improve the efficiency of single-disk data reconstruction.

[0118] In one exemplary embodiment, the user repair data includes multiple components; one possible implementation of the aforementioned step S304, "obtaining reconstruction data of the faulty disk based on the user repair data," is described below, such as... Figure 7 As shown, it includes the following steps:

[0119] S701 writes the repair data of each user into the data block of the hot spare disk inserted in the slot of the previous failed disk, resulting in multiple user data columns.

[0120] One set of user repair data corresponds to one data block. A data block is allocated from the hot spare disk to store one set of user repair data as a column of user data blocks. This process continues until all user repair data in the faulty disk being reconstructed is written to the hot spare disk, resulting in multiple columns of user data corresponding to the faulty disk.

[0121] In another scenario, data blocks matching the number of user-repaired data can be retrieved from the hot spare disk, and then each user's repaired data can be written into each data block in batches to obtain multiple user data columns corresponding to the faulty disk.

[0122] S702, based on the data column length threshold, combine each user data column to obtain at least one reconstructed strip.

[0123] The data column length threshold refers to the maximum number of user data columns allowed in a single stripe in the data storage system. In this embodiment, to ensure that the data storage system can read the reconstructed data, it is necessary to combine each user data column according to the data column length threshold to obtain at least one reconstructed stripe, such that the number of user repair data columns in the reconstructed stripe does not exceed the data column length threshold.

[0124] It is important to emphasize that a validation data column is reserved in the reconstructed strip. The validation data block of the validation data column is used to fill the validation data corresponding to the user data column of the reconstructed strip.

[0125] In practical applications, a single failed disk typically comprises multiple failed data blocks. Therefore, the user data columns obtained from hot spare space user data reconstruction will also be multiple. This can lead to two scenarios: either the number of user data columns exceeds the data column length threshold, or the number of user data columns is less than or equal to the data column length threshold. The methods for obtaining reconstruction stripes in these two scenarios are explained below.

[0126] If the number of user data columns is less than or equal to the data column length threshold, then the user data columns are directly combined to obtain a reconstructed strip.

[0127] If the number of user data columns exceeds a data column length threshold, the multiple user data columns are divided into at least one first user data column set and one second user data column set. The number of user data columns in each first user data column set matches the data column length threshold. The second user data column set includes all user data columns except those in the first user data column set. The number of user data columns in each first user data column set is less than or equal to the data column length threshold. Next, the user data columns in each first user data column set are combined to obtain a first reconstructed stripe corresponding to each first user data column set. The user data columns in the second user data column set are then combined to obtain a second reconstructed stripe for the second user data column set.

[0128] S703 encodes the user repair data in each reconstruction strip to obtain the verification data for each reconstruction strip.

[0129] For each reconstructed stripe, all user repair data in that reconstructed stripe is used as input for stripe EC encoding to obtain verification data. Then, the verification data is written into the data block on the reserved verification data column of the reconstructed stripe to obtain the complete reconstructed stripe.

[0130] S704 identifies the user repair data and the verification data of each reconstruction stripe as the reconstruction data for the faulty disk.

[0131] When a single failed disk comprises multiple reconstruction stripes, the user data and checksum data of each reconstruction stripe are aggregated to obtain the reconstruction data of the failed disk, taking each reconstruction stripe as a unit.

[0132] In this embodiment, the user repair data is written into the data block of the hot spare disk inserted into the slot of the previous failed disk, resulting in multiple user data columns. Based on the data column length threshold, the user data columns are combined to obtain at least one reconstruction stripe, ensuring that the reconstruction stripe meets the data read and write requirements of the data storage system and guarantees the readability of the data in the reconstruction stripe. Then, the user repair data in each reconstruction stripe is encoded to obtain the verification data of each reconstruction stripe, and the content of the verification data column in the reconstruction stripe is supplemented. Finally, the user repair data and the verification data of each reconstruction stripe are summarized to determine that the reconstruction data of the failed disk has both integrity and reliability.

[0133] In the data reconstruction process of the disk array, the entire data reconstruction task is decomposed into multiple disk-by-disk reconstruction tasks for failed disks and a final data reconstruction task for non-failed disks. The aforementioned embodiments have described in detail the disk-by-disk reconstruction task for failed disks; next, the final data reconstruction task for non-failed disks will be described.

[0134] In one exemplary embodiment, such as Figure 8 As shown, based on the reconstruction data of each failed disk, the disk array is reconstructed, including the following steps:

[0135] S801, retrieve the data stripe corresponding to each faulty data block in each faulty disk.

[0136] It should be noted that data stripes include both faulty and non-faulty data blocks. After reconstructing the faulty data blocks in the data stripe, the data in the data stripe is no longer complete, and the checksum data in the data stripe is no longer valid. Based on this, this embodiment of the application uses data stripes as the reassembly unit to perform the final data reconstruction task of the non-faulty disk, that is, to reassemble the data in the data stripe excluding the faulty data.

[0137] For a single failed disk, the method for obtaining the data stripes corresponding to each failed data block in the failed disk is the same as described above. Figure 3 The steps in step S301 of the illustrated embodiment are the same and will not be repeated here.

[0138] The difference between the two is that the purpose of the aforementioned step S301 is to reconstruct the faulty data of the faulty disk, specifically to perform a reconstruction on a single disk, while the purpose of this embodiment is to reconstruct the data of the non-faulty disk, which can be to treat the normal data in each data stripe as a whole and perform a data reconstruction.

[0139] S802, remove faulty data blocks from each data stripe to obtain multiple stripes to be reassembled.

[0140] To reiterate, the data stripe includes faulty data blocks, but these faulty data blocks are in an unreadable state. Therefore, after reconstructing the data in the faulty data blocks, they can be directly removed from the data stripe to update the column width of the data stripe, resulting in the stripe to be reconstructed.

[0141] For any data stripe, since the data stripe includes faulty data blocks, the column width of the stripe to be reconstructed after removing the faulty data blocks is also smaller than the column width of the data stripe.

[0142] In one scenario, if the strip to be reassembled does not contain user data and only contains verification data, then the verification data no longer has value, and the strip to be reassembled can be released.

[0143] S803 reassembles the user data in each stripe to be reassembled, obtaining the reassembled data of the disk array.

[0144] According to the data column length threshold of the data storage system, the user data is reorganized to obtain multiple reorganized stripes. Each reorganized stripe has a reserved checksum column for storing checksum data corresponding to the user data columns of each reorganized stripe. In this embodiment, the data on each reorganized stripe is determined as the reorganized data of the disk array.

[0145] The data column length threshold refers to the maximum number of user data columns allowed in a single stripe in the data storage system. In this embodiment, to ensure that the data storage system can read the reconstructed data, the number of user data columns in each reconstructed stripe is no greater than the data column length threshold. Based on this, there are no restrictions on the method of reconstructing the user data.

[0146] S804 determines the reconstruction data of the disk array based on the reconstruction data of each failed disk and the reorganization data of the disk array.

[0147] The reconstruction data of each faulty disk and the reorganization data of the disk array are combined to obtain the reconstruction data of the disk array.

[0148] In this embodiment, based on obtaining the data stripes corresponding to each faulty data block in each faulty disk, the faulty data blocks in each data stripe are removed to obtain multiple stripes to be reconstructed, and the user data in each stripe to be reconstructed is further determined. Next, the user data in each stripe to be reconstructed is integrated to obtain the user data for the entire disk array. Finally, combined with the reconstruction data of each faulty disk, the reconstruction data of the disk array is obtained. In this embodiment, during the data reconstruction process, the user data in each data stripe is treated as a whole, and a reconstruction task is performed to obtain the reconstruction data for the entire disk array, improving data reconstruction efficiency and shortening the disk array data reconstruction time to a certain extent.

[0149] The aforementioned embodiments do not restrict the method of reorganizing user data, provided that the number of user data columns in the reorganized strip does not exceed the data column length threshold. Below, we will describe one feasible method for reorganizing user data based on the number of user data columns in each strip to be reorganized before reorganization.

[0150] In one exemplary embodiment, such as Figure 9 As shown, the user data in each stripe to be reassembled is reassembled to obtain the reassembled data of the disk array, including the following steps:

[0151] S901, obtain the number of user data columns in each strip to be reorganized.

[0152] The number of user data columns refers to the sum of user data columns in all strips to be reorganized.

[0153] The user data blocks in each reassembled strip are used as user data columns for each reassembled strip. Then, the user data columns of each strip to be reassembled are superimposed to obtain the number of user data columns.

[0154] S902, if the number of user data columns is less than or equal to the data column length threshold, then the user data in each strip to be reorganized is determined as the reorganized user data.

[0155] If the number of user data columns is less than or equal to the data column length threshold, it means that the number of all user data columns can be integrated into one stripe. In this case, the user data in each stripe to be reassembled will be combined into one reassembled stripe, and the data in the reassembled stripe will be determined as the reassembled user data of the disk array.

[0156] S903 encodes the recombined user data to obtain recombined verification data.

[0157] EC encoding is performed on each recombined user data to obtain the recombination check data corresponding to each recombined user data. Furthermore, the recombination check data is filled into the recombination strip to which the recombined user data belongs to obtain a complete recombination strip.

[0158] S904: Based on the reorganized user data and reorganization verification data, the reorganization data of the disk array is obtained.

[0159] If the number of user data columns is less than or equal to the data column length threshold, the reconstructed user data and reconstructed verification data are in one reconstructed stripe, and all data in this stripe is identified as the reconstructed data of the disk array.

[0160] In this embodiment of the application, when the number of user data columns is less than or equal to the data column length threshold, each stripe to be reassembled is combined into a reassembled stripe, and the user data in each stripe to be reassembled is encoded to obtain the reassembled verification data of the reassembled stripe, thereby realizing the efficient integration of normal user data in each data stripe in the disk array.

[0161] Below, as in relation to the above Figure 9 The parallel embodiments illustrate another possible way to reorganize user data based on the number of user data columns in each strip to be reorganized before reorganization.

[0162] In one exemplary embodiment, such as Figure 10 As shown, the user data in each stripe to be reassembled is reassembled to obtain the reassembled data of the disk array, including:

[0163] S1001, if the number of user data columns in each strip to be reassembled is less than the data column length threshold, the user data in each strip to be reassembled is divided into a first reassembled user data set and a second reassembled user data set; both the first reassembled user data set and the second reassembled user data set match the data column length threshold.

[0164] If the number of user data columns in each strip to be reassembled is greater than the data column length threshold, the user data columns are divided into a first reassembled user data set and a second reassembled user data set according to the data column length threshold.

[0165] Both the first and second recombined user datasets match the data column length threshold, meaning that the number of user data columns in the first recombined user dataset is no greater than the data column length threshold, and the number of user data columns in the second recombined user dataset is no greater than the data column length threshold.

[0166] Optionally, there may be multiple first recombined user data sets, and the number of user data columns in each first recombined user data set is consistent with the data column length threshold. The number of second recombined user data sets is one, and the number of user data columns in the second recombined user data set is less than the data column length threshold.

[0167] Taking a total of 90 user data sets for each bandwidth to be reorganized, i.e., 80 user data columns, as an example, if the data column length threshold is 20, then the 90 data sets are divided into 4 first reorganized user data sets and one second reorganized user data set. Each first reorganized user data set includes 20 user data sets, and each second reorganized user data set includes 10 user data sets.

[0168] S1002, the user data in the first recombined user data set is encoded to obtain the first recombined verification data, and the user data in the second recombined user data set is encoded to obtain the second recombined verification data.

[0169] Each recombined user dataset corresponds to a recombined stripe.

[0170] EC encoding is performed on the user data in each first recombined user data set to obtain the first recombined check data corresponding to each first recombined user data set, which is the data in the check data column of the recombined stripe corresponding to each recombined user data set.

[0171] The user data in the second recombined user data set is encoded using EC encoding to obtain the second recombined check data corresponding to the second recombined user data set, which is also the data in the check data column of the recombined stripe corresponding to the second recombined user data set.

[0172] S1003, the user data in the first recombined user data set and the user data in the second recombined user data set are determined as recombined user data, and the first recombined verification data and the second recombined verification data are determined as recombined verification data.

[0173] S1004: Obtain the reorganization data of the disk array based on the reorganization user data and the reorganization verification data.

[0174] Use the reconstructed user data and reconstructed verification data as the reconstructed data for the disk array.

[0175] In this embodiment, when the number of user data columns exceeds the data column length threshold, the user data in each stripe to be reassembled is re-divided into a first reassembled user data set and a second reassembled user data set to achieve grouping of user data; then, the user data in the first reassembled user data set is encoded to obtain the first reassembled verification data, and the user data in the second reassembled user data set is encoded to obtain the second reassembled verification data, thus setting valid verification data for the reassembled user data. The reassembly process is divided into two stages: user data reassembly and verification data calculation, thereby achieving standardized processing of non-faulty user data of the disk array.

[0176] In an exemplary embodiment, taking the presence of multiple failed disks in a disk array as an example, a data reconstruction method for a multi-disk failure scenario is provided. The disk array reconstruction task is decomposed into multiple single-disk reconstruction subtasks and a cleanup task. These subtasks and the cleanup task are executed sequentially to complete the data reconstruction of the entire disk array. The method steps for the single-disk reconstruction subtasks and the cleanup task are described below:

[0177] The single-disk rebuild subtask is described below. The process of a single-disk rebuild subtask includes the following steps:

[0178] (1) Obtain all the original stripes to which each data fault block belongs in a single faulty disk to be reconstructed, and process them one by one.

[0179] (2) Based on the position of each original stripe in its original stripe, each data fault block is divided into two categories: data column fault blocks and check column fault blocks.

[0180] (3) For a single data fault block of a data column fault block, allocate a data block from the hot spare space, and obtain the user recovery data corresponding to the data fault block by EC decoding of the undamaged data in the original stripe to which the data fault block belongs; then write the user recovery data into the hot spare space data block; and combine it with the hot spare space data blocks that have been written with other user recovery data (hot spare space data blocks generated by the same operation in other stripes), allocate and calculate EC encoding check data, and form a new stripe. Then, set the reconstructed data column fault block in the original stripe as an invalid data block, and do not allow the data storage system to read it. In this way, when reading the data of the data column fault block in the future, the recovered user recovery data can be read directly.

[0181] (4) For the data fault block of the check column fault block, set the data fault block in the original strip as an invalid data block and do not allow the system to read it.

[0182] (5) Repeat the above steps until all original stripe data reconstruction is completed, release the disk slot where the faulty disk to be reconstructed is located, insert a new hot spare disk, and continue to start the next single disk reconstruction subtask for the next faulty disk to be reconstructed.

[0183] After all the single-disk reconstruction subtasks of the faulty disks to be reconstructed have completed reconstruction, the cleanup task can begin. The cleanup task reconstructs the original stripes processed by all the single-disk reconstruction subtasks. Its purpose is to remove the data fault blocks that were previously set as invalid data blocks and then reassemble them. There are various reassembly schemes, and this application embodiment does not limit them.

[0184] Next, we will describe one possible way to implement the cleanup task, including the following steps:

[0185] (1) For stripes after removing invalid data blocks, release the non-faulty check column data blocks (which can be released after reassembly).

[0186] (2) If the number of remaining data columns m is greater than the data column length threshold M supported by the current system, select (m - M) data blocks into the data block pool to be processed. The remaining M data blocks serve as the new data columns of the original stripe, and apply for N parity column data blocks from the hot spare disk and calculate EC to form a stripe of M + N.

[0187] (3) For the (m - M) data blocks placed in the data block pool to be processed, continue to compare them with the data column length threshold M supported by the current system. Group them in sets of M, and additionally allocate N parity data blocks. After EC encoding calculation, form multiple stripes of M + N; finally, assume there are t remaining data blocks to be processed (t < M), allocate N parity blocks, and after EC calculation, form a stripe of t + N to complete the reconstruction task.

[0188] (4) If the number of remaining data columns m is less than or equal to the data column length threshold M supported by the current system, apply for N parity column data blocks from the hot spare disk, calculate EC and write them to form a stripe of m + N.

[0189] (5) If the number of remaining data columns is 0, directly release the original stripe.

[0190] The embodiment of this application uses the limited hot spare space to preemptively reconstruct the data of the data columns, ensuring the data reading efficiency of the front-end IO during the data reconstruction stage of the data storage system, more quickly completing the reconstruction of a single failed disk, and quickly releasing the slot, so that in a storage system with tight slots and insufficient hot spare space, a new disk can be inserted into the released slot as soon as possible to supplement the hot spare space, thereby ensuring that the reconstruction task can be successfully completed in the multi-disk reconstruction scenario with insufficient hot spare space and improving the reliability of the system.

[0191] Taking the disk array including two failed disks, that is, there are two failed data blocks in a single stripe, and the data column length threshold supported by the current system is 3 columns as an example, the data reconstruction process is divided into three stages: performing two single-disk reconstruction tasks and one finishing task. Taking the original stripe including 4 data columns and 2 parity columns as an example, Figure 11 It shows the reconstruction schematic diagrams in each data reconstruction stage for scenarios where the failed data blocks are of various types.

[0192] Figure 11(a) represents a scenario where two data blocks (data block 3 and data block 4) in the original stripe fail. During the first single-disk reconstruction task, EC decoding is performed on the data in data block 3 to obtain user recovery data, which is then written to the hot spare space. EC encoding is performed on the user recovery data obtained from other stripes to obtain check data, thus forming a new stripe. Data block 3 in the original stripe is set as an invalid data block. During the second single-disk reconstruction task, EC decoding is performed on the data in data block 4 to obtain user recovery data, which is then written to the hot spare space. EC encoding is performed on the user recovery data obtained from other stripes to obtain check data, thus forming a new stripe. Data block 4 in the original stripe is set as an invalid data block. During the cleanup task, data blocks 3 and 4 are removed from the original stripe, and EC is recalculated for data blocks 1 and 2 to obtain check data, resulting in a 2+2 stripe.

[0193] Figure 11 (b) represents a scenario where two data blocks (data block 3 and data block 5) in the original stripe fail. During the first single-disk reconstruction task, the data in data block 3 is EC-decoded to obtain user recovery data, which is then written to the hot spare space. This data is EC-encoded with the user recovery data obtained from other stripes to obtain checksum data, thus forming a new stripe. Data block 3 in the original stripe is set as an invalid data block. During the second single-disk reconstruction task, data block 5 in the original stripe is set as an invalid data block. During the cleanup task, data block 3 is removed from the original stripe, and data blocks 1, 2, and 4 are re-EC-encoded to obtain checksum data, resulting in a 3+2 stripe.

[0194] Figure 11 (c) represents a scenario where two data blocks (data block 4 and data block 5) in the original stripe fail. During the first single-disk reconstruction task, data block 5 in the original stripe is set as an invalid data block. During the second single-disk reconstruction task, the data in data block 4 is EC-decoded to obtain user recovery data, which is written to the hot spare space. This data is then EC-encoded with the user recovery data obtained from other stripes to obtain checksum data, forming a new stripe. Data block 4 in the original stripe is set as an invalid data block. During the cleanup task, data block 4 is removed from the original stripe, and data blocks 1, 2, and 3 are re-EC-encoded to obtain checksum data, resulting in a 3+2 stripe.

[0195] Figure 11(d) represents a scenario where two data blocks (block 5 and block 6) in the original stripe fail. During the first single-disk reconstruction task, block 5 in the original stripe is set as an invalid data block. During the second single-disk reconstruction task, block 6 in the original stripe is set as an invalid data block. During the cleanup task, blocks 1, 2, and 3 are combined into a new stripe and re-EC encoded to obtain a 3+2 stripe; block 4 is combined with user data from other failed disks into a new stripe and re-EC encoded to obtain a 3+2 stripe.

[0196] In this embodiment of the application, during the disk-by-disk data reconstruction process, the data to be reconstructed in the original stripe is marked as invalid data blocks to avoid premature stripe EC encoding and writing, thereby improving the efficiency of single-disk reconstruction. Furthermore, the reconstruction of bad disk data in the parity column is skipped, and the data migration of the data column is prioritized. This ensures that the limited hot spare space is used as much as possible to reconstruct the data column data, avoiding the risk of downgraded data reading due to insufficient hot spare space causing incomplete reconstruction.

[0197] To verify the effectiveness of the data reconstruction method provided in this application embodiment, taking a storage system with a stripe distribution of 4 (data columns) + 2 (parity columns) + 1 (hot spare column), comprising 700 stripes, evenly distributed, and containing two faulty disks as an example, a multi-dimensional comparison is made between the traditional disk-by-disk reconstruction scheme and the data reconstruction method in this embodiment to obtain data comparison results. The traditional disk-by-disk scheme refers to reconstructing the parity column data once for each data column moved during the reconstruction process. This application embodiment, however, first reconstructs the data columns of all faulty disks and then calculates the parity column data once at the end.

[0198] Please refer to Table 1, which is a comparison chart of the experimental results of this scheme and the traditional scheme.

[0199] Table 1

[0200]

[0201] As can be seen from Table 1, compared with the traditional reconstruction scheme, the reconstruction method of this application embodiment occupies less hot spare space, requires less data read / write on the reconstruction disk and less data for EC calculation, and thus has higher reconstruction efficiency. It can be applied to reconstruction scenarios with insufficient hot spare space and multiple disk failures.

[0202] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0203] Based on the same inventive concept, this application also provides a data reconstruction apparatus for implementing the data reconstruction method described above. The solution provided by this apparatus is similar to the implementation scheme described in the above method; therefore, the specific limitations in one or more data reconstruction apparatus embodiments provided below can be found in the limitations of the data reconstruction method described above, and will not be repeated here.

[0204] In one exemplary embodiment, such as Figure 12 As shown, a data reconstruction device is provided, including: a data acquisition module 1201 and a data reconstruction module 1202, wherein:

[0205] The data acquisition module 1201 is used to sequentially reconstruct the data of each failed disk in the disk array to obtain the reconstruction data of each failed disk. During the data reconstruction process, each time the data of a failed disk is reconstructed, the reconstruction data of the corresponding failed disk is stored in the hot spare disk inserted in the slot of the previous failed disk, and the slot of the corresponding failed disk is released for inserting a new hot spare disk.

[0206] The data reconstruction module 1202 is used to reconstruct the data of the disk array based on the reconstruction data of each faulty disk.

[0207] In an exemplary embodiment, the data acquisition module 1201 includes: a first stripe acquisition unit, a fault block determination unit, a data repair unit, and a repair data reconstruction unit, wherein:

[0208] The first stripe acquisition unit is used to acquire the data stripe corresponding to each faulty data block in any faulty disk.

[0209] The fault block determination unit is used to determine the user data fault blocks in the faulty disk based on the position of each faulty data block in the data stripe;

[0210] The data repair unit is used to recover the data in the user data fault block to obtain the user repaired data;

[0211] The data repair and reconstruction unit is used to obtain reconstructed data for the faulty disk based on the user's repair data.

[0212] In an exemplary embodiment, the first stripe acquisition unit includes: an original stripe acquisition subunit and a data block invalidation subunit, wherein:

[0213] The raw stripe acquisition subunit is used to acquire the raw stripe corresponding to each faulty data block in each faulty disk.

[0214] The invalid data block sub-unit is used to set the faulty data blocks in each original stripe as invalid data blocks, thereby obtaining the data stripe corresponding to each faulty data block.

[0215] In an exemplary embodiment, the data repair unit includes: a normal data block acquisition subunit and a repair data acquisition subunit, wherein:

[0216] The normal data block acquisition subunit is used to acquire, for any user data fault block, normal data blocks other than the user data fault block in the data stripe to which the user data fault block belongs;

[0217] The data acquisition subunit is used to decode the data in each normal data block and recover the user repair data corresponding to the user data fault block.

[0218] In one exemplary embodiment, the user-repaired data includes multiple sub-units; the data repair reconstruction unit includes: a data writing sub-unit, a data column combination sub-unit, a data encoding sub-unit, and a data determination sub-unit, wherein:

[0219] The data writing sub-unit is used to write the repair data of each user into the data block of the hot spare disk inserted into the slot of the previous failed disk, resulting in multiple user data columns;

[0220] The data column combination subunit is used to combine each user data column according to the data column length threshold to obtain at least one reconstructed strip;

[0221] The data encoding subunit is used to encode the user repair data in each reconstruction strip to obtain the verification data of each reconstruction strip;

[0222] The data determination subunit is used to determine the user repair data and the verification data of each reconstruction stripe as the reconstruction data of the faulty disk.

[0223] In an exemplary embodiment, the data reconstruction module 1202 includes: a second stripe acquisition unit, a data block removal unit, a data reassembly unit, and a reconstructed data determination unit, wherein:

[0224] The second stripe acquisition unit is used to acquire the data stripe corresponding to each faulty data block in each faulty disk.

[0225] The data block removal unit is used to remove faulty data blocks from each data stripe, resulting in multiple stripes to be reassembled.

[0226] The data reorganization unit is used to reorganize the user data in each stripe to be reorganized to obtain the reorganized data of the disk array;

[0227] The reconstruction data determination unit is used to determine the reconstruction data of the disk array based on the reconstruction data of each failed disk and the reorganization data of the disk array.

[0228] In an exemplary embodiment, the data reconstruction unit includes: a data column acquisition subunit, a reconstructed data determination subunit, a first encoding subunit, and a first reconstruction subunit, wherein:

[0229] The data column acquisition sub-unit is used to obtain the number of user data columns in each strip to be reassembled;

[0230] The data reorganization determination sub-unit is used to determine the user data in each strip to be reorganized as the reorganized user data if the number of user data columns is less than or equal to the data column length threshold.

[0231] The first encoding subunit is used to encode the recombined user data to obtain recombined verification data;

[0232] The first reassembly subunit is used to obtain the reassembly data of the disk array based on the reassembly user data and the reassembly verification data.

[0233] In an exemplary embodiment, the data reassembly unit includes: a data partitioning subunit, a second encoding subunit, a data determination subunit, and a second reassembly subunit, wherein:

[0234] The data partitioning subunit is used to divide the user data in each strip to be reassembled into a first reassembled user data set and a second reassembled user data set if the number of user data columns in each strip to be reassembled is greater than the data column length threshold; both the first reassembled user data set and the second reassembled user data set match the data column length threshold.

[0235] The second encoding subunit is used to encode the user data in the first recombined user data set to obtain the first recombined verification data, and to encode the user data in the second recombined user data set to obtain the second recombined verification data.

[0236] The data determination subunit is used to determine user data in the first recombined user data set and user data in the second recombined user data set as recombined user data, and to determine the first recombined verification data and the second recombined verification data as recombined verification data.

[0237] The second reorganization subunit is used to obtain the reorganization data of the disk array based on the reorganization user data and the reorganization verification data.

[0238] Each module in the aforementioned data reconstruction device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the operations corresponding to each module.

[0239] In one exemplary embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 13 As shown, this computer device includes a processor, memory, input / output interfaces (I / O), and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides the environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The database stores data reconstruction data. The I / O interfaces are used for exchanging information between the processor and external devices. The communication interface is used for communicating with external terminals via a network connection. When the computer program is executed by the processor, it implements a data reconstruction method.

[0240] Those skilled in the art will understand that Figure 13 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0241] In one exemplary embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above-described method embodiments.

[0242] In one exemplary embodiment, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps in the above-described method embodiments.

[0243] In one exemplary embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above-described method embodiments.

[0244] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data must comply with relevant regulations.

[0245] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, artificial intelligence (AI) processors, etc., and are not limited to these.

[0246] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0247] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A data reconstruction method, characterized in that, The method includes: Data reconstruction is performed sequentially on each faulty disk in the disk array to obtain reconstruction data for each faulty disk. During the data reconstruction process, for each faulty disk, the reconstruction data of the corresponding faulty disk is stored in the hot spare disk inserted in the slot of the previous faulty disk, and the slot of the corresponding faulty disk is released for inserting a new hot spare disk. Based on the reconstruction data of each of the faulty disks, the disk array is reconstructed.

2. The method according to claim 1, characterized in that, The step of sequentially reconstructing data for each faulty disk in the disk array to obtain reconstructed data for each faulty disk includes: For any faulty disk, obtain the data stripe corresponding to each faulty data block in the faulty disk; Based on the position of each faulty data block in the data stripe, the user data faulty blocks in the faulty disk are determined; The data in the faulty user data block is recovered to obtain the user repair data; Based on the user's repair data, the reconstruction data of the faulty disk is obtained.

3. The method according to claim 2, characterized in that, The step of obtaining the data stripe corresponding to each faulty data block in the faulty disk includes: Obtain the original stripe corresponding to each faulty data block in each of the faulty disks; The faulty data blocks in each of the original stripes are set as invalid data blocks to obtain the data stripes corresponding to each faulty data block.

4. The method according to claim 2, characterized in that, The step of recovering the data in the faulty user data block to obtain user repair data includes: For any user data fault block, obtain the normal data blocks other than the user data fault block in the data stripe to which the user data fault block belongs; The data in each of the normal data blocks is decoded to recover the user repair data corresponding to the user data fault block.

5. The method according to claim 2, characterized in that, The user repair data includes multiple components; the reconstruction data of the faulty disk obtained based on the user repair data includes: Each user's repair data is written into a data block in the hot spare disk inserted into the slot of the previous failed disk, resulting in multiple user data columns; Based on the data column length threshold, each of the user data columns is combined to obtain at least one reconstructed strip; The user repair data in each of the reconstructed strips is encoded to obtain the verification data of each of the reconstructed strips; The user repair data and the verification data of each of the reconstruction stripes are determined as the reconstruction data of the faulty disk.

6. The method according to any one of claims 1-5, characterized in that, The data reconstruction of the disk array based on the reconstruction data of each of the failed disks includes: Obtain the data stripe corresponding to each faulty data block in each of the faulty disks; Faulty data blocks are removed from each of the data stripes to obtain multiple stripes to be reassembled; The user data in each of the stripes to be reassembled is reassembled to obtain the reassembled data of the disk array; The reconstruction data of the disk array is determined based on the reconstruction data of each of the faulty disks and the reorganization data of the disk array.

7. The method according to claim 6, characterized in that, The reorganization of user data in each of the stripes to be reorganized to obtain reorganized data for the disk array includes: Obtain the number of user data columns in each of the strips to be reassembled; If the number of user data columns is less than or equal to the data column length threshold, then the user data in each of the strips to be reconstructed is determined as reconstructed user data; The recombined user data is encoded to obtain recombined verification data; The reorganization data of the disk array is obtained based on the reorganized user data and the reorganization verification data.

8. The method according to claim 6, characterized in that, The reorganization of user data in each of the stripes to be reorganized to obtain reorganized data for the disk array includes: If the number of user data columns in each strip to be reassembled is greater than the data column length threshold, the user data in each strip to be reassembled is divided into a first reassembled user data set and a second reassembled user data set; both the first reassembled user data set and the second reassembled user data set match the data column length threshold. The user data in the first recombined user data set is encoded to obtain the first recombined verification data, and the user data in the second recombined user data set is encoded to obtain the second recombined verification data; The user data in the first reconstructed user data set and the user data in the second reconstructed user data set are determined as reconstructed user data, and the first reconstructed verification data and the second reconstructed verification data are determined as reconstructed verification data; The reorganization data of the disk array is obtained based on the reorganized user data and the reorganization verification data.

9. A data reconstruction apparatus, characterized in that, The device includes: The data acquisition module is used to sequentially reconstruct the data of each faulty disk in the disk array to obtain the reconstruction data of each faulty disk. During the data reconstruction process, each time the data reconstruction of a faulty disk is performed, the reconstruction data of the corresponding faulty disk is stored in the hot spare disk inserted in the slot of the previous faulty disk, and the slot of the corresponding faulty disk is released for inserting a new hot spare disk. The data reconstruction module is used to reconstruct the disk array based on the reconstruction data of each of the faulty disks.

10. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 8.

11. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 8.

12. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 8.

Images