Fault data repair method and computer equipment

Abstract

The present application relates to a method for repairing faulty data and a computer device. The method comprises: obtaining fault information of faulty disks in different available zones in a distributed storage system; each available zone stores a local check block; the local check block is obtained by encoding the data block and the global check block in the corresponding available zone; based on the fault information, the local check block, the data block and the global check block stored in each available zone, determining the repair data required for the faulty data corresponding to any faulty disk; repairing the faulty data in the corresponding faulty disk by using the repair data. The method can improve the efficiency of repairing faulty data.

Description

Technical Field

[0001] The present application relates to the field of distributed storage technology, and in particular to a method and computer device for repairing faulty data. Background Art

[0002] In a distributed storage system, different data is stored in different disks. Disk failures are very common, and the efficiency of repairing data stored in failed disks is very critical for distributed storage systems.

[0003] In the related art, an erasure code encoding algorithm (for example, a linear block code (Reed-Solomon, RS) algorithm) is used to encode the data to be stored, and the encoded data is stored in different disks. When a disk fails, the erasure code decoding algorithm and the data stored in other disks are used to repair the data in the failed disk.

[0004] However, the data fault repair method in the related art has the problem of low efficiency in repairing faulty data. Summary of the invention

[0005] Based on this, it is necessary to provide a fault data repair method and computer device to improve the efficiency of fault data repair in response to the above technical problems.

[0006] In a first aspect, the present application provides a method for repairing faulty data, comprising:

[0007] Obtaining fault information of faulty disks in different availability zones in a distributed storage system; each availability zone stores a local check block; the local check block is obtained based on the data block and global check block encoding in the corresponding availability zone;

[0008] Determine the repair data required for the fault data corresponding to any faulty disk based on the fault information, the local check blocks, data blocks, and global check blocks stored in each availability zone;

[0009] The faulty data in the corresponding faulty disk is repaired by using each repair data.

[0010] In one embodiment, based on the fault information, the local check blocks, data blocks and global check blocks stored in each available zone, determining the repair data required for the fault data corresponding to any faulty disk includes:

[0011] According to the fault information, the fault data type and the fault data quantity corresponding to the faulty disk in the distributed storage system are obtained; the fault data type includes a faulty local check block, a faulty data block, and a faulty global check block;

[0012] Obtaining non-faulty local check blocks except faulty local check blocks in the local check blocks, non-faulty data blocks except faulty data blocks in the data blocks, and non-faulty global check blocks except faulty global check blocks in the global check blocks stored in each available zone;

[0013] The repair data required for the faulty data corresponding to any faulty disk is determined based on the non-faulty local check blocks, non-faulty data blocks, and non-faulty global check blocks stored in each available zone.

[0014] In one embodiment, determining the repair data required for the fault data corresponding to any faulty disk according to the non-faulty local check blocks, non-faulty data blocks, and non-faulty global check blocks stored in each available zone includes:

[0015] If both the type of fault data and the amount of fault data in the distributed storage system meet the first preset condition, determining the repair data required for the fault data according to the non-faulty local check blocks, non-faulty data blocks, and non-faulty global check blocks stored in the available zone;

[0016] The first preset condition includes:

[0017] The fault data type is any one of a fault local check block, a fault data block and a fault global check block, and the number of fault data in the distributed storage system is one or two.

[0018] In one embodiment, determining the repair data required for the fault data corresponding to any faulty disk according to the non-faulty local check blocks, non-faulty data blocks, and non-faulty global check blocks stored in each available zone includes:

[0019] If both the type of fault data and the amount of fault data in the distributed storage system meet the second preset condition, determining the first repair data corresponding to the faulty data block and / or the faulty global check block according to other available zones outside the available zone and the non-faulty local check blocks, non-faulty data blocks, and non-faulty global check blocks stored in the available zones;

[0020] And, using the non-faulty data blocks, non-faulty global check blocks, non-faulty local check blocks, repair data blocks corresponding to the faulty data blocks and / or repair global check blocks corresponding to the faulty global check blocks stored in the available zone as second repair data corresponding to the faulty local check blocks;

[0021] The second precondition includes:

[0022] The number of faulty data in the distributed storage system is three, the three faulty data are in the same availability zone, and the three faulty data in the availability zone include at least one faulty local check block.

[0023] In one embodiment, repairing the faulty data in the corresponding faulty disk by using the repair data includes:

[0024] Using a global decoding method, predicting a repair data block corresponding to the faulty data block and / or a repair global check block corresponding to the faulty global check block according to the first repair data;

[0025] A local decoding method is used to predict a repaired local check block corresponding to the failed local check block according to the second repair data.

[0026] In one embodiment, determining the repair data required for the fault data corresponding to any faulty disk according to the non-faulty local check blocks, non-faulty data blocks, and non-faulty global check blocks stored in each available zone includes:

[0027] If both the type of fault data and the amount of fault data in the distributed storage system meet the third preset condition, then determine the repair data required for the fault data corresponding to any faulty disk in the available zone based on other available zones outside the available zone and non-faulty data blocks, non-faulty global check blocks, and non-faulty local check blocks stored in the available zone;

[0028] The third preset condition includes at least one of the following:

[0029] The number of fault data in the distributed storage system is at least four, and the fault data type is any one of a fault local check block, a fault data block, and a fault global check block;

[0030] The number of faulty data in the distributed storage system is three, the three faulty data are in the same availability zone, and the three faulty data in the availability zone do not include a faulty local check block.

[0031] In one embodiment, determining the repair data required for the faulty data corresponding to any faulty disk in the available zone according to other available zones outside the available zone and the non-faulty data blocks, non-faulty global check blocks, and non-faulty local check blocks stored in the available zone includes:

[0032] Determine third repair data corresponding to the faulty data block and / or the faulty global check block according to the non-faulty data blocks, non-faulty global check blocks, and non-faulty local check blocks stored in other available zones and available zones;

[0033] And, the non-faulty data blocks, non-faulty global check blocks, non-faulty local check blocks, repair data blocks corresponding to the faulty data blocks and / or repaired global check blocks corresponding to the faulty global check blocks stored in the available area are used as the fourth repair data corresponding to the local check blocks.

[0034] In one embodiment, repairing the faulty data in the corresponding faulty disk by using each repair data includes:

[0035] Using a global decoding method, predicting a repair data block corresponding to the faulty data block and / or a repair global check block corresponding to the faulty global check block according to the third repair data;

[0036] A local decoding method is used to predict a repaired local check block corresponding to a local check block in an available area according to the fourth repair data.

[0037] In one embodiment, the method further comprises:

[0038] Global encoding is performed on multiple data blocks to obtain a global check block; multiple data blocks are obtained by segmenting the data to be stored;

[0039] According to the number of available zones, the global check block and multiple data blocks are divided into different available zones;

[0040] Locally encode the data blocks and global check blocks in each available zone to obtain local check blocks in each available zone;

[0041] The data blocks, global check blocks, and local check blocks in each availability zone are combined and sorted, and then stored in multiple disks in different availability zones in the distributed storage system.

[0042] In a second aspect, the present application also provides a fault data repair device, comprising:

[0043] The acquisition module is used to obtain the fault information of the faulty disks in different available zones in the distributed storage system; each available zone stores a local check block; the local check block is obtained based on the data block and the global check block encoding in the corresponding available zone;

[0044] A determination module, used to determine the repair data required for the fault data corresponding to any faulty disk based on the fault information, the local check blocks, the data blocks and the global check blocks stored in each available zone;

[0045] The repair module is used to repair the faulty data in the corresponding faulty disk by using the repair data.

[0046] In a third aspect, the present application further provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the content of any one embodiment of the fault data repair method in the first aspect is implemented.

[0047] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the content of any one embodiment of the fault data repair method in the first aspect above.

[0048] In a fifth aspect, the present application further provides a computer program product, including a computer program, which, when executed by a processor, implements the content of any one embodiment of the fault data repair method in the first aspect above.

[0049] The above-mentioned fault data repair method and computer device obtain the fault information of the faulty disks in different available zones in the distributed storage system; each available zone stores a local check block; the local check block is obtained based on the data block and the global check block encoding in the corresponding available zone; based on the fault information, the local check block, the data block and the global check block stored in each available zone, the repair data required for the faulty data corresponding to any faulty disk is determined; the faulty data in the corresponding faulty disk is repaired by each repair data. In each available zone in the method, a local check block is added, and the local check block is obtained by encoding the data block and the global check block in the available zone, indicating that the local check block implies the relationship between the data block and the global check block in the available zone. Based on this, when a disk failure occurs, the faulty data corresponding to most of the faulty disks can be repaired by the local check block, the data block and the global check block stored in the available zone where the faulty disk is located, that is, most of the data fault repair does not need to rely on the data of other available zones, which reduces the data transmission between the available zones and can improve the efficiency of faulty data repair. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] FIG1 is a schematic diagram of a fault data repair process in one embodiment;

[0051] FIG2 is a schematic diagram of an application environment of a method for repairing faulty data in one embodiment;

[0052] FIG3 is a schematic diagram of a first flow chart of a method for repairing faulty data in one embodiment;

[0053] FIG4 is a schematic diagram of a second flow chart of a method for repairing faulty data in one embodiment;

[0054] FIG5 is a schematic diagram of a repair process of fault data in an embodiment;

[0055] FIG6 is a schematic diagram of a repair process of two fault data in one embodiment;

[0056] FIG7 is a schematic diagram of a third flow chart of a method for repairing faulty data in one embodiment;

[0057] FIG8 is a schematic diagram of a repair process of three fault data in one embodiment;

[0058] FIG9 is a schematic diagram of a fourth flow chart of a method for repairing faulty data in one embodiment;

[0059] FIG10 is a schematic diagram of a repair process of multiple fault data in one embodiment;

[0060] FIG11 is a schematic diagram of a fifth flow chart of a method for repairing faulty data in one embodiment;

[0061] FIG12 is a schematic diagram of a coding result of a Hitchhiker code in one embodiment;

[0062] FIG13 is a schematic diagram of a division result in one embodiment;

[0063] FIG14 is a schematic diagram of a coding result of a CLAY code in one embodiment;

[0064] FIG15 is a schematic diagram of the correspondence between blocks of an available area and OSDs in one embodiment;

[0065] FIG16 is a schematic diagram of an OSD arrangement method in one embodiment;

[0066] FIG17 is a schematic diagram of a storage process of data to be stored in one embodiment;

[0067] FIG18 is a structural block diagram of a fault data repair device in one embodiment. DETAILED DESCRIPTION

[0068] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0069] With the rapid development of cloud computing technology and the establishment of a new generation of cloud platforms, the demand for data storage is increasing. In the new generation of cloud platforms, the storage system stores data in a distributed manner. For example, the data is divided into multiple data blocks, and each data block is stored in a corresponding disk. When a disk fails, since user data is stored in different disks in a distributed manner, the data in other disks can be used to predict the data in the failed disk, which can provide data fault tolerance.

[0070] Among them, replicas and erasure codes are commonly used fault-tolerance methods in distributed storage systems. Replicas are complete copies of data without additional operations, but the storage efficiency is low. Erasure codes can greatly improve storage efficiency while ensuring high reliability. In distributed storage systems, disk failures are very common, making data repair efficiency very critical to the storage system.

[0071] In the related art, an erasure code encoding algorithm (for example, a linear block code (Reed-Solomon, RS) algorithm) is used to encode the data to be stored, and the encoded data is stored in different disks. When a disk fails, the erasure code decoding algorithm and the data stored in other disks are used to repair the data in the failed disk.

[0072] It should be noted that the distributed storage system includes multiple availability zones, for example, different availability zones can be in different geographical locations. Each availability zone includes multiple disks, and after the data to be stored is divided into multiple data blocks, the multiple data blocks are stored in disks in different availability zones.

[0073] FIG1 is a schematic diagram of a fault data repair process, in which D1-D12 represent data blocks after the data to be stored is segmented, G1-G6 represent global check blocks corresponding to the data blocks, and AZ1-AZ3 represent three available zones. The left side of the arrow in the figure represents a schematic diagram of data storage in a distributed storage system, which includes three available zones AZ1-AZ3, wherein four data blocks D1-D4 and two global check blocks G1-G2 are stored in 6 disks in the available zone AZ1, four data blocks D5-D8 and two global check blocks G3-G4 are stored in 6 disks in the available zone AZ2, and four data blocks D9-D12 and two global check blocks G5-G6 are stored in 6 disks in the available zone AZ3. The right side of the arrow in the figure indicates that the disk corresponding to the data block D8 in the available zone AZ2 is faulty. At this time, it is necessary to predict the faulty data block D8 corresponding to the faulty disk with the help of D1-D4 stored in the available zone AZ1, D9-D12 stored in the available zone AZ3, and D5-D7 stored in the available zone AZ2.

[0074] During the fault data repair process, the related technical methods need to use data blocks stored in different availability zones to repair the data blocks of the faulty disk. Due to the large amount of data transmission between different availability zones and the slow data transmission rate, the fault repair efficiency is low.

[0075] In view of the above problems, the present application provides a faulty data repair method and a computer device, by which the faulty data repair method can improve the efficiency of faulty data repair. The faulty data repair method can be applied to a new generation of cloud platforms, for example, object storage products, file storage products, and block storage products in a distributed storage system. It can also be applied to cloud platforms, for example, object storage products, file storage products, and block storage products in other public clouds, private clouds, and hybrid cloud platforms.

[0076] The fault data repair method provided by the embodiment of the present application can be applied in the application environment as shown in Figure 2. The computer device in the application environment may include a processor, a memory and a network interface connected by a system bus or by wireless. The database of the computer device is used to store data in the fault data repair process. The network interface of the computer device is used to communicate with an external terminal through a network connection, and the computer program is executed by the processor to implement a fault data repair method.

[0077] In an exemplary embodiment, as shown in FIG3 , a method for repairing faulty data is provided, which is described by taking the method applied to the computer device in FIG2 as an example, and includes the following steps 201 to 203. Among them:

[0078] S201, obtaining fault information of faulty disks in different availability zones in a distributed storage system; each availability zone stores a local check block; the local check block is obtained based on the data block and the global check block encoding in the corresponding availability zone.

[0079] The distributed storage system is composed of multiple availability zones, and each availability zone includes multiple disks. A disk can store a data block, a global check block, or a local check block. The failed disk can be one or more disks in any availability zone. When a disk fails, the data in the disk cannot be obtained. The failed disk can store data blocks, global check blocks, or local check blocks. No matter what type of data is stored, the data cannot be obtained.

[0080] In an embodiment of the present application, a computer device can obtain memory information of disks in different availability zones in a distributed storage system at multiple times. If the memory information at multiple times does not change, the disk is determined to be a non-faulty disk. If the memory information at multiple times changes, the disk is determined to be a faulty disk. After multiple faulty disks in different availability zones are determined, the number, type, and location of the faulty disks in each availability zone can be calculated, and this information can be used as fault information of the faulty disks. Alternatively, assuming that the faulty disks in the distributed storage system have been identified as faulty, the computer device can determine the fault information of the faulty disks in different availability zones in the distributed storage system based on the fault identification information of each availability zone. The embodiment of the present application does not limit the method for obtaining fault information of faulty disks in different availability zones in a distributed storage system.

[0081] S202, based on the fault information, the local check blocks, the data blocks and the global check blocks stored in each available zone, determine the repair data required for the fault data corresponding to any faulty disk.

[0082] The repair data required for the fault data refers to basic data used to predict the real data corresponding to the fault data. The repair data may be non-fault data or repaired fault data.

[0083] In an embodiment of the present application, after obtaining the fault information of the fault data in each available zone, the computer device can determine whether the number of faults in the distributed storage system is too large based on the fault information of each available zone. If the number of faults in the distributed storage system is small, the other data in the available zone except the fault data is used as repair data. For example, the available zone includes data blocks D1-D4, global check blocks G1-G2, and local check blocks P1-P2. Assuming that the fault data refers to the fault data block D3, then the data blocks D1, D2 and D4, the global check blocks G1-G2, and the local check blocks P1-P2 can all be used as the repair data required for the fault data block D3.

[0084] Optionally, the computer device may also input the fault information, local check blocks, data blocks and global check blocks stored in each available zone into a preset repair data determination model, and analyze the repair data determination model to determine the repair data required for the fault data corresponding to any faulty disk.

[0085] S203: Repair the faulty data in the corresponding faulty disk using the repair data.

[0086] In an embodiment of the present application, when the amount of faulty data in the distributed storage system is small, the computer device can use the repair data to predict the data in the faulty disk in the available area and replace the faulty data in the faulty disk with the predicted data.

[0087] When the amount of faulty data in the distributed storage system is large, the computer device can use the repair data corresponding to the data block and the global check block to predict the faulty data block and the faulty global check block in the available area, and replace the faulty data block and the faulty global check block with the predicted data. After that, the repaired faulty data block and the repaired faulty global check block are added to the repair data of the local check block, and the faulty local check block in the available area is predicted using the new repair data, and the predicted data replaces the faulty local check block.

[0088] In the above fault data repair method, the fault information of the faulty disks in different available zones in the distributed storage system is obtained; each available zone stores a local check block; the local check block is obtained based on the data block and the global check block encoding in the corresponding available zone; based on the fault information, the local check block, the data block and the global check block stored in each available zone, the repair data required for the faulty data corresponding to any faulty disk is determined; and the faulty data in the corresponding faulty disk is repaired by each repair data. In each available zone in the method, a local check block is added, and the local check block is obtained by encoding the data block and the global check block in the available zone, indicating that the local check block implies the relationship between the data block and the global check block in the available zone. Based on this, when a disk failure occurs, the faulty data corresponding to most of the faulty disks can be repaired by the local check block, the data block and the global check block stored in the available zone where the faulty disk is located, that is, most of the data fault repair does not need to rely on the data of other available zones, which reduces the data transmission between the available zones and can improve the efficiency of faulty data repair.

[0089] Repair data is crucial to the repair process of fault data. Next, an embodiment is used to introduce the process of obtaining the repair data. As shown in FIG4 , the specific content of the repair data required for determining the fault data corresponding to any faulty disk based on the fault information, the local check blocks, the data blocks, and the global check blocks stored in each available zone includes:

[0090] S301, according to the fault information, obtaining the fault data type and fault data quantity corresponding to the faulty disk in the distributed storage system; the fault data type includes a faulty local check block, a faulty data block and a faulty global check block.

[0091] In the embodiment of the present application, after obtaining the fault information of the faulty disk, the computer device can determine the available zone where the faulty disk is located and the type of faulty data corresponding to the faulty disk by analyzing the fault information. Then, the faulty disks in all available zones are combined to determine the type of faulty data and the amount of faulty data corresponding to the faulty disks in the distributed storage system.

[0092] S302, obtaining non-faulty local check blocks except faulty local check blocks in local check blocks, non-faulty data blocks except faulty data blocks in data blocks, and non-faulty global check blocks except faulty global check blocks in global check blocks stored in each available zone.

[0093] In the embodiment of the present application, the repair data of the fault data is non-fault data or repaired fault data. Therefore, in the process of determining the repair data, it is necessary to first obtain the non-fault data in each available zone. The non-fault data may include non-fault data blocks, non-fault global check blocks, and non-fault local check blocks.

[0094] For any available zone, the computer device can obtain other local check blocks except the faulty local check blocks from the local check blocks in the available zone, and use the other local check blocks as non-faulty local check blocks. Obtain other data blocks except the faulty data blocks from the data blocks in the available zone, and use the other data blocks as non-faulty local check blocks. And obtain other global check blocks except the faulty global check blocks from the global check blocks in the available zone, and use the other global check blocks as non-faulty global check blocks.

[0095] S303, determining the repair data required for the fault data corresponding to any faulty disk according to the non-faulty local check blocks, non-faulty data blocks and non-faulty global check blocks stored in each available zone.

[0096] In the embodiment of the present application, if the number of faulty data of the faulty disk in the distributed storage system is small, the non-faulty local check blocks, non-faulty data blocks, and non-faulty global check blocks stored in the available zone where the faulty disk is located are used as the repair data required for the faulty data. If the number of faulty data of the faulty disk in the distributed storage system is large, the non-faulty local check blocks, non-faulty data blocks, and non-faulty global check blocks stored in other available zones and the available zone are used as the repair data required for the faulty data.

[0097] In the above fault data repair method, according to the fault information, the fault data type and the number of fault data corresponding to the faulty disk in the distributed storage system are obtained; the fault data type includes a faulty local check block, a faulty data block and a faulty global check block; the non-faulty local check blocks other than the faulty local check blocks in the local check blocks stored in each available zone, the non-faulty data blocks other than the faulty data blocks in the data blocks and the non-faulty global check blocks other than the faulty global check blocks in the global check blocks are obtained; according to the non-faulty local check blocks, non-faulty data blocks and non-faulty global check blocks stored in each available zone, the repair data required for the faulty data corresponding to any faulty disk is determined. The method can obtain the overall fault situation of the distributed storage system through the fault information in each available zone, so that the repair data required for the faulty data corresponding to each faulty disk can be accurately determined from an overall perspective based on the overall fault situation and non-faulty data.

[0098] In the process of determining the repair data, it is necessary to consider the type and quantity of fault data in the distributed storage system. Different numbers of fault data and different types of fault data require different repair data. Therefore, the following introduces the process of determining the repair data through three different situations.

[0099] First, the case where the number of faulty data in the distributed storage system is one or two and the type of faulty data is not limited is introduced. Then, in one embodiment, the specific content of the repair data required for determining the faulty data corresponding to any faulty disk based on the non-faulty local check blocks, non-faulty data blocks and non-faulty global check blocks stored in each available zone includes:

[0100] If both the type of fault data and the amount of fault data in the distributed storage system meet the first preset condition, determining the repair data required for the fault data according to the non-faulty local check blocks, non-faulty data blocks, and non-faulty global check blocks stored in the available zone;

[0101] The first preset condition includes:

[0102] The fault data type is any one of a fault local check block, a fault data block and a fault global check block, and the number of fault data in the distributed storage system is one or two.

[0103] In the embodiment of the present application, the situation where the amount of faulty data is small may be that the distributed storage system includes one or two faulty disks.

[0104] The most common failure in a distributed storage system is a faulty disk. In this case, it is necessary to avoid the participation of less data, select the minimum amount of data, and avoid data transmission between multiple availability zones as much as possible. This will help improve the efficiency of faulty data repair and reduce resource consumption.

[0105] When there is a faulty disk in the distributed storage system, for the availability zone where the faulty disk is located, the availability zone includes a faulty disk. When the faulty disk corresponds to a faulty data block, the computer device can use half of the data of the non-faulty data blocks, global check blocks, and local check blocks in the availability zone as repair data for the faulty data block. When the faulty disk corresponds to a faulty global check block, the computer device can use half of the data of the data blocks, non-faulty global check blocks, and local check blocks in the availability zone as repair data for the faulty global check block. When the faulty disk corresponds to a faulty local check block, the computer device can use half of the data of the non-faulty local check blocks, data blocks, and global check blocks in the availability zone as repair data for the faulty local check block.

[0106] In the case where the above-mentioned distributed storage system includes a faulty disk, the computer device can use the multi-layer joint (CLAY code) decoding method and repair data to decode the faulty data of the faulty disk and obtain the repaired faulty data corresponding to the faulty disk. FIG5 is a schematic diagram of a repair process of faulty data, which includes four parts, namely the leftmost figure (S11), the figure on the right side of the first arrow (S12), the figure on the right side of the second arrow (S13) and the figure on the right side of the third arrow (S14). It can be seen from S11 that the available area AZ1 includes data blocks D1-D4, global check blocks Q1-Q2 and local check blocks P1-P2. It can be seen from S12 that the available area AZ1 includes a faulty disk, which is the disk corresponding to the faulty data block D1. Then, data blocks D2-D4 are non-faulty data blocks, global check blocks Q1-Q2 are non-faulty global check blocks, and local check blocks P1-P2 are non-faulty local check blocks. S13 uses the data blocks D2-D4, global check blocks Q1-Q2 and local check blocks P1-P2 stored in the availability zone AZ1 as repair data, and repairs the faulty data block D1 through the repair data. S14 refers to the availability zone AZ1 after repair. In addition, it can be seen from the figure that since the availability zone AZ1 includes a faulty disk, only half of the data needs to be repaired during the data repair process of the faulty disk, which reduces the amount of repair data required for faulty data repair and can further improve the efficiency of faulty data repair.

[0107] When the distributed storage system includes two faulty disks, and the two faulty disks are in different availability zones, each of the two availability zones includes a faulty disk. For this situation, the method for obtaining repair data can refer to the content of only one faulty disk in the availability zone, and no further details are given here. For example, referring to FIG. 5 , assuming that data block D1 in availability zone AZ1 and data block D5 in availability zone AZ2 fail at the same time, both the faulty data block D1 and the faulty data block D5 can be repaired by decoding the CLAY code.

[0108] When the distributed storage system includes two faulty disks, and the two faulty disks are in the same available zone, all data in the other disks in the available zone except the two faulty disks can be used as repair data, and the fault repair is performed by decoding the CLAY code. FIG6 is a schematic diagram of the repair process of two faulty data, which also includes four parts, namely the leftmost figure (S21), the figure on the right side of the first arrow (S22), the figure on the right side of the second arrow (S23), and the figure on the right side of the third arrow (S24). It can be seen from S21 that the available zone AZ1 includes data blocks D1-D4, global check blocks Q1-Q2, and local check blocks P1-P2. It can be seen from S22 that the available zone AZ1 includes two faulty disks, which are the disks corresponding to the faulty data blocks D1 and D3. S23 uses the data blocks D2 and D4, global check blocks Q1-Q2, and local check blocks P1-P2 stored in the available zone AZ1 as repair data, and repairs the faulty data blocks D1 and D3 through the repair data. S24 refers to the repaired available zone AZ1. Different from the case where availability zone AZ1 includes one faulty disk, when availability zone AZ1 includes two faulty disks, the entire amount of data to be repaired needs to be involved in the data repair process for the two faulty disks to ensure the accuracy of the data repair.

[0109] In the above-mentioned fault data repair method, if the type of fault data and the amount of fault data in the distributed storage system both meet the first preset condition, the repair data required for the fault data is determined based on the non-fault local check blocks, non-fault data blocks and non-fault global check blocks stored in the available area; wherein the first preset condition includes: the type of fault data is any one of the fault local check blocks, fault data blocks and fault global check blocks, and the amount of fault data in the distributed storage system is one or two. This method is limited by the first preset condition and can accurately determine whether the amount of fault data in the distributed storage system is small. And when the amount of fault data is small, other data in the available area where the fault data is located is used as repair data, that is, in this case, there is no data transmission between available areas, and the repair process is only implemented within the available area, which can improve the subsequent efficiency of fault data repair.

[0110] Next, the case where there are three faulty data in the distributed storage system, the three faulty data are in the same availability zone, and the faulty data type includes at least one faulty local check block is described. In one embodiment, the specific content of the repair data required for the faulty data corresponding to any faulty disk determined based on the non-faulty local check blocks, non-faulty data blocks, and non-faulty global check blocks stored in each availability zone includes:

[0111] If both the type of fault data and the amount of fault data in the distributed storage system meet the second preset condition, determining the first repair data corresponding to the faulty data block and / or the faulty global check block according to other available zones outside the available zone and the non-faulty local check blocks, non-faulty data blocks, and non-faulty global check blocks stored in the available zones;

[0112] And, using the non-faulty data blocks, non-faulty global check blocks, non-faulty local check blocks, repair data blocks corresponding to the faulty data blocks and / or repair global check blocks corresponding to the faulty global check blocks stored in the available zone as second repair data corresponding to the faulty local check blocks;

[0113] The second precondition includes:

[0114] The number of faulty data in the distributed storage system is three, the three faulty data are in the same availability zone, and the three faulty data in the availability zone include at least one faulty local check block.

[0115] In an embodiment of the present application, when there are three faulty data in a distributed storage system, and the three faulty data are in the same availability zone, and the availability zone includes a faulty local check block, the repair data required for the faulty data block, the faulty global check block, and the faulty local check block are completely different. For the faulty data blocks and the faulty global check blocks in the availability zone, since some or all of the local check blocks in the availability zone are faulty local check blocks, it is impossible to predict the faulty data blocks and the faulty global check blocks through the local check blocks. In this case, it is necessary to use the data of other availability zones outside the availability zone to determine the repair data.

[0116] For the faulty local check block in an availability zone, when the faulty data block and the faulty global check block have been repaired, the repaired faulty data block, the repaired faulty global check block, the non-faulty data block, the non-faulty global check block, and the non-faulty local check block in the availability zone are used as the repair data for the faulty local check block.

[0117] Exemplarily, assuming that the faulty data includes the faulty data block and all the faulty local check blocks, then, for the faulty data block, all the data of the non-faulty data blocks in the available zone where the faulty data block is located and half of the data of the non-faulty data blocks in other available zones can be used as the repair data required for the faulty data block. For all the faulty local check blocks, after the faulty data block is repaired, the repaired faulty data block, the non-faulty data blocks and the global check blocks in the available zone where the faulty local check block is located can be used as the repair data.

[0118] In the above-mentioned fault data repair method, if the type of fault data and the number of fault data in the distributed storage system both meet the second preset condition, the first repair data corresponding to the fault data block and / or the fault global check block is determined according to other available zones outside the available zone and the non-fault local check blocks, non-fault data blocks and non-fault global check blocks stored in the available zone; and the non-fault data blocks, non-fault global check blocks, non-fault local check blocks, repair data blocks corresponding to the fault data blocks and / or repair global check blocks corresponding to the fault global check blocks stored in the available zone are used as the second repair data corresponding to the fault local check block; wherein the second preset condition includes: the number of fault data in the distributed storage system is three, the three fault data are in the same available zone, and the three fault data in the available zone include at least one fault local check block. The method is limited by the second preset condition and can accurately determine whether the number of fault data in the distributed storage system is moderate and whether there is a fault local check block. If so, the first repair data corresponding to the fault data block and / or the fault global check block can be accurately obtained with the help of data in other areas outside the available zone and data in the available zone. On this basis, the second repair data corresponding to the faulty local check block can be accurately determined through the repaired data and non-faulty data in the available area.

[0119] Furthermore, after obtaining the first repair data corresponding to the faulty data block and / or the faulty global check block and the second repair data corresponding to the faulty local check block, the faulty data can be repaired using the first repair data and the second repair data. Then, the specific content of repairing the faulty data in the corresponding faulty disk using each repair data is introduced through an embodiment, as shown in FIG7 , and the specific content includes the following steps:

[0120] S401, using a global decoding method, predicting a repair data block corresponding to a faulty data block and / or a repair global check block corresponding to a faulty global check block according to first repair data.

[0121] The global decoding method may be a Hitchhiker decoding method.

[0122] In an embodiment of the present application, in the process of using the first repair data and the second repair data to respectively repair the faulty data blocks and / or the faulty global check blocks and the faulty local check blocks in the available area, it is necessary to first repair the faulty data blocks and / or the faulty global check blocks, and then repair the faulty local check blocks.

[0123] Based on this, for the faulty data block, the computer device can use the Hitchhiker code decoding method to predict the data corresponding to the faulty data block according to the first repair data, and use the predicted data as the repair data block corresponding to the faulty data block. For the faulty global check block, the computer device can also use the Hitchhiker code decoding method to predict the data corresponding to the faulty global check block according to the first repair data, and use the predicted data as the repaired global check block corresponding to the faulty global check block.

[0124] S402: using a local decoding method, predicting a repaired local check block corresponding to the failed local check block according to the second repair data.

[0125] The local decoding method may be a coupled layer coding (CLAY code) decoding method.

[0126] In the embodiment of the present application, for the faulty local check block, the computer device can use the decoding method of the CLAY code to predict the data corresponding to the faulty local check block according to the second repair data, and use the predicted data as the repaired local check block. It should be noted that the second repair data includes the repair data block corresponding to the faulty data block and / or the repaired global check block corresponding to the faulty global check block.

[0127] Exemplarily, FIG8 is a schematic diagram of the repair process of three faulty data, and the repair process includes three parts, namely, the leftmost figure (S31), the figure on the right side of the first arrow (S32), and the figure on the right side of the second arrow (S33). It can be seen from S31 that the available area AZ1 includes a faulty data block D1 and two faulty local check blocks P1-P2. It can be seen from S32 that for the faulty data block D1, the entire amount of data blocks D2-D4 and global check blocks Q1-Q2 in the available area AZ1, the half amount of data blocks D5-D8 in the available area AZ2, and the half amount of data blocks D9-D12 in the available area AZ3 are used as repair data to repair the faulty data block D1. After the repair is completed, it can be seen from S33 that for the faulty local check blocks P1 and P2, the entire amount of data of the repaired faulty data blocks D1, data blocks D2-D4, and global check blocks Q1-Q2 can be used as repair data to repair the faulty local check blocks P1 and P2.

[0128] In the above fault data repair method, a global decoding method is used to predict the repair data block corresponding to the fault data block and / or the repair global check block corresponding to the fault global check block according to the first repair data; and a local decoding method is used to predict the repair local check block corresponding to the fault local check block according to the second repair data. In the fault data repair process, the repair data block and / or the fault global check block are first repaired using the first repair data of the available area and other available areas. On the basis of the repair, based on the repair result and other data in the area, the repair local check block corresponding to the fault local check block can be accurately predicted, thereby improving the accuracy of the repair process.

[0129] Finally, an embodiment is used to illustrate the situation where there are a large number of faulty data in a distributed storage system. The specific content of the repair data required for determining the faulty data corresponding to any faulty disk based on the non-faulty local check blocks, non-faulty data blocks, and non-faulty global check blocks stored in each available zone includes:

[0130] If both the type of fault data and the amount of fault data in the distributed storage system meet the third preset condition, then determine the repair data required for the fault data corresponding to any faulty disk in the available zone based on other available zones outside the available zone and non-faulty data blocks, non-faulty global check blocks, and non-faulty local check blocks stored in the available zone;

[0131] The third preset condition includes at least one of the following:

[0132] The number of fault data in the distributed storage system is at least four, and the fault data type is any one of a fault local check block, a fault data block, and a fault global check block;

[0133] The number of faulty data in the distributed storage system is three, the three faulty data are in the same availability zone, and the three faulty data in the availability zone do not include a faulty local check block.

[0134] In an embodiment of the present application, when the number of faulty data in the distributed storage system is large, it is also necessary to first repair the faulty data blocks and / or the faulty global check blocks. After the repair is completed, the local check blocks are repaired based on the repair results, non-faulty data blocks and non-faulty global check blocks in the available area. It should be noted that regardless of whether the local check blocks are faulty, all local check blocks need to be repaired using the decoding method of the CLAY code.

[0135] Based on this, when determining the third repair data corresponding to the faulty data block and / or the faulty global check block and the fourth repair data corresponding to the local check block, in one embodiment, the third repair data corresponding to the faulty data block and / or the faulty global check block can be determined based on other available zones and non-faulty data blocks, non-faulty global check blocks and non-faulty local check blocks stored in the available zones; and the non-faulty data blocks, non-faulty global check blocks, non-faulty local check blocks, repair data blocks corresponding to the faulty data blocks and / or repaired global check blocks corresponding to the faulty global check blocks stored in the available zones are used as the fourth repair data corresponding to the local check blocks.

[0136] In the above-mentioned fault data repair method, if the type of fault data and the number of fault data in the distributed storage system both meet the third preset condition, the third repair data corresponding to the fault data block and / or the fault global check block is determined according to the non-fault data blocks, non-fault global check blocks and non-fault local check blocks stored in other available zones and available zones; and the repair data blocks corresponding to the non-fault data blocks, non-fault global check blocks, non-fault local check blocks, fault data blocks and / or the repair global check blocks corresponding to the fault global check blocks stored in the available zones are used as the fourth repair data corresponding to the local check blocks; wherein the third preset condition includes at least one of the following: the number of fault data in the distributed storage system is at least four, and the type of fault data is any one of the fault local check blocks, the fault data blocks and the fault global check blocks; the number of fault data in the distributed storage system is three, the three fault data are in the same available zone, and the three fault data in the available zone do not include the fault local check blocks. The method is limited by the third preset condition and can accurately determine whether the number of fault data in the distributed storage system is large. If so, the third repair data corresponding to the faulty data block and / or the faulty global check block can be accurately obtained by using the data in other areas outside the available zone and the data in the available zone. On this basis, the fourth repair data corresponding to the local check block can be accurately determined by the repaired data and non-faulty data in the available zone.

[0137] Further, after obtaining the third repair data corresponding to the faulty data block and / or the faulty global check block and the fourth repair data corresponding to the local check block, the third repair data and the fourth repair data can be used to repair the faulty data. Then, the specific content of repairing the faulty data in the corresponding faulty disk by using each repair data is introduced through an embodiment, as shown in FIG9 , and the specific content includes the following steps:

[0138] S501 , using a global decoding method, predicting a repair data block corresponding to a faulty data block and / or a repair global check block corresponding to a faulty global check block according to third repair data.

[0139] Exemplarily, FIG10 shows a schematic diagram of a repair process of multiple faulty data, and the repair process includes four parts, namely, the upper left figure (S41), the upper right figure (S42), the lower right figure (S43) and the lower left figure (S44). It can be seen from S41 that the distributed storage system includes four faulty disks, and the faulty data corresponding to the four faulty disks are the faulty data blocks D3 and D4 in the available zone AZ1, the faulty data block D6 in the available zone AZ2, and the faulty data block D11 in the available zone AZ3. For multiple faulty data blocks in the distributed storage system, the faulty data of sub-strip a in each available zone can be repaired by decoding the RS code, and the data in S42 is the content after the faulty data of sub-strip a is repaired. Then, the data in each available zone is calculated by an XOR operation, the XOR operation is canceled, and then the faulty data of sub-strip b is repaired by decoding the RS code. The data in S43 is the content after the faulty data of sub-strip b is repaired. It can be understood that the repair method of the faulty global check block is exactly the same as that of the faulty data block.

[0140] S502: Using a local decoding method, predict a repaired local check block corresponding to a local check block in an available area according to fourth repair data.

[0141] In the embodiment of the present application, referring to FIG. 10 , after the faulty data block in each available area is repaired, the decoding method of the CLAY code is used to repair all the local check blocks in the available area according to the fourth repair data to obtain a repaired local check block. S44 represents the content after the local check block is repaired. It should be noted here that regardless of whether the local check block has a fault, it needs to be repaired using the decoding method of the CLAY code.

[0142] In the above-mentioned fault data repair method, a global decoding method is used to predict the repair data block corresponding to the fault data block and / or the repair global check block corresponding to the fault global check block according to the third repair data; and a local decoding method is used to predict the repair local check block corresponding to the local check block in the available area according to the fourth repair data. In the process of repairing the fault data, the repair data block and / or the fault global check block are first repaired using the third repair data of the available area and other available areas. On the basis of the repair, based on the repair result and other data in the area, the repair local check block corresponding to the local check block can be accurately predicted, thereby improving the accuracy of the repair process.

[0143] The above embodiments are all for describing the situation where a disk failure has occurred in an available zone in a distributed storage system, that is, multiple data have been stored in various disks. Next, an embodiment is used to describe how data is stored in multiple disks. As shown in FIG11 , the method further includes:

[0144] S601, globally encode multiple data blocks to obtain a global check block; the multiple data blocks are obtained by segmenting the data to be stored.

[0145] In an embodiment of the present application, after the data to be stored is acquired, the data to be stored may be split once according to stripes, and after the splitting is completed, an encoding operation is performed on each stripe of data. After the encoding is completed, each stripe of data continues to be split to obtain multiple data blocks, and then the multiple data blocks are encoded in turn. For example, when the stripe size is set to 1MB and the data to be stored is 4MB, one split can split the data to be stored into 4 stripes of data. For example, the 4 stripes of data are A, B, C, and D. Taking stripe data D as an example, a second split can split stripe data D into 12 data blocks, namely data blocks D1-D12.

[0146] Furthermore, after obtaining multiple data blocks, the computer device can encode the multiple data blocks using a global encoding method (i.e., an encoding method of a Hitchhiker code), divide the stripe data into two sub-stripes, respectively recorded as sub-strip a and sub-strip b, and evenly divide the k data blocks of sub-strip a into mg-2 groups, where m represents the total number of check blocks and g represents the total number of global check blocks. Then, the data blocks in these available areas are XORed to obtain global check blocks.

[0147] FIG12 is a schematic diagram showing the encoding result of the Hitchhiker code, in which the left side is a plurality of data blocks, including sub-strip a and sub-strip b, and the right side is a global check block generated by an XOR operation.

[0148] S602: Divide the global check block and the multiple data blocks into different available zones according to the number of available zones.

[0149] In an embodiment of the present application, a computer device can divide data blocks and global check blocks into different available areas according to the order of multiple data blocks and the order of global check blocks. Exemplarily, assuming that the number of available areas is 3, the number of global check blocks is 6, and the number of data blocks is 12, four data blocks and two global check blocks can be divided into a group. Figure 13 is a schematic diagram of the division result, which includes three available areas, namely, available area AZ1, available area AZ2 and available area AZ3, available area AZ1 includes four data blocks D1-D4 and two global check blocks f1 (a), f1 (b) and f2 (a), f2 (b), available area AZ2 includes four data blocks D5-D8 and two global check blocks f3 (a), f3 (b) and f4 (a), f4 (b), available area AZ3 includes four data blocks D9-D12 and two global check blocks f5 (a), f5 (b) and f6 (a), f6 (b).

[0150] S603: Locally encode the data blocks and global check blocks in each available area to obtain local check blocks in each available area.

[0151] In the embodiment of the present application, the computer device can locally encode the data blocks and global check blocks in each available area using a local encoding method (i.e., the encoding method of the CLAY code), use the data blocks and global check blocks in each available area as input blocks of the CLAY code, and output p / z local check blocks after encoding. Where p represents the number of all local check blocks, and z represents the number of available areas.

[0152] FIG14 is a schematic diagram showing the encoding result of the CLAY code. It can be seen from the figure that after the data block and the global check block in the available area AZ1 are encoded by the CLAY code, the local check codes P1 and P2 in the available area AZ1 are obtained.

[0153] S604: After combining and sorting the data blocks, global check blocks and local check blocks in each available zone, the data blocks are stored in multiple disks in different available zones in the distributed storage system.

[0154] In the embodiment of the present application, after the CLAY code is used to complete the encoding, there is a logical grouping relationship within each available area. However, for a distributed storage system, it is necessary to integrate each available area into the same data structure (Map), in which the code of each available area is used as the key and the data of each block is used as the value. When each available area is merged into the Map, the data block D, the global check block Q and the local check block P are taken out according to the number of the available area, and then the blocks after the combined sorting are stored in multiple disks in different available areas.

[0155] The above combined sorting process can use the interval coding (Ordered Statistics Decoder, OSD) method to encode the blocks in each available zone in sequence. The data blocks, global check blocks and local check blocks all need to be distributed according to the coding logic.

[0156] Figure 15 is a schematic diagram showing the correspondence between blocks in available zones and OSDs. It can be seen from the figure that data blocks D1-D4, global check blocks Q1-Q2 and local check blocks P1-P2 in available zone AZ1, data blocks D5-D8, global check blocks Q3-Q4 and local check blocks P3-P4 in available zone AZ2, data blocks D9-D12, global check blocks Q5-Q6 and local check blocks P5-P6 in available zone AZ3 correspond to OSD1-OSD24 respectively.

[0157] In order to store the corresponding numbered blocks in the corresponding devices, the OSD list needs to be rearranged when calculating the OSD list. FIG16 is a schematic diagram of the OSD arrangement method, in which the strip data D is divided into 12 data blocks, namely D1-D12, and the global check blocks Q1-Q6 are obtained by the global coding method, and the local check blocks P1-P6 are obtained by the local coding method. Then, according to the corresponding relationship in FIG15, the OSD corresponding to each block is determined.

[0158] FIG17 is a schematic diagram showing the storage process of the data to be stored. After the stripe data D is obtained, the stripe data D is segmented to obtain data blocks D1-D12. The multiple data blocks D1-D12 are globally encoded to obtain global check blocks Q1-Q6. According to the number of available zones, the global check blocks and multiple data blocks are divided into different available zones, that is, the data blocks D1-D4 and the global check blocks Q1-Q2 are divided into the available zone DZ1, the data blocks D5-D8 and the global check blocks Q3-Q4 are divided into the available zone DZ2, and the data blocks D9-D12 and the global check blocks Q5-Q6 are divided into the available zone DZ3. The data blocks and the global check blocks in each available zone are locally encoded to obtain the local check blocks in each available zone, that is, the available zone DZ1 includes the local check blocks P1-P2, the available zone DZ2 includes the local check blocks P3-P4, and the available zone DZ3 includes the local check blocks P5-P6. The data blocks, global check blocks, and local check blocks in each availability zone are combined and sorted, and then stored in multiple disks in different availability zones in the distributed storage system.

[0159] In the above fault data repair method, multiple data blocks are globally encoded to obtain a global check block; multiple data blocks are obtained by data segmentation of the data to be stored; the global check block and multiple data blocks are divided into different available zones according to the number of available zones; the data blocks and global check blocks in each available zone are locally encoded to obtain local check blocks in each available zone; the data blocks, global check blocks and local check blocks in each available zone are combined and sorted, and then stored in multiple disks in different available zones in the distributed storage system. In the encoding process, this method first performs global encoding and then divides the available zones, so that the global check block can be accurately obtained, and then the local check block can be accurately determined based on the multiple data blocks and the global check block. By combining and sorting the multiple data blocks, the global check block and the local check block, each block can be accurately stored in different disks.

[0160] In an exemplary embodiment, as shown in FIG18 , a fault data repair device is provided, comprising: an acquisition module 11, a determination module 12 and a repair module 13, wherein:

[0161] The acquisition module 11 is used to obtain the fault information of the faulty disks in different available zones in the distributed storage system; each available zone stores a local check block; the local check block is obtained based on the data block and the global check block encoding in the corresponding available zone;

[0162] A determination module 12, configured to determine the repair data required for the fault data corresponding to any faulty disk based on the fault information, the local check blocks, the data blocks and the global check blocks stored in each available zone;

[0163] The repair module 13 is used to repair the faulty data in the corresponding faulty disk by using the repair data.

[0164] In an exemplary embodiment, the determination module includes a first acquisition unit, a second acquisition unit and a determination unit, wherein:

[0165] A first acquisition unit is used to acquire the fault data type and fault data quantity corresponding to the faulty disk in the distributed storage system according to the fault information; the fault data type includes a faulty local check block, a faulty data block and a faulty global check block;

[0166] A second acquisition unit is used to acquire non-faulty local check blocks except faulty local check blocks in the local check blocks stored in each available zone, non-faulty data blocks except faulty data blocks in the data blocks, and non-faulty global check blocks except faulty global check blocks in the global check blocks;

[0167] The determination unit is used to determine the repair data required for the fault data corresponding to any faulty disk according to the non-faulty local check blocks, non-faulty data blocks and non-faulty global check blocks stored in each available zone.

[0168] In an exemplary embodiment, the above-mentioned determination unit is also used to determine the repair data required for the fault data based on the non-fault local check blocks, non-fault data blocks and non-fault global check blocks stored in the available area when the fault data type and the fault data quantity in the distributed storage system both meet the first preset condition; wherein the first preset condition includes: the fault data type is any one of the fault local check blocks, fault data blocks and fault global check blocks, and the number of fault data in the distributed storage system is one or two.

[0169] In an exemplary embodiment, the above-mentioned determination unit is also used to determine the first repair data corresponding to the faulty data block and / or the faulty global check block based on other available zones outside the available zone and the non-faulty local check blocks, non-faulty data blocks and non-faulty global check blocks stored in the available zone when both the type of faulty data and the number of faulty data in the distributed storage system meet the second preset condition; and use the non-faulty data blocks, non-faulty global check blocks, non-faulty local check blocks, repair data blocks corresponding to the faulty data blocks and / or repair global check blocks corresponding to the faulty global check blocks stored in the available zone as the second repair data corresponding to the faulty local check block; wherein the second preset condition includes: the number of faulty data in the distributed storage system is three, the three faulty data are in the same available zone, and the three faulty data in the available zone include at least one faulty local check block.

[0170] In an exemplary embodiment, the above-mentioned repair module includes a first global repair unit and a first local repair unit, wherein:

[0171] A first global repair unit, configured to predict, by using a global decoding method, a repair data block corresponding to a faulty data block and / or a repair global check block corresponding to a faulty global check block according to first repair data;

[0172] The first local repair unit is used to predict the repaired local check block corresponding to the faulty local check block according to the second repair data by using a local decoding method.

[0173] In an exemplary embodiment, the above-mentioned determination unit is also used to determine the repair data required for the fault data corresponding to any faulty disk in the available zone based on other available zones outside the available zone and the non-faulty data blocks, non-faulty global check blocks and non-faulty local check blocks stored in the available zone, when both the type of faulty data and the number of faulty data in the distributed storage system meet the third preset condition; wherein the third preset condition includes at least one of the following: the number of faulty data in the distributed storage system is at least four, and the type of faulty data is any one of a faulty local check block, a faulty data block and a faulty global check block; the number of faulty data in the distributed storage system is three, the three faulty data are in the same available zone, and the three faulty data in the available zone do not include a faulty local check block.

[0174] In an exemplary embodiment, the determination unit is further configured to determine third repair data corresponding to the faulty data block and / or the faulty global check block based on the non-faulty data blocks, non-faulty global check blocks, and non-faulty local check blocks stored in other available zones and available zones;

[0175] And, the non-faulty data blocks, non-faulty global check blocks, non-faulty local check blocks, repair data blocks corresponding to the faulty data blocks and / or repaired global check blocks corresponding to the faulty global check blocks stored in the available area are used as the fourth repair data corresponding to the local check blocks.

[0176] In an exemplary embodiment, the above-mentioned repair module further includes: a second global repair unit and a second local repair unit, wherein:

[0177] A second global repair unit, configured to predict, by using a global decoding method, a repair data block corresponding to the faulty data block and / or a repaired global check block corresponding to the faulty global check block according to the third repair data;

[0178] The second local repair unit is used to predict the repaired local check block corresponding to the local check block in the available area according to the fourth repair data by using a local decoding method.

[0179] In an exemplary embodiment, the above-mentioned fault data repair device further includes: a global encoding module, a partitioning module, a local encoding module and a storage module, wherein:

[0180] A global encoding module is used to globally encode multiple data blocks to obtain a global check block; the multiple data blocks are obtained by segmenting the data to be stored;

[0181] A partitioning module is used to partition the global check block and multiple data blocks into different availability zones according to the number of availability zones;

[0182] A local encoding module is used to locally encode the data blocks and global check blocks in each available zone to obtain a local check block in each available zone;

[0183] The storage module is used to combine and sort the data blocks, global check blocks and local check blocks in each available zone, and store them in multiple disks in different available zones in the distributed storage system.

[0184] In an exemplary embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, the content of any one embodiment of the above-mentioned fault data repair method is implemented.

Claims

1. A method for repairing faulty data, characterized in that: The method comprises: Obtaining fault information of faulty disks in different available zones in a distributed storage system; each available zone stores a local check block; the local check block is obtained based on the data block and the global check block encoding in the corresponding available zone; According to the fault information, the fault data type and the fault data quantity corresponding to the faulty disk in the distributed storage system are obtained; the fault data type includes a faulty local check block, a faulty data block and a faulty global check block; Acquire non-faulty local check blocks other than faulty local check blocks in the local check blocks stored in each available zone, non-faulty data blocks other than faulty data blocks in the data blocks, and non-faulty global check blocks other than faulty global check blocks in the global check blocks; If the type of fault data and the amount of fault data in the distributed storage system both meet the first preset condition, the second preset condition or the third preset condition, determining the repair data required for the fault data corresponding to any faulty disk according to the non-faulty local check blocks, non-faulty data blocks and non-faulty global check blocks stored in each available area; The first preset condition includes: the fault data type is any one of a fault local check block, a fault data block and a fault global check block, and the number of fault data in the distributed storage system is one or two; The second preset condition includes: the number of faulty data in the distributed storage system is three, the three faulty data are in the same availability zone, and the three faulty data in the availability zone include at least one faulty local check block; The third preset condition includes at least one of the following: the number of fault data in the distributed storage system is at least four, and the fault data type is any one of a fault local check block, a fault data block, and a fault global check block; the number of fault data in the distributed storage system is three, the three fault data are in the same available zone, and the three fault data in the available zone do not include a fault local check block; The faulty data in the corresponding faulty disk is repaired by using the repair data.

2. The method according to claim 1, characterized in that The repair data refers to basic data used to predict real data corresponding to the fault data.

3. The method according to claim 1, characterized in that If the type of fault data and the amount of fault data in the distributed storage system both meet a first preset condition; The determining, based on the non-faulty local check blocks, non-faulty data blocks, and non-faulty global check blocks stored in each available zone, the repair data required for the faulty data corresponding to any faulty disk comprises: The repair data required for the faulty data is determined according to the non-faulty local check blocks, non-faulty data blocks and non-faulty global check blocks stored in the available area.

4. The method according to claim 1, characterized in that: If the type of fault data and the amount of fault data in the distributed storage system both meet a second preset condition; The determining, based on the non-faulty local check blocks, non-faulty data blocks, and non-faulty global check blocks stored in each available zone, the repair data required for the faulty data corresponding to any faulty disk comprises: The first repair data corresponding to the faulty data block and / or the faulty global check block is determined according to other available zones outside the available zone and non-faulty local check blocks, non-faulty data blocks and non-faulty global check blocks stored in the available zone.

5. The method according to claim 4, characterized in that The repairing of the faulty data in the corresponding faulty disk by using the repair data includes: Using a global decoding method, predicting a repair data block corresponding to the faulty data block and / or a repair global check block corresponding to the faulty global check block according to the first repair data; A local decoding method is used to predict a repaired local check block corresponding to the failed local check block according to the second repair data.

6. The method according to claim 1, characterized in that If the type of fault data and the amount of fault data in the distributed storage system both meet a third preset condition; The determining, based on the non-faulty local check blocks, non-faulty data blocks, and non-faulty global check blocks stored in each available zone, the repair data required for the faulty data corresponding to any faulty disk comprises: Determine the repair data required for the faulty data corresponding to any faulty disk in the available zone according to other available zones outside the available zone and the non-faulty data blocks, non-faulty global check blocks and non-faulty local check blocks stored in the available zone.

7. The method according to claim 6, characterized in that The step of determining the repair data required for the fault data corresponding to any faulty disk in the available zone according to other available zones outside the available zone and the non-faulty data blocks, non-faulty global check blocks, and non-faulty local check blocks stored in the available zone comprises: Determine third repair data corresponding to the faulty data block and / or the faulty global check block according to the other available zones and the non-faulty data blocks, the non-faulty global check blocks, and the non-faulty local check blocks stored in the available zone; And, the non-faulty data blocks, non-faulty global check blocks, non-faulty local check blocks, repair data blocks corresponding to the faulty data blocks and / or repaired global check blocks corresponding to the faulty global check blocks stored in the available area are used as the fourth repair data corresponding to the local check blocks.

8. The method according to claim 7, characterized in that The repairing of the faulty data in the corresponding faulty disk by using the repair data includes: Using a global decoding method, predicting a repair data block corresponding to the faulty data block and / or a repair global check block corresponding to the faulty global check block according to the third repair data; A local decoding method is used to predict a repaired local check block corresponding to the local check block in the available area according to the fourth repair data.

9. The method according to any one of claims 1 to 8, characterized in that: The method further comprises: Performing global encoding on multiple data blocks to obtain a global check block; the multiple data blocks are obtained by segmenting the data to be stored; According to the number of available zones, the global check block and the multiple data blocks are divided into different available zones; Locally encode the data blocks and global check blocks in each available zone to obtain local check blocks in each available zone; After the data blocks, global check blocks and local check blocks in each available zone are combined and sorted, they are stored in multiple disks in different available zones in the distributed storage system.

10. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 9 are implemented.

Images