A fully distributed repair method based on erasure codes

Abstract

The invention discloses a fully distributed repair method based on erasure codes, aiming at improving the repair efficiency of failure data. The technical proposal is to construct a distributed storage system composed of a control node and a storage node, and the control node is provided with a data writing management program and a data repairing management program. A data writing program and a data repairing program are installed on the storage node; The data writing management program divides the data object into blocks and calculates the encoding, and writes the original data block and theredundant data block into different storage nodes. The data repair management program selects a new storage node LN for the failed data block D'i. The data repair management program and the data repair program cooperate with each other, and after decoding calculation, the data sub-blocks which have been repaired successfully are obtained and stored in the LN. The data repair program of LN combinesthe data sub-blocks to get the D'i recovered Di. The invention can fully utilize the uplink bandwidth of each storage node, effectively avoid the bottleneck bandwidth in the network, and improve therepair efficiency of invalid data.

Description

technical field [0001] The invention relates to the data fault tolerance technology of the distributed storage system, in particular to the fully distributed repair method based on the erasure correction code. Background technique [0002] Data fault tolerance is one of the key technologies for distributed storage of massive data, and data fault tolerance is crucial to improving system availability and reliability. Currently, data fault-tolerant technologies mainly include fault-tolerant technologies based on replication and fault-tolerant technologies based on erasure codes. [0003] The fault-tolerant technology based on replication is simple and intuitive, easy to implement and deploy, and needs to create several copies of the same size for each data object, and the storage space overhead is relatively large. The error-tolerant technology based on erasure code divides the data object into multiple data blocks, uses the erasure code algorithm to encode the original data b...

AI Technical Summary

Problems solved by technology

Therefore, the traditional repair method selects k available data blocks from k+m-1 available data blocks, uploads data from the storage nodes where the k available data blocks are located, and only uses the uplink of the storage nodes where the k available data blocks are located. Bandwidth, does not make full use of the uplink bandwidth of the storage nodes where all k+m-1 available data blocks are located for fully distributed repair, and its repair efficiency is often limited by the uplink bandwidth of the storage nodes where the k available data blocks are located

Images