A Linear Repair Method of Invalid Data Based on Erasure Code

Abstract

The invention discloses a linear repair method for invalid data based on an erasure code, and aims to design a repair method for invalid data aiming at the characteristics of the erasure code. The scheme is to build a distributed storage system consisting of a control node and N storage nodes. The control node is equipped with a task management program and a result recovery program, and the storage node is equipped with a repair program; the task management program selects a new storage node LN for the invalid data block; Select available data blocks from the available data blocks of the data object to construct a linear repair path; the repair program performs decoding calculations, and transmits and merges the decoding calculation results along the linear repair path; the LN repair program receives data from the last storage node of the linear repair path The calculation result is decoded, and the result recovery program receives the repair success information of the LN. The invention can effectively avoid the bottleneck bandwidth in the network, shorten the transmission distance of repair data, reduce the bandwidth cost of invalid data repair, and improve the repair efficiency of invalid data.

Description

technical field [0001] The invention relates to a data fault-tolerant method of a distributed storage system, in particular to a method for restoring failed data based on erasure codes. Background technique [0002] Data fault tolerance is one of the key technologies for distributed storage of massive data. Data fault tolerance is very important to improve the availability and reliability of the system. At present, data fault-tolerant technologies mainly include replication-based fault-tolerant technologies and erasure-code-based fault-tolerant technologies. [0003] The replication-based fault-tolerant technology is simple and intuitive, easy to implement and deploy. It needs to create several copies of the same size for each data object, and the storage space overhead is large. The fault-tolerant technology based on erasure code divides the data object into multiple data blocks, and obtains redundant data blocks by using the erasure code algorithm to encode the original d...

AI Technical Summary

Problems solved by technology

[0014] The bandwidth cost of the star repair method is high, and the repair time is limited by the bottleneck bandwidth between the new storage node and the available storage node of the expired data block

Compared with the star repair method, the tree repair method can effectively avoid the bottleneck bandwidth between the new storage node of the invalid data block and the available storage node; however, the repair efficiency of the tree repair method is greatly affected by the spanning tree structure, It is prone to high repair bandwidth costs, and its repair time is still limited by the bottleneck bandwidth between the parent node and the child node in the spanning tree

Images