Erasure code based failure data linear restoration method

Abstract

The invention discloses an erasure code based failure data linear restoration method for the purpose of designing a failure data restoration method aiming at erasure code characteristics. The scheme involves constructing a distribution storage system composed of a control node and N storage nodes. The control node is provided with a task management program and a result recovery program and the storage nodes are provided with restoration programs; the task management program selects a new storage node LN for a failure data block; an available data block is selected from available data blocks of a data object to construct a linear restoration path; the restoration programs perform decoding calculating and transmit and merge decoding results along the linear restoration path; and the restoration program of the LN receives a decoding calculation result from the last storage node of the linear restoration path, and the result recovery program receives the restoration success information of the LN. By using the method provided by the invention, the bottleneck bandwidth in a network can be effectively avoided, the transmission distance of data restoration is shortened, the bandwidth cost of failure data restoration is reduced, and the restoration efficiency of failure data is improved.

Description

technical field [0001] The invention relates to a data fault-tolerant method of a distributed storage system, in particular to a method for repairing invalid data based on an erasure code. Background technique [0002] Data fault tolerance is one of the key technologies for distributed storage of massive data, and data fault tolerance is crucial to improving system availability and reliability. Currently, data fault-tolerant technologies mainly include fault-tolerant technologies based on replication and fault-tolerant technologies based on erasure codes. [0003] The fault-tolerant technology based on replication is simple and intuitive, easy to implement and deploy, and needs to create several copies of the same size for each data object, and the storage space overhead is relatively large. The error-tolerant technology based on erasure code divides the data object into multiple data blocks, and encodes the original data block by using the erasure code algorithm to obtain ...

AI Technical Summary

Problems solved by technology

[0014] The bandwidth cost of the star repair method is high, and the repair time is limited by the bottleneck bandwidth between the new storage node and the available storage node of the expired data block

Compared with the star repair method, the tree repair method can effectively avoid the bottleneck bandwidth between the new storage node of the invalid data block and the available storage node; however, the repair efficiency of the tree repair method is greatly affected by the spanning tree structure, It is prone to high repair bandwidth costs, and its repair time is still limited by the bottleneck bandwidth between the parent node and the child node in the spanning tree

Images