Distributed block storage system based on erasure codes

Abstract

The invention relates to a distributed block storage system based on erasure codes, which comprises a block access client, a metadata service cluster and a storage server cluster; the block access client receives an upper-layer application request and converts the upper-layer application request into a read-write request for a rear-end storage object based on an erasure code configuration strategy; the storage server cluster comprises a plurality of storage service nodes and is used for receiving and responding to the read-write request; the metadata service cluster monitors the states of all storage service nodes to form a cluster state; and each storage service node performs data distribution by adopting a three-layer distributed Hash algorithm logic. Compared with the prior art, the method has the advantages that the throughput performance which is not output to a copy storage system can be provided, so that the influence on the read-write request performance of the front-end application in abnormal scenes such as node / disk faults and capacity expansion is very small.

Description

technical field [0001] The invention relates to a distributed block storage system, in particular to a distributed block storage system based on erasure codes. Background technique [0002] In a distributed block storage system, a multi-copy storage method is generally adopted, that is, the same data is stored in multiple copies in different fault domains. By improving the data redundancy method, the data security caused by the unreliability of general hardware is solved. But there is also the problem of low storage space utilization. For example, distributed block storage in the market generally adopts a three-copy strategy to store data, and its space utilization rate is only 1 / 3. In contrast, erasure code technology can achieve higher storage space utilization without reducing redundancy. For example, the erasure code storage system with 4:2 configuration has a storage space utilization rate of 2 / 3 , Compared with the copy technology, the space utilization rate has been...

AI Technical Summary

Problems solved by technology

[0003] At present, most distributed storage systems use the distributed consistent hash algorithm or the CRUSH pseudo-random distribution algorithm for data distribution. When a failure occurs, it cannot solve the problem that the distribution location of some storage objects that have not failed has changed.

The data blocks of the erasure code are in order. Compared with the copy storage system, this change will undoubtedly have a greater impact on the read and write access of the front-end application, and the heavy load scenario may even cause the entire storage system to be unavailable.

Moreover, most of the current erasure code storage systems are mainly used in the field of object storage, and generally do not support block access methods, and there are problems that do not support overwriting

Images