Parallel erasure code encoding and decoding method and system

Abstract

The invention discloses a parallel erasure code encoding and decoding method and system. The parallel erasure code encoding and decoding method comprises the steps that an I / O reading thread reads a data bar of original data and obtains a data bar stored in a data buffer area; the data buffer area is added into a coding ready data buffer area queue, and a judgment is made as to whether the codingready queue is empty or not by a data coding thread; when the coding ready queue is not empty, the coding ready data buffer queue is taken out and a coding buffer is acquired; when the coding ready queue is empty, a coding buffer area is allocated to a memory to store the coding ready data buffer area queue; the data encoding thread encodes the data bar to obtain an encoded bar, and transmits theencoded bar to a write-in ready data buffer queue; the I / O write-in thread judges whether the write-in ready data buffer queue and the coding buffer queue are empty or not; and when the write-ready data buffer queue and the coding buffer queue are not empty, the I / O write-in thread writes the coding bar and the data bar into different files respectively.

Description

technical field [0001] The invention relates to the field of data storage, in particular to a parallel erasure code encoding and decoding method and system. Background technique [0002] Erasure code technology stores data objects in the system through encoding, thereby improving the fault-tolerant performance of the system. Compared with copy-based fault-tolerant technology, erasure code technology can store less data while maintaining the same fault-tolerant capability, so it can effectively improve the utilization rate of storage space and reduce storage overhead. However, the erasure code-based storage system needs to encode the data before the data enters the system, and when reading the data, the stored data needs to be encoded to obtain the original data object. Therefore, the efficiency of encoding and decoding is crucial to improving the efficiency of erasure code-based storage systems. [0003] Jerasure is an open source erasure code library implemented in C / C++ ...

AI Technical Summary

Problems solved by technology

[0004] The technical problem to be solved by the present invention is that the serial execution efficiency of the existing erasure correction coding and decoding is low, and the purpose is to provide a parallel correction Erasure code encoding and decoding method and system to solve the above problems

Images