Construction method of heterogeneous part repetition code based on Huffman tree

Abstract

The invention discloses a construction method of a (Huffman Variable Fractional Repetition, HVFR) code. The construction method comprises the following steps of carrying out statistical analysis on the trajectory data of a distributed system within a certain time, and dividing the trajectory data into data blocks with different access frequencies; taking the data blocks with different access frequencies as the leaf nodes with determined weights of a Huffman tree, and constructing the Huffman tree according to a Huffman algorithm; determining the repetition degrees of the data blocks with different access frequencies according to the constructed Huffman tree; carrying out mDS coding on the data blocks with different access frequencies, and finally, constructing a heterogeneous FR code by using the paired balance design. According to the method, the bottleneck of parallel access of hot data is avoided, the load balancing is achieved, meanwhile, the system storage efficiency is improved,the repair locality and the repair bandwidth of the fault nodes are reduced, the calculation complexity is low, and the construction is simpler and more visual.

Description

technical field [0001] The invention belongs to the field of computers and relates to a method for constructing a heterogeneous partial repetition code based on a Huffman tree. Background technique [0002] In distributed storage systems, Rouayheb and Ramchandran proposed a Fractional Repetition (FR) code for accurate repair in 2010. FR codes can tolerate low-complexity code-free repair of multiple faulty nodes, minimize repair bandwidth overhead and disk I / O overhead, and significantly improve system fault repair performance. However, traditional FR codes are mainly aimed at static distributed storage systems, and the storage capacity of nodes and the repetition degree of data blocks remain unchanged. However, most of the actual distributed storage systems are dynamic storage systems, and the storage capacity of nodes is not the same, and the access to data by distributed storage systems is often unbalanced, "hot" data is often accessed, and "cold" data is rarely is visit...

AI Technical Summary

Problems solved by technology

However, most of the actual distributed storage systems are dynamic storage systems, and the storage capacity of nodes is not the same, and the access to data by distributed storage systems is often unbalanced, "hot" data is often accessed, and "cold" data is rarely was visited

If only one storage mechanism is used, it will often limit the performance of certain aspects of the storage system, such as the storage space occupied by the system, the overall data access throughput, etc.

Images