A method and device for adaptively storing files

Abstract

The invention discloses a method and device for self-adaptively storing files. The method includes: determining the hotness attribute of the file, which represents the frequency at which the file is accessed; according to the hotness attribute of the file, , perform encoding storage or transcoding storage processing on the file. Therefore, according to the method and device for adaptively storing files according to the embodiments of the present invention, the file can be stored and processed using the corresponding encoding and decoding technology according to the hotness and coldness attributes of the file, so the impact of the hotness and coldness of the file on the storage overhead can be fully considered. , Restoration cost requirements, and the codec technology used has high reliability and fast codec speed, therefore, it can improve the comprehensive performance of multiple dimensions of storage overhead, restoration cost, reliability and codec speed.

Description

technical field [0001] The present invention relates to the field of storage, and more particularly, to a method and device for adaptively storing files. Background technique [0002] In a distributed file system, in order to reduce storage overhead, Erasure Code (EC) technology can be used to encode and decode data instead of multi-copy replication. Each EC technology has its own advantages, and some are reliable. High performance, some with low recovery cost, and some with low storage overhead. For example, the dot product codec (Product Code, PC) algorithm uses both horizontal and vertical coding methods, and only performs XOR operations to obtain high codec speed and low recovery cost, but the storage overhead is high, and local reconstruction codec (Locally Reconstruction Code, LRC) algorithm has low storage overhead, but the recovery cost is high when multiple nodes fail. Therefore, the comprehensive performance of the existing EC technology still needs to be improve...

AI Technical Summary

Problems solved by technology

For example, the dot product codec (Product Code, PC) algorithm uses both horizontal and vertical coding methods, and only performs XOR operations to obtain high codec speed and low recovery cost, but the storage overhead is high, and local reconstruction codec (Locally Reconstruction Code, LRC) algorithm has low storage overhead, but the recovery cost is high when multiple nodes fail

Therefore, the comprehensive performance of the existing EC technology still needs to be improved.

Images