Metadata storage system for cluster file system and metadata management method

Abstract

The invention provides a metadata storage system for a cluster file system and a metadata management method. The metadata management method comprises the following steps of: storing metadata of the cluster file system by using a dynamic random access memory (DRAM), solid-state disk (SSD) and hard disk drive (HDD) three-layer architecture, and sequentially writing the metadata into metadata buffer regions in a DRAM; and when a metadata buffer region is overwritten, sequentially writing the metadata back to an SSD to form a new metadata storage unit, and storing metadata index information in a head structure into the DRAM in the form of a data index unit. By storing the metadata by adopting a rewriting mode, the metadata performance of the cluster file system can be improved effectively; furthermore, the invention also provides a data migration strategy based on access favor; therefore, more frequently accessed metadata are retained on the SSD; and the performance of reading operation of the metadata is improved.

Description

technical field [0001] The invention relates to the system structure of a cluster file system, in particular to metadata storage and a related data migration mechanism in the cluster file system. Background technique [0002] With the development of information technology, human beings generate, save and process more and more data. The number of files increases significantly with the increase of data volume, which gives the metadata storage system of cluster file system based on traditional dynamic random access memory (Dynamic Random Access Memory, DRAM) + hard disk drive (Hard Disk Drive, HDD) storage architecture Serious challenges are posed: [0003] In terms of storage scale, the total amount of metadata storage is getting larger and larger. The number of files to be saved in the cluster file system is increasing, especially with the rapid development of Internet applications, files exist in various forms, such as emails, photos, videos, reports, etc. With the increa...

AI Technical Summary

Problems solved by technology

For the first and third strategies, since the SSD will receive a large number of small-grained random I / O and cause rapid wear and tear, the service life of the SSD cannot be guaranteed

The second strategy of using HDD as SSD write buffer has two main problems for the metadata storage application of the cluster file system: one is that the data is finally placed on the SSD. When the data scale is large, a large number of SSDs are required. This increases the system cost; second, HDD is used as a write buffer, and some metadata read requests may be sent to HDD, which greatly increases the delay of metadata read requests. The read operation is a synchronous operation, and the delay of read requests is too large. The performance of this system, therefore, the second strategy also cannot meet the metadata storage system requirements of the cluster file system

Images