Metadata management method and system and medium

A management method and metadata technology, applied in the field of metadata management, can solve problems such as difficult concurrent processing and difficulty in improving metadata access performance, and achieve the effects of reducing the number of IO requests, improving overall performance, and low latency

CN109933570AActive Publication Date: 2019-06-25SUN YAT SEN UNIV +1
15 Cites 14 Cited by

Patent Information

Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Publication Date
2019-06-25

Smart Images

  • Figure 1
    Figure 1
  • Figure 2
    Figure 2
  • Figure 3
    Figure 3
Patent Text Reader

Abstract

The invention discloses a metadata management method, system and a medium. All directories and files in a file system are organized by adopting a tree structure, and all the directories and sub-filesunder a single directory are used as a whole and are sequentially stored in a continuous address space of bottom-layer storage equipment according to a creation sequence. When the metadata is updated,updating operations of all the subdirectories / subfiles in one directory are added to the tail of the directory in sequence in a log form, so that only one IO request is initiated in each metadata operation. When metadata retrieval is carried out, a sub-catalog is analyzed under a parent catalog, all data on a continuous address space corresponding to the parent catalog are firstly read, and thenmultiple acceleration methods are adopted for searching and matching. According to the method, the large directory problem can be effectively solved, directory path analysis has lower delay, the IO request number in the metadata access process can be reduced, and metadata access of a file system can be accelerated.
Need to check novelty before this filing date? Find Prior Art

Description

technical field

[0001] The invention relates to the field of metadata management of a file system for large-scale data storage, and in particular to a metadata management method, system and medium for accelerating metadata access of a file system and improving the overall performance of file system data access. Background technique

[0002] In large-scale storage file systems, metadata access has always been an important factor restricting IO performance. Metadata access is on the critical path of the IO process, and its performance determines the upper limit of the data access performance of the entire file system; more than half of the IO requests received by the storage device are generated by metadata access, and optimizing metadata access will bring Huge benefits; applications often issue a large number of irregular small requests for file system metadata, and this IO feature poses a huge challenge for metadata optimization. The mainstream file systems have made a lot ...

Examples

Embodiment 1

[0055] The key to the metadata management method is metadata update and metadata and on-demand. The key to the implementation of this embodiment is how to save the metadata of the directory / file on the storage device and how to quickly retrieve it from the parent directory. Specific subdirectories / subfiles.

[0056] Such as figure 1 As shown, the implementation steps of updating metadata in the metadata management method of this embodiment include:

[0057] 1) Organize all directories and files in the file system in a tree structure, and store all directories and subfiles under a single directory as a whole in the order of creation in the continuous address space of the underlying storage device;

[0058] 2) When updating file system metadata, update all subdirectories / subfiles under a directory in the form of a log, and all metadata updates are appended to the end of the continuous address space corresponding to the directory in the order in which they occur, so that Only o...

Embodiment 2

[0091]This embodiment is basically the same as Embodiment 1, the main differences are: this embodiment is a subset of Embodiment 1, this embodiment only includes the relevant content of metadata update in Embodiment 1, and provides the corresponding metadata Data management system and computer readable medium.

Embodiment 3

[0093] This embodiment is basically the same as Embodiment 1, the main differences are: this embodiment is a subset of Embodiment 1, this embodiment only includes the relevant content of metadata retrieval in Embodiment 1, and provides the corresponding metadata Data management system and computer readable medium.