System and method for prefetching file layout through readdir++ in cluster file system

Abstract

The invention discloses a system and method for prefetching a file layout through readdir++ in a cluster file system. The system comprises a client side module (1) and a server module (2). The client side module (1) is used for obtaining or returning catalogue reading authorization from the server module (2); after the catalogue reading authentication is obtained, catalogue reading requests are sent to the server module (2); a webpage which is sent by the server module (2) and contains the file layout is stored in a local cache, and when the client side module (1) reads files in a catalogue, the file layout stored in the local cache is used directly. The server module (2) is used for authorizing the catalogue reading authorization to the client side module (1) or recalling the catalogue reading authorization from the client side module (1); when the catalogue reading requests are received, metadata information including the file layout is encapsulated in the webpage, and the webpage is sent to the client side module (1). Therefore, network interaction overheads for acquiring the file layout in the file reading process can be reduced, and the reading access performance of the massive small files can be improved greatly.

Description

technical field [0001] The invention relates to a metadata prefetching mechanism in a cluster file system, in particular to a system and method for prefetching file layout by readdir++ in a cluster file system. Background technique [0002] With the advent of the era of big data, the amount of global data information is increasing rapidly. There are more and more small-sized files in fields such as e-commerce, social networking, and scientific computing. Therefore, efficiently managing "massive small files" and providing low-latency small file access services are new issues facing the distributed file system. [0003] In recent years, the separation of metadata and data services has become the mainstream trend of distributed file systems. The advantage of this separation structure is that: the client directly accesses the storage device in an out-of-band manner, and can obtain high access performance for accessing large files. [0004] However, the case of small file acce...

AI Technical Summary

Problems solved by technology

The readdir+ technology used by pNFS obviously cannot effectively reduce the network interaction information overhead of file layout acquisition

Images