System and method for ensuring data validity in parallel network file system

Abstract

The invention discloses a system and a method for ensuring data validity in a parallel network file system. The system comprises: a client, which is used for sending a read-only directory authorization application, receiving the granted read-only directory authorization, and marking the read-only directory version number, and access the file in the read-only directory on the disk and the data page of the file; the metadata server is used to receive the read-only directory authorization application sent by the client, and grant the client read-only directory authorization; the client Return the read-only directory authorization to the metadata server after completing the access, and clear the version number of the read-only directory.

Description

technical field [0001] The invention relates to a cache validity mechanism after pre-reading data between files in a parallel network file system, in particular to a system and method for ensuring data validity in a parallel network file system. Background technique [0002] With the advent of the era of big data, the amount of global data information is increasing rapidly. There are more and more small-sized files in fields such as e-commerce, social networking, and scientific computing. According to the 2007 statistics of the Pacific Northwest National Laboratory of the United States, among the more than 12 million files stored internally, more than 80% of the files are smaller than 1MB, and more than 50% of the files are smaller than 32KB. In a sense, there is currently no purely large file application. Human beings are gradually entering the "era of massive small files". [0003] Parallel Network File System (pNFS) adopts the separation structure of metadata and data ...

AI Technical Summary

Problems solved by technology

[0005] However, compared with metadata prefetching, the biggest difficulty in data prefetching is that even with good physical locality, it is impossible to accurately determine which file the prefetched data page belongs to, and it may even prefetch files in other directories. data pages, which creates a significant barrier to extending directory authority to "data validity assurance"

Images