Non-volatile memory based file metadata storage method and system

Abstract

The application provides a non-volatile memory-based file metadata storage method and system, which comprises the following steps: initializing a metadata storage structure composed of a directory, a directory subset, a metadata block and a metadata item in a non-volatile memory, wherein all the metadata blocks are stored in a metadata storage pool file of the non-volatile memory; obtaining file metadata to be stored, selecting a non-saturated directory subset in the directory as a target subset, continuing to select a non-saturated metadata block in the target subset as a target data block, selecting a free metadata item in the target data block as a target item; and writing the file metadata to be stored into the target item. By using the design structure and the related method, the file system metadata access performance can be improved, and the problem of insufficient file system concurrent access performance in a big data application environment can be effectively solved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of file system, in particular to a file metadata storage method and system based on non-volatile memory. BACKGROUND

[0002] Files are the main logical carriers of current data storage, and the design of file systems plays a key role in the performance of data-intensive applications. The efficiency of the metadata system, as the control and management module of the file system, determines the overall performance of the file system. Traditional file systems such as EXT4 and XFS use a hierarchical structure for metadata storage and management. Active metadata is stored in high-performance DRAM memory cache, and inactive metadata is stored in large-capacity solid-state disk (SSD) storage devices. The hierarchical metadata storage structure helps to take advantage of different storage media and achieve a high cost performance, but it has been unable to meet the explosive growth of demand for file data processing capabilities in the big data era. The contradiction between application requirements and existing file systems mainly reflects in:

[0003] 1. High-performance computing, big data, and artificial intelligence applications require microsecond-level file access latency and millions of concurrent access frequencies per second. The metadata storage structure based on block devices cannot meet the high bandwidth and low latency requirements of applications.

[0004] 2. Massive small file access scenarios such as machine learning and digital media generate a large amount of active file metadata. DRAM memory has limited capacity and cannot load all active metadata, resulting in frequent exchange of metadata between memory and slow devices, which reduces access performance.

[0005] 3. In large-scale storage clusters, partial node failure is a high-probability event, and high reliability and fault recovery capability of file data are necessary functions. Memory is a volatile medium, and to ensure the integrity and consistency of metadata information, complex information fault recovery mechanisms such as logs must be introduced, further reducing system performance.

[0006] 4. Due to the huge gap in random access performance within disk block devices (more than 100 times), the precondition for memory devices to achieve effective acceleration is that cache access must have a very high hit rate (more than 95%). File system metadata access is mainly small-grained random access, which cannot objectively support the requirement of high hit rate, greatly weakening the performance optimization effect of the cache hierarchy.

[0007] To solve the above-mentioned contradiction, a new storage system needs to be used to replace the original storage structure of the existing file system metadata module, to fundamentally solve the problem of metadata access in terms of performance and reliability, to adapt to the needs of applications. In recent years, the progress of storage device technology has made non-volatile memory (NVM) a product that can be used in bulk, providing a technical basis for new storage structures. The main features of non-volatile memory (NVM) media are:

[0008] 1. High random access performance. Through actual testing, the random access performance of NVM is more than 30 times that of solid state disk (SSD), and can reach 40% of the random access performance of DRAM memory;

[0009] 2. Access in byte units. NVM access mode is basically the same as memory, and application programs can run directly in the NVM environment without modification, without the need for address mode conversion;

[0010] 3. Large capacity, low unit capacity cost, only 1 / 3 of the memory;

[0011] 4. Support for data persistent storage, information is not lost after the device is turned off;

[0012] 5. NVM device has high access performance, close to DRAM device, but needs to be initialized through file mapping before it can be used by application programs. The NVM mapping time overhead is more than 50us, which is much higher than the NVM metadata write and persistent time delay (about 3us). For massive file creation and other metadata allocation-intensive application scenarios, NVM initialization delay seriously restricts application performance.

[0013] The above-mentioned features determine that non-volatile memory can be used as a low-cost capacity expansion and data persistent storage function expansion of memory devices. With limited cost, the introduction of non-volatile memory devices can replace the functions of memory cache and persistent storage devices in the file system metadata module. According to the functional characteristics and performance advantages of NVM devices, performance optimization design can significantly improve the file system metadata access performance while ensuring high data reliability, meeting the needs of high-performance file access of big data applications. SUMMARY

[0014] The present application addresses the technical challenges of high-performance computing and big data applications in file access performance, introduces non-volatile memory (NVM) devices in file system design, and proposes a metadata access method for file system metadata module optimization design. Using the design structure and related methods proposed in the present application, the file system metadata access performance can be significantly improved, effectively solving the problem of insufficient file system concurrent access performance in big data application environment.

[0015] In order to solve the problems in the prior art, the application provides a file metadata storage method based on non-volatile memory, which comprises the following steps:

[0016] Step 1: initializing a metadata storage structure composed of a directory, a directory subset, a metadata block and a metadata item in the non-volatile memory, wherein all the metadata blocks are stored in a metadata storage pool file of the non-volatile memory;

[0017] Step 2: obtaining file metadata to be stored, selecting a non-saturated directory subset in the directory as a target subset, continuing to select a non-saturated metadata block in the target subset as a target data block, and selecting a free metadata item in the target data block as a target item;

[0018] Step 3: setting the state of the target item as allocated, initializing the target item, and writing the file metadata to be stored into the target item and persisting the file metadata.

[0019] The file metadata storage method based on the non-volatile memory, wherein the storage space of the metadata block comprises a storage unit for storing the metadata item, and the metadata storage pool file is created and mapped once in the system initialization stage.

[0020] The file metadata storage method based on the non-volatile memory, wherein in the metadata storage structure, the directory and the directory subset are independently stored in the form of NVM files on the non-volatile memory; the NVM file of the directory is mapped to the current process address space, and the mapping address is recorded in a global directory hash table; the NVM file of the directory subset is mapped to the current process address space, and the mapping address is recorded in a global directory subset hash table.

[0021] The file metadata storage method based on the non-volatile memory, wherein the step 2 comprises the following steps:

[0022] Step 21: judging whether there is a non-saturated directory subset in the directory, if yes, taking the non-saturated directory subset as the target subset, otherwise, creating a new NVM file of the directory subset, mapping the new NVM file to the current process address space, recording the mapping address in a global directory subset hash table, and taking the new directory subset as the target subset;

[0023] Step 22: judging whether there is a non-saturated metadata block in the target subset, if yes, taking the non-saturated metadata block as the target data block, otherwise, allocating a metadata block from the metadata storage pool to the target subset, and taking the allocated metadata block as the target data block.

[0024] The application further provides a file metadata storage system based on the non-volatile memory, which comprises the following steps:

[0025] an initial module, configured to initialize a metadata storage structure composed of a directory, a directory subset, a metadata block and a metadata item in the non-volatile memory, wherein all the metadata blocks are stored in a metadata storage pool file of the non-volatile memory;

[0026] a selection module, configured to acquire file metadata to be stored, select a directory subset that is not saturated in the directory as a target subset, continue to select a metadata block that is not saturated in the target subset as a target data block, and select a free metadata item in the target data block as a target item;

[0027] a storage module, configured to set a state of the target item as allocated, initialize the target item, and write the file metadata to be stored into the target item and persist the file metadata.

[0028] The non-volatile memory-based file metadata storage system, wherein the storage space of the metadata block contains a storage unit for storing the metadata item, and the metadata storage pool file is created and mapped once in a system initialization phase.

[0029] The non-volatile memory-based file metadata storage system, wherein in the metadata storage structure, the directory and the directory subset are independently stored in the form of NVM files on the non-volatile memory; the NVM file of the directory is mapped to a current process address space, and a mapping address is recorded in a global directory hash table; the NVM file of the directory subset is mapped to the current process address space, and a mapping address is recorded in a global directory subset hash table.

[0030] The non-volatile memory-based file metadata storage system, wherein the selection module comprises:

[0031] a first judgment module, configured to judge whether there is a directory subset that is not saturated in the directory, if yes, the directory subset that is not saturated is selected as a target subset, otherwise, an NVM file of the directory subset is newly created, mapped to a current process address space, a mapping address is recorded in a global directory subset hash table, and the newly created directory subset is selected as the target subset;

[0032] a second judgment module, configured to judge whether there is a metadata block that is not saturated in the target subset, if yes, the metadata block that is not saturated is selected as a target data block, otherwise, a metadata block is allocated from the metadata storage pool to the target subset, and the allocated metadata block is selected as the target data block.

[0033] The application further provides a storage medium for storing a program for executing any one of the non-volatile memory-based file metadata storage methods.

[0034] The application further provides a client for the nonvolatile memory file metadata storage system.

[0035] From the above scheme, the application has the advantages of:

[0036] The application provides a file system metadata storage structure and method based on a nonvolatile memory (NVM) device. The performance advantages of the nonvolatile memory device are utilized to propose a single-layer cache-free metadata storage structure, and the metadata access process is optimized. In view of the delay problem of small-granularity resource allocation after the introduction of the NVM device, an efficient metadata storage space allocation and initialization method is proposed. The optimized metadata system can significantly improve the file system metadata access performance on the premise of ensuring high reliability. In actual tests, the metadata access performance can be improved by more than 5 times. BRIEF DESCRIPTION OF DRAWINGS

[0037] Figure 1a It is a mixed structure diagram of the adopted DRAM memory and SSD solid state disk;

[0038] Figure 1b It is a single-layer structure cache-free file system metadata storage structure diagram based on NVM;

[0039] Figure 2 It is a metadata hierarchical scalable storage structure diagram based on NVM;

[0040] Figure 3 It is a large-scale file initialization creation flowchart based on NVM. DETAILED DESCRIPTION

[0041] The application mainly includes the following key technical points:

[0042] Key point 1: Establish a cache-free single-layer metadata storage structure based on NVM. The performance advantages and capacity advantages of the NVM device are utilized to directly access all metadata in the file system through the NVM, avoid frequent data exchange between the memory and the hard disk, greatly reduce the file metadata access delay, and improve the aggregate access performance.

[0043] Key point 2: Use a hierarchical metadata storage structure, and design an optimized metadata allocation and initialization method according to the access characteristics of the metadata storage units at different levels. On the premise of ensuring the dynamic expansion of the metadata storage, the metadata storage management time overhead introduced by the use of the NVM device is reduced, and the overall metadata access performance is improved.

[0044] In order to make the above features and effects of the application more explicit and easy to understand, the following embodiments are specifically described, and the detailed description is as follows in combination with the accompanying drawings.

[0045] The metadata storage structure of the current mainstream file system adopts a hybrid structure of DRAM memory and SSD solid state disk, as shown in Figure 1a . In the hybrid structure, the memory is used as a cache, and the hot metadata frequently accessed is stored, and the SSD is used for persistent storage of all metadata. The design goal of such a structure is to combine the advantages of high random access performance of DRAM memory and large capacity of SSD device, and to provide the application with metadata access performance close to DRAM memory. However, in actual design, the overall performance of any cache structure system is limited by the cache hit rate and the performance ratio of fast / slow devices in the system. According to the formula: Ta=H*Ta1+(1-H)*Ta2, where H is the hit rate, Ta is the overall access time, Ta1 is the access time of the cache device, and Ta2 is the access time of the slow device.

[0046] Through performance analysis, it can be known that when the access time of the slow device in the system and the access time of the cache device are greatly different, in order to make the overall performance close to the performance of the cache device, a very high cache hit rate is required. In actual tests, the random access time of the SSD is more than 100 times that of the DRAM memory (Ta2>=100Ta1). Even if only the overall performance is required to reach 1 / 3 of the DRAM (Ta<=3Ta1), the hit rate needs to reach 98%. In the actual application scenario of mass files, the metadata access mode of the file system is mainly high-frequency small-granularity random access, and the metadata access cache hit rate is less than 90%, which leads to the fact that in actual application, the overall performance of the metadata access of the cache structure is less than 10% of the performance of the DRAM device, and cannot meet the high-performance and low-latency requirements of the application.

[0047] In order to solve the problems of random access performance, data reliability and data consistency, and improve the metadata access performance, the present application scheme proposes the following structure and method.

[0048] The metadata storage structure of the file system without cache based on the single-layer structure of NVM. The structure is shown in the accompanying Figure 1b . Compared with the two-layer (multi-layer) cache structure, the single-layer storage structure is simple, and the hit rate is 100%, and the metadata does not need to be exchanged between the cache device and the slow persistent device, and only needs to be persisted through the CPU instruction when writing to ensure the reliability of the metadata. Therefore, under the metadata storage structure based on NVM, the access performance is only limited by the performance of the medium. According to actual tests, the overall performance of the random access of NVM can reach 40% of that of DRAM, and can meet the requirements of the application.

[0049] Hierarchical scalable metadata storage structure and optimized storage resource allocation method. In order to solve the initialization time overhead caused by NVM memory allocation, the present application proposes a hierarchical scalable metadata storage structure, as shown in Figure 2The metadata storage structure in the system is stored in four-level structure units of directory -> directory subset -> metadata block -> metadata (meta), and each level structure is logically dependent on the upper level structure. In the storage structure, the directory subset and the directory to which it belongs are independently stored in the form of a file on the NVM device without a containing relationship. All metadata blocks are stored in an NVM metadata storage pool file. The metadata storage space in the metadata block contains metadata storage units. The metadata storage pool file is created and mapped once in the system initialization stage. After the directory and the directory subset are created, the corresponding NVM file needs to be mapped to the current process address space, and the corresponding address needs to be recorded in the global hash table, so that the application program can be accessed. The mapping process of the metadata block and the metadata has been completed in the file system initialization stage, and after allocation, only the state needs to be persisted as "allocated" to enable access. The advantages of hierarchical storage of metadata in NVM are as follows: 1) For directories and directory subsets, which have the characteristics of large internal metadata capacity, small quantity, low creation frequency, and the need for dynamic expansion, the independent file storage and mapping method is used for allocation, which can realize system dynamic expansion with only a small increase in average latency. 2) For metadata blocks and metadata, which have the characteristics of small capacity, high creation frequency, and single structure, the centralized storage and unified mapping method is used for allocation and management, which can significantly reduce the time cost of metadata allocation and initialization.

[0050] Figure 3 An example is given to illustrate a common batch file creation process of a large capacity directory with a flat structure in a big data application. 1 million regular files are created, 64 directory subsets are established during the process, 512 metadata blocks are created (allocated) for each directory subset, and each metadata block contains 32 fixed metadata. Therefore, in Figure 3 In the above-mentioned method embodiment, branch 1 is executed 64 times, each time about 50us, for creating a directory subset. Branch 2 is executed 64*512 = 32768 times, each time about 3us, for allocating a metadata block. Branch 3 is executed 1 million times for allocating NVM metadata space. The metadata allocation and initialization time during the process is 3.24 seconds, and the average time for each metadata to be allocated and initialized is about 3us, which is close to the time for DRAM to allocate memory, achieving the expected optimization effect.

[0051] The following is a system embodiment corresponding to the above-mentioned method embodiment. The technical details mentioned in the above-mentioned embodiment are still valid in this embodiment. In order to reduce repetition, they will not be repeated here. Correspondingly, the technical details mentioned in this embodiment can also be applied in the above-mentioned embodiment.

[0052] The application further provides a non-volatile memory-based file metadata storage system, which comprises the following modules:

[0053] An initialization module is configured to initialize a metadata storage structure composed of a directory, a directory subset, a metadata block and a metadata item in the non-volatile memory, wherein all the metadata blocks are stored in a metadata storage pool file of the non-volatile memory.

[0054] A selection module is configured to obtain file metadata to be stored, select a non-saturated directory subset in the directory as a target subset, continue to select a non-saturated metadata block in the target subset as a target data block, and select a free metadata item in the target data block as a target item.

[0055] A storage module is configured to set the state of the target item as allocated, initialize the target item, and write the file metadata to be stored into the target item and persist the file metadata.

[0056] The non-volatile memory-based file metadata storage system, wherein the storage space of the metadata block comprises storage units for storing metadata items, and the metadata storage pool file is created and mapped once in the system initialization stage.

[0057] The non-volatile memory-based file metadata storage system, wherein in the metadata storage structure, the directory and the directory subset are independently stored in the form of NVM files on the non-volatile memory; the NVM file of the directory is mapped to the current process address space, and the mapping address is recorded in a global directory hash table; the NVM file of the directory subset is mapped to the current process address space, and the mapping address is recorded in a global directory subset hash table.

[0058] The non-volatile memory-based file metadata storage system, wherein the selection module comprises:

[0059] A first judgment module is configured to judge whether there is a non-saturated directory subset in the directory, if yes, the non-saturated directory subset is selected as the target subset, otherwise, a NVM file of the directory subset is newly created, mapped to the current process address space, and the mapping address is recorded in the global directory subset hash table, and the newly created directory subset is selected as the target subset.

[0060] A second judgment module is configured to judge whether there is a non-saturated metadata block in the target subset, if yes, the non-saturated metadata block is selected as the target data block, otherwise, a metadata block is allocated from the metadata storage pool to the target subset, and the allocated metadata block is selected as the target data block.

[0061] The application further provides a storage medium for storing a program for executing the non-volatile memory-based file metadata storage method.

[0062] The present application also provides a client for any of the nonvolatile memory file metadata storage systems.

Claims

1. A method for storing file metadata based on non-volatile memory, characterized in that, include: Step 1: Establish a cache-free single-layer metadata storage structure based on NVM. Initialize the metadata storage structure consisting of directories, directory subsets, metadata blocks, and metadata items in non-volatile memory. All metadata blocks are centrally stored in the metadata storage pool file of this non-volatile memory. The storage space of this metadata block contains metadata item storage units for storing metadata. This metadata storage pool file is created and mapped once during the system initialization phase. In this metadata storage structure, the directory and its subset are stored independently as NVM files on the non-volatile memory; the NVM files of the directory are mapped to the current process address space, and the mapped addresses are recorded in the global directory hash table; The NVM files of the directory subset are mapped to the address space of the current process, and the mapped addresses are recorded in the global directory subset hash table. For directories and directory subsets, which have large internal metadata capacity, small quantity, low creation frequency, and require dynamic expansion, an independent file storage and mapping method is used for allocation. For metadata blocks and metadata, which have small capacity, high creation frequency, and simple structure, a centralized storage and unified mapping method is used for allocation and management. Step 2: Obtain the file metadata to be stored, select an unsaturated subset of directories in the directory as the target subset, continue to select unsaturated metadata blocks in the target subset as the target data blocks, and select free metadata items in the target data blocks as target items; Step 3: Set the status of the target item to allocated, initialize the target item, and write the metadata of the file to be stored into the target item and persist it.

2. The file metadata storage method based on non-volatile memory as described in claim 1, characterized in that, Step 2 includes: Step 21: Determine if there is an unsaturated subset of directories in the directory. If there is, use the unsaturated subset of directories as the target subset. Otherwise, create a new NVM file for the new subset of directories, map it to the current process address space, record the mapping address in the global subset hash table, and use the new subset of directories as the target subset. Step 22: Determine whether there are unsaturated metadata blocks in the target subset. If there are, use the unsaturated metadata block as the target data block. Otherwise, allocate a metadata block from the metadata storage pool to the target subset and use the allocated metadata block as the target data block.

3. A file metadata storage system based on non-volatile memory, characterized in that, include: The initial module is used to establish a cacheless single-layer metadata storage structure based on NVM. It initializes a metadata storage structure consisting of directories, directory subsets, metadata blocks, and metadata items in non-volatile memory, where all metadata blocks are centrally stored in the metadata storage pool file of this non-volatile memory. The storage space of this metadata block contains metadata item storage units for storing metadata. This metadata storage pool file is created and mapped once during the system initialization phase. In this metadata storage structure, the directory and its subset are stored independently as NVM files on the non-volatile memory; the NVM files of the directory are mapped to the current process address space, and the mapped addresses are recorded in the global directory hash table; The NVM files of the directory subset are mapped to the address space of the current process, and the mapped addresses are recorded in the global directory subset hash table. For directories and directory subsets, which have large internal metadata capacity, small quantity, low creation frequency, and require dynamic expansion, an independent file storage and mapping method is used for allocation. For metadata blocks and metadata, which have small capacity, high creation frequency, and simple structure, a centralized storage and unified mapping method is used for allocation and management. The selection module is used to obtain the file metadata to be stored, select an unsaturated subset of directories in the directory as the target subset, continue to select unsaturated metadata blocks in the target subset as the target data block, and select free metadata items in the target data block as the target items; The storage module is used to set the status of the target item to allocated, initialize the target item, and write and persist the metadata of the file to be stored into the target item.

4. The file metadata storage system based on non-volatile memory as described in claim 3, characterized in that, This selection module includes: The first judgment module is used to determine whether there is an unsaturated directory subset in the directory. If it exists, the unsaturated directory subset is used as the target subset. Otherwise, a new NVM file for the directory subset is created and mapped to the current process address space. The mapped address is recorded in the global directory subset hash table and the newly created directory subset is used as the target subset. The second judgment module is used to determine whether there is an unsaturated metadata block in the target subset. If there is, the unsaturated metadata block is used as the target data block. Otherwise, a metadata block is allocated from the metadata storage pool to the target subset and the allocated metadata block is used as the target data block.

5. A storage medium for storing a program that executes a file metadata storage method based on non-volatile memory as described in claim 1 or 2.

6. A client for a file metadata storage system based on non-volatile memory as described in claim 3 or 4.

Images