Metadata management method and related equipment

Abstract

The embodiment of the invention provides a metadata management method and related equipment, which are used for reducing read-write operation of metadata on a PM (Program Management) so as to improve the reliability of single writing of the metadata. The method provided by the embodiment of the invention comprises the following steps: generating a file creation package and a data updating package in the metadata single writing file system; after the file creating package and the data updating package are subjected to crash consistency processing, the file creating package and the data updating package are written into a persistent memory managed by the metadata single-writing file system, a data storage address of the user data is determined, and expanded index metadata for indexing the user data in the data updating package is determined according to the data storage address; analyzing the file creating package and the data updating package to establish a file index node through the file index node number, and establishing a mapping relation between the data storage address and the file index node number by expanding index metadata; and inserting the file index node corresponding to the user file into the index node table in the memory to complete access to the user file.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of computer technology, and in particular to a metadata management method and related equipment. Background Art

[0002] Current file systems for new persistent memory (PM) use traditional crash consistency technologies (such as log writes and soft updates) to ensure file system reliability. However, these traditional crash consistency technologies involve multiple, small, sequential updates of metadata. This significantly complicates input / output (I / O) operations on PMs, significantly wastes PM bandwidth, and reduces PM cost-effectiveness. Summary of the Invention

[0003] The embodiments of the present application provide a metadata management method and related equipment for reducing the read and write operations of metadata in PM to improve the reliability of single-write metadata.

[0004] A first aspect of an embodiment of the present application provides a metadata management method, including:

[0005] Based on a user's file write operation on a user file in an application, a file creation package and a data update package are generated in the metadata single-write file system; wherein the file creation package includes a file inode number and directory entry information corresponding to the user file, and the directory entry information includes the file name of the user file; and the data update package includes index data, the index data being used to index user data, the user data being data written by the user file to the metadata single-write file system;

[0006] After the file creation package and the data update package have undergone crash consistency processing, the file creation package and the data update package that have undergone crash consistency processing are written into the persistent memory managed by the metadata single-write file system, the data storage address of the user data in the persistent memory is determined, and based on the data storage address, the extended index metadata that indexes the user data in the data update package is determined; wherein the extended index metadata is used to replace the index data in the data update package;

[0007] Parsing the file creation package and the data update package stored in the persistent memory to establish a file index node pointing to the file name of the user file through the file index node number, and establishing a mapping relationship between the data storage address and the file index node number under the file index node through the extended index metadata;

[0008] The file index node corresponding to the user file is inserted into the index node table, so that when the user performs a file reading operation on the target user file in the application program, the user data corresponding to the target user file can be located through the index node table to complete the access to the target user file.

[0009] Optionally, writing the file creation package and the data update package that have undergone crash consistency processing into the persistent memory of the metadata single-write file system includes:

[0010] Constructing a data block in the persistent memory, storing the user data in the data block, and determining a data block pointer pointing to the data block; wherein the data block pointer is used to describe the data storage address pointed to by the user data in the persistent memory;

[0011] Based on the memory operation instruction, the index data in the data update package is aligned according to the data block, and the data correspondence between the index data after the block alignment and the data block pointer is recorded;

[0012] Obtaining a data update package after crash consistency processing based on the data correspondence, the data block pointer, and the index data;

[0013] Writing the crash-consistent data update package into the persistent memory.

[0014] Optionally, the step of establishing a file index node pointing to the file name of the user file by using the file index node number includes:

[0015] A package translation layer defining the metadata single-write file system;

[0016] Creating a metadata update package; wherein the metadata update package includes dynamic attribute information, the dynamic attribute information records file attribute information corresponding to the file operation of the user file; the file creation package also includes static attribute information, the static attribute information records the file index node number and the directory entry information;

[0017] In the package translation layer, a metadata update package node for managing the metadata update package and a file creation package node for managing the file creation package are abstractly created; wherein the metadata update package node includes a metadata update package pointer pointing to the storage address of the metadata update package in the persistent memory; the file creation package node includes a file creation package pointer pointing to the storage address of the file creation package in the persistent memory and a name pointer pointing to the storage address of the file name in the persistent memory;

[0018] Based on the dynamic attribute information and the static attribute information, the file creation package node and the metadata update package node are merged to obtain the file index node.

[0019] Optionally, establishing a mapping relationship between the data storage address and the file index node number under the file index node through the extended index metadata includes:

[0020] In the packet translation layer, a data update package node for managing the data update package is abstractly created; wherein the data update package node includes a data update package pointer pointing to a storage address of the data update package in the persistent memory;

[0021] Establishing a data node linked list of the data update package node at the file index node, so as to linearly set a file mapping relationship between the data update package and the file logical address of the user file through the data node linked list, where the file logical address is the storage address of the user file in the application program;

[0022] A linear index table is constructed based on the file mapping relationship, the extended index metadata, and the mapping relationship between the data storage address and the file index node number to index different data update package nodes in the same user file; wherein, the linear index table is used by the user to access the file logical address.

[0023] Optionally, after defining the packet translation layer of the metadata single-write file system, the method further includes:

[0024] At the file index node, establishing a creation node linked list of the file creation package node;

[0025] By creating the node linked list and the hash table, setting the directory mapping relationship between all file creation packages to obtain a directory linked list;

[0026] The directory linked list is abstracted to obtain a directory object, so that the user can search for the file name of the user file through the directory object.

[0027] Optionally, the method for generating the index node table includes:

[0028] In the packet translation layer, all file index nodes are obtained, and all the file index node numbers are mapped one-to-one with the file names of all the user files through a hash table to form a file table object, and the file table object is adjusted according to the file attribute information to obtain an index node table.

[0029] Optionally, writing the file creation package and the data update package that have undergone crash consistency processing into the persistent memory of the metadata write-only file system includes:

[0030] Determine free blocks in the persistent memory and construct a free block index of a red-black tree; wherein the free blocks are data blocks to which the file metadata is not written, and the free block index includes a free block number corresponding to each free block on the root node of the red-black tree;

[0031] When allocating the data update package and file creation package that have undergone crash consistency processing to the target free block of any of the red-black tree root nodes, subtracting the free block number corresponding to the target free block, allocating the data update package and file creation package that have undergone crash consistency processing to the target free block in the form of a data block, and determining the target free block as the target data block;

[0032] The target free block number of each data update package and file creation package after allocation is recorded through a global coarse-grained dimension, and the bitmap is used to complete the bitmap index of the data update package and file creation package after crash consistency processing and the target data block, so as to insert the data update package and file creation package after crash consistency processing into the red-black tree root node; wherein, the corresponding bit of the bitmap is 1.

[0033] Optionally, the method further includes:

[0034] Based on the user's file recycling operation on the user file, a file deletion package is generated in the metadata write-only file system; wherein the file deletion includes a file deletion instruction, and the file deletion instruction is used to delete the file index node number or the directory entry information;

[0035] When the data update package and file creation package that have undergone crash consistency processing are recycled, if the data update package or file creation package is not stored in the target data block, the corresponding position of the bitmap is set to 0, and based on the causal dependency between the file deletion package and the file creation package, the space block where the file creation package that has undergone crash consistency processing is located is marked as invalid, and the space block where the original data update package covered by the data update package that has undergone crash consistency processing is located is marked as invalid, and the file creation package and the original data update package are recycled.

[0036] A second aspect of an embodiment of the present application provides a metadata management system, including:

[0037] a generating unit configured to generate a file creation package and a data update package in the metadata single-write file system based on a file write operation of a user on a user file in an application; wherein the file creation package includes a file inode number and directory entry information corresponding to the user file, the directory entry information including the file name of the user file; and the data update package includes index data for indexing user data, the user data being data written by the user file to the metadata single-write file system;

[0038] a writing unit, configured to write the file creation package and the data update package that have undergone crash consistency processing into the persistent memory managed by the metadata single-write file system after crash consistency processing has been performed on the file creation package and the data update package, determine the data storage address of the user data in the persistent memory, and determine, based on the data storage address, the extended index metadata in the data update package that indexes the user data; wherein the extended index metadata is used to replace the index data in the data update package;

[0039] a parsing unit, configured to parse the file creation package and the data update package stored in the persistent memory, so as to establish a file index node pointing to the file name of the user file through the file index node number, and to establish a mapping relationship between the data storage address and the file index node number under the file index node through the extended index metadata;

[0040] The writing unit is further configured to insert the file index node corresponding to the user file into the index node table, so that when the user performs a file reading operation on the target user file in the application, the user data corresponding to the target user file can be located through the index node table to complete the access to the target user file.

[0041] The metadata management system provided in the second aspect of the embodiment of the present application is used to execute the metadata management method described in the first aspect.

[0042] A third aspect of an embodiment of the present application provides a metadata management device, including:

[0043] CPU, memory, input and output interfaces, wired or wireless network interfaces, and power supply;

[0044] The memory is a transient storage memory or a persistent storage memory;

[0045] The central processing unit is configured to communicate with the memory and execute instructions in the memory to perform the metadata management method described in the first aspect.

[0046] A fourth aspect of an embodiment of the present application provides a computer-readable storage medium, which includes instructions. When the instructions are executed on a computer, the computer executes the metadata management method described in the first aspect.

[0047] A fifth aspect of the embodiments of the present application provides a computer program product, which includes instructions. When the instructions are executed on a computer, the computer executes the metadata management method described in the first aspect.

[0048] It can be seen from the above technical solutions that the embodiments of the present application have the following advantages: through a metadata management method disclosed in the embodiments of the present application, by designing a metadata package (such as a file creation package or a data update package, which is a form of metadata package), the metadata is stored in the persistent memory in the form of a metadata package, ensuring that only the metadata package is written during each file operation, thereby merging multiple small-scale metadata write operations of multiple metadata into a small-scale metadata write operation, thereby reducing PM read and write operations and improving the cost-effectiveness of PM. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments recorded in the present application. For ordinary technicians in this field, other drawings can also be obtained based on these drawings.

[0050] Figure 1 A flowchart of a metadata management method disclosed in an embodiment of the present application;

[0051] Figure 2 A flowchart of another metadata management method disclosed in an embodiment of the present application;

[0052] Figure 3 A flowchart of another metadata management method disclosed in an embodiment of the present application;

[0053] Figure 4 A flowchart of another metadata management method disclosed in an embodiment of the present application;

[0054] Figure 5 This is a system architecture diagram of a metadata single-write file system disclosed in an embodiment of the present application;

[0055] Figure 6 A metadata package conversion and translation design diagram disclosed in an embodiment of the present application;

[0056] Figure 7 This is a diagram illustrating an architecture implementation of a metadata package single-write file system disclosed in an embodiment of the present application;

[0057] Figure 8 A schematic diagram of the structure of a metadata management system disclosed in an embodiment of the present application;

[0058] Figure 9 This is a structural diagram of a metadata management device disclosed in an embodiment of the present application. DETAILED DESCRIPTION

[0059] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0060] It should be noted in advance that in the embodiments of this application, the main purpose is to design a metadata management method for a high-performance file system for a new type of persistent memory. Specifically, the core of this management method lies in the metadata package single-write file system architecture, and four designs are carried out around this architecture: metadata package abstract construction, metadata package conversion and translation, metadata package recycling management, and metadata package fast recovery. These designs ensure the practicality, efficiency, and reliability of metadata package single-write. For details, please refer to the subsequent Figures 1 to 4 The embodiment shown.

[0061] To solve the technical problems described in the above background technology, please refer to Figure 1 , Figure 1 This is a flowchart of a metadata management method disclosed in an embodiment of the present application, including steps 101 to 104.

[0062] 101. Based on a file write operation of a user on a user file in an application, a file creation package and a data update package are generated in a metadata write-only file system.

[0063] Since the core of metadata single-write is to minimize the writing of metadata, a metadata single-write file system architecture is proposed to design different metadata packages for different file operations, thereby ensuring that only a single metadata is written for each file operation, so as to minimize the read and write and sequential barriers to PM. Specifically, it is necessary to create a file creation package and a data update package in the metadata single-write file system based on the user's file write operation in the application. It should be noted that the file creation package includes the file index node number and directory entry information corresponding to the user file, and the directory entry information includes the file name of the user file; the data update package includes index data, which is used to index user data, and the user data is the data written by the user file in the metadata single-write file system.

[0064] In one specific embodiment, see Figure 5 , Figure 5 This is a system architecture diagram of a metadata single-write file system disclosed in an embodiment of this application. Figure 5 As shown, for metadata operations, it is necessary to create a file Create, that is, a metadata single-write file system (which can also be described as a single-write file system, which will not be described in detail later) first generates a Create metadata package ( Figure 5 In step ① of generating a package), the file creation package includes 64 bytes of static file attributes, such as the creation time and file inode number (file index node number), 128 bytes of directory entry information (name), and the remaining 64 bytes of metadata packet header (hdr). It can also include records of the magic number, size, type, etc. of the packet. Furthermore, the data update package can include the attribute attr, index idx (the above-mentioned index data) or data packet header (hdr), etc. At the same time, the metadata single-write file system can allocate data blocks in persistent memory based on the user files written by the user, thereby writing the corresponding user data into the data blocks, thereby completing the writing of the data (metadata) of the user files.

[0065] 102. After the file creation package and the data update package have been crash-consistently processed, the file creation package and the data update package that have undergone crash-consistency processing will be written into the persistent memory managed by the metadata single-write file system, and the data storage address of the user data in the persistent memory will be determined. Based on the data storage address, the extended index metadata that indexes the user data in the data update package will be determined; wherein, the extended index metadata is used to replace the index data in the data update package.

[0066] Then, the file creation package and data update package need to be crash-consistently processed and then written to the persistent memory managed by the metadata single-write file system. At the same time, the data storage address of the user data in the persistent memory needs to be determined, and the extended index metadata that indexes the user data in the persistent memory needs to be determined based on the index data and the data storage address. It should be noted that the extended index metadata can be used to replace the index data in the data update package.

[0067] In one specific embodiment, the file creation package or data update package can be flushed back through the CPU cache once, and then written into the persistent memory PM after performing a memory barrier (clwb+sfence). Figure 5② (shown as a persistent package in Figure 2) completes the Write Once File System (WOFS) layout. This metadata write-once file system also includes various other types of packages, such as rename packages, delete packages, chmod packages, truncate packages, and create packages. It should be noted that within the memory barrier (clwb+sfence), CLWB (Cache Line Write Back) and SFENCE instructions are used for memory operations, primarily to ensure data correctness and consistency in memory. The CLWB instruction writes dirty data in a cache line back to main memory and ensures that the data cannot be read again before it is written back. Specifically, the CLWB instruction writes dirty data in the cache line at the specified address back to main memory and ensures that the cache line cannot be accessed again until the write-back operation is complete. This is often used to ensure data persistence, especially when using non-volatile memory (NVM). The SFENCE (Store Fence) instruction ensures that all previous store operations have completed before the instruction is executed. The SFENCE instruction prevents the processor from reordering stores that preceded it, thereby ensuring data correctness and consistency. The SFENCE instruction is often used with the CLWB instruction to ensure that no new read operations affect the data before the write-back operation completes. For ease of description, this will not be discussed further.

[0068] Furthermore, since the user data is already stored in the PM, the data update package needs to update the corresponding pointer to point to the user data. Therefore, the index relationship needs to be updated using the original index data and the data storage address of the user data in the PM, i.e., the extended index metadata generated above. It should be noted that the extended index metadata at this time is a form of metadata in the PM.

[0069] 103. Parse the file creation package and data update package stored in the persistent memory to establish a file index node pointing to the file name of the user file through the file index node number, and establish a mapping relationship between the data storage address under the file index node and the file index node number by extending the index metadata.

[0070] Furthermore, the file creation package and data update package stored in the persistent memory are parsed, so as to establish a file index node pointing to the file name of the user file through the file index node number, and by expanding the index metadata, establish a mapping relationship between the data storage address under the file index node and the file index node number.

[0071] In one specific embodiment, after writing is completed, it is necessary to parse the metadata package (specifically parse the file creation package and data update package in this embodiment) to obtain the physical address or the file number pointed to, so as to organize the file (inode) table in the PM for subsequent fast access, that is, Figure 5 Step ③ of parsing the packet into easily accessible metadata objects is shown in Figure 3. It's understandable that when a hard drive is formatted, the operating system automatically divides it into two areas. One is the data area (block area), which stores file data; the other is the inode area (inode table), which stores the information contained in the inode. The inode area is stored in an array, with the corresponding array subscript being the inode code. Each element is a structure (the structure's contents are described above). The inode structure contains a pointer to the block area, which stores the file content. In other words, an inode can be understood as a (file) index node. This file index node establishes an association between the file index node and the user file through a mapping or index relationship between the file index number, extended index metadata, and data storage.

[0072] 104. Insert the file index node corresponding to the user file into the index node table, so that when the user performs a file read operation on the target user file in the application, the user data corresponding to the target user file can be located through the index node table to complete the access to the target user file.

[0073] In this way, the file index node corresponding to the user file can be written into the index node table, so that when the user triggers the file reading operation of the target user file in the application, the user data corresponding to the target user file can be located through the index node table to complete the access to the target user file.

[0074] In one specific embodiment, the file inode is inserted into the already established inode table in the metadata file system. (If the currently written user file is the first user file written to the metadata file system, the inode table is directly established using the file inode. However, in this case, the inode table only has one file inode and its corresponding user file.) Thus, users can access user files using the file inode number of the directly corresponding user file.

[0075] A metadata management method disclosed in this embodiment stores metadata in the form of a metadata package in persistent memory by designing a metadata package (such as a file creation package or a data update package, which is a form of metadata package), ensuring that only the metadata package is written during each file operation, thereby merging multiple small-scale metadata write operations of multiple metadata into a small-scale metadata write operation, thereby reducing PM read and write operations and improving the cost-effectiveness of PM.

[0076] To further understand the above step 102, please refer to Figure 2 , Figure 2 This is a flowchart of another metadata management method disclosed in an embodiment of the present application, including steps 201 to 203.

[0077] 201. Construct a data block in persistent memory, store user data in the data block, and determine a data block pointer pointing to the data block.

[0078] It should be noted that the process of storing user data in data blocks can be either when the user file is written to the metadata file system or when a data update package is created and used to write the user data to the PM. Specifically, data blocks are constructed in persistent memory, user data is stored in the data blocks, and data block pointers pointing to the data blocks are determined. The data block pointers describe the data storage addresses that the user data points to in persistent memory.

[0079] In one specific embodiment, a data block is first constructed in the PM, and then the user data is written to the data block in the persistent memory. Figure 4 The embodiment shown is not described in detail here. For step 201 in this embodiment, that is, Figure 5 ②-(a) shown. It is also understood that the user data can also be stored and allocated in the form of metadata, that is, allocated by a block allocator and aligned in a data block manner, i.e., 4k alignment. Furthermore, data blocks have specific storage addresses in the PM, so data update packets can index the aforementioned data blocks by setting data block pointers. Furthermore, the data block pointer can also index data blocks within a storage range.

[0080] 202. Based on the memory operation instruction, align the index data in the data update package by data block, and record the data correspondence between the index data after block alignment and the data block pointer.

[0081] Then, through the memory operation instructions in the metadata file system, the index data in the data update package can be aligned according to the data blocks, and the data correspondence between the index data after block alignment and the data block pointer can be recorded at the same time.

[0082] In one specific embodiment, since the data update package may include information about the index relationship of the user data, the data update package may be written into the PM, thereby facilitating subsequent users to locate the user data (write data) through the index information in the package. Figure 5 ②-(b) shown. At the same time, the data update package can be aligned with the data block, thereby recording the data correspondence between the index data and the data block pointer after block alignment. The data correspondence can be understood as the index relationship between the data update package and the user data.

[0083] 203. Based on the data correspondence, the data block pointer, and the index data, a data update package that has undergone crash consistency processing is obtained, and the data update package that has undergone crash consistency processing is written into the persistent memory.

[0084] Then, the data update package that has undergone crash consistency processing can be obtained through the data correspondence, data block pointer, and index data. Then, the data update package that has undergone crash consistency processing can be written to the persistent memory.

[0085] In one specific embodiment, after the index relationship between the data update package and the data blocks corresponding to the user data is established, the data update package can be written into the PM.

[0086] According to the metadata management method disclosed in this embodiment, since data operations use different I / O mechanisms, metadata packages and file metadata are associated through index relationships. Therefore, when accessing metadata subsequently, the data update package can be searched first, and then the written metadata can be located based on the index information in the data update package.

[0087] To further understand the above step 103, please refer to Figure 3 , Figure 3 This is a flowchart of another metadata management method disclosed in an embodiment of the present application, including steps 301 to 310.

[0088] 301. Define the package translation layer for the metadata single-write file system.

[0089] Since traditional file operations represent file system operations through traditional file system metadata such as inode, the maintenance of crash consistency must be protected against a large amount of random metadata, which introduces additional reads and writes. That is, this embodiment classifies file system operations into database add, delete, modify, and check (CRUD) operations, which are file creation, file reading, file writing / file updating, and file deletion. Since the read operation does not change the file system state, the construction of read-related packages can be omitted. In the update operation, it can be further divided into file data update and file metadata modification. Therefore, only four basic metadata packages need to be designed: file creation package, file deletion package, data update package, and metadata update package. The above-mentioned file creation package, file deletion package, data update package, and metadata update package are explained below. Among them,

[0090] File creation packet (approximately 256 bytes): The file creation packet (also known as the create packet) is primarily used for operations that require the creation of a new inode, such as creating a file and creating a hard link. The file creation packet contains 64 bytes of static file attributes, such as the creation time and file inode number, 128 bytes of directory entry information, and the remaining 64 bytes of the metadata packet header, which records the packet's magic number, size, type, and so on.

[0091] File deletion packet (approximately 64 bytes): This packet is primarily used for inode deletion operations, such as rm and unlink. It contains the inode attributes of the parent directory to be modified, such as the modification time and the size of the directory after deletion, as well as the attributes of the target inode to be deleted, including the link count and the directory number of the inode to be deleted.

[0092] Data update packet (approximately 64 bytes): This packet is primarily used for operations requiring new block allocation, such as file writes and file preallocation. It records extent index information, enabling indexing of a contiguous range of data, and records the modification time and size of the inode after the write.

[0093] Metadata update package (about 64 bytes): The metadata update package is mainly used for metadata operations, such as changing the file mode chmod and changing the file owner chown, etc. It records these updated fields, including: mode file mode, user state ID (uid), etc.

[0094] For the convenience of description, the above-mentioned file creation package, file deletion package, data update package and metadata update package will not be described in detail later.

[0095] In this embodiment, it is necessary to first define the packet translation layer of the metadata single-write file system.

[0096] In one specific embodiment, since the structure of the single-write data packet destroys the metadata objects that the traditional file system relies on, it is necessary to parse the data packet to provide metadata objects (such as inodes) that are compatible with the traditional file system. To achieve this goal, it is necessary to introduce a packet translation layer for the metadata packet. For easier understanding, please refer to Figure 6 , Figure 6 This is a metadata package conversion and translation design diagram disclosed in the embodiment of this application. Figure 6 As shown, the Packet Translation Level (PTL) is first defined. The basic management unit of the PTL is the metadata package node (i.e., C-node / A-node / W-node in the diagram), which is subsequently referred to as the file index node. It contains key data parsed from the metadata package and the metadata package PM address (pkg addr in the diagram). This address is used for subsequent metadata package access and recovery. Specifically, the file creation package node (C-node) contains the static file attributes, the file name storage address, and the metadata package PM storage address. The metadata update package node (A-node) records the file's dynamic attributes (such as mode) and the metadata update package PM storage address. The data update package node (W-node) records the location (extent) of the index data block, as well as the size (size) and write time (time) of the corresponding write. These will be described separately below.

[0097] Furthermore, the metadata package nodes mentioned above are further organized into metadata objects compatible with traditional file systems.

[0098] It should be noted that after executing step 301, steps 302 to 304, or steps 305 to 307, or steps 308 to 310 may be executed respectively.

[0099] 302. Create a metadata update package.

[0100] For steps 302 to 304, the main description is Figure 6 (a) The inode table, or the implementation process of the file table object, requires first creating a metadata update package (the file creation package is already created when the user writes the file). It should be noted that the metadata update package includes dynamic attribute information, which records the file attributes corresponding to the user's file operations; the file creation package also includes static attribute information, which records the file inode number and directory entry information.

[0101] In one specific embodiment, see Figure 6The file creation package in the PM includes ino, bar, and h, which correspond to static attributes, the user file name, and the storage address of the file creation package in the PM. The metadata update package in the PM includes attr and h, which correspond to dynamic attributes and the storage address of the metadata update package in the PM.

[0102] 303. In the package translation layer, abstractly create a metadata update package node for managing metadata update packages, and a file creation package node for managing file creation packages.

[0103] Then, in the package translation layer, a metadata update package node is abstractly created to manage metadata update packages, and a file creation package node is abstractly created to manage file creation packages. It should be noted that the metadata update package node includes a metadata update package pointer pointing to the persistent memory address where the metadata update package is stored; the file creation package node includes a file creation package pointer pointing to the persistent memory address where the file creation package is stored, and a name pointer pointing to the persistent memory address where the file name is stored.

[0104] In one specific embodiment, the package translation layer first parses the metadata update package and the file creation package to obtain the metadata update package (A-node) and the file creation package (C-node).

[0105] Among them, see Figure 6 As you can see, the file creation package in PM includes ino, bar, and h. Correspondingly, in PTL, they correspond to static attributes, name pointers (pointers to the storage address pointed to by the file name of the user file), and package pointers (pointers to the storage address of the file creation package). The association between static attributes and ino is obtained through parsing; the name pointer and bar, or the package pointer and h, are obtained through pointers in PTL (volatile memory, etc.). The metadata update package in PM includes attr and h. Correspondingly, in PTL, they correspond to dynamic attributes and package pointers (pointers to the storage address of the metadata update package). The association between dynamic attributes and attr is obtained through parsing; the package pointer and h are obtained through pointers in PTL (volatile memory, etc.).

[0106] 304. Based on the dynamic attribute information and the static attribute information, the file creation package node and the metadata update package node are merged to obtain a file index node.

[0107] Then, the file creation package node and the metadata update package node may be merged through the dynamic attribute information and the static attribute information to obtain a file index node.

[0108] In one specific embodiment, the metadata update package node and the file creation package node are merged to form an inode, ie, a file index node, because the two packages have static and dynamic attributes of the file.

[0109] 305. In the package translation layer, abstractly create a data update package node for managing the data update package.

[0110] For steps 305 to 307, the main description is Figure 6 (b) The underlying file abstraction, that is, the implementation of file abstraction.

[0111] Then, in the packet translation layer, a data update package node is abstractly created to manage the data update package. It should be noted that the data update package node includes a data update package pointer pointing to the storage address of the data update package in the persistent memory. Furthermore, since the data update package also records the modification time of the inode after writing and the size information of the inode after writing, Figure 6 As shown, the data update package node can also obtain the modification time and size (i.e., size & time) of the corresponding inode by parsing the data update package. Correspondingly, the data update package node also has the data segment of the data block (user data) indexed by the data update package.

[0112] 306. Establish a data node linked list of the data update package node at the file index node, so as to linearly set a file mapping relationship between the data update package and the file logical address of the user file through the data node linked list. The file logical address is the storage address of the user file in the application.

[0113] 307. Construct a linear index table based on the file mapping relationship, the extended index metadata, and the mapping relationship between the data storage address and the file index node number to index different data update package nodes in the same user file.

[0114] Specifically, first, based on the file index node, a data node linked list of the data update package node is established, and then through the data node linked list, the file mapping relationship between the data update package and the file logical address of the user file is linearly set. The file logical address is the storage address of the user file in the application.

[0115] In one specific embodiment, the packet translation layer needs to find the data update node of the same file to provide file abstraction. Specifically, Figure 6 As shown in (b), the packet translation layer maintains a data list of nodes of the data update package for each file inode, namely the data link.

[0116] Among them, see Figure 6As can be seen, the data update package in the PM, located under the (b) underlying file abstraction, includes data segments 1 and h. Data segment 1 points to data 1 (user data) via a pointer (persistent memory) in the PM. The corresponding data update package nodes for other data update packages are similar and will not be detailed here. Correspondingly, in the PTL, data segment 1 in the PM corresponds to data segment 1 in the PTL, and h corresponds to the storage address of "size & time" in the PM. The above is obtained through parsing. Furthermore, the package pointer is associated with h via a pointer (volatile) in the PTL.

[0117] Then, a linear index table can be constructed through the file mapping relationship, extended index metadata, and the mapping relationship between the data storage address and the file index node number to index different data update package nodes in the same user file. It should be noted that the linear index table is used for users to access the file logical address.

[0118] In one specific embodiment, it can be further combined with Figure 7 As shown, the upper-level data access further builds a linear table on top of the datalist to index different data update package nodes (maintaining the mapping from the file logical address to the data update package node without traversing the linked list) to achieve fast file logical address access.

[0119] Furthermore, in other feasible technical solutions, in order to avoid the multi-core requisition bottleneck, the translation process is designed to be performed independently by each core, and each core is responsible for parsing and accessing the data packets of the core, such as Figure 7 As shown in the PTL, the details are not repeated here.

[0120] 308. At the file index node, a creation node linked list of the file creation package node is established.

[0121] For step 304-step 305, the main description is Figure 6 (c) Low-level directory abstraction, or the abstraction process of directory objects, is a process that first establishes a linked list of creation nodes for the file creation package based on the file index node. It should be noted that the linked list of creation nodes references the corresponding node storage addresses of the file creation package in the metadata single-write file system.

[0122] In one specific embodiment, the Figure 6 As shown in the figure, the packet translation layer maintains a node list (dent list) of file creation packages for each directory inode, that is, a directory necklace (a node list containing multiple file creation packages), which is used to index the inode addresses of all subfiles / directories under the directory.

[0123] Among them, see Figure 6As you can see, the file creation package in PM, located under the (c) underlying directory abstraction, includes 2, `a`, h or 3, `b`, h. Correspondingly, in PTL, they correspond to static attributes, name pointers, and package pointers, respectively. The association between the static attributes and 2 or 3 is obtained through parsing, that is, the file index node number corresponding to different user files. For example, the file index node number of the currently written user file is "ino 1", and the file index node number of the last written user file is "2". Different file creation package nodes are associated through pointers (volatile) in the PTL.

[0124] 309. By creating a node linked list and a hash table, the directory mapping relationship between all file creation packages is set to obtain a directory linked list.

[0125] Thus, the directory mapping relationship between all file creation packages in the metadata package can be set by creating a node linked list and a hash table, so as to obtain a directory linked list through the directory mapping relationship.

[0126] In one specific embodiment, please refer to Figure 7 , Figure 7 This diagram illustrates the architecture implementation of a metadata package single-write file system disclosed in an embodiment of this application. Upper-level directory access further constructs a hash table on top of the dent list to index different file creation package nodes (maintaining a mapping from the file name hash value BKDR to the file creation package node without traversing the linked list), enabling fast file name lookup.

[0127] 310. Abstract the directory list to obtain a directory object so that the user can find the file name of the user file through the directory object.

[0128] Then, the directory list can be abstracted to obtain a directory object, so that the user can find the file name of the user file through the directory object.

[0129] In one specific embodiment, the directory chain list is abstracted into an easily accessible metadata object, i.e., a directory object. Then, the user can complete the search operation of the file name of the user file through the directory object.

[0130] In combination with the above steps 301-310, all file index nodes are obtained in the packet translation layer, and then all file index node numbers are mapped one-to-one with the file names of all user files through a hash table to form a file table object, and the file table object is adjusted according to the file attribute information to obtain the index node table, that is, the inode table.

[0131] The metadata management method disclosed in this embodiment can reduce the complexity of engineering implementation as much as possible, improve the maintainability of the system, meet the needs of single-write metadata, and improve the I / O performance of the file system.

[0132] In other feasible technical solutions, it is also necessary to complete the recycling management of metadata packages, see Figure 4 , Figure 4 This is a flowchart of another metadata management method disclosed in an embodiment of the present application, including steps 401 to 405.

[0133] 401. Determine free blocks in persistent memory and construct a free block index of a red-black tree.

[0134] It should be noted that traditional file system metadata is generally fixedly allocated and does not require dynamic management. However, single-write metadata packages are generated by file system operations and require additional layout, allocation, and recycling management. Figure 7 As shown in the figure, for the management of single-write packets, this embodiment of the application introduces a non-logging layout on the PM (the gray area in the PM). This layout allocates blocks / packets on the PM, similar to malloc / free. It should be noted that a red-black tree is a self-balancing binary search tree, commonly used to implement free block indexing in efficient memory management. The system needs to track available memory blocks (free blocks) to quickly allocate and reclaim memory.

[0135] Therefore, in this embodiment, it is necessary to first determine the free blocks in the persistent memory and build the free block index of the red-black tree. It should be noted that the free blocks are data blocks that have not been written into the file metadata, and the space block index includes the free block number corresponding to each free block on the root node of the red-black tree.

[0136] In one specific embodiment, first, the allocator in the metadata-write-only file system can construct a free block index based on a red-black tree. The space block index needs to correspond to the block number of the free block. It should be noted that the free block is a data block but currently has no user data written to it.

[0137] 402. When allocating the data update package and file creation package that have undergone crash consistency processing to the target free block of any red-black tree root node, subtract the free block number corresponding to the target free block, and allocate the data update package and file creation package that have undergone crash consistency processing to the target free block in the form of a data block, and determine the target free block as the target data block.

[0138] It should be noted that steps 402 and 403 are the metadata package allocation and indexing steps implemented in the metadata package allocation phase or the metadata package fast recovery phase. Specifically, when allocating crash-consistent data update packages and file creation packages to the target free block of any red-black tree root node, the free block number corresponding to the target free block is subtracted, and the crash-consistent data update packages and file creation packages are allocated to the target free block as data blocks, and the target free block is determined to be the target data block.

[0139] In one specific embodiment, the metadata write-only file system can allocate space at any location, thereby fully utilizing the high-speed I / O and erase-free features of PM. Figure 7 As shown in the PM section, a metadata package can be inserted into a free block in any format, and the free block after insertion is determined to be the target data block. In a feasible technical solution, the metadata package often exists in the form of a combined package, and then the metadata package is written to the PM by forming a package group. For details, see step 403.

[0140] Furthermore, the packet translation layer can parse the above packet group, thereby completing the parsing of the metadata packet.

[0141] 403. Record the target free block number of each data update package and file creation package after allocation through the global coarse-grained dimension, and use the bitmap to complete the bitmap index of the data update package and file creation package after crash consistency processing and the target data block, so as to insert the data update package and file creation package after crash consistency processing into the root node of the red-black tree.

[0142] Corresponding to step 402, the target free block number of each data update package and file creation package after allocation can be recorded using a global coarse-grained dimension. A bitmap index is then used to complete the bitmap indexing of the crash-consistent data update package and file creation package and the target data block, thereby inserting the crash-consistent data update package and file creation package into the root node of the red-black tree. At this point, the corresponding bit in the bitmap is 1.

[0143] In one specific embodiment, since traditional file systems can perform fast file system crash recovery by scanning fixed metadata areas, metadata-write-only file systems need to scan metadata packages on all media, perform package parsing, and rebuild the metadata package translation layer. Figure 7As shown in the PM section, to speed up this process, a coarse-grained allocation method is used to allocate and index metadata packets. Specifically, metadata packets (such as data update packets and file creation packets, which will not be described in detail later) are allocated in the form of data blocks (package groups pkg-group in the figure), and the data blocks of each metadata packet are recorded in real time through a global coarse-grained bitmap. Specifically, the metadata single-write file system first allocates a data block in units of 4KB blocks, and then uses a u64 bitmap to index the 64 64B slots within the 4KB block. Allocating a metadata packet is equivalent to setting the corresponding position of the u64 bitmap to 1.

[0144] Furthermore, the global coarse-grained bitmap can be set to 1 during the initial data block allocation. Because a data block can store multiple metadata packages, the overhead of updating the global coarse-grained bitmap is negligible. Consequently, during file system crash recovery, only the data blocks corresponding to the bitmap need to be scanned, significantly reducing the amount of PM media that needs to be scanned during recovery and accelerating file system crash recovery.

[0145] 404. Based on the user's file recovery operation on the user file, a file deletion package is generated in the metadata write-only file system.

[0146] Additionally, steps 404 and 405 are file recovery operations. Specifically, a file deletion package can be generated in the metadata write-only file system based on the user's file recovery operation. It should be noted that file deletion includes a file deletion instruction, which is used to delete the file inode number or directory entry information.

[0147] In one specific embodiment, when a metadata package (which can be understood as the recycling of user files due to the mapping relationship between metadata packages and user data) is recycled, a combined package can be provided, including a file deletion package, a file creation package, or a data update package. The file deletion package will then mark the space of the file creation package as invalid (i.e., a causal dependency: the file deletion package causes the creation package to be recycled). The data update package will also mark the overwritten data package (the original data update package) as invalid (i.e., a causal dependency: the new data update package causes the old data update package to be recycled). The same applies to the metadata update package. This completes the metadata package recycling operation.

[0148] In other feasible technical solutions, for other combined metadata packages (e.g., a rename operation consisting of a file creation package and a file deletion package), the PM space occupied by the package is reclaimed only when the space of all sub-packages that make up the combined package is marked as invalid. Furthermore, to avoid multi-core requisition bottlenecks, each single-core CPU is designed with an allocator to allocate and reclaim data packages for that core.

[0149] 405. When recycling the data update package and file creation package that have undergone crash consistency processing, if the data update package or file creation package is not stored in the target data block, the corresponding position of the bitmap is set to 0, and based on the causal dependency relationship between the file deletion package and the file creation package, the space block where the file creation package that has undergone crash consistency processing is located is marked as invalid, and the space block where the original data update package covered by the data update package that has undergone crash consistency processing is located is marked as invalid, and the file creation package and the original data update package are recycled.

[0150] Therefore, when the data update package and file creation package that have undergone crash consistency processing are recycled, the corresponding position of the bitmap is set to 0, and based on the causal dependency between the file deletion package and the file creation package, the space block where the file creation package that has undergone crash consistency processing is located is marked as invalid, and the space block where the original data update package covered by the data update package that has undergone crash consistency processing is located is marked as invalid, and the file creation package and the original data update package are recycled.

[0151] In one specific embodiment, as can be seen in step 403, when a u64 bitmap is used to index the 64 64B slots within the 4KB block and metadata packets are retrieved, the corresponding positions in the u64 bitmap are set to 0. Furthermore, the bitmap is set to 0 only when there are no more metadata packets (e.g., file deletion packets, file creation packets, metadata update packets, or data update packets) within the single data block.

[0152] Through the metadata management method disclosed in this embodiment, the file system can allocate space at any location, fully leveraging the PM's high-speed I / O and erase-free features. Simultaneously, space is quickly recovered through soft garbage collection. Soft garbage collection infers causal dependencies between operations and directly frees up space for invalid data packets, returning it to the allocator.

[0153] Therefore, combined with the above Figures 1 to 4 In the illustrated embodiment, in the metadata single-write file system architecture of the present application, crash-consistent metadata I / O is performed only once, minimizing overhead and fully utilizing the throughput performance of the new persistent memory. Furthermore, the non-logging layout management method and soft garbage collection mechanism minimize the garbage collection overhead of metadata packets, making PM space management controllable and efficient.

[0154] It should be understood that, although the various steps in the flowcharts involved in the various embodiments described above are displayed in sequence as indicated by the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and these steps can be executed in other orders. Moreover, at least a portion of the steps in the flowcharts involved in the various embodiments described above may include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a portion of the steps or stages in other steps.

[0155] See also Figure 8 , Figure 8 This is a schematic diagram of the structure of a metadata management system disclosed in an embodiment of the present application.

[0156] A generating unit 801 is configured to generate a file creation package and a data update package in the metadata-write-only file system based on a user's file write operation on a user file in an application program. The file creation package includes a file inode number and directory entry information corresponding to the user file, and the directory entry information includes the file name of the user file. The data update package includes index data, which is used to index user data. The user data is data written by the user file to the metadata-write-only file system.

[0157] The writing unit 802 is configured to write the file creation package and the data update package after crash consistency processing to the persistent memory managed by the metadata single-write file system, determine the data storage address of the user data in the persistent memory, and determine the extended index metadata for indexing the user data in the data update package based on the data storage address; wherein the extended index metadata is used to replace the index data in the data update package;

[0158] The parsing unit 803 is configured to parse the file creation package and the data update package stored in the persistent memory to establish a file index node pointing to the file name of the user file through the file index node number, and to establish a mapping relationship between the data storage address under the file index node and the file index node number by extending the index metadata;

[0159] The writing unit 802 is also used to insert the file index node corresponding to the user file into the index node table, so that when the user performs a file reading operation on the target user file in the application, the user data corresponding to the target user file can be located through the index node table to complete the access to the target user file.

[0160] Exemplarily, the system further includes: a determination unit 804, an alignment unit 805, and an acquisition unit 806;

[0161] The determining unit 804 is configured to construct a data block in the persistent memory, store the user data in the data block, and determine a data block pointer pointing to the data block; wherein the data block pointer is used to describe the data storage address pointed to by the user data in the persistent memory;

[0162] An alignment unit 805 is configured to align the index data in the data update packet by data blocks based on the memory operation instruction, and record the data correspondence between the index data and the data block pointers after the block alignment;

[0163] An acquisition unit 806 is configured to obtain a data update package that has undergone crash consistency processing based on the data correspondence, the data block pointer, and the index data;

[0164] The writing unit 802 is specifically configured to write the data update package that has undergone crash consistency processing into the persistent memory.

[0165] Exemplarily, the system further includes: a definition unit 807 and a creation unit 808;

[0166] A definition unit 807, configured to define a packet translation layer for a metadata write-only file system;

[0167] The creation unit 808 is configured to create a metadata update package; wherein the metadata update package includes dynamic attribute information, which records file attribute information corresponding to the user's file operation; the file creation package also includes static attribute information, which records the file inode number and directory entry information;

[0168] The creation unit 808 is further configured to abstractly create, in the package translation layer, a metadata update package node for managing metadata update packages and a file creation package node for managing file creation packages. The metadata update package node includes a metadata update package pointer pointing to the storage address of the metadata update package in persistent memory. The file creation package node includes a file creation package pointer pointing to the storage address of the file creation package in persistent memory and a name pointer pointing to the storage address of the file name in persistent memory.

[0169] The acquiring unit 806 is specifically configured to merge the file creation package node and the metadata update package node based on the dynamic attribute information and the static attribute information to obtain a file index node.

[0170] Exemplarily, the system further includes: an establishing unit 809;

[0171] The creation unit 808 is specifically configured to abstractly create a data update package node for managing the data update package in the packet translation layer; wherein the data update package node includes a data update package pointer pointing to the storage address of the data update package in the persistent memory;

[0172] An establishing unit 809 is configured to establish a data node linked list of data update package nodes at the file index node, so as to linearly set a file mapping relationship between the data update package and the file logical address of the user file through the data node linked list, where the file logical address is the storage address of the user file in the application program;

[0173] Establishment unit 809 is also used to construct a linear index table based on the file mapping relationship, extended index metadata and the mapping relationship between the data storage address and the file index node number to index different data update package nodes in the same user file; wherein the linear index table is used for users to access the file logical address.

[0174] Exemplarily, the system further includes: a setting unit 810;

[0175] The establishing unit 809 is further used to establish a creation node linked list of the file creation package node based on the file index node;

[0176] A setting unit 810 is used to set the directory mapping relationship between all file creation packages by creating a node link table and a hash table to obtain a directory link table;

[0177] The acquisition unit 806 is further configured to abstract the directory linked list to obtain a directory object, so that the user can search for the file name of the user file through the directory object.

[0178] Exemplarily, the system includes:

[0179] The acquisition unit 806 is specifically used in the packet translation layer to obtain all file index nodes, map all file index node numbers to the file names of all user files one by one through a hash table, form a file table object, and adjust the file table object through file attribute information to obtain an index node table.

[0180] Exemplarily, the system includes:

[0181] The determining unit 804 is specifically configured to determine free blocks in the persistent memory and construct a free block index of the red-black tree; wherein the free blocks are data blocks to which file metadata is not written, and the free block index includes the free block number corresponding to each free block on the root node of the red-black tree;

[0182] The determining unit 804 is further configured to, when allocating the crash-consistent data update package and the file creation package to a target free block of any red-black tree root node, subtract the free block number corresponding to the target free block, allocate the crash-consistent data update package and the file creation package to the target free block in the form of a data block, and determine the target free block as the target data block;

[0183] The write unit 802 is specifically used to record the target free block number of each data update package and file creation package after allocation through a global coarse-grained dimension, and use a bitmap to complete the bitmap index of the data update package and file creation package after crash consistency processing and the target data block, so as to insert the data update package and file creation package after crash consistency processing into the root node of the red-black tree; wherein, the corresponding bit of the bitmap is 1.

[0184] Exemplarily, the system further includes: a recovery unit 811;

[0185] The generating unit 801 is further configured to generate a file deletion packet in the metadata write-only file system based on a file recycling operation of a user on a user file; wherein the file deletion includes a file deletion instruction, and the file deletion instruction is configured to delete a file inode number or directory entry information;

[0186] The recycling unit 811 is used to, when recycling the data update package and the file creation package that have undergone crash consistency processing, if the data update package or the file creation package is not stored in the target data block, set the corresponding position of the bitmap to 0, and based on the causal dependency relationship between the file deletion package and the file creation package, mark the space block where the file creation package that has undergone crash consistency processing is located as invalid, mark the space block where the original data update package covered by the data update package that has undergone crash consistency processing is located as invalid, and recycle the file creation package and the original data update package.

[0187] See below Figure 9 The structural diagram of a metadata management device disclosed in an embodiment of the present application includes:

[0188] CPU 901, memory 905, input / output interface 904, wired or wireless network interface 903 and power supply 902;

[0189] The memory 905 is a temporary storage memory or a permanent storage memory;

[0190] The CPU 901 is configured to communicate with the memory 905 and execute the instructions in the memory 905 to perform the aforementioned Figures 1 to 4 The metadata management method in any of the illustrated embodiments.

[0191] The embodiment of the present application also provides a chip system, which includes at least one processor and a communication interface, wherein the communication interface and the at least one processor are interconnected through a line, and the at least one processor is used to run a computer program or instruction to execute the aforementioned Figures 1 to 4 The metadata management method in any of the illustrated embodiments.

[0192] The embodiment of the present application also provides a computer-readable storage medium, which includes instructions. When the instructions are executed on a computer, the computer executes the aforementioned Figures 1 to 4 The metadata management method in any of the illustrated embodiments.

[0193] The present application also provides a computer program product comprising instructions, which, when executed on a computer, causes the computer to execute the aforementioned Figures 1 to 4 The metadata management method in any of the illustrated embodiments.

[0194] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0195] In the several embodiments provided in this application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of the units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0196] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0197] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0198] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, read-only memory), random access memory (RAM, random access memory), disk or optical disk, and other media that can store program code.

Claims

1. A metadata management method, characterized in that: include: Based on a user's file write operation on a user file in an application, a file creation package and a data update package are generated in the metadata single-write file system; wherein the file creation package includes a file inode number and directory entry information corresponding to the user file, and the directory entry information includes the file name of the user file; and the data update package includes index data, the index data being used to index user data, the user data being data written by the user file to the metadata single-write file system; After the file creation package and the data update package have undergone crash consistency processing, the file creation package and the data update package that have undergone crash consistency processing are written into the persistent memory managed by the metadata single-write file system, the data storage address of the user data in the persistent memory is determined, and based on the data storage address, the extended index metadata that indexes the user data in the data update package is determined; wherein the extended index metadata is used to replace the index data in the data update package; Parsing the file creation package and the data update package stored in the persistent memory to establish a file index node pointing to the file name of the user file through the file index node number, and establishing a mapping relationship between the data storage address and the file index node number under the file index node through the extended index metadata; The file index node corresponding to the user file is inserted into the index node table, so that when the user performs a file reading operation on the target user file in the application program, the user data corresponding to the target user file can be located through the index node table to complete the access to the target user file.

2. The metadata management method according to claim 1, characterized in that: The step of writing the file creation package and the data update package that have undergone crash consistency processing into the persistent memory of the metadata single-write file system includes: Constructing a data block in the persistent memory, storing the user data in the data block, and determining a data block pointer pointing to the data block; wherein the data block pointer is used to describe the data storage address pointed to by the user data in the persistent memory; Based on the memory operation instruction, the index data in the data update package is aligned according to the data block, and the data correspondence between the index data after the block alignment and the data block pointer is recorded; Obtaining a data update package after crash consistency processing based on the data correspondence, the data block pointer, and the index data; Writing the crash-consistent data update package into the persistent memory.

3. The metadata management method according to claim 1, wherein: The step of establishing a file index node pointing to the file name of the user file by using the file index node number includes: A package translation layer defining the metadata single-write file system; Creating a metadata update package; wherein the metadata update package includes dynamic attribute information, the dynamic attribute information records file attribute information corresponding to the file operation of the user file; the file creation package also includes static attribute information, the static attribute information records the file index node number and the directory entry information; In the package translation layer, a metadata update package node for managing the metadata update package and a file creation package node for managing the file creation package are abstractly created; wherein the metadata update package node includes a metadata update package pointer pointing to the storage address of the metadata update package in the persistent memory; the file creation package node includes a file creation package pointer pointing to the storage address of the file creation package in the persistent memory and a name pointer pointing to the storage address of the file name in the persistent memory; Based on the dynamic attribute information and the static attribute information, the file creation package node and the metadata update package node are merged to obtain the file index node.

4. The metadata management method according to claim 3, characterized in that: The mapping relationship between the data storage address and the file index node number is established under the file index node by using the extended index metadata, including: In the packet translation layer, a data update package node for managing the data update package is abstractly created; wherein the data update package node includes a data update package pointer pointing to a storage address of the data update package in the persistent memory; Establishing a data node linked list of the data update package node at the file index node, so as to linearly set a file mapping relationship between the data update package and the file logical address of the user file through the data node linked list, where the file logical address is the storage address of the user file in the application program; A linear index table is constructed based on the file mapping relationship, the extended index metadata, and the mapping relationship between the data storage address and the file index node number to index different data update package nodes in the same user file; wherein, the linear index table is used by the user to access the file logical address.

5. The metadata management method according to claim 3, characterized in that: After defining the packet translation layer of the metadata single-write file system, the method further includes: At the file index node, establishing a creation node linked list of the file creation package node; By creating the node linked list and the hash table, setting the directory mapping relationship between all file creation packages to obtain a directory linked list; The directory linked list is abstracted to obtain a directory object, so that the user can search for the file name of the user file through the directory object.

6. The metadata management method according to claim 1, wherein: The method for generating the index node table includes: In the packet translation layer, all file index nodes are obtained, and all the file index node numbers are mapped one-to-one with the file names of all the user files through a hash table to form a file table object, and the file table object is adjusted according to the file attribute information to obtain an index node table.

7. The metadata management method according to claim 1, characterized in that: Writing the file creation package and the data update package after crash consistency processing into the persistent memory of the metadata single-write file system includes: Determine free blocks in the persistent memory and construct a free block index of a red-black tree; wherein the free blocks are data blocks to which the file metadata is not written, and the space block index includes a free block number corresponding to each free block on the root node of the red-black tree; When allocating the data update package and file creation package that have undergone crash consistency processing to the target free block of any of the red-black tree root nodes, subtracting the free block number corresponding to the target free block, allocating the data update package and file creation package that have undergone crash consistency processing to the target free block in the form of a data block, and determining the target free block as the target data block; The target free block number of each data update package and file creation package after allocation is recorded through a global coarse-grained dimension, and the bitmap is used to complete the bitmap index of the data update package and file creation package after crash consistency processing and the target data block, so as to insert the data update package and file creation package after crash consistency processing into the red-black tree root node; wherein, the corresponding bit of the bitmap is 1.

8. The metadata management method according to claim 7, characterized in that: The method further comprises: Based on the user's file recycling operation on the user file, a file deletion package is generated in the metadata write-only file system; wherein the file deletion includes a file deletion instruction, and the file deletion instruction is used to delete the file index node number or the directory entry information; When the data update package and file creation package that have undergone crash consistency processing are recycled, if the data update package or file creation package is not stored in the target data block, the corresponding position of the bitmap is set to 0, and based on the causal dependency between the file deletion package and the file creation package, the space block where the file creation package that has undergone crash consistency processing is located is marked as invalid, and the space block where the original data update package covered by the data update package that has undergone crash consistency processing is located is marked as invalid, and the file creation package and the original data update package are recycled.

9. A metadata management system, characterized in that: The system comprises: a generating unit configured to generate a file creation package and a data update package in the metadata single-write file system based on a file write operation of a user on a user file in an application; wherein the file creation package includes a file inode number and directory entry information corresponding to the user file, the directory entry information including the file name of the user file; and the data update package includes index data for indexing user data, the user data being data written by the user file to the metadata single-write file system; a writing unit, configured to write the file creation package and the data update package that have undergone crash consistency processing into the persistent memory managed by the metadata single-write file system after crash consistency processing has been performed on the file creation package and the data update package, determine the data storage address of the user data in the persistent memory, and determine, based on the data storage address, the extended index metadata in the data update package that indexes the user data; wherein the extended index metadata is used to replace the index data in the data update package; a parsing unit, configured to parse the file creation package and the data update package stored in the persistent memory, so as to establish a file index node pointing to the file name of the user file through the file index node number, and to establish a mapping relationship between the data storage address and the file index node number under the file index node through the extended index metadata; The writing unit is further configured to insert the file index node corresponding to the user file into the index node table, so that when the user performs a file reading operation on the target user file in the application, the user data corresponding to the target user file can be located through the index node table to complete the access to the target user file.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium includes instructions, and when the instructions are executed on a computer, the computer is caused to execute the metadata management method according to any one of claims 1 to 8.

Images