Distributed file system metadata index optimization method based on remote direct memory access

Abstract

The invention discloses a distributed file system metadata index optimization method based on remote direct memory access, which comprises directory metadata index based on path name hash: defining a plurality of layers of directories in a directory tree as key directories, and regarding the path as a character string. After calculation of the hash function, determining the storage position of the data in the memory of the metadata server of the distributed file system; and optimizing the memory position of the file metadata based on the popularity, namely placing the file metadata which is frequently accessed at a position adjacent to the key directory metadata to which the file metadata belongs. A metadata management module in a metadata server of the distributed file system regularly checks the popularity of the file and carries out cold and hot migration according to the situation; and client cache optimization oriented to key directory metadata: a metadata cache of a distributed file system client caches the metadata and directory entries of the key directory. Optimally assuming that the metadata and directory entries of the key directory in the cache are not invalid, and updating the cache according to the result of the next-level metadata search. According to the method, the remote reading performance of the metadata index of the distributed file system is optimized by fully utilizing the characteristic of remote direct memory access, the network transmission frequency is reduced, and the load of the metadata server is effectively reduced.

Description

[0001] The present invention relates to the field of distributed file systems, and in particular to a distributed file system metadata index optimization method, and in particular to a distributed file system metadata index optimization method based on remote direct memory access. Background Art

[0002] Distributed file systems are widely used in cloud servers, streaming media applications, social networks, and other fields. They efficiently manage and store massive amounts of data, provide reliable data access services, and offer powerful data management and storage solutions for users and businesses. They allow files to be distributed across multiple nodes, improving data reliability and performance. With the expansion of application data volumes and the emergence of new technologies, distributed file systems have seen further development and widespread use.

[0003] Metadata servers in current distributed file systems are a common bottleneck. As data volumes and user access continue to grow, the number of metadata requests that metadata servers must process also increases. Furthermore, metadata servers communicate with each client using the Transmission Control Protocol (TCP). The memory copy overhead and user-to-kernel switching overhead introduced by TCP further increase the metadata server load.

[0004] The current distributed file system metadata index design has problems such as high network transmission protocol software stack overhead and excessive metadata requests that the metadata server needs to process.

[0005] Remote Direct Memory Access (RDMA) is a computer network communication technology that allows two computers on a network to transfer data directly between each other without the intervention of the operating system. RDMA technology allows data to be transferred directly from the sender's memory to the receiver's memory without passing through the operating system's kernel space, thereby significantly reducing data transmission latency and CPU overhead. RDMA works based on dedicated hardware on the network adapter (network interface card, NIC), which provides a mechanism to implement data transmission that bypasses the operating system kernel. Through RDMA, the sender can send the memory address and length of the data to the receiver, and the receiver can read the data directly from the sender's memory without the need for an intermediate copy operation.

[0006] RDMA technology allows us to improve the existing distributed file system metadata index to support distributed file system clients to read directly from the memory of the distributed file system metadata server, further reducing the load on the distributed file system metadata server. Summary of the Invention

[0007] In view of the defects of the prior art, the object of the present invention is to provide a distributed file system metadata index optimization method based on remote direct memory access.

[0008] According to the present invention, a distributed file system metadata index optimization method based on remote direct memory access is provided, comprising:

[0009] Directory metadata indexing based on pathname hashing: Define directories at levels N0, N1, and so on in the directory tree as key directories. Treat the path to the key directory as a string and calculate the string using a hash function to obtain the location of the key directory metadata in the metadata memory pool of the distributed file system metadata server. The distributed file system metadata server is responsible for writing directory metadata to the corresponding location in the metadata memory pool. The distributed file system client uses remote direct memory access (RDMA) technology and a one-sided read primitive to directly obtain the required directory metadata from this location.

[0010] Popularity-based file metadata memory location optimization: The metadata memory pool of the distributed file system metadata server is divided into three areas: the block area, the directory entry area, and the inode area. In the block area, a 4096-byte memory block is pre-allocated for metadata in each key directory, allowing frequently accessed file metadata to be placed adjacent to the metadata of the key directory to which it belongs. The metadata management module regularly checks file popularity based on client feedback and dynamically migrates file metadata stored in the pre-allocated blocks.

[0011] Client-side caching optimization for key directory metadata: Distributed file system clients cache metadata and directory entries for key directories. This cache uses an optimistic access method, assuming that the locally cached metadata and directory entries for key directories are valid. Therefore, it attempts to retrieve metadata for next-level directories / files using remote direct memory technology. If the retrieved metadata for next-level directories / files is incorrect, the system updates the key directory metadata to ensure consistency in the client's meta-cache.

[0012] Preferably, the directory metadata index based on pathname hash includes:

[0013] Component A1: When establishing a connection with a distributed file system client, the distributed file system metadata server informs the client of the specific values ​​of the key directory levels N0 and N1 set by the metadata server, the hash function used, and the conflict resolution method;

[0014] Component A2: The client metadata indexing module of the distributed file system client sends a write request to the distributed file system metadata server via the TCP communication module. Upon receiving the request, the metadata management module of the distributed file system metadata server uses a hash function to calculate the in-memory location of the metadata corresponding to the request and makes the appropriate modifications. If the modification is successful, the server returns a successful write request message to the client.

[0015] Component A3: When the distributed file system client reads remote metadata, the client metadata index module first determines the number of key directory layers used, and uses a hash function to calculate the location of the metadata in the distributed file system metadata server memory. Then, the distributed file system client uses the RDMA communication module and the one-sided read primitive to obtain the required metadata from the distributed file system metadata server memory. After obtaining the metadata, the distributed file system client will verify it to confirm whether it is the required metadata. If the verification fails, it means that a hash conflict may have occurred, and the client will retry to read the metadata according to the conflict handling method agreed upon in Component A1;

[0016] Preferably, the file metadata memory location optimization based on popularity includes:

[0017] Component B1: The metadata memory pool of the distributed file system metadata server is divided into three areas: the block area, the directory entry area, and the index node area. The block area is divided into several memory blocks of 4096 bytes in size, each of which stores one directory metadata and several file metadata. The distributed file system client uses 4096 bytes as the granularity to remotely directly access the metadata in the block area; the directory entry area stores the directory entries of the directory, and the directory entries record the location of the corresponding directory / file metadata in the metadata memory pool of the distributed file system meta server; the index node area stores the directory metadata and file metadata that are not stored in the block area;

[0018] Component B2: The metadata management module of the distributed file system metadata server places the metadata of frequently accessed files and the directory metadata of the directory to which they belong in adjacent locations in the memory, that is, in the same block in the block area. The metadata management module maintains a heat table for each key directory in the block area, recording the access heat of the file metadata under the directory. The distributed file system client will periodically send the heat information of the client's access to the metadata server. The metadata management module will update the heat table based on this information and migrate the metadata of infrequently accessed files to the index node area according to the heat table, while the metadata of frequently accessed files will be migrated to the block area;

[0019] Preferably, the client cache optimization step for key directory metadata includes:

[0020] Step C1: Find metadata and directory entries of the key directory in the client metadata cache;

[0021] Step C2: For the cache search process in step C1, if no corresponding cache is found, a hash function is used to calculate the location of the metadata in the metadata memory pool of the distributed file system metadata server, and remote direct memory access is used to obtain the metadata and directory entries, and the metadata are written into the client metadata cache, and the cache LRU list is updated;

[0022] Step C3: For the cache lookup process in step C1, if the corresponding cache is found, assuming that the directory metadata and directory entries in the local cache are valid, remote direct memory access is performed to the metadata of the next-level directory / file using the one-sided read primitive through the RDMA communication module;

[0023] Step C4: For the remote direct memory access process in step C3, if the path name of the obtained metadata is consistent with the required path name, it is determined that the directory metadata and directory entries cached locally in step C3 are valid, and the cache LRU linked list is updated;

[0024] Step C5: For the remote direct memory access process in step C3, if the pathname of the obtained metadata is inconsistent with the required pathname, the directory metadata and directory entries cached locally in step C3 are determined to be invalid, and the cache is deleted. A hash function is used to calculate the location of the metadata in the metadata memory pool of the distributed file system metadata server. The metadata and directory entries are obtained through remote direct memory access and written into the client metadata cache, and the cache LRU list is updated.

[0025] Compared with the prior art, the present invention has the following beneficial effects:

[0026] 1. Low load on the distributed file system metadata server. This invention uses RDMA technology to reduce the overhead of the network transmission software stack and offloads the metadata reading work to the client, reducing the load on the distributed file system metadata server;

[0027] 2. Low network communication. The directory metadata indexing based on pathname hashing proposed in this invention uses pathnames to calculate the location of key metadata without querying the directory tree, significantly reducing the number of remote accesses. The client-side cache optimization for key directory metadata proposed in this invention caches key metadata on the client, preventing it from being frequently accessed.

[0028] 3. High efficiency in reading hotspot file metadata. The present invention proposes a method of optimizing the memory location of file metadata based on popularity. When reading a directory, the metadata of frequently accessed hotspot files in that directory is also read. This reduces the number of remote accesses required to read hotspot file metadata and speeds up the reading of hotspot file metadata. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 It is the overall framework diagram of the present invention.

[0030] Figure 2 The figure is a schematic diagram of a directory metadata index structure based on path name hash according to a specific embodiment of the present invention.

[0031] Figure 3 Schematic diagram of the file metadata memory location optimization structure based on popularity according to a specific embodiment of the present invention.

[0032] Figure 4 The figure is a schematic diagram of the steps of optimizing client cache for key directory metadata according to a specific embodiment of the present invention. DETAILED DESCRIPTION

[0033] The present invention is further described below in conjunction with the accompanying drawings and specific embodiments. It should be understood that these embodiments are only used to illustrate the present invention and are not used to limit the scope of the present invention. After reading the present invention, modifications of various equivalent forms of the present invention made by those skilled in the art all fall within the scope defined by the claims attached to this application.

[0034] According to the present invention, a distributed file system metadata index optimization method based on remote direct memory access is provided. Figure 1 As shown, including:

[0035] Directory metadata indexing based on pathname hashing: Define directories at levels N0, N1, and so on in the directory tree as key directories. Treat the path to the key directory as a string and calculate the string using a hash function to obtain the location of the key directory metadata in the metadata memory pool of the distributed file system metadata server. The distributed file system metadata server is responsible for writing directory metadata to the corresponding location in the metadata memory pool. The distributed file system client uses remote direct memory access (RDMA) technology and a one-sided read primitive to directly obtain the required directory metadata from this location.

[0036] Popularity-based file metadata memory location optimization: The metadata memory pool of the distributed file system metadata server is divided into three areas: the block area, the directory entry area, and the inode area. In the block area, a 4096-byte memory block is pre-allocated for metadata in each key directory, allowing frequently accessed file metadata to be placed adjacent to the metadata of the key directory to which it belongs. The metadata management module regularly checks file popularity based on client feedback and dynamically migrates file metadata stored in the pre-allocated blocks.

[0037] Client-side caching optimization for key directory metadata: Distributed file system clients cache metadata and directory entries for key directories. This cache uses an optimistic access method, assuming that the locally cached metadata and directory entries for key directories are valid. Therefore, it attempts to retrieve metadata for next-level directories / files using remote direct memory technology. If the retrieved metadata for next-level directories / files is incorrect, the system updates the key directory metadata to ensure consistency in the client's meta-cache.

[0038] Specifically, the directory metadata index based on the path name hash includes the following components:

[0039] Component A1: If Figure 2 As shown, when the distributed file system metadata server establishes a connection with the distributed file system client, it uses the TCP communication module to inform the client of the specific values ​​of the key directory layer number N0, N1 set by the metadata server, the hash function used and the conflict handling method, and establishes an RDMA connection with the client;

[0040] Component A2: The client metadata indexing module of the distributed file system client sends a write request to the distributed file system metadata server via the TCP communication module. Upon receiving the request, the metadata management module of the distributed file system metadata server uses a hash function to calculate the in-memory location of the metadata corresponding to the request and makes the appropriate modifications. If the modification is successful, the server returns a successful write request message to the client.

[0041] Component A3: If Figure 2As shown, when the distributed file system client reads remote metadata, the client metadata index module first determines the number of key directory layers used, and uses a hash function to calculate the location of the metadata in the distributed file system metadata server memory. Then, the distributed file system client uses the RDMA communication module and the one-sided read primitive to obtain the required metadata from the distributed file system metadata server memory. Finally, after obtaining the metadata, the distributed file system client will verify it to confirm whether it is the required metadata. If the verification fails, it means that a hash conflict may have occurred, and the client will retry to read the metadata according to the conflict handling method agreed upon in component A1;

[0042] Specifically, the file metadata memory location optimization based on popularity includes:

[0043] Component B1: e.g. Figure 3 As shown, the metadata memory pool of the distributed file system metadata server is divided into three areas: the block area, the directory entry area, and the index node area. The block area is divided into several memory blocks of 4096 bytes in size, each of which stores a directory metadata and several file metadata. The distributed file system client uses 4096 bytes as the granularity to remotely directly access the metadata in the block area; the directory entry area stores the directory entries of the directory, and the directory entries record the location of the corresponding directory / file metadata in the metadata memory pool of the distributed file system meta server; the index node area stores the directory metadata and file metadata that are not stored in the block area;

[0044] Component B2: e.g. Figure 3 As shown, the metadata management module of the distributed file system metadata server places the metadata of frequently accessed files and the directory metadata of the directory to which they belong in adjacent locations in the memory, that is, in the same block in the block area. The metadata management module maintains a heat table for each key directory in the block area, recording the access heat of the file metadata under the directory. The distributed file system client will periodically send the heat information of the client's access to the metadata server. The metadata management module will update the heat table based on this information, and migrate the metadata of files that are not frequently accessed to the index node area according to the heat table, while the metadata of files that are frequently accessed will be migrated to the block area;

[0045] Specifically, the client cache optimization for key directory metadata, such as Figure 4 As shown, including:

[0046] Step C1: Find metadata and directory entries of the key directory in the client metadata cache;

[0047] Step C2: For the cache search process in step C1, if no corresponding cache is found, a hash function is used to calculate the location of the metadata in the metadata memory pool of the distributed file system metadata server, and remote direct memory access is used to obtain the metadata and directory entries, and the metadata are written into the client metadata cache, and the cache LRU list is updated;

[0048] Step C3: For the cache lookup process in step C1, if the corresponding cache is found, assuming that the directory metadata and directory entries in the local cache are valid, remote direct memory access is performed to the metadata of the next-level directory / file using the one-sided read primitive through the RDMA communication module;

[0049] Step C4: For the remote direct memory access process in step C3, if the path name of the obtained metadata is consistent with the required path name, it is determined that the directory metadata and directory entries cached locally in step C3 are valid, and the cache LRU linked list is updated;

[0050] Step C5: For the remote direct memory access process in step C3, if the pathname of the obtained metadata is inconsistent with the required pathname, the directory metadata and directory entries cached locally in step C3 are determined to be invalid, and the cache is deleted. A hash function is used to calculate the location of the metadata in the metadata memory pool of the distributed file system metadata server. The metadata and directory entries are obtained through remote direct memory access and written into the client metadata cache, and the cache LRU list is updated.

[0051] According to the present invention, a distributed file system metadata index optimization method based on remote direct memory access is provided, comprising:

[0052] Directory metadata indexing based on pathname hashing: This method uses pathname hashing to obtain the metadata location in the metadata memory pool of the distributed file system metadata server. This reduces the number of remote direct memory accesses required to obtain the directory and reduces the load on the metadata server.

[0053] Optimizes the memory location of file metadata based on popularity: Frequently accessed file metadata and directory metadata are placed adjacent to each other in memory, reducing the number of remote direct memory accesses when accessing hot file metadata and lowering the load on the metadata server.

[0054] Client-side caching optimization for key directory metadata: Caching metadata of key nodes reduces frequent access to key node metadata during path traversal, reducing the load on the metadata server.

[0055] In the description of this application, it should be understood that the terms "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", etc., indicating the orientation or position relationship, are based on the orientation or position relationship shown in the accompanying drawings, and are only for the convenience of describing this application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on this application.

[0056] The above describes specific embodiments of the present invention. It should be understood that the present invention is not limited to the specific embodiments described above, and those skilled in the art may make various changes or modifications within the scope of the claims, which do not affect the essence of the present invention. The embodiments of this application and the features in the embodiments may be combined with each other in any manner unless there is a conflict.

Claims

1. A distributed file system metadata index optimization method based on remote direct memory access, characterized in that: include: Directory metadata indexing based on path name hashing: Define the N0, N1, and other level directories in the directory tree as key directories. Treat the path of the key directory as a string and calculate the string through a hash function to obtain the location of the key directory metadata in the metadata memory pool of the distributed file system metadata server. The distributed file system metadata server is responsible for writing the directory metadata to the corresponding location in the metadata memory pool. The distributed file system client uses Remote Direct Memory Access (RIMA) technology and a one-sided read primitive to directly obtain the required directory metadata from this location. Popularity-based file metadata memory location optimization: The metadata memory pool of the distributed file system metadata server is divided into three areas: the block area, the directory entry area, and the inode area. In the block area, a 4096-byte memory block is pre-allocated for metadata in each key directory, allowing frequently accessed file metadata to be placed adjacent to the metadata of the key directory to which it belongs. The metadata management module regularly checks file popularity based on client feedback and dynamically migrates file metadata stored in the pre-allocated blocks. Client-side caching optimization for key directory metadata: Distributed file system clients cache metadata and directory entries for key directories. This cache uses an optimistic access method, assuming that the locally cached metadata and directory entries for key directories are valid. Therefore, it attempts to retrieve metadata for next-level directories / files using remote direct memory technology. If the retrieved metadata for next-level directories / files is incorrect, the system updates the key directory metadata to ensure consistency in the client's meta-cache.

2. The distributed file system metadata index optimization method based on remote direct memory access according to claim 1, characterized in that: Directory metadata index based on pathname hashes: Component A1: When establishing a connection with a distributed file system client, the distributed file system metadata server informs the client of the specific values ​​of the key directory levels N0 and N1 set by the metadata server, the hash function used, and the conflict resolution method; Component A2: The client metadata indexing module of the distributed file system client sends a write request to the distributed file system metadata server via the TCP communication module. Upon receiving the request, the metadata management module of the distributed file system metadata server uses a hash function to calculate the in-memory location of the metadata corresponding to the request and makes the appropriate modifications. If the modification is successful, the server returns a successful write request message to the client. Component A3: When a distributed file system client reads remote metadata, the client's metadata indexing module first determines the key directory level used and uses a hash function to calculate the location of the metadata in the distributed file system metadata server's memory. The distributed file system client then uses the RDMA communication module and a one-sided read primitive to retrieve the required metadata from the distributed file system metadata server's memory. After obtaining the metadata, the distributed file system client verifies it to confirm that it is the desired metadata. If verification fails, indicating a possible hash conflict, the client will retry reading the metadata according to the conflict resolution method agreed upon in Component A1.

3. The distributed file system metadata index optimization method based on remote direct memory access according to claim 1, characterized in that: Optimize the memory location of file metadata based on popularity: Component B1: The metadata memory pool of the distributed file system metadata server is divided into three areas: the block area, the directory entry area, and the inode area. The block area is divided into several 4096-byte memory blocks, each of which stores one directory metadata and several file metadata. The distributed file system client uses 4096-byte granularity to remotely access metadata in the block area. The directory entry area stores the directory entries of the directory, and the directory entries record the location of the corresponding directory / file metadata in the metadata memory pool of the distributed file system meta server; The inode area stores directory metadata and file metadata that are not stored in the block area; Component B2: The metadata management module of the distributed file system metadata server places the metadata of frequently accessed files and the directory metadata of the directory to which they belong in adjacent locations in the memory, that is, in the same block in the block area. The metadata management module maintains a popularity table for each key directory in the block area, recording the access popularity of the file metadata under that directory. The distributed file system client periodically sends popularity information about its accesses to the metadata server. The metadata management module updates the popularity table based on this information and migrates the metadata of infrequently accessed files to the inode area, while the metadata of frequently accessed files is migrated to the block area.

4. The distributed file system metadata index optimization method based on remote direct memory access according to claim 1, characterized in that: Client-side caching optimization for key directory metadata; Step C1: Find metadata and directory entries of the key directory in the client metadata cache; Step C2: For the cache search process in step C1, if no corresponding cache is found, a hash function is used to calculate the location of the metadata in the metadata memory pool of the distributed file system metadata server, and remote direct memory access is used to obtain the metadata and directory entries, and the metadata are written into the client metadata cache, and the cache LRU list is updated; Step C3: For the cache search process in step C1, if the corresponding cache is found, assuming that the directory metadata and directory entries in the local cache are valid, use the unilateral read primitive through the RDMA communication module to remotely access the metadata of the next-level directory / file; Step C4: For the remote direct memory access process in step C3, if the path name of the obtained metadata is consistent with the required path name, it is determined that the directory metadata and directory entries cached locally in step C3 are valid, and the cache LRU linked list is updated; Step C5: For the remote direct memory access process in step C3, if the pathname of the obtained metadata is inconsistent with the required pathname, the directory metadata and directory entries cached locally in step C3 are determined to be invalid, and the cache is deleted. A hash function is used to calculate the location of the metadata in the metadata memory pool of the distributed file system metadata server. The metadata and directory entries are obtained through remote direct memory access and written into the client metadata cache, and the cache LRU list is updated. A distributed file system metadata index optimization method based on remote direct memory access, characterized by comprising: Directory metadata indexing based on pathname hashing: This method uses pathname hashing to obtain the metadata location in the metadata memory pool of the distributed file system metadata server. This reduces the number of remote direct memory accesses required to obtain the directory and reduces the load on the metadata server. Optimizes the memory location of file metadata based on popularity: Frequently accessed file metadata and directory metadata are placed adjacent to each other in memory, reducing the number of remote direct memory accesses when accessing hot file metadata and lowering the load on the metadata server. Client-side cache optimization for key directory metadata: Caching metadata of key nodes reduces frequent access to key node metadata during path traversal and reduces the load on the metadata server.

Images