Metadata storage method and distributed file system

Abstract

The present application provides a metadata storage method and a distributed file system, wherein the method is applied to a distributed file system DFS, and the distributed file system DFS includes multiple metadata servers; the method disassembles (i.e., decouples) each directory metadata into two parts, namely, corresponding access metadata and content metadata, and then, under the premise of ensuring that the mapping relationship remains unchanged, recombines and stores the access metadata, content metadata, and file metadata of each directory metadata corresponding to the directory tree, thereby ensuring that the directory list operation of the DFS is performed on one MDS. The present application solves the problem in the prior art that frequent directory list operations lead to low performance of the entire distributed file system.

Description

Technical Field

[0001] The present application relates to the technical field of data storage, and in particular to a metadata storage method and a distributed file system. Background Art

[0002] Distributed File System (DFS) organizes and manages files in the entire DFS through a directory tree with a hierarchical tree structure consisting of directory metadata and file metadata. The global directory tree view corresponding to the directory tree is as follows: Figure 1 As shown in the figure, the global directory tree view only displays the directory name of each directory metadata and the file name of each file metadata in the directory tree, as well as the hierarchical mapping relationship between directory metadata and the hierarchical mapping relationship between directory metadata and file metadata. Figure 2 As shown. The directory metadata and file metadata in DFS are distributedly stored on multiple metadata servers (MDS) 221 in the metadata server (MDS) cluster 22, and each MDS 221 manages and processes the directory metadata or file metadata stored therein. The file content data of the files in DFS is stored on multiple data servers (DS) 231 in the data server (DS) cluster 23. The client 211 obtains the file metadata of the target file by accessing the MDS cluster 22 and obtaining the file metadata of the target file from the MDS cluster 22, and then indexes to the DS 231 storing the file content data of the target file based on the file metadata, and then performs access operations (such as reading) on ​​the file content data of the target file on the DS 231. Among them, directory metadata includes directory name, directory globally unique identifier (ID), directory permission information, directory timestamp, directory items (i.e., sub-file names and sub-directory name lists under the directory). For example, directory metadata of a specified directory name includes the specified directory name, the ID of the specified directory name, permission information of the specified directory name, the timestamp of the specified directory name, and directory items of the specified directory name; file metadata includes file name, file globally unique identifier, file permission information, and file content data storage information (such as the identifier of the DS that stores file content data).

[0003] In the process of the client 211 obtaining the file metadata of the target file from the MDS cluster 22, the MDS 221 associated with the target file in the MDS cluster 22 needs to parse the metadata (i.e., directory metadata and / or file metadata) stored therein, obtain the file metadata of the target file, and return it to the client 211. At present, the common storage method of DFS metadata on multiple MDSs 221 in the MDS cluster 22 is as follows: the DFS directory tree is hierarchically (e.g., Figure 1 The metadata of the DFS directory tree is divided into different levels (as shown in FIG. 2 ), and metadata of different levels is stored on different MDSs 221 of the MDS cluster 22. Each MDS is responsible for managing the metadata stored therein, and all MDSs 221 in the MDS cluster 22 jointly maintain the metadata of a DFS directory tree and the hierarchical mapping relationship between metadata, and perform metadata processing based on the access request of the client 211. The metadata storage method is simple in design and has low implementation difficulty, and the entire DFS metadata processing throughput is the sum of the throughputs of all MDSs 221 in the MDS cluster 22, which effectively guarantees the metadata processing performance of the DFS.

[0004] Since DFS has frequent directory list operations (such as creating or deleting files, directory command (ls) operations, etc.), when DFS adopts the existing metadata storage method, the frequent directory list operations will lead to poor performance of the entire DFS. Summary of the invention

[0005] The present application provides a metadata storage method and a distributed file system to solve the problem in the prior art that frequent directory list operations lead to poor performance of the entire DFS.

[0006] In a first aspect, the present application provides a metadata storage method, which is applied to a distributed file system DFS, wherein the distributed file system DFS includes multiple metadata servers;

[0007] The method comprises:

[0008] Decouple the metadata of each directory in the DFS directory tree in the following manner to obtain the access metadata and content metadata corresponding to each directory metadata: decouple the directory metadata of the specified directory name into the access metadata of the specified directory name and the content metadata of the specified directory name; wherein the access metadata of the specified directory name includes the specified directory name and the globally unique identifier of the specified directory name; the content metadata of the specified directory name includes the directory entry of the specified directory name;

[0009] The access metadata and content metadata corresponding to each directory metadata in the directory tree, as well as each file metadata, are grouped into a metadata group in the following manner: the access metadata of the specified directory name, the content metadata of the parent directory of the specified directory name, and the global unique identifier of the parent directory of the specified directory name are grouped into a metadata group; and the content metadata of the specified directory name, the global unique identifier of the specified directory name, and the access metadata of the subdirectory of the specified directory name and / or the file metadata of the subfile are grouped into a metadata group;

[0010] The multiple metadata groups of the directory tree are evenly distributed and stored in the multiple metadata servers.

[0011] Optionally, the evenly distributing and storing the multiple metadata groups of the directory tree on the multiple metadata servers includes:

[0012] Performing consistent hash calculations on the metadata group and the metadata server respectively to obtain hash values ​​corresponding to the metadata group and the metadata server respectively;

[0013] Based on the hash value, the metadata group is correspondingly stored in the metadata server.

[0014] Optionally, performing consistent hash calculation on the metadata group to obtain a hash value corresponding to the metadata group includes:

[0015] The metadata group is subjected to consistent hashing calculation in the following manner to obtain a hash value corresponding to the metadata group:

[0016] Perform consistent hash calculation on the globally unique identifier of the parent directory of the specified directory name to obtain a hash value of the access metadata of the specified directory name;

[0017] Performing consistent hash calculation on the globally unique identifier of the specified directory name to obtain a hash value of the content metadata of the specified directory name;

[0018] A consistent hash calculation is performed on the globally unique identifier of the parent directory to which the file metadata belongs to obtain a hash value of the file metadata.

[0019] Optionally, storing the metadata group in the metadata server includes:

[0020] The metadata groups are stored in the metadata server in the form of key-value pairs.

[0021] Optionally, the access metadata of the specified directory name also includes permission information of the specified directory name; the content metadata of the specified directory name also includes a timestamp of the specified directory name;

[0022] The step of storing the metadata group in the metadata server in the form of key-value pairs includes:

[0023] The metadata groups are stored in the metadata server in the following key-value pairs:

[0024] A key-value pair consisting of a globally unique identifier of the specified directory name and the permission information of the specified directory name as a value, and a globally unique identifier of the parent directory of the specified directory name and the specified directory name as a key;

[0025] A key-value pair consisting of a directory entry of the specified directory name and a timestamp of the specified directory name as a value, and a globally unique identifier of the specified directory name as a key;

[0026] A key-value pair is composed of the file metadata as a value and the globally unique identifier of the parent directory to which the file metadata belongs and the file name as a key.

[0027] In a second aspect, the present application provides a distributed file system, including: multiple clients, multiple metadata servers, and multiple data servers; wherein:

[0028] The client is used to send an access request to the metadata server to access the metadata server and obtain a request result returned by the metadata server; and is also used to access the file content data stored in the data server corresponding to the request result based on the request result;

[0029] The metadata server is used to store the directory metadata and file metadata of the distributed file system using the method described above; and is also used to perform metadata processing on the metadata group stored in the metadata server based on the access request of the client, obtain the request result, and return the request result to the client;

[0030] The metadata server is further used to maintain the consistency of the client cache item and the metadata of the distributed file system in the following manner:

[0031] After receiving the access request from the client, the metadata server checks the validity of the client cache item corresponding to the access request, obtains the check result, and performs the following consistency maintenance operations based on the check result:

[0032] If the checking result is that the client cache item corresponding to the access request is valid, returning the request result corresponding to the access request and an invalidation instruction for invalidating the invalid cache item in the client cache to the client;

[0033] If the checking result is that the client cache item corresponding to the access request is invalid, a termination instruction indicating the termination of the access request and an invalidation instruction for invalidating the invalid cache item in the client cache are returned to the client.

[0034] Optionally, the system further comprises: a coordination server;

[0035] The coordination server is used to identify a directory orphan loop in the directory tree of the distributed file system when multiple clients perform directory renaming operations concurrently, and send a termination instruction to the client to terminate the directory renaming operation when the directory orphan loop is identified;

[0036] The directory orphan loop is an isolated mapping loop formed by a mapping relationship between multiple directory metadata, or an isolated mapping loop formed by a mapping relationship between multiple directory metadata and file metadata.

[0037] In a third aspect, the present application provides a server, including:

[0038] Processor and memory;

[0039] The memory stores executable instructions executable by the processor;

[0040] The processor executes the executable instructions stored in the memory, so that the processor executes the method as described above.

[0041] In a fourth aspect, the present application provides a storage medium, wherein the storage medium stores computer-executable instructions, and the computer-executable instructions are used to implement the method described above when executed by a processor.

[0042] In a fifth aspect, the present application provides a program product, including a computer program, which implements the method described above when executed by a processor.

[0043] The metadata storage method and distributed file system provided by the present application decompose the directory metadata into two parts: directory access metadata and directory content metadata. Then, under the premise of ensuring that the mapping relationship remains unchanged, the decomposed access metadata, directory content metadata, and file metadata of each directory corresponding to the directory tree are recombined and stored, ensuring that the directory list operation of DFS is performed on one MDS. The present application solves the problem in the prior art that frequent directory list operations lead to poor performance of the entire DFS. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0045] Figure 1 A schematic diagram of the existing global directory tree view;

[0046] Figure 2 This is the architecture diagram of the existing distributed file system;

[0047] Figure 3A flowchart of a metadata storage method provided in an embodiment of the present application;

[0048] Figure 4 A diagram of the distributed file system architecture provided in the embodiment of the present application;

[0049] Figure 5 A schematic diagram of a directory orphan loop provided for an embodiment of the present application;

[0050] Figure 6 A server structure diagram provided for an embodiment of the present application.

[0051] The above drawings have shown clear embodiments of the present application, which will be described in more detail later. These drawings and text descriptions are not intended to limit the scope of the present application in any way, but to illustrate the concept of the present application to those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION

[0052] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0053] Distributed File System (DFS) has the advantages of cost saving, convenient management, good scalability, high reliability and high availability, and has been widely used. Figure 1 The directory tree corresponding to the global directory tree view shown in the figure organizes and manages the files in the entire distributed file system. When the distributed file system adopts the existing metadata storage method, the directory tree of the distributed file system is organized as follows Figure 1 The metadata of different levels are stored in Figure 2When the metadata of the DFS directory tree is on different MDS 221 of the MDS cluster 22 shown, the hierarchical mapping relationship between the metadata is collaboratively maintained by multiple MDS 221 in the MDS cluster 22. When it is necessary to perform a directory list operation on the distributed file system (such as creating or deleting a file), it is necessary to create or delete the corresponding file metadata for the newly created or deleted file. When creating or deleting file metadata, it is bound to be necessary to create or modify the mapping relationship between the file metadata and the directory metadata of its parent directory. However, in the prior art, the file metadata and the directory metadata of its parent directory are stored on different MDS 221 because they belong to different hierarchies. When creating or modifying the mapping relationship between the file metadata and the directory metadata of its parent directory, the two MDS221 corresponding to the file metadata and the directory metadata of its parent directory need to perform communication coordination of the distributed protocol to complete the processing of the relevant file metadata and directory metadata on the two MDS 221. For example, when creating or deleting file metadata, two MDS 221 distributed atomic commit protocols (such as two-phase commit) where the file metadata and the directory metadata of its parent directory are located are required to ensure atomic updates of data on multiple metadata servers, and distributed lock protocols (such as two-phase locks) are required to ensure isolation between different operations. These distributed protocols require complex network communications with high latency; competition between locks leads to a decrease in concurrency and throughput, which in turn affects the throughput performance of the distributed file system. However, with the continuous development of network technology, the number of files carried by the distributed file system has not increased, and there are situations in which the client 211 in the distributed file system frequently creates or deletes files, performs directory command modification operations, and other directory list operations. Distributed file systems for e-commerce businesses such as Taobao and JD.com frequently upload or delete product information files due to their extremely large merchant groups. Therefore, frequent directory list operations will cause the performance of the entire distributed file system (such as throughput performance) to become low.

[0054] Since directory metadata includes the directory name, the directory's globally unique identifier (ID), the directory's permission information, the directory's timestamp, and directory entries; file metadata includes the file name, the file's globally unique identifier, the file's permission information, and the file content data storage information (such as the identifier of the DS that stores the file). The hierarchical mapping relationship between the directory metadata of the previous level and the directory metadata of the next level is achieved through the correspondence between the directory entries in the directory metadata of the previous level and the directory names in the directory metadata of the next level; similarly, the hierarchical mapping relationship between the directory metadata of the previous level and the file metadata of the next level is achieved through the correspondence between the directory entries in the directory metadata of the previous level and the file names in the file metadata of the next level. For example, as Figure 1The directory metadata, file metadata and mapping relationships in the directory tree corresponding to the global directory tree view are as shown in Table 1 before the newly created file f3:

[0055] Table 1 Directory metadata, file metadata and mapping relationship

[0056]

[0057] The examples of directory metadata and file metadata in Table 1 are as follows:

[0058] d1 represents the name of directory d1; IDd1 represents the globally unique identifier of directory d1, that is, the ID of directory d1; d1 permission information represents the permission information of directory d1; d1 timestamp represents the timestamp of directory d1; d1 directory entry represents the directory entry of directory d1.

[0059] f1 represents the name of file f1; IDf1 represents the globally unique identifier of file f1, that is, the ID of file f1; f1 permission information represents the permission information of file f1; f1 timestamp represents the timestamp of file f1; f1 content storage information represents the file content data storage information of file f1, such as the storage address or the identifier of the data server storing the file content data of file f1.

[0060] The other directory metadata and file metadata representation methods in this application are similar to the above-mentioned directory metadata example description and file metadata example description, and will not be repeated here.

[0061] As shown in Table 1, in the prior art, the directory list operation is an operation on the hierarchical mapping relationship between two directory metadata or the hierarchical mapping relationship between directory metadata and file metadata (such as creation, modification, and deletion). For example, as shown in Table 1 above, when it is necessary to create a new file f3 in DFS and implement the index from directory d3 to file f3, that is, to create a hierarchical mapping relationship between the directory metadata of the new directory d3 (that is, the directory name is d3) and the file metadata of the file f3 (that is, the name of the file f3 is f3), it is necessary to modify the directory item in the directory metadata of the directory d3 (such as adding f3 to the directory item), and at the same time, it is necessary to create the file metadata of the new file f3. Therefore, the operation of creating a new file f3 requires multiple MDSs (that is, at least two MDSs, such as the MDS storing the directory metadata of the directory d3 and the MDS storing the file metadata of the file f3) to coordinate the communication of the distributed protocol.

[0062] That is to say, the metadata storage method of the prior art is to disassemble the directory tree from the hierarchical mapping relationship between two directory metadata, or between directory metadata and its associated file metadata, and then perform distributed storage.

[0063] In this regard, in order to solve the problem that after adopting the existing technology for metadata storage, the directory list operation requires multiple MDSs to perform communication coordination of the distributed protocol, the present application proposes a metadata storage method, by disassembling (i.e., decoupling) each directory metadata into two parts corresponding to the access metadata and the content metadata, and then, under the premise of ensuring that the mapping relationship remains unchanged, the access metadata, content metadata, and file metadata corresponding to the directory metadata of the directory tree are recombined and stored, ensuring that the directory list operation of the DFS is performed on one MDS, thereby solving the problem of frequent directory list operations in the prior art leading to poor performance of the entire DFS.

[0064] The metadata storage method provided by the present application is described below in conjunction with some embodiments.

[0065] The distributed file system architecture of the embodiment of the present application can also be as follows Figure 2 The architecture shown in Figure 2 As shown, the distributed file system includes: a client cluster 21, a metadata server (MDS) cluster 22, and a data server (DS) cluster 23. Among them, the MDS cluster 22 includes multiple metadata servers (MDS) 221, such as Figure 2 The metadata server 1, metadata server 2, metadata server 3, ..., metadata server n shown; the DS cluster 23 includes multiple data servers (DS) 231, such as Figure 2 The data server 1, data server 2, data server 3, ..., data server m shown; the client cluster 21 includes multiple clients 211, such as Figure 2 The client 1, client 2, client 3, ..., client p are shown; and m, n, p are all natural numbers. The distributed system organizes and manages the files in the entire DFS through a directory tree. The directory metadata and file metadata in the DFS are evenly distributed and stored on multiple MDS 221, and each MDS 221 manages and processes the directory metadata or file metadata stored by each. The file content data of the files in the DFS is stored on multiple DS 231. The client 211 can access at least one MDS 221 and obtain the file metadata of the target file from the MDS 221, and then communicate with the DS 231 corresponding to the file metadata based on the file metadata to implement access operations (such as reading) on ​​the file content data of the target file stored on the DS 231. The storage method of the directory metadata and file metadata corresponding to the directory tree of the distributed file system on the MDS cluster 22 is as shown in FIG. Figure 3 shown.

[0066] Figure 3 A flowchart of a metadata storage method provided in an embodiment of the present application. Figure 3The execution subject of the embodiment shown may be Figure 2 Any of the client 211, MDS 221, and DS 231 shown. The metadata storage method provided by the present application is applied to a distributed file system DFS, which includes multiple metadata servers. Figure 3 As shown, the method includes:

[0067] S301. Decouple the metadata of each directory in the DFS directory tree in the following manner to obtain the access metadata and content metadata corresponding to each directory metadata: decouple the directory metadata of the specified directory name into the access metadata of the specified directory name and the content metadata of the specified directory name; wherein the access metadata of the specified directory name includes the specified directory name and the globally unique identifier of the specified directory name; and the content metadata of the specified directory name includes the directory entry of the specified directory name.

[0068] Exemplarily, the client 211 decouples each directory metadata in the DFS directory tree in the following manner to obtain the access metadata and content metadata corresponding to each directory metadata:

[0069] The directory metadata of the specified directory name is decoupled into the access metadata of the specified directory name and the content metadata of the specified directory name; wherein the access metadata of the specified directory name includes the specified directory name and the globally unique identifier of the specified directory name; and the content metadata of the specified directory name includes the directory items of the specified directory name.

[0070] For example, assuming that the designated directory name is K, the client 211 decouples the metadata of each directory in the DFS directory tree in the following manner to obtain the access metadata and content metadata corresponding to each directory metadata:

[0071] The directory metadata of directory K is decoupled into the access metadata of directory K and the content metadata of directory K; wherein the access metadata of directory K includes the name of directory K and the globally unique identifier of directory K; and the content metadata of directory K includes the directory items of directory K.

[0072] S302. The access metadata and content metadata corresponding to each directory metadata in the directory tree, as well as each file metadata, are organized into a metadata group in the following manner: the access metadata of the specified directory name, the content metadata of the parent directory of the specified directory name, and the globally unique identifier of the parent directory of the specified directory name are organized into a metadata group; and the content metadata of the specified directory name, the globally unique identifier of the specified directory name, and the access metadata of the subdirectory of the specified directory name and / or the file metadata of the subfile are organized into a metadata group.

[0073] Exemplarily, the client 211 groups the access metadata and content metadata corresponding to each directory metadata in the directory tree, and each file metadata into a metadata group in the following manner:

[0074] Client 211 composes a metadata group with the access metadata of the specified directory name, the content metadata of the parent directory of the specified directory name, and the globally unique identifier of the parent directory of the specified directory name; client 211 also composes a metadata group with the content metadata of the specified directory name, the globally unique identifier of the specified directory name, and the access metadata of the subdirectory of the specified directory name and / or the file metadata of the subfile.

[0075] For example, assuming that the designated directory name is K, the client 211 assembles the access metadata and content metadata corresponding to each directory metadata in the directory tree, as well as each file metadata, into a metadata group in the following manner:

[0076] Client 211 combines the access metadata of directory K, the content metadata of the parent directory of directory K, and the globally unique identifier of the parent directory of directory K into a metadata group; client 211 also combines the content metadata of directory K, the globally unique identifier of directory K, and the access metadata of the subdirectory of directory K and / or the file metadata of the subfile into a metadata group.

[0077] S303: Evenly distribute and store multiple metadata groups of the directory tree in multiple metadata servers.

[0078] Exemplarily, the client 211 evenly distributes and stores multiple metadata groups of the directory tree in multiple metadata servers 221 .

[0079] Optionally, the client 211 performs consistent hashing on the metadata group and the metadata server to obtain the hash values ​​corresponding to the metadata group and the metadata server. The client 211 stores the metadata group in the metadata server 221 based on the hash values, ie, the hash values ​​corresponding to the metadata group and the metadata server.

[0080] Furthermore, the client 211 performs consistent hash calculation on the metadata group in the following manner to obtain a hash value corresponding to the metadata group:

[0081] Perform consistent hash calculation on the global unique identifier of the parent directory of the specified directory name (that is, the parent directory ID of the specified directory name) to obtain the hash value of the access metadata of the specified directory name;

[0082] Perform consistent hash calculation on the globally unique identifier of the specified directory name (i.e., the directory ID of the specified directory name) to obtain a hash value of the content metadata of the specified directory name;

[0083] A consistent hash calculation is performed on the globally unique identifier of the parent directory to which the file metadata belongs (that is, the parent directory ID of the file) to obtain a hash value of the file metadata.

[0084] Exemplarily, assuming that the specified directory name is K, the client 211 performs consistent hash calculation on the metadata group in the following manner to obtain a hash value corresponding to the metadata group:

[0085] Perform consistent hash calculation on the globally unique identifier of the parent directory of directory K (i.e., the parent directory ID of directory K) to obtain the hash value of the access metadata of directory K;

[0086] Perform consistent hash calculation on the globally unique identifier of directory K (i.e., directory ID of directory K) to obtain a hash value of the content metadata of directory K;

[0087] A consistent hash calculation is performed on the globally unique identifier of the parent directory to which the file metadata belongs (that is, the parent directory ID of the file) to obtain a hash value of the file metadata.

[0088] Optionally, the client 211 stores the metadata group in the form of key-value pairs in the metadata server 221 .

[0089] Optionally, the access metadata of the specified directory name also includes the permission information of the specified directory name; the content metadata of the specified directory name also includes the timestamp of the specified directory name. The client 211 uses the following key-value pairs to store the metadata groups in the metadata server 221:

[0090] A key-value pair consisting of a globally unique identifier of a specified directory name and permission information of the specified directory name as a value, and a globally unique identifier of the parent directory of the specified directory name and the specified directory name as a key;

[0091] A key-value pair consisting of a directory entry with a specified directory name and a timestamp with a specified directory name as the value, and a globally unique identifier with a specified directory name as the key;

[0092] A key-value pair consisting of the file metadata as the value and the globally unique identifier of the parent directory to which the file metadata belongs and the file name as the key.

[0093] For example, assuming that the specified directory name is K, the access metadata of directory K also includes the permission information of directory K; the content metadata of directory K also includes the timestamp of directory K. The client 211 uses the following key-value pairs to store the metadata groups in the metadata server 221:

[0094] A key-value pair consisting of a globally unique identifier of directory K and permission information of directory K as a value, and a globally unique identifier of the parent directory of directory K and the name of directory K as a key;

[0095] A key-value pair consisting of the directory entry of directory K and the timestamp of directory K as the value and the globally unique identifier of directory K as the key;

[0096] A key-value pair consisting of the file metadata as the value and the globally unique identifier of the parent directory to which the file metadata belongs and the file name as the key.

[0097] For example, it is assumed that the global directory tree view corresponding to the directory tree of the distributed file system R is as follows: Figure 1 As shown, the directory metadata and file metadata corresponding to the directory tree are shown in Table 1, and the distributed file system R includes four metadata servers: metadata server 1, metadata server 2, metadata server 3, and metadata server 4. When the directory metadata and file metadata of the distributed file system R are stored in metadata server 1, metadata server 2, metadata server 3, and metadata server 4 by using the metadata storage method provided by the prior art and the present application, the respective storage conditions are shown in Table 2 and Table 3 respectively.

[0098] Table 2 Metadata storage using existing technology

[0099]

[0100] Table 3 Metadata storage using the method provided by this application

[0101]

[0102] The directory tree corresponding to the directory metadata and file metadata before the newly created file f3 is as follows: Figure 1 The directory tree corresponding to the global directory tree view shown.

[0103] After the metadata of the distributed file system R is stored using the prior art, when a directory list operation is performed on the distributed file system R, the creation of a new file f3 as shown in Table 2 above involves at least the distributed protocol communication coordination between the metadata server 2 and the metadata server 3.

[0104] After adopting the metadata storage method provided by the present invention to store metadata of the distributed file system R, when performing a directory list operation on the distributed file system R, a new file f3 is created as shown in Table 3 above. Since the file metadata of file f3 and its hierarchical mapping relationship with directory d3 only need to be performed on one metadata server, metadata server 4, distributed protocol communication coordination between multiple metadata servers is not involved.

[0105] The following is a description of the operation effect of a distributed file system after metadata storage is performed using the metadata storage method provided by the present invention in conjunction with a specific example. Assume that the system architecture of a distributed file system Q is as follows: Figure 2 As shown in Table 3, the distributed file system Q uses the metadata storage method provided by the present invention to store metadata. The metadata storage of the distributed file system Q is specifically shown in Table 3. The directory tree of the distributed file system Q is shown in Table 3. Figure 1 The directory tree corresponding to the global directory tree view shown in the figure. When client 1 wants to access file f2, the operation process of distributed file system Q is as follows:

[0106] Client 1 performs a consistent hash operation on the root directory ID to obtain the code of the metadata server where the access metadata of directory d1 is located, that is, metadata server 1 (such as MDS-1); based on MDS-1, client 1 sends access request 1 (such as root directory ID, d1, user ID) to metadata server 1;

[0107] Based on the root directory ID and d1 in access request 1, metadata server 1 finds the access metadata of directory d1, and based on the user ID in access request 1, checks whether the user corresponding to the user ID has the permission to access directory d1. If the check result is that the permission is granted, metadata server 1 returns IDd1 and a confirmation signal (such as an ACK signal) to client 1; if the check result is that the permission is denied, metadata server 1 returns a non-confirmation signal (such as a NACK signal) to client 1 to terminate access request 1 of client 1. Assume in this example that metadata server 1 returns IDd1 and an ACK signal to client 1.

[0108] Next, client 1 performs a consistent hash operation on IDd1 to obtain the code (e.g., MDS-2) of the metadata server where the access metadata of directory d3 is located, namely, metadata server 2; based on MDS-2, client 1 sends access request 2 (e.g., IDd1, d3, user ID) to metadata server 2;

[0109] Continuing, metadata server 2 finds the access metadata of directory 3 based on IDd1 and d3 in access request 2, and checks whether the user corresponding to the user ID has the permission to access directory d3 based on the user ID in access request 2. If the check result shows that the user has the permission, metadata server 2 returns IDd3 and an ACK signal to client 1;

[0110] Next, client 1 performs a consistent hash operation on IDd3 to obtain the code (e.g., MDS-4) of the metadata server where the file metadata of file f2 is located, namely, metadata server 4; based on MDS-4, client 1 sends access request 3 (e.g., IDd3, f2, user ID) to metadata server 4;

[0111] Continuing, metadata server 4 finds the file metadata of file f2 based on IDd3 and f2 in access request 3, and checks whether the user corresponding to the user ID has the permission to access file f2 based on the user ID in access request 3. If the check result shows that the user has the permission, metadata server 4 returns the file metadata of file f2 and the ACK signal to client 1;

[0112] Finally, based on the f2 content storage information in the file metadata of the file f2, the client 1 can communicate with the data server storing the file content data of the file f2 and read the file content data of the file f2 stored on the data server.

[0113] Therefore, after adopting the metadata storage method provided by the present application to store metadata of the distributed file system, frequent directory listing operations will not have an adverse effect on the performance (such as throughput performance) of the distributed file system. The metadata storage method provided by the present application solves the problem in the prior art that frequent directory listing operations will lead to low performance (such as throughput performance) of the entire DFS. In addition, in a distributed file system that adopts the metadata storage method provided by the present application, when a client accesses the file content data of a file in the distributed file system, the client only needs to perform a consistent hash operation and then communicate with the relevant metadata server in a targeted manner, without the need for information exchange between metadata servers, which greatly improves the performance of the entire distributed file system in dealing with concurrent access to file content data by multiple clients.

[0114] The embodiment of the present application also provides a distributed file system. Figure 4 A diagram of the distributed file system architecture provided in an embodiment of the present application. Figure 5 A schematic diagram of a directory orphan loop provided in an embodiment of the present application. Figure 4 and Figure 5 The distributed file system provided by this application is described. Figure 4 As shown, the distributed file system includes: multiple clients 211, multiple metadata servers 221, and multiple data servers 231.

[0115] The client 211 is used to send an access request to the metadata server 221 to access the metadata server 221 and obtain the request result returned by the metadata server 221; and is also used to access the file content data stored in the data server 231 corresponding to the request result based on the request result;

[0116] Metadata server 221, used to adopt the method provided by this application (such as Figure 3The method provided in the embodiment shown in the figure) stores directory metadata and file metadata of the distributed file system; is also used to perform metadata processing on the metadata group stored in the metadata server 221 based on the access request of the client 211, obtain the request result, and return the request result to the client 211;

[0117] The metadata server 221 is also used to maintain the consistency of the cache items of the client 211 and the metadata of the distributed file system in the following manner:

[0118] After receiving the access request from the client 211, the metadata server 221 checks the validity of the cache item of the client 211 corresponding to the access request, obtains the check result, and performs the following consistency maintenance operations based on the check result:

[0119] If the check result is that the cache item of the client 211 corresponding to the access request is valid, a request result corresponding to the access request and an invalidation instruction for invalidating the invalid cache item in the cache of the client 211 are returned to the client;

[0120] If the check result is that the cache item of the client 211 corresponding to the access request is invalid, a termination instruction indicating the termination of the access request and an invalidation instruction for invalidating the invalid cache item in the client cache are returned to the client 211.

[0121] Exemplarily, the technical effect of the method for maintaining the consistency of the cache items of the client 211 and the metadata of the distributed file system provided in the embodiment of the present application is described as follows:

[0122] In the prior art, the metadata server 221 maintains the consistency of the cache items of the client 211 with the metadata of the distributed file system in the following manner: once the directory tree in the distributed file system changes, such as when at least one metadata server 221 performs a directory change operation that causes the directory tree to change, such as directory renaming, directory list operation, directory permission modification operation, etc., then the metadata server 221 that performs the directory change operation will send an invalidation instruction to each client 211 at the same time after the directory change operation is completed, for invalidating the cache items that have expired in the client 211. That is, each time the metadata server 221 performs a directory change operation, it will send an invalidation instruction to each client 211 at the same time, and each time the client 211 receives the invalidation instruction, it will invalidate the corresponding cache item. When there are frequent directory change operations in the distributed file system, the metadata server 221 has a large overhead in maintaining the consistency of the cache items of the client 211 with the metadata of the distributed file system in the entire distributed file system.

[0123] Since in actual applications, not every cache item in the cache of client 211 is a frequently used cache item, there is a situation where a cache item in the cache of most clients 211 will not be used after being cached in client 211. For such cache items of client 211 that will not be used, there is actually no need to perform consistency maintenance. Therefore, the method for maintaining the consistency of the cache items of client 211 and the metadata of the distributed file system provided in the embodiment of the present application is to perform validity checks and consistency maintenance on the cache items in client 211 when client 211 uses the cache items, thereby ensuring that the metadata server 221 avoids the consistency maintenance overhead of the cache items that will not be used in the cache of client 211 without affecting the performance of the client 211.

[0124] Optionally, the distributed file system provided by the present application also includes: a coordination server 41.

[0125] The coordination server 41 is used to identify a directory orphan loop in the directory tree of the distributed file system when multiple clients 211 perform directory renaming operations concurrently, and send a termination instruction to the client 211 to terminate the directory renaming operation when a directory orphan loop is identified.

[0126] The directory orphan loop is an isolated mapping loop formed by a mapping relationship between multiple directory metadata, or an isolated mapping loop formed by a mapping relationship between multiple directory metadata and file metadata.

[0127] An example of the coordination server 41 identifying a directory orphan loop and sending a termination instruction to the client 211 to terminate the directory renaming operation when the directory orphan loop is identified is as follows:

[0128] For example, Figure 5 As shown, Figure 5 a is the directory tree path diagram (i.e., the hierarchical mapping relationship trend diagram) of the distributed file system S before the directory rename operation. The directory tree path diagram includes the following two paths: the root directory (i.e. Figure 5 The directory represented by " / " in the figure) → directory A → directory B → directory C, and the root directory → directory D → directory E → directory F; Figure 5 b and Figure 5 c is the path diagram of the client M and the client N performing directory renaming operations on the distributed file system S at the same time; Figure 5 d is when the system architecture of the distributed file system S is Figure 2 In the system architecture shown, a global directory tree view is formed after client M and client N concurrently perform directory renaming operations on distributed file system S.

[0129] like Figure 5As shown, if the system architecture of the distributed file system S is Figure 2 In the system architecture shown in the figure, when client M and client N perform directory renaming operations on distributed file system S at the same time, client M and client N can only check whether their own directory renaming operations will cause directory orphan loops, but cannot know whether concurrent directory renaming operations of different clients will jointly cause directory orphan loops. Figure 5 b and Figure 5 After the directory renaming operation shown in c, the distributed file system S appears as follows Figure 5 d shows a directory orphan loop of B→C→E→F→B, which results in the loss of directory metadata related to the directory orphan loop or directory metadata and file metadata related to the directory orphan loop. For example, the directory metadata of directory B, directory C, directory E, and directory F cannot be indexed by path resolution when the distributed file system S is running.

[0130] If the system architecture of the distributed file system S is Figure 4 The system architecture shown in FIG. 1 is for the case where multiple clients 211 perform directory renaming operations concurrently, such as client M and client N performing the following operations on the distributed file system S at the same time. Figure 5 b and Figure 5 c, the coordination server 41 identifies the directory orphan loop, and when the directory orphan loop is identified, sends a termination instruction to the client 211 to terminate the directory renaming operation. Exemplarily, before the directory renaming operation results of the client M and the client N take effect, the client M and the client N will verify through the coordination server 41 whether the directory renaming operations of the client M and the client N at the same time will jointly cause the directory orphan loop. If the coordination server 41 verifies and identifies that the directory renaming operations of the client M and the client N at the same time will jointly cause the directory orphan loop, the coordination server 1 sends termination instructions to the client M and the client N respectively to terminate the directory renaming operations of the client M and the client N, and prevent the directory renaming operation results of the client M and the client N from taking effect; further, if the coordination server 41 verifies and identifies that the directory renaming operations of the client M and the client N at the same time will not jointly cause the directory orphan loop, the coordination server 1 sends confirmation instructions to the client M and the client N respectively to allow the directory renaming operation results of the client M and the client N to take effect.

[0131] Among them, the coordination server 41 identifies the directory orphan loop in the directory tree formed when the distributed file system performs directory renaming operations concurrently on multiple clients 211 in the following manner: the coordination server 41 locally maintains a directory renaming directed graph, which records the source path and destination path of the directory renaming operation currently being executed. The coordination server 41 identifies the directory orphan loop in the directory renaming directed graph to realize the identification of the directory orphan loop formed when multiple clients 211 perform directory renaming operations concurrently.

[0132] Optionally, during the directory renaming operation performed by the client 211, the source path and destination path of the directory renaming operation will be saved in the directory renaming directed graph by the coordination server 41, and will be deleted by the coordination server 41 after the directory renaming operation is completed (such as after the coordination server 41 sends an abort instruction or a confirmation instruction to the client 211).

[0133] Optionally, if multiple clients 211 perform directory renaming operations concurrently without forming a directory orphan loop, after the coordination server 41 sends confirmation instructions to the multiple clients 211 that perform the directory renaming operations respectively, the coordination server 41 will save the respective directory renaming operations concurrently performed by the multiple clients 211, and generate an incremental version number for each directory renaming operation. Then, the coordination server 41 sends the operation version number and operation information of each directory renaming operation to the invalid list of each metadata server 221. Among them, the operation information of the directory renaming operation includes the operation type, the source path and the destination path. After receiving the operation version number and operation information of the directory renaming operation sent by the coordination server 41, the metadata server 221 associated with the directory renaming operation processes the target metadata according to the operation version number and operation information of the directory renaming operation. The processing of the target metadata is as follows: the metadata server corresponding to the source path of the directory renaming operation reads and deletes the access metadata key-value pair of the directory corresponding to the directory renaming operation, and the metadata server corresponding to the destination path creates the access metadata key-value pair of the directory corresponding to the directory renaming operation.

[0134] Optionally, the metadata server 221 may use the operation version number and operation information of the directory rename operation on the invalidation list stored therein to check whether the client 211 cache item corresponding to the access request sent by the client 211 is valid when the client 211 accesses the metadata server 221. For example, when the client 211 accesses the metadata server 221, the metadata server 221 may use the operation version number and operation information of the directory rename operation on the invalidation list stored therein to compare with the client 211 cache item corresponding to the access request sent by the client 211. If the comparison result is consistent, it means that the client 211 cache item corresponding to the access request sent by the client 211 is valid; if the comparison result is inconsistent, it means that the client 211 cache item corresponding to the access request sent by the client 211 has expired.

[0135] The distributed file system provided in the embodiment of the present application not only has the following features: Figure 3 The technical principle and technical effect of the illustrated embodiment also reduce the network overhead of the metadata server for maintaining the consistency of the client cache items. The distributed file system provided by the embodiment of the present application also solves the problem of a directory orphan loop formed by multiple clients performing directory renaming operations concurrently in the prior art, resulting in the loss of directory metadata related to the directory orphan loop or directory metadata and file metadata related to the directory orphan loop.

[0136] The embodiment of the present application also provides a server. Figure 6 This is a server structure diagram provided in the embodiment of the present application. Figure 6As shown, the server includes a processor 61 and a memory 62, and the memory 62 stores executable instructions of the processor 61, so that the processor 61 can be used to execute the technical solution of the above method embodiment, and its implementation principle and technical effect are similar, and this embodiment will not be repeated here. It should be understood that the above processor 61 can be a central processing unit (English: Central Processing Unit, referred to as: CPU), or other general-purpose processors, digital signal processors (English: Digital Signal Processor, referred to as: DSP), application-specific integrated circuits (English: Application Specific Integrated Circuit, referred to as: ASIC), etc. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc. The steps of the method disclosed in the invention can be directly embodied as being executed by a hardware processor, or executed by a combination of hardware and software modules in the processor. The memory 62 may include a high-speed random access memory (English: Random Access Memory, referred to as: RAM), and may also include a non-volatile memory (English: Non-volatile memory, referred to as: NVM), such as at least one disk storage, and can also be a U disk, a mobile hard disk, a read-only memory, a disk or an optical disk, etc.

[0137] The embodiment of the present application also provides a storage medium, in which computer executable instructions are stored. When these computer executable instructions are executed by a processor, the metadata storage method provided by the present application is implemented. The storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (English: Static Random-Access Memory, referred to as: SRAM), electrically erasable programmable read-only memory (English: Electrically-Erasable Programmable Read-Only Memory, referred to as: EEPROM), erasable programmable read-only memory (English: Erasable Programmable Read-Only Memory, referred to as: EPROM), programmable read-only memory (English: Programmable Read-Only Memory, referred to as: PROM), read-only memory (English: Read-Only Memory, referred to as: ROM), magnetic memory, flash memory, magnetic disk or optical disk. The storage medium can be any available medium that can be accessed by a general or special-purpose computer.

[0138] An exemplary storage medium is coupled to a processor so that the processor can read information from the storage medium and write information to the storage medium. Of course, the storage medium can also be a component of the processor. The processor and the storage medium can be located in an application specific integrated circuit (English: Application Specific Integrated Circuits, referred to as: ASIC). Of course, the processor and the storage medium can also exist as discrete components in an electronic device or a main control device.

[0139] An embodiment of the present application also provides a program product, such as a computer program, which implements the metadata storage method covered by the present application when executed by a processor.

[0140] Those skilled in the art can understand that all or part of the steps of implementing the above-mentioned method embodiments can be completed by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, the steps of the above-mentioned method embodiments are executed; and the aforementioned storage medium includes: ROM, RAM, disk or optical disk and other media that can store program codes.

[0141] Finally, it should be noted that the above implementation modes are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned implementation modes, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned implementation modes, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the implementation modes of the present invention.

Claims

1. A metadata storage method, characterized in that: Applied to a distributed file system DFS, wherein the distributed file system DFS includes a plurality of metadata servers; The method comprises: Decouple the metadata of each directory in the DFS directory tree in the following manner to obtain the access metadata and content metadata corresponding to each directory metadata: decouple the directory metadata of the specified directory name into the access metadata of the specified directory name and the content metadata of the specified directory name; wherein the access metadata of the specified directory name includes the specified directory name and the globally unique identifier of the specified directory name; and the content metadata of the specified directory name includes the directory entry of the specified directory name; The access metadata and content metadata corresponding to each directory metadata in the directory tree, as well as each file metadata, are combined into a metadata group in the following manner: the access metadata of the specified directory name, the content metadata of the parent directory of the specified directory name, and the global unique identifier of the parent directory of the specified directory name are combined into a metadata group; and the content metadata of the specified directory name, the global unique identifier of the specified directory name, the access metadata of the subdirectory of the specified directory name, and the file metadata of the subfile are combined into a metadata group; Evenly distributing and storing the multiple metadata groups of the directory tree in the multiple metadata servers; The step of evenly distributing and storing the plurality of metadata groups of the directory tree on the plurality of metadata servers comprises: Performing consistent hash calculations on the metadata group and the metadata server respectively to obtain hash values ​​corresponding to the metadata group and the metadata server respectively; Based on the hash value, the metadata group is correspondingly stored in the metadata server.

2. The method according to claim 1, characterized in that: The performing consistent hash calculation on the metadata group to obtain a hash value corresponding to the metadata group includes: The metadata group is subjected to consistent hashing calculation in the following manner to obtain a hash value corresponding to the metadata group: Perform consistent hash calculation on the globally unique identifier of the parent directory of the specified directory name to obtain a hash value of the access metadata of the specified directory name; Performing consistent hash calculation on the globally unique identifier of the specified directory name to obtain a hash value of the content metadata of the specified directory name; A consistent hash calculation is performed on the globally unique identifier of the parent directory to which the file metadata belongs to obtain a hash value of the file metadata.

3. The method according to claim 1, characterized in that The storing the metadata group in the metadata server accordingly includes: The metadata groups are stored in the metadata server in the form of key-value pairs.

4. The method according to claim 3, characterized in that The access metadata of the specified directory name also includes permission information of the specified directory name; the content metadata of the specified directory name also includes a timestamp of the specified directory name; The step of storing the metadata group in the metadata server in the form of key-value pairs includes: The metadata groups are stored in the metadata server in the following key-value pairs: A key-value pair consisting of a globally unique identifier of the specified directory name and the permission information of the specified directory name as a value, and a globally unique identifier of the parent directory of the specified directory name and the specified directory name as a key; A key-value pair consisting of a directory entry of the specified directory name and a timestamp of the specified directory name as a value, and a globally unique identifier of the specified directory name as a key; A key-value pair is composed of the file metadata as a value and the globally unique identifier of the parent directory to which the file metadata belongs and the file name as a key.

5. A distributed file system, characterized in that: include: Multiple clients, multiple metadata servers, multiple data servers; wherein, The client is used to send an access request to the metadata server to access the metadata server and obtain a request result returned by the metadata server; and is also used to access the file content data stored in the data server corresponding to the request result based on the request result; The metadata server is used to store the directory metadata and file metadata of the distributed file system using the method described in any one of claims 1 to 4; and is also used to perform metadata processing on the metadata group stored in the metadata server based on the access request of the client, obtain the request result, and return the request result to the client; The metadata server is further used to maintain the consistency of the client cache item and the metadata of the distributed file system in the following manner: After receiving the access request from the client, the metadata server checks the validity of the client cache item corresponding to the access request, obtains the check result, and performs the following consistency maintenance operations based on the check result: If the checking result is that the client cache item corresponding to the access request is valid, returning the request result corresponding to the access request and an invalidation instruction for invalidating the invalid cache item in the client cache to the client; If the checking result is that the client cache item corresponding to the access request is invalid, a termination instruction indicating the termination of the access request and an invalidation instruction for invalidating the invalid cache item in the client cache are returned to the client.

6. The distributed file system according to claim 5, characterized in that: The system also includes: a coordination server; The coordination server is used to identify a directory orphan loop in the directory tree of the distributed file system when multiple clients perform directory renaming operations concurrently, and send a termination instruction to the client to terminate the directory renaming operation when the directory orphan loop is identified; The directory orphan loop is an isolated mapping loop formed by a mapping relationship between multiple directory metadata, or an isolated mapping loop formed by a mapping relationship between multiple directory metadata and file metadata.

7. A server, characterized in that: include: Processor and memory; The memory stores executable instructions executable by the processor; The processor executes the executable instructions stored in the memory, so that the processor executes the method according to any one of claims 1 to 4.

8. A storage medium, characterized in that: The storage medium stores computer-executable instructions, which are used to implement the method according to any one of claims 1 to 4 when executed by a processor.

9. A program product, comprising a computer program, wherein when the computer program is executed by a processor, the method according to any one of claims 1 to 4 is implemented.

Images