Method, device and equipment for processing directory metadata applied to distributed file system

Abstract

The disclosure provides a processing method, device and equipment applied to directory metadata of a distributed file system, relates to the technical field of computers, in particular to the field of big data and distributed file systems. The specific implementation scheme is as follows: when a processing request is acquired, a first index shard corresponding to directory metadata indicated by the processing request can be determined according to the processing request, the first index shard is loaded, and a metadata shard corresponding to the directory metadata indicated by the processing request is determined according to the first index shard. Information in the determined metadata shard is adjusted. A second index shard related to the directory metadata indicated by the processing request is determined according to the processing request, and index information corresponding to the directory metadata in the second index shard is adjusted.

Description

Technical Field

[0001] This disclosure relates to big data in computer technology, distributed file systems, and more particularly to a method, apparatus, and device for processing directory metadata applied to a distributed file system. Background Technology

[0002] As data volumes increase, distributed file systems are becoming increasingly widespread. Currently, the index unit of a device node in a distributed file system contains only one index shard. This index shard is used to write index information for all directory metadata of that device node. When reading or writing data from this device node, the entire index shard needs to be loaded, which can easily lead to reduced read / write efficiency.

[0003] Therefore, improving the read and write efficiency of this distributed file system has become an urgent problem to be solved. Summary of the Invention

[0004] This disclosure provides a method, apparatus, and device for processing directory metadata applied to a distributed file system.

[0005] According to a first aspect of this disclosure, a method for processing directory metadata applied to a distributed file system is provided. The distributed file system includes multiple device nodes, each device node corresponding to a metadata unit and an index unit. The metadata unit includes a metadata fragment corresponding to the directory metadata in the device node. The index unit includes multiple index fragments, each index fragment used to store index information for a portion of the directory metadata, including:

[0006] In response to a processing request, a first index fragment corresponding to the directory metadata indicated by the processing request is determined and loaded; and a metadata fragment corresponding to the directory metadata indicated by the processing request is determined based on the first index fragment; wherein, the processing request represents an adjustment to the directory metadata in the device node;

[0007] Adjust the information in the determined metadata fragment, and adjust the index information in the second index fragment related to the directory metadata indicated by the processing request that corresponds to the directory metadata.

[0008] According to a second aspect of this disclosure, an apparatus for processing directory metadata in a distributed file system is provided. The distributed file system includes multiple device nodes, each device node corresponding to a metadata unit and an index unit. The metadata unit includes metadata fragments corresponding to directory metadata in the device node. The index unit includes multiple index fragments, each index fragment used to store index information for a portion of the directory metadata, including:

[0009] A determining unit is configured to, in response to a processing request, determine and load a first index fragment corresponding to the directory metadata indicated by the processing request; and determine a metadata fragment corresponding to the directory metadata indicated by the processing request based on the first index fragment; wherein the processing request represents an adjustment to the directory metadata in the device node;

[0010] The adjustment unit is used to adjust the information in the determined metadata fragment, and to adjust the index information corresponding to the directory metadata in the second index fragment related to the directory metadata indicated by the processing request.

[0011] According to a third aspect of this disclosure, an electronic device is provided, comprising:

[0012] At least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of the first aspect.

[0013] According to a fourth aspect of this disclosure, a non-transitory computer-readable storage medium is provided storing computer instructions, wherein the computer instructions are configured to cause the computer to perform the method described in the first aspect.

[0014] According to a fifth aspect of this disclosure, a computer program product is provided, the computer program product comprising: a computer program stored in a readable storage medium, wherein at least one processor of an electronic device can read the computer program from the readable storage medium, and the at least one processor executes the computer program to cause the electronic device to perform the method described in the first aspect.

[0015] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0016] The accompanying drawings are provided to better understand this solution and do not constitute a limitation of this disclosure. Wherein:

[0017] Figure 1 This is a schematic diagram based on the first embodiment of the present disclosure;

[0018] Figure 2 This is a schematic diagram according to the second embodiment of the present disclosure;

[0019] Figure 3 This is a schematic diagram according to the third embodiment of the present disclosure;

[0020] Figure 4 This is a schematic diagram according to the fourth embodiment of the present disclosure;

[0021] Figure 5 This is a schematic diagram according to the fifth embodiment of the present disclosure;

[0022] Figure 6 This is a schematic diagram according to the sixth embodiment of the present disclosure;

[0023] Figure 7 This is a schematic diagram according to the seventh embodiment of the present disclosure;

[0024] Figure 8 This is a schematic block diagram of an electronic device used to implement the embodiments of this disclosure;

[0025] Figure 9 This is a schematic diagram of the structure of a distributed file system that can implement the embodiments of this disclosure. Detailed Implementation

[0026] The exemplary embodiments of this disclosure are described below with reference to the accompanying drawings, including various details of the embodiments to aid understanding, and should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this disclosure. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0027] With the increasing volume of data, distributed file systems are becoming more and more widely used. Current distributed file system nodes typically consist of two parts: a metadata unit (Dentry cell) and an index unit (Dtree cell). The index unit includes a storage area for writing directories. However, due to the limitation on the number of directories stored in this storage area, full memory can generally only support up to 1 billion directories, not hundreds of billions of files. Furthermore, limited by the single storage area's capacity for storing directories, the queuing-per-second (QPS) rate it can provide is only in the millions, insufficient for the demands of hundreds of billions of files.

[0028] To further improve the storage capacity and QPS of distributed file systems, this disclosure proposes a novel architecture for index units to address the poor scalability and low QPS issues inherent in current D-tree single-point distributed file systems. Specifically, this disclosure optimizes the index units by pre-creating N index shards during file system creation. Operations such as creating directories (mkdir) and deleting directories (rmdir) are routed to fixed shards using a full path hash, and the corresponding index information is created or deleted within those shards. Path modification (mv) requires searching and processing across all index shards. This method of routing to specific index shards based on full path hashes effectively improves QPS. Furthermore, this sharded path storage method effectively increases file storage capacity.

[0029] For example, taking a query for directory c as an example, the query request includes directory metadata for directory c, and the target path of this directory metadata is / a / b / c. First, based on the full path / a / b / c, the index shard can be determined using a full path hash. Then, through a Remote Procedure Call (RPC), this index shard can be used to retrieve the metadata encoding of directory c. Finally, another RPC call is used to call the metadata shard corresponding to the metadata encoding to retrieve the attribute information of directory c.

[0030] This disclosure provides a method, apparatus, and device for processing directory metadata in a distributed file system, applicable to big data and distributed file systems in the field of computer technology, to improve the read and write efficiency of the distributed file system.

[0031] It should be noted that the data in this embodiment is not specific to any particular user and does not reflect the personal information of any particular user. It should also be noted that the data in this embodiment comes from a publicly available dataset.

[0032] The collection, storage, use, processing, transmission, provision, and disclosure of user personal information involved in the technical solution disclosed herein comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0033] To help readers gain a deeper understanding of the implementation principles of this disclosure, the following will be discussed in conjunction with... Figures 2-5 right Figure 1 The illustrated embodiments are further refined.

[0034] Figure 1 This is a schematic diagram based on the first embodiment of the present disclosure, as shown below. Figure 1As shown, this disclosure provides a method for processing directory metadata applied to a distributed file system. The distributed file system used in this disclosure includes multiple device nodes. Each device node corresponds to a metadata unit and an index unit. The metadata unit includes a metadata fragment corresponding to the directory metadata in the device node. The index unit includes multiple index fragments. The index fragments are used to store index information for a portion of the directory metadata. The method includes:

[0035] 110. In response to a processing request, determine and load the first index fragment corresponding to the directory metadata indicated by the processing request; and determine the metadata fragment corresponding to the directory metadata indicated by the processing request based on the first index fragment. The processing request represents an adjustment to the directory metadata in the device node.

[0036] In this embodiment, a user's processing request can be obtained. This processing request is used to adjust directory metadata in the device node. Optionally, the adjustment of the directory metadata can involve creating, deleting, or modifying the path of the directory metadata. Optionally, the processing request can typically be an operation code. Optionally, the processing request must at least include the directory metadata to be processed and path information. Optionally, when the processing request is used to instruct the creation or deletion of the directory metadata, the path information must at least include a target path. The target path is the path of the directory metadata to be deleted. Alternatively, the target path is the path of the directory metadata to be created. Optionally, when the processing request is used to instruct the modification of the path of the directory metadata, the path information must at least include an initial path and a final path. The initial path is the path of the directory metadata to be moved. The final path is the path the directory metadata needs to reach after being moved.

[0037] Based on the processing request, the first index fragment corresponding to the directory metadata indicated by the processing request can be determined. Then, the first index fragment can be loaded, and within that first index fragment, the index information corresponding to the path information contained in the processing request can be retrieved. This index information can indicate the corresponding metadata fragment, thereby determining the metadata fragment corresponding to the directory metadata indicated by the processing request.

[0038] 120. Adjust the information in the determined metadata fragments, and adjust the index information corresponding to the directory metadata in the second index fragment related to the directory metadata indicated by the processing request.

[0039] In this embodiment, a metadata fragment can be determined according to step 110, and the metadata fragment can be adjusted to realize the creation, deletion, and modification of information in the metadata fragment. Optionally, the information in the metadata fragment can be the attribute information of the directory metadata corresponding to the metadata fragment. In addition, according to the processing request, one or more second index fragments related to the directory metadata of the processing request can be determined in the index unit. After determining the second index fragment, the index information in the second index fragment can be modified to ensure the consistency of data between the index unit and the metadata unit. Optionally, during the modification of the metadata fragment and the second index fragment, the processing request is successful only if the modification of all full data fragments and the second index fragment corresponding to the processing request is successful.

[0040] In this embodiment, by determining and loading the first index fragment corresponding to the directory metadata indicated by the processing request, the metadata fragment can be determined, thereby enabling the modification of the metadata fragment. This process partially loads the data in the index unit by loading the first index fragment, reducing the amount of data that needs to be loaded and improving loading efficiency. Furthermore, by modifying the index information in the second index fragment related to the directory metadata indicated by the processing request, the corresponding modification of the index fragment is achieved, improving the data modification efficiency in the index unit and ensuring the consistency of data between the index unit and the metadata unit.

[0041] Figure 2 This is a schematic diagram based on the second embodiment of the present disclosure, as shown below. Figure 2 As shown, this disclosure provides a method for processing directory metadata applied to a distributed file system, the method comprising:

[0042] 210. Calculate the index code based on the path information in the directory metadata. The index code value represents the encoding of the index fragment.

[0043] In this embodiment, the path information of the directory metadata can first be obtained from the processing request. Based on this path information, the index code can be calculated. This index code is the code for the index fragment. There is a one-to-one correspondence between the index code and the index fragment.

[0044] Optionally, the information contained in the path information may differ for different processing requests; therefore, the calculation process of the index encoding may differ for different processing requests.

[0045] Optionally, when processing a request instructing the deletion of directory metadata, or when creating directory metadata, the path information of the directory metadata includes the target path. In this case, it is necessary to calculate the index code of the directory metadata and the index code of its parent directory. The calculation process may include:

[0046] 2111. Calculate the first index code based on the target path.

[0047] In this embodiment, a first index code for the directory metadata can be calculated based on the full path information of the target path. The index fragment indicated by the first index code corresponds to the directory path. That is, when processing a request to delete directory metadata, the index fragment indicated by the first index code contains the target path. When processing a request to create directory metadata, the index fragment indicated by the first index code is the index fragment stored in the target path.

[0048] 2112. Based on the target path of the metadata, determine the target parent path of the parent directory of the target metadata in the target path.

[0049] In this embodiment, the parent directory of the target metadata can be determined based on the target path of the metadata. The parent directory is the path that remains after deleting the directory corresponding to the target metadata from the target path. This path is the target parent path. For example, when the target path is / a / b / c, the target parent path is / a / b.

[0050] 2113. Calculate the second index code based on the parent path.

[0051] In this embodiment, the second index code of the directory metadata can be calculated based on the full path information of the target parent path. This second index code indicates the index fragment and the target parent path. That is, the index fragment indicated by the second index code contains the target parent path.

[0052] Through steps 2111 to 2113 above, the calculation of index codes is implemented when processing requests to delete or create directory metadata. This can include a first index code for the index shard indicating the path to the directory metadata, and a second index code for the index shard indicating the path to the parent directory of the directory metadata. The calculation of these two index codes confirms the index shards that need to be loaded, thereby avoiding the need to load the entire index unit, reducing the amount of data loaded, and improving data loading efficiency.

[0053] Optionally, when a request indicates a path for modifying directory metadata, the path information of the directory metadata includes an initial path and a final path. The initial path indicates the path before the directory metadata is modified, and the final path indicates the path after the directory metadata is modified. In this case, it is necessary to determine the index code of the parent directory where the directory metadata was located before modification based on the initial path, and the index code of the parent directory where the directory metadata is located after modification based on the final path. The calculation process may include:

[0054] 2121. Based on the final path of the metadata, determine the first parent path of the parent directory of the directory metadata in the final path.

[0055] 2122. Calculate the third index code based on the first parent path.

[0056] 2123. Based on the initial path of the metadata, determine the second parent path of the parent directory of the directory metadata in the initial path.

[0057] 2124. The fourth index code is calculated based on the second parent path.

[0058] The implementation of steps 2121 and 2123 is similar to that of step 2112, and the implementation of steps 2122 and 2124 is similar to that of step 2113, and will not be repeated here. Through the execution of steps 2121 to 2124, when processing a request indicating the path to modify directory metadata, the paths of the parent directories of the directory metadata on the final path and the initial path are obtained, and the index codes corresponding to these two parent directories are calculated. The calculation of these two index codes can confirm the index fragments that need to be loaded, thereby avoiding the need to load the entire index unit, reducing the amount of data loaded, and improving data loading efficiency.

[0059] Optionally, taking the target path as an example, the process of calculating the first index code based on the target path may include the following steps:

[0060] 2131. Calculate the MD5 value of the target path.

[0061] In this embodiment, the MD5 value can be calculated based on the target path. The calculation of the MD5 value can be accomplished using a preset message digest algorithm.

[0062] 2132. Determine the index code based on the MD5 value and the number of index fragments preset in the index unit.

[0063] In this embodiment, after determining the MD5 value, the final index code can be obtained by taking the modulo operation based on the preset number of index shards in the index unit. Optionally, the number of index shards in the index unit can be preset when the device node of the distributed file system is created. This modulo operation can realize the allocation of the target path to the index shard. This allocation method can improve the data balancing effect and avoid the index information being concentrated in a few index shards.

[0064] Optionally, the first index code and the second index code mentioned above may be the same. Also, the third index code and the fourth index code may be the same.

[0065] 220. Based on the index encoding, load the first index fragment corresponding to the directory metadata indicated by the processing request.

[0066] In this embodiment, after determining at least one index code according to step 210, the loading of the first index fragment can be completed according to the index code.

[0067] In this embodiment, by determining and loading at least one first index fragment corresponding to the directory metadata indicated by the processing request, partial data loading within the index unit is achieved. This avoids the need to load the entire index unit, reduces the amount of data loaded, and improves the index unit loading efficiency in response to the processing request. Furthermore, this helps improve the path resolution efficiency of the distributed file system and increases the QPS limit.

[0068] Figure 3 This is a schematic diagram based on the third embodiment of the present disclosure, as shown below. Figure 3 As shown, this disclosure provides a method for processing directory metadata applied to a distributed file system, the method comprising:

[0069] 310. Based on the path information of the directory metadata indicated in the processing request, determine the metadata encoding corresponding to the directory metadata in the index fragment. Metadata represents the encoding of the metadata fragment.

[0070] In this embodiment, the processing request includes path information of directory metadata. Based on this path information, the metadata encoding corresponding to the path information can be determined in the index fragment. This metadata encoding is the encoding of the metadata fragment. Each metadata encoding corresponds one-to-one with a metadata fragment.

[0071] Optionally, the specific process of determining the metadata encoding corresponding to the path information in the index shard may include:

[0072] 311. Based on the path information of the directory metadata indicated by the processing request, determine the index information corresponding to the path information in the index fragment.

[0073] In this embodiment, the path information corresponding to the directory metadata can be determined according to the second embodiment. Based on this path information, index information can be retrieved from the directory metadata. Based on this index information...

[0074] 312. Determine the metadata encoding corresponding to the index information in the index shard.

[0075] In this embodiment, the index shard may store the correspondence between the index information and the metadata encoding. Based on this correspondence, the metadata encoding corresponding to the index information can be determined after the index information is determined.

[0076] Through steps 311 and 312 above, a query can be completed in the index shard based on the path information to obtain the index information and the data encoding corresponding to the index information, thereby realizing the determination of the metadata shard, improving the determination efficiency of the metadata shard, and improving the parsing efficiency of the path.

[0077] Optionally, when the processing request instructs the deletion of directory metadata, or the creation of directory metadata, the index code calculated in the second embodiment may include a first index code and a second index code. The first index code is the index code corresponding to the target path of the directory metadata indicated by the processing request. A first index fragment can be determined based on the first index code. Within the first index fragment, a search can be performed based on the target path to obtain the index information of the target path and the metadata code corresponding to the index information. This metadata code can point to the metadata fragment of the directory metadata. The second index code is the index code corresponding to the target parent path of the parent directory of the directory metadata indicated by the processing request. A second first index fragment can be determined based on the second index code. Within this first index fragment, a search can be performed based on the target parent path to obtain the index information of the target parent path and the metadata code corresponding to the index information. This metadata code can point to the metadata fragment of the parent directory of the directory metadata.

[0078] Optionally, when the processing request indicates a path for modifying directory metadata, the index code calculated in the second embodiment may include a third index code and a fourth index code. The third index code is the index code corresponding to the first parent path of the parent directory of the directory metadata in the final path. A first index fragment can be determined based on the third index code. Within this first index fragment, a search can be performed based on the first parent path to obtain the index information of the first parent path and the metadata code corresponding to that index information. This metadata code can point to the metadata fragment of the parent directory of the directory metadata in the final path. The fourth index code is the index code corresponding to the second parent path of the parent directory of the directory metadata in the initial path. A second first index fragment can be determined based on the fourth index code. Within this first index fragment, a search can be performed based on the second parent path to obtain the index information of the second parent path and the metadata code corresponding to that index information. This metadata code can point to the metadata fragment of the second parent path of the parent directory of the directory metadata in the initial path.

[0079] 320. Based on the metadata encoding, determine the metadata fragment corresponding to the directory metadata indicated by the processing request.

[0080] In this embodiment, after determining at least one metadata code according to step 310, the corresponding metadata fragment can be obtained in the metadata unit according to the metadata code, thereby realizing the reading and writing of the metadata fragment.

[0081] In this embodiment, the metadata encoding is determined by querying the index information in the first index shard based on the path information, and the metadata shard is read and written based on the metadata encoding. This improves the efficiency of determining the metadata shard and the efficiency of parsing the path in the distributed file system.

[0082] Figure 4 This is a schematic diagram based on the fourth embodiment of the present disclosure, as shown below. Figure 4 As shown, this disclosure provides a method for processing directory metadata in a distributed file system. After determining the metadata fragments, this method can adjust the information within those fragments. Based on three different processing requests—deletion, creation, and modification—the adjustment process for the metadata fragments can include:

[0083] 410. When a request is processed indicating that directory metadata should be deleted, delete the metadata fragment corresponding to the directory metadata and modify the information in the metadata fragment corresponding to the parent directory of the directory metadata.

[0084] In this embodiment, when the processing request indicates the deletion of the directory metadata, two metadata fragments can be obtained according to the second and third embodiments. One metadata fragment is the metadata fragment corresponding to the directory metadata, and the other metadata fragment can be the metadata fragment corresponding to the parent directory of the directory metadata. Since the directory metadata needs to be deleted in response to the processing request, the metadata allocation corresponding to the directory metadata also becomes meaningless after the directory metadata is deleted. Therefore, the metadata fragment corresponding to the directory metadata can be deleted accordingly. When the directory metadata is deleted, the subdirectories associated with the parent directory of the directory metadata change. Therefore, in order to ensure that the attribute information of the parent directory of the directory metadata matches the actual situation, the information in the metadata fragment of the parent directory of the directory metadata needs to be updated.

[0085] Optionally, the above-described metadata fragmentation process may include the following steps:

[0086] 411. Delete the first metadata shard.

[0087] In this embodiment, the metadata fragment corresponding to the directory metadata is designated as the first metadata fragment. This first metadata fragment can be deleted from this metadata unit.

[0088] 412. Subtract the amount of directory metadata from the information in the second metadata fragment.

[0089] In this embodiment, the metadata fragment corresponding to the parent directory of the directory metadata is designated as the second metadata fragment. The quantity of the directory metadata can be subtracted from the information of this second metadata fragment. For example, the information of this second metadata fragment may include the number of subdirectories under the parent directory. When the directory metadata is deleted, the number of subdirectories will decrease by one.

[0090] Step 410 enables the processing of metadata fragments when a request is made to delete directory metadata, thereby ensuring the execution of the request, improving the response efficiency of the request, and enhancing the user experience.

[0091] 420. When the processing request indicates that directory metadata should be created, create the metadata fragment corresponding to the directory metadata and modify the information in the metadata fragment corresponding to the parent directory of the directory metadata.

[0092] In this embodiment, when the processing request instructs the creation of the directory metadata, two metadata fragments can be obtained according to the second and third embodiments. One metadata fragment is the metadata fragment corresponding to the directory metadata, and the other metadata fragment can be the metadata fragment corresponding to the parent directory of the directory metadata. Since the directory metadata needs to be created in response to the processing request, the metadata fragment corresponding to the directory metadata needs to be created within the metadata unit. After the directory metadata is created, the associated subdirectories of the parent directory of the directory metadata change. Therefore, to ensure that the attribute information of the parent directory of the directory metadata matches the actual situation, the information in the metadata fragment of the parent directory of the directory metadata needs to be updated.

[0093] Optionally, the above-described metadata fragmentation process may include the following steps:

[0094] 421. Create the first metadata shard.

[0095] In this embodiment, the metadata fragment corresponding to the directory metadata is designated as the first metadata fragment. This first metadata fragment can be created within this metadata unit.

[0096] 422. Increase the amount of directory metadata in the information of the second metadata fragment.

[0097] In this embodiment, the metadata fragment corresponding to the parent directory of the directory metadata is designated as the second metadata fragment. The number of directory metadata entries can be incremented in the information of this second metadata fragment. For example, the information of the second metadata fragment may include the number of subdirectories under the parent directory. When the directory metadata is created, the number of subdirectories will be incremented by one.

[0098] Step 420 enables the processing of metadata fragments when a processing request is used to create directory metadata, thereby ensuring the execution of the processing request, improving the response efficiency of the processing request, and enhancing the user experience.

[0099] 430. When processing a request indicating that the directory metadata should be modified, modify the metadata fragment corresponding to the parent directory of the directory metadata indicated by the final path and the metadata fragment corresponding to the parent directory of the directory metadata indicated by the initial path.

[0100] In this embodiment, when the processing request indicates modification of the directory metadata, according to the second and third embodiments, two metadata fragments can be obtained. One metadata fragment is the metadata fragment corresponding to the parent directory of the directory metadata indicated by the final path, and the other metadata fragment can be the metadata fragment corresponding to the parent directory of the directory metadata indicated by the initial path. In response to the processing request, the directory metadata changes its location, which is equivalent to changing the parent directory of the directory metadata. That is, it is deleted from the parent directory corresponding to the initial path and added to the parent directory corresponding to the final path. For the directory metadata itself, its subdirectories remain unchanged. Therefore, the metadata fragments of the directory metadata itself do not change.

[0101] Optionally, the above-described metadata fragmentation process may include the following steps:

[0102] 431. Increase the amount of directory metadata in the information of the third metadata fragment.

[0103] In this embodiment, the metadata fragment corresponding to the parent directory of the directory metadata indicated by the final path is designated as the third metadata fragment. The number of directory metadata entries can be incremented in the information of this third metadata fragment. For example, the information of this third metadata fragment may include the number of subdirectories under the parent directory. When the directory metadata is moved to the parent directory indicated by the final path, the number of subdirectories will be incremented by one.

[0104] 432. Subtract the amount of directory metadata from the information in the second metadata fragment.

[0105] In this embodiment, the metadata fragment corresponding to the parent directory of the directory metadata indicated by the initial path is designated as the fourth metadata fragment. The number of directory metadata can be subtracted from the information of this fourth metadata fragment. For example, the information of this fourth metadata fragment may include the number of subdirectories under the parent directory. When the directory metadata is moved out of the parent directory corresponding to the initial path, the number of subdirectories will decrease by one.

[0106] Step 430 enables the processing of metadata fragments when a request is made to modify the directory metadata, thereby ensuring the execution of the request, improving the response efficiency of the request, and enhancing the user experience.

[0107] In this embodiment, by modifying the metadata fragment, the response to the processing request in the metadata unit is realized, which improves the response efficiency of the processing request and enhances the user experience.

[0108] Figure 5 This is a schematic diagram based on the fifth embodiment of the present disclosure, as shown below. Figure 5 As shown, this disclosure provides a method for processing directory metadata in a distributed file system. This method can process a second index shard by determining the second index shard associated with the directory metadata indicated by the processing request. Depending on three different processing requests—deletion, creation, and modification—the determination and adjustment process of the second index shard may include:

[0109] 510. When processing a request to delete directory metadata, the index information corresponding to the target path in the index shard corresponding to the target path of the directory metadata to be deleted.

[0110] In this embodiment, when a processing request indicates the deletion of directory metadata, the path information of the processing request may include a target path. This target path is the path to the directory metadata that needs to be deleted. Based on this target path, the index code corresponding to the target path can be determined according to the index fragment determination method shown in the second embodiment, thereby determining the second index fragment. When the directory metadata is deleted, the index information of the directory metadata will become ineffective. Therefore, the index information corresponding to the target path can be deleted from the index fragment.

[0111] Optionally, the above process of determining and processing index shards may specifically include the following steps:

[0112] 511. Determine the second index shard based on the target path of the directory metadata.

[0113] In this embodiment, the MD5 value of the directory metadata can be determined based on the target path. Then, the index code is determined based on the modulus of the preset number of index shards within the MD5 value range. The corresponding second index shard is then determined based on this index code.

[0114] 512. Delete the index information corresponding to the target path of the directory metadata in the second index shard.

[0115] In this embodiment, the target path can be queried in the second index shard to obtain the index information corresponding to the target path. The index information is then deleted from the second index shard. Optionally, deleting the index information includes deleting the metadata encoding stored in the second index shard corresponding to the index information. Optionally, deleting the index information includes deleting all information stored in the second index shard corresponding to the index information.

[0116] Step 510 enables the processing of index fragments in the index unit when a request is made to delete directory metadata, thereby ensuring the execution of the processing request, improving the response efficiency of the processing request, and enhancing the user experience.

[0117] 520. When a request is processed to indicate the creation of directory metadata, create the index information corresponding to the target path in the index shard corresponding to the target path of the directory metadata.

[0118] In this embodiment, when a processing request instructs the creation of directory metadata, the path information of the processing request may include a target path. This target path is the path to the directory metadata to be created. Based on this target path, the index code corresponding to the target path can be determined according to the index shard determination method shown in the second embodiment, thereby determining the second index shard. When the directory metadata is created, the index information of the target path of the directory metadata needs to be created synchronously in the index shard, thereby ensuring that the newly created directory metadata has corresponding index information.

[0119] Optionally, the above process of determining and processing index shards may specifically include the following steps:

[0120] 521. Determine the second index shard based on the target path of the directory metadata.

[0121] In this embodiment, the MD5 value of the directory metadata can be determined based on the target path. Then, the index code is determined based on the modulus of the preset number of index shards within the MD5 value range. The corresponding second index shard is then determined based on this index code.

[0122] 522. Create index information corresponding to the target path of directory metadata in the index shard.

[0123] In this embodiment, index information for the directory path can be created in the second index shard. Optionally, after creating the index information, the metadata encoding of the metadata shard of the directory metadata that has already been created can be obtained from the metadata shard. The metadata encoding is stored in the second index shard, and the correspondence between the metadata encoding and the index information is also stored.

[0124] Step 520 enables the processing of index shards in the index unit when a processing request is used to create directory metadata, thereby ensuring the execution of the processing request, improving the response efficiency of the processing request, and enhancing the user experience.

[0125] 530. When processing a request indicating that a directory's metadata needs to be modified, delete the index information containing the directory metadata from each index shard; and create the index information for the path in the index shard corresponding to the path, based on the modified paths of the directory metadata and its subdirectories.

[0126] In this embodiment, when a processing request indicates a directory whose metadata needs to be modified, the path information of the processing request may include a final path and an initial path. The final path is the path after the directory metadata has been modified, and the initial path is the path before the directory metadata has been modified.

[0127] When the directory metadata is moved out of the initial path, all subdirectories containing the directory metadata will also be moved out. Existing index information containing the directory metadata in this index unit will become invalid. Therefore, index information containing the directory metadata can be deleted from each index shard.

[0128] After the directory metadata is moved to the final path, the paths of the subdirectories within that directory metadata will be modified accordingly. The index information for both the directory metadata and its subdirectories needs to be recreated. Therefore, the index information for that path can be created in the corresponding index shard based on the modified paths of the directory metadata and its subdirectories.

[0129] Optionally, the above process of determining and processing index shards may specifically include the following steps:

[0130] 531. If an index shard contains directory metadata, then the index shard is determined to be the second index shard, and the index information containing directory metadata in the second index shard is deleted.

[0131] In this embodiment, the determination of whether the directory metadata is contained in the index information can be achieved by traversing the index information in each index shard. When the index shard contains the directory metadata, the index shard can be identified as the second index shard. When the index information contains the directory metadata, the index information can be deleted.

[0132] 532. Based on the final path and the initial path, determine the modified paths of the directory metadata and its subdirectories.

[0133] In this embodiment, the subdirectory path of the subdirectory of the directory metadata before modification can be obtained. Using this final path, the directory metadata and the portion preceding it in the subdirectory path are replaced to obtain the modified path of the subdirectory. The modified path of the directory metadata is the final path.

[0134] 533. Based on the modified paths of the directory metadata and its subdirectories, create index information for the modified paths in the corresponding second index shard.

[0135] In this embodiment, based on the modified path obtained in step 532, the corresponding index fragment can be determined one by one in the index unit, and the index information of the modified path can be created in the index fragment. Optionally, after the index information of the modified path is created, the metadata of the directory and its subdirectories can also be encoded and written into the corresponding position of the index information of the modified path.

[0136] Step 530 enables the processing of index fragments in the index unit when a request is made to delete directory metadata, thereby ensuring the execution of the processing request, improving the response efficiency of the processing request, and enhancing the user experience.

[0137] In this embodiment, by modifying the metadata shards, the response to the processing request in the index unit is realized, which improves the response efficiency of the processing request and enhances the user experience.

[0138] Figure 6 This is a schematic diagram based on the sixth embodiment of the present disclosure, as shown below. Figure 6 As shown, this disclosure provides a directory metadata processing device 600 applied to a distributed file system. The distributed file system includes multiple device nodes, each device node corresponding to a metadata unit and an index unit. The metadata unit includes metadata fragments corresponding to the directory metadata in the device node, and the index unit includes multiple index fragments. The index fragments are used to store index information for a portion of the directory metadata. This includes:

[0139] The determining unit 610 is configured to, in response to a processing request, determine and load the first index fragment corresponding to the directory metadata indicated by the processing request, and determine the metadata fragment corresponding to the directory metadata indicated by the processing request based on the first index fragment. The processing request represents an adjustment to the directory metadata in the device node.

[0140] The adjustment unit 620 is used to adjust the information in the determined metadata fragments and to adjust the index information corresponding to the directory metadata in the second index fragment related to the directory metadata indicated by the processing request.

[0141] The apparatus in this embodiment can execute the technical solutions in the above method. Its specific implementation process and technical principles are the same, and will not be repeated here.

[0142] Figure 7 This is a schematic diagram based on the seventh embodiment of the present disclosure, as shown below. Figure 7 As shown, this disclosure provides a directory metadata processing apparatus 700 applied to a distributed file system. The distributed file system includes multiple device nodes, each device node corresponding to a metadata unit and an index unit. The metadata unit includes metadata fragments corresponding to the directory metadata in the device node, and the index unit includes multiple index fragments. The index fragments are used to store index information for a portion of the directory metadata. The apparatus includes:

[0143] The determining unit 710 is configured to, in response to a processing request, determine and load the first index fragment corresponding to the directory metadata indicated by the processing request, and determine the metadata fragment corresponding to the directory metadata indicated by the processing request based on the first index fragment. The processing request represents an adjustment to the directory metadata in the device node.

[0144] The adjustment unit 720 is used to adjust the information in the determined metadata fragments and to adjust the index information corresponding to the directory metadata in the second index fragment related to the directory metadata indicated by the processing request.

[0145] Optionally, the path information in the request includes directory metadata. The determining unit 710 includes:

[0146] The first calculation module 711 is used to calculate the index code based on the path information of the directory metadata. The index code value represents the code of the index fragment.

[0147] The first determining module 712 is used to determine and load the first index fragment corresponding to the directory metadata indicated by the processing request based on the index encoding.

[0148] Optionally, when the processing request instructs the deletion of directory metadata, or when creating directory metadata, the path information of the directory metadata includes the target path. The calculation module 711 includes:

[0149] The first calculation submodule 7111 is used to calculate the first index code based on the target path.

[0150] The first determining submodule 7112 is used to determine the target parent path of the parent directory of the metadata in the target path based on the target path of the metadata.

[0151] The second calculation submodule 7113 is used to calculate the second index code based on the parent path.

[0152] Optionally, the first calculation submodule 7111 is specifically used for:

[0153] Calculate the MD5 value of the target path.

[0154] The index encoding is determined based on the MD5 value and the number of index fragments preset in the index unit.

[0155] Optionally, when the processing request indicates a path to modify directory metadata, the path information in the directory metadata includes an initial path and a target path. The calculation module 711 includes:

[0156] The second determining submodule 7114 is used to determine the first parent path of the parent directory of the metadata in the target path based on the target path of the metadata.

[0157] The third calculation submodule 7115 is used to calculate the third index code based on the first parent path.

[0158] The third determining submodule 7116 is used to determine the second parent path of the parent directory of the metadata in the initial path based on the initial path of the metadata.

[0159] The fourth submodule 7117 is used to calculate the fourth index code based on the second parent path.

[0160] Optionally, the path information in the request includes directory metadata. The determining unit 710 includes:

[0161] The second determining module 713 is used to determine the metadata encoding corresponding to the directory metadata in the index fragment based on the path information of the directory metadata indicated by the processing request. The metadata represents the encoding of the metadata fragment.

[0162] The third determining module 714 is used to determine the metadata fragment corresponding to the directory metadata indicated by the processing request based on the metadata encoding.

[0163] Optionally, the second determining module 713 includes:

[0164] The first determining submodule 7131 is used to determine the index information corresponding to the path information in the index fragment based on the path information of the directory metadata indicated by the processing request.

[0165] The second determining submodule 7132 is used to determine the metadata encoding corresponding to the index information in the index shard.

[0166] Optionally, when a processing request indicates the deletion of directory metadata, the metadata fragment includes a first metadata fragment corresponding to the directory metadata and a second metadata fragment corresponding to the parent directory of the directory metadata. The adjustment unit 720 includes:

[0167] The first deletion module 7211 is used to delete the first metadata fragment.

[0168] The first adjustment module 7212 is used to subtract the amount of directory metadata from the information in the second metadata fragment.

[0169] Optionally, when the processing request indicates the creation of directory metadata, the metadata fragment includes a first metadata fragment corresponding to the directory metadata and a second metadata fragment corresponding to the parent directory of the directory metadata. Adjustment unit 720 includes:

[0170] The first creation unit 7213 is used to create the first metadata fragment.

[0171] The second adjustment module 7214 is used to increase the amount of directory metadata in the information of the second metadata fragment.

[0172] Optionally, when processing a request indicating a directory to modify directory metadata, the metadata fragment includes a third metadata fragment corresponding to the parent directory indicated by the final path of the directory metadata, and a fourth metadata fragment corresponding to the parent directory indicated by the initial path of the directory metadata. The adjustment unit 720 includes:

[0173] The third adjustment module 7215 is used to increase the amount of directory metadata in the information of the third metadata fragment.

[0174] The fourth adjustment module 7216 is used to subtract the amount of directory metadata from the information in the second metadata fragment.

[0175] Optionally, when processing a request instructing the deletion of directory metadata, the path information of the directory metadata includes the target path. Adjustment unit 720 includes:

[0176] The first determining module 7221 is used to determine the second index fragment based on the target path of the directory metadata.

[0177] The first deletion module 7222 is used to delete the index information corresponding to the target path of the directory metadata in the second index fragment.

[0178] Optionally, when the processing request instructs the creation of directory metadata, the path information of the directory metadata includes the target path. Adjustment unit 720 includes:

[0179] The second determining module 7223 is used to determine the second index fragment based on the target path of the directory metadata.

[0180] The first creation module 7224 is used to create index information corresponding to the target path of directory metadata in the index shard.

[0181] Optionally, when processing a request indicating a directory for modifying directory metadata, the path information in the directory metadata includes both the final path and the initial path. The adjustment unit 720 includes:

[0182] The second deletion module 7225 is used to determine that if the index shard contains directory metadata, the index shard is a second index shard and the index information corresponding to the directory metadata in the second index shard is deleted.

[0183] The third determining module 7226 is used to determine the subdirectory path of the subdirectory of the directory metadata based on the final path and the initial path.

[0184] The second creation module 7227 is used to determine the second index shard corresponding to the subdirectory path based on the subdirectory path of the directory metadata, and to create index information of the subdirectory path in each index shard.

[0185] The apparatus in this embodiment can execute the technical solutions in the above method. Its specific implementation process and technical principles are the same, and will not be repeated here.

[0186] According to embodiments of this disclosure, this disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0187] According to embodiments of this disclosure, this disclosure also provides a computer program product comprising: a computer program stored in a readable storage medium, at least one processor of an electronic device being able to read the computer program from the readable storage medium, and the at least one processor executing the computer program causing the electronic device to perform the scheme provided in any of the above embodiments.

[0188] Figure 8 A schematic block diagram of an example electronic device 800 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0189] like Figure 8As shown, device 800 includes a computing unit 801, which can perform various appropriate actions and processes based on a computer program stored in read-only memory (ROM) 802 or a computer program loaded from storage unit 808 into random access memory (RAM) 803. RAM 803 may also store various programs and data required for the operation of device 800. The computing unit 801, ROM 802, and RAM 803 are interconnected via bus 804. Input / output (I / O) interface 805 is also connected to bus 804.

[0190] Multiple components in device 800 are connected to I / O interface 805, including: input unit 806, such as keyboard, mouse, etc.; output unit 807, such as various types of monitors, speakers, etc.; storage unit 808, such as disk, optical disk, etc.; and communication unit 809, such as network card, modem, wireless transceiver, etc. Communication unit 809 allows device 800 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0191] The computing unit 801 can be various general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 801 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 801 performs the various methods and processes described above, such as the method for processing directory metadata applied to a distributed file system. For example, in some embodiments, the method for processing directory metadata applied to a distributed file system can be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 808. In some embodiments, part or all of the computer program can be loaded and / or installed on device 800 via ROM 802 and / or communication unit 809. When the computer program is loaded into RAM 803 and executed by the computing unit 801, one or more steps of the method for processing directory metadata applied to a distributed file system described above can be performed. Alternatively, in other embodiments, computing unit 801 may be configured by any other suitable means (e.g., by means of firmware) to perform a method of processing directory metadata applied to a distributed file system.

[0192] Figure 9 This is a schematic diagram illustrating the structure of a distributed file system that can implement embodiments of the present disclosure. This distributed file system may include multiple device nodes. Each device node can be located within an electronic device. For example... Figure 9As shown, a device node in this distributed file system can include a metadata unit and an index unit. The metadata unit can include multiple metadata shards. Each metadata shard can correspond to a directory metadata location. The index unit can include multiple index shards. The number of index shards can be set when the device node is created. Each index shard can store index information for a portion of the directory metadata. Furthermore, the view of directory metadata and file metadata in this device node can be as follows: Figure 9 The tree structure is shown in the image. Boxes represent directory metadata, and circles represent file metadata. The view can include a tree structure of all directory and file metadata for that device node.

[0193] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0194] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0195] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0196] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0197] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0198] Computer systems can include clients and servers. Clients and servers are generally geographically separated and typically interact via communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. A server can be a cloud server, also known as a cloud computing server or cloud host, a hosting product within the cloud computing service system that addresses the shortcomings of traditional physical hosts and VPS (Virtual Private Server) services, such as high management difficulty and weak business scalability. Servers can also be servers for distributed systems or servers incorporating blockchain technology.

[0199] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this disclosure can be achieved, and this is not limited herein.

[0200] The specific embodiments described above do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure should be included within the scope of protection of this disclosure.

Claims

1. A method for processing directory metadata of a distributed file system, applied to a distributed file system, wherein a device node of the distributed file system is provided with a metadata unit containing a plurality of metadata shards and an index unit containing a plurality of index shards, the method comprising: determining and loading a first index shard corresponding to directory metadata indicated by a processing request in response to the processing request; and determining a metadata shard corresponding to the directory metadata indicated by the processing request according to the first index shard, wherein the processing request represents an adjustment to the directory metadata in the device node, the index shard is used to store index information of part of the directory metadata, the metadata shard is determined based on a metadata encoding corresponding to first index information in the first index shard, and the first index information is information in the first index shard corresponding to path information of the directory metadata; adjusting information in the determined metadata shard and adjusting index information corresponding to the directory metadata in a second index shard related to the directory metadata indicated by the processing request, so that the index unit and the metadata unit are consistent; wherein the path information of the directory metadata is included in the processing request; the determining and loading of the first index shard corresponding to the directory metadata indicated by the processing request comprises: calculating an index encoding according to the path information of the directory metadata; the index encoding value represents the encoding of the index shard; loading the first index shard corresponding to the directory metadata indicated by the processing request according to the index encoding; the second index shard is determined according to an index encoding of a parent directory of the directory metadata, the index encoding of the parent directory is calculated according to a target parent path of the directory metadata, and the target parent path is determined according to the path information of the directory metadata; the metadata shard includes a metadata shard corresponding to the directory metadata and a metadata shard corresponding to a parent directory of the directory metadata; when the processing request indicates deleting the directory metadata or creating the directory metadata, the path information of the directory metadata includes a target path; the target path indicates the path of the directory metadata; the calculating of the index encoding according to the path information of the directory metadata comprises: calculating the first index encoding according to the target path; determining a target parent path of a parent directory of the directory metadata in the target path according to the target path of the metadata; and calculating the second index encoding according to the target parent path; the calculating of the first index encoding according to the target path comprises: calculating an MD5 value of the target path; and determining the index encoding according to the MD5 value and a preset number of the index shards in the index unit; when the processing request indicates modifying the path of the directory metadata, the path information of the directory metadata includes an initial path and a final path. ​ ​ ​ ​ ​ ​ ​ 2. The method of claim 1, wherein, ​ ​ ​ ​ ​ ​ 3. The method of claim 2, wherein, ​ ​ ​ 4. The method of claim 1, wherein, ​ The initial path indicates a path when the directory metadata is not modified, and the final path indicates a path after the directory metadata is modified. The index code is calculated according to the path information of the directory metadata, and the method comprises the following steps: According to the final path of the metadata, a first parent path of a parent directory of the directory metadata in the final path is determined; According to the first parent path, a third index code is calculated; According to the initial path of the metadata, a second parent path of a parent directory of the directory metadata in the initial path is determined; According to the second parent path, a fourth index code is calculated.

5. The method of any one of claims 1-4, wherein, The path information of the directory metadata is included in the processing request; The metadata slice corresponding to the directory metadata indicated by the processing request is determined according to the first index slice, and the method comprises the following steps: According to the path information of the directory metadata indicated by the processing request, the metadata code corresponding to the directory metadata is determined in the first index slice; the metadata represents the code of the metadata slice; According to the metadata code, the metadata slice corresponding to the directory metadata indicated by the processing request is determined.

6. The method of claim 5, wherein, According to the path information of the directory metadata indicated by the processing request, the index information corresponding to the path information is determined in the index slice; The metadata code corresponding to the index information in the index slice is determined. When the processing request indicates to delete the directory metadata, the metadata slice comprises a first metadata slice corresponding to the directory metadata and a second metadata slice corresponding to a parent directory of the directory metadata; 7. The method of any one of claims 1-4, 6, wherein, The information in the determined metadata slice is adjusted, and the method comprises the following steps: The first metadata slice is deleted; The number of the directory metadata is subtracted from the information in the second metadata slice. When the processing request indicates to create the directory metadata, the metadata slice comprises a first metadata slice corresponding to the directory metadata and a second metadata slice corresponding to a parent directory of the directory metadata; 8. The method of any one of claims 1-4, 6, wherein, The information in the determined metadata slice is adjusted, and the method comprises the following steps: The first metadata slice is created; The number of the directory metadata is added to the information in the second metadata slice. When the processing request indicates to modify the directory of the directory metadata, the metadata slice comprises a third metadata slice corresponding to a parent directory indicated by a final path of the directory metadata and a fourth metadata slice corresponding to a parent directory indicated by an initial path of the directory metadata; 9. The method of any one of claims 1-4, 6, wherein, The information in the determined metadata slice is adjusted, and the method comprises the following steps: The number of the directory metadata is added to the information in the third metadata slice; The number of the directory metadata is subtracted from the information in the fourth metadata slice. When the processing request indicates to delete the directory metadata, the target path is included in the path information of the directory metadata; 10. The method of any one of claims 1-4, 6, wherein, ​ The adjusting the index information corresponding to the directory metadata in the second index shard indicated by the processing request comprises: determining a second index shard according to a target path of the directory metadata; deleting the index information corresponding to the target path of the directory metadata in the second index shard.

11. The method of any one of claims 1-4, 6, wherein, When the processing request indicates creating the directory metadata, the path information of the directory metadata comprises a target path; The adjusting the index information corresponding to the directory metadata in the second index shard indicated by the processing request comprises: determining a second index shard according to a target path of the directory metadata; creating the index information corresponding to the target path of the directory metadata in the index shard.

12. The method of any one of claims 1-4, 6, wherein, When the processing request indicates modifying a directory of the directory metadata, the path information of the directory metadata comprises a final path and an initial path; The adjusting the index information corresponding to the directory metadata in the second index shard indicated by the processing request comprises: if the index shard contains the directory metadata, determining the index shard as a second index shard, and deleting the index information containing the directory metadata in the second index shard; determining modified paths of the directory metadata and sub-directories thereof according to the final path and the initial path; creating index information of the modified paths in the corresponding second index shard according to the modified paths of the directory metadata and sub-directories thereof.

13. A processing device for directory metadata of a distributed file system, applied to a distributed file system, wherein a device node of the distributed file system is provided with a metadata unit containing a plurality of metadata shards and an index unit containing a plurality of index shards, and the device comprises: a determining unit configured to, in response to a processing request, determine and load a first index shard corresponding to directory metadata indicated by the processing request, and determine a metadata shard corresponding to the directory metadata indicated by the processing request according to the first index shard, wherein the processing request represents an adjustment to the directory metadata in the device node, the index shard is used to store index information of part of the directory metadata, and the metadata shard is determined based on a metadata encoding corresponding to first index information in the first index shard, wherein the first index information corresponds to path information of the directory metadata; an adjusting unit configured to adjust information in the determined metadata shard, and adjust index information corresponding to the directory metadata in a second index shard related to the directory metadata indicated by the processing request, so that data in the index unit and the metadata unit are consistent; wherein the processing request comprises path information of the directory metadata; The determining module is specifically configured to: calculate an index code according to the path information of the directory metadata, wherein the index code value represents an encoding of the index shard; load the first index shard corresponding to the directory metadata indicated by the processing request according to the index code; ​ The second index shard is determined according to an index code of a parent directory of the directory metadata, the index code of the parent directory is calculated according to a target parent path of the directory metadata, and the target parent path is determined according to path information of the directory metadata. The metadata shard includes a metadata shard corresponding to the directory metadata and a metadata shard corresponding to a parent directory of the directory metadata. 14.An electronic device, comprising: at least one processor; and a memory connected to the at least one processor in communication; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-12. 15.A distributed file system, comprising a plurality of device nodes, wherein a metadata unit and an index unit are arranged in the device nodes, the metadata unit includes a metadata shard corresponding to directory metadata in the device nodes, and the index unit includes a plurality of index shards, the index shards are used to store index information of part of the directory metadata; The device node is arranged in the electronic device of claim 14.

16. A non-transitory computer readable storage medium having stored thereon computer instructions, wherein, The computer instructions are used to enable the computer to perform the method according to any one of claims 1-12. 17.A computer program product, comprising a computer program, which, when executed by a processor, implements the steps of the method of any one of claims 1-12.

Images