Directory metadata processing method, device and equipment applied to distributed file system

Abstract

The invention provides a directory metadata processing method, device and equipment applied to a distributed file system, and relates to the technical field of computers, in particular to the field of big data and distributed file systems. According to the specific implementation scheme, when a processing request is obtained, a first index fragment corresponding to directory metadata indicated by the processing request can be determined according to the processing request, the first index fragment is loaded, and a metadata fragment corresponding to the directory metadata indicated by the processing request is determined according to the first index fragment. And adjusting the information in the determined metadata fragments. And determining a second index fragment related to the directory metadata indicated by the processing request according to the processing request, and adjusting index information corresponding to the directory metadata in the second index fragment.

Description

Technical Field

[0001] The present disclosure relates to big data in computer technology and a distributed file system, and in particular to a method, device and apparatus for processing directory metadata applied to a distributed file system. Background Art

[0002] As the amount of data increases, the application of distributed file systems is becoming more and more widespread. At present, the index unit of a device node in a distributed file system only includes one index shard. The index shard is used to write the index information of all directory metadata of the device node. When reading and writing data from the device node, the entire index shard needs to be loaded, which easily leads to reduced reading and writing efficiency.

[0003] Therefore, how to improve the read and write efficiency of the distributed file system becomes an urgent problem to be solved. Summary of the invention

[0004] The present disclosure provides a method, apparatus and device for processing directory metadata applied to a distributed file system.

[0005] According to a first aspect of the present disclosure, a method for processing directory metadata applied to a distributed file system is provided, wherein the distributed file system includes a plurality of device nodes, the device nodes correspond to metadata units and index units, the metadata units include metadata slices corresponding to the directory metadata in the device nodes, the index units include a plurality of index slices, and the index slices are used to store index information of part of the directory metadata, including:

[0006] In response to a processing request, determining and loading a first index slice corresponding to the directory metadata indicated by the processing request; and determining a metadata slice corresponding to the directory metadata indicated by the processing request based on the first index slice; wherein the processing request represents an adjustment to the directory metadata in the device node;

[0007] The information in the determined metadata slice is adjusted, and the index information corresponding to the directory metadata in the second index slice related to the directory metadata indicated by the processing request is adjusted.

[0008] According to a second aspect of the present disclosure, a processing device for directory metadata applied to a distributed file system is provided, wherein the distributed file system includes a plurality of device nodes, the device nodes correspond to metadata units and index units, the metadata units include metadata slices corresponding to the directory metadata in the device nodes, the index units include a plurality of index slices, and the index slices are used to store index information of part of the directory metadata, including:

[0009] A determination unit, configured to determine and load, in response to a processing request, a first index slice corresponding to the directory metadata indicated by the processing request; and determine, based on the first index slice, a metadata slice corresponding to the directory metadata indicated by the processing request; wherein the processing request represents an adjustment to the directory metadata in the device node;

[0010] An adjusting unit is used to adjust the information in the determined metadata slice, and to adjust the index information corresponding to the directory metadata in the second index slice related to the directory metadata indicated by the processing request.

[0011] According to a third aspect of the present disclosure, there is provided an electronic device, including:

[0012] At least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the method described in the first aspect.

[0013] According to a fourth aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to cause the computer to execute the method described in the first aspect.

[0014] According to a fifth aspect of the present disclosure, a computer program product is provided, comprising: a computer program, wherein the computer program is stored in a readable storage medium, at least one processor of an electronic device can read the computer program from the readable storage medium, and the at least one processor executes the computer program so that the electronic device executes the method described in the first aspect.

[0015] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] The accompanying drawings are used to better understand the present solution and do not constitute a limitation of the present disclosure.

[0017] Figure 1 is a schematic diagram according to a first embodiment of the present disclosure;

[0018] Figure 2 is a schematic diagram according to a second embodiment of the present disclosure;

[0019] Figure 3 is a schematic diagram according to a third embodiment of the present disclosure;

[0020] Figure 4 is a schematic diagram according to a fourth embodiment of the present disclosure;

[0021] Figure 5 is a schematic diagram according to a fifth embodiment of the present disclosure;

[0022] Figure 6 is a schematic diagram according to a sixth embodiment of the present disclosure;

[0023] Figure 7 is a schematic diagram according to a seventh embodiment of the present disclosure;

[0024] Figure 8 is a schematic block diagram of an electronic device for implementing an embodiment of the present disclosure;

[0025] Fig. 9 It is a schematic diagram of the structure of a distributed file system that can implement the embodiment of the present disclosure. DETAILED DESCRIPTION

[0026] The following is a description of exemplary embodiments of the present disclosure in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0027] With the increase in data volume, the application of distributed file systems is becoming more and more widespread. The device nodes of current distributed file systems usually include two parts: metadata unit (Dentry cell) and index unit (Dtree cell). Among them, the index unit includes a storage area for writing directories. However, due to the limitation of the number of directories stored in this storage area, the full memory can generally only support up to 1 billion directories, and cannot reach 100 billion files. In addition, due to the limitation of storing directories in this single storage area, the query rate per second (QPS) of path resolution that it can provide can only support up to the million level, which cannot meet the demand of the 100 billion level.

[0028] In order to further improve the storage capacity and QPS of the distributed file system, the present disclosure proposes a new architecture of the index unit to solve the problems of poor scalability and low QPS of the distributed file system that currently exists in the single point of Dtree. Specifically, the index unit is optimized in the present disclosure, and N index shards are created in advance when creating a file system. When creating a directory (mkdir), deleting a directory (rmdir), and other operations, they will be routed to a fixed shard in the manner of the full path Hash, and then the corresponding index information will be created or deleted in the corresponding index shard. For modifying the path (mv), all index shards need to be searched and processed. The method of performing path resolution by routing to a specific index shard according to the full path Hash can effectively improve QPS. And the sharded path storage method can effectively increase the storage capacity of files.

[0029] For example, take the query of directory c as an example. The directory metadata included in the query request is directory c, and the target path of the directory metadata is / a / b / c. First, the index shard can be determined by the full path Hash method based on the full path / a / b / c. Then, the metadata code of directory c can be queried through this index shard through a remote procedure call (RPC). Finally, another RPC call is made to the metadata shard corresponding to the metadata code to realize the query of the attribute information of directory c.

[0030] The present invention discloses a method, apparatus and device for processing directory metadata applied to a distributed file system, which are applied to big data and distributed file systems in the field of computer technology to improve the reading and writing efficiency of the distributed file system.

[0031] It should be noted that the data in this embodiment is not targeted at a specific user and cannot reflect the personal information of a specific user. It should be noted that the data in this embodiment comes from a public data set.

[0032] In the technical solution of the present disclosure, the collection, storage, use, processing, transmission, provision and disclosure of user personal information involved are in compliance with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0033] In order to enable readers to more deeply understand the implementation principle of the present disclosure, the following Figure 2-Figure 5 right Figure 1 The illustrated embodiment is further refined.

[0034] Figure 1 is a schematic diagram according to the first embodiment of the present disclosure, such as Figure 1As shown, the present disclosure provides a method for processing directory metadata applied to a distributed file system. The distributed file system used in the present disclosure includes multiple device nodes. Each device node corresponds to a metadata unit and an index unit. The metadata unit includes metadata slices corresponding to the directory metadata in the device node. The index unit includes multiple index slices. The index slices are used to store index information of part of the directory metadata. The method includes:

[0035] 110. In response to a processing request, determine and load a first index slice corresponding to the directory metadata indicated by the processing request; and determine a metadata slice corresponding to the directory metadata indicated by the processing request based on the first index slice. The processing request represents adjusting the directory metadata in the device node.

[0036] In this embodiment, a processing request of a user can be obtained. The processing request is used to adjust the directory metadata in the device node. Optionally, the adjustment of the directory metadata can create the directory metadata, delete the directory metadata, and modify the path of the directory metadata. Optionally, the processing request can usually be an operation code. Optionally, the processing request needs to include at least the directory metadata and path information to be processed. Optionally, when the processing request is used to indicate the creation of the directory metadata or the deletion of the directory metadata, the path information can at least include the target path. The target path is the path of the directory metadata to be deleted. Or, the target path is the path of the directory metadata to be created. Optionally, when the processing request is used to indicate the modification of the path of the directory metadata, the path information can at least include an initial path and a final path. The initial path is the path of the directory metadata to be moved. The final path is the path to which the directory metadata needs to reach after the move.

[0037] According to the processing request, the first index slice corresponding to the directory metadata indicated by the processing request can be determined. Further, the first index slice can be loaded, and in the first index slice, according to the path information included in the processing request, the index information corresponding to the path information can be queried in the first index slice. The index information can indicate the corresponding metadata slice, thereby determining the metadata slice corresponding to the directory metadata indicated by the processing request.

[0038] 120. Adjust information in the determined metadata slice, and adjust index information corresponding to the directory metadata in a second index slice related to the directory metadata indicated by the processing request.

[0039] In this embodiment, the metadata slice can be determined according to step 110, and the metadata slice can be adjusted to achieve the creation and deletion of the metadata slice, as well as the modification of the information in the metadata slice. Optionally, the information in the metadata slice can be the attribute information of the directory metadata corresponding to the metadata slice. In addition, according to the processing request, one or more second index slices related to the directory metadata of the processing request can also be determined in the index unit. After determining the second index slice, the index information in the second index slice can be modified to ensure the consistency of the data in the index unit and the metadata unit. Optionally, in the process of completing the modification of the metadata slice and the second index slice, the processing request is executed successfully only when the modification of all the full data slices and the second index slices corresponding to the processing request is successful.

[0040] In this embodiment, by determining and loading the first index shard corresponding to the directory metadata indicated by the processing request, the metadata shard can be determined, thereby realizing the modification of the metadata shard. This process can realize partial loading of data in the index unit by loading the first index shard, reduce the data to be loaded, and improve loading efficiency. In addition, by modifying the index information in the second index shard related to the directory metadata indicated by the processing request, the corresponding modification of the index shard is realized, the modification efficiency of the data in the index unit is improved, and the consistency of the data in the index unit and the metadata unit is ensured.

[0041] Figure 2 is a schematic diagram according to the second embodiment of the present disclosure, such as Figure 2 As shown, the present disclosure provides a method for processing directory metadata applied to a distributed file system, the method comprising:

[0042] 210. Calculate the index code according to the path information of the directory metadata. The index code value represents the code of the index fragment.

[0043] In this embodiment, the path information of the directory metadata can be first obtained from the processing request. Based on the path information, the index code can be calculated. The index code is the code of the index fragment. The index code corresponds to the index fragment one by one.

[0044] Optionally, for different processing requests, the information included in the path information is different. Therefore, for different processing requests, the calculation process of the index code is different.

[0045] Optionally, when the processing request indicates to delete the directory metadata, or to create the directory metadata, the path information of the directory metadata includes the target path. At this time, it is necessary to calculate the index code of the directory metadata and the index code of the parent directory of the directory metadata. The calculation process may include:

[0046] 2111. Calculate and obtain a first index code according to the target path.

[0047] In this embodiment, the first index code of the directory metadata can be calculated based on the full path information of the target path. The index slice indicated by the first index code corresponds to the directory path. That is, when the processing request indicates to delete the directory metadata, the index slice indicated by the first index code includes the target path. When the processing request indicates to create the directory metadata, the index slice indicated by the first index code is the index slice stored in the target path.

[0048] 2112. Determine a target parent path of a parent directory of target metadata in the target path according to the target path of the metadata.

[0049] In this embodiment, the parent directory of the target metadata can be determined according to the target path of the metadata. The parent directory is the path that remains after deleting the directory corresponding to the target metadata in the target path. The path is the target parent path. For example, when the target path is / a / b / c, the target parent path is / a / b.

[0050] 2113. According to the parent path, calculate and obtain the second index code.

[0051] In this embodiment, the second index code of the directory metadata can be calculated based on the full path information of the target parent path. The index fragment indicated by the second index code and the target parent path. That is, the index fragment indicated by the second index code contains the target parent path.

[0052] Through the above steps 2111 to 2113, the calculation of the index code is realized when processing a request indicating the deletion of directory metadata, or creating directory metadata. At this time, it can include a first index code for indicating the index slice where the path of the directory metadata is located, and a second index code for the index slice where the path of the parent directory of the directory metadata indicated by the user is located. The calculation of the two index codes can realize the confirmation of the index slice that needs to be loaded, thereby avoiding the need to load the entire index unit, reducing the amount of data loaded, and improving the efficiency of data loading.

[0053] Optionally, when the processing request indicates to modify the path of the directory metadata, the path information of the directory metadata includes an initial path and a final path. The initial path indicates the path of the directory metadata when it is not modified, and the final path indicates the path of the directory metadata after it is modified. At this time, it is necessary to determine the index code of the parent directory where the directory metadata is located when it is not modified based on the initial path. And, based on the final path, determine the index code of the parent directory where the directory metadata is located after the modification is completed. The calculation process may include:

[0054] 2121. According to the final path of the metadata, determine the first parent path of the parent directory of the directory metadata in the final path.

[0055] 2122. Calculate and obtain a third index code according to the first parent path.

[0056] 2123. According to the initial path of the metadata, determine the second parent path of the parent directory of the directory metadata in the initial path.

[0057] 2124. Calculate and obtain a fourth index code according to the second parent path.

[0058] The implementation of the above steps 2121 and 2123 is similar to the above step 2112, and the implementation of steps 2122 and 2124 is similar to the above step 2113, which will not be repeated here. Through the execution of the above steps 2121 to 2124, when the processing request indicates to modify the path of the directory metadata, the path of the parent directory of the directory metadata on the final path and the initial path is obtained, and the index codes corresponding to the two parent directories are calculated. The calculation of the two index codes can realize the confirmation of the index fragments that need to be loaded, thereby avoiding the need to load the entire index unit, reducing the amount of data loaded, and improving the efficiency of data loading.

[0059] Optionally, taking the target path as an example, the process of specifically calculating and obtaining the first index code according to the target path may include the following steps:

[0060] 2131. Calculate the MD5 value of the target path.

[0061] In this embodiment, an MD5 value may be calculated according to the target path. The calculation of the MD5 value may be completed by a preset information digest algorithm.

[0062] 2132. Determine the index code according to the MD5 value and the number of index fragments preset in the index unit.

[0063] In this embodiment, after determining the MD5 value, the modulus can be calculated according to the number of index shards preset in the index unit to obtain the final index code. Optionally, the number of index shards in the index unit can be preset when the device node of the distributed file system is created. The modulus operation can realize the allocation of the target path to the index shard. This allocation method can improve the balancing effect of data and avoid the index information being concentratedly written into several index shards.

[0064] Optionally, the first index code and the second index code may be the same. Also, the third index code and the fourth index code may be the same.

[0065] 220. According to the index code, load the first index fragment corresponding to the directory metadata indicated by the processing request.

[0066] In this embodiment, after at least one index code is determined according to step 210, the loading of the first index fragment can be completed according to the index code.

[0067] In this embodiment, by determining and loading at least one first index shard corresponding to the directory metadata indicated by the processing request, the loading of part of the data in the index unit is realized, avoiding the need to load the entire index unit, reducing the amount of data loaded, and improving the efficiency of loading the index unit in response to the processing request. In turn, it helps to improve the path resolution efficiency of the distributed file system and increase the QPS upper limit.

[0068] Figure 3 is a schematic diagram according to a third embodiment of the present disclosure, Figure 3 As shown, the present disclosure provides a method for processing directory metadata applied to a distributed file system, the method comprising:

[0069] 310. According to the path information of the directory metadata indicated by the processing request, determine the metadata encoding corresponding to the directory metadata in the index slice. The metadata represents the encoding of the metadata slice.

[0070] In this embodiment, the processing request includes the path information of the directory metadata. According to the path information, the metadata code corresponding to the path information can be determined in the index slice. The metadata code is the code of the metadata slice. The metadata code corresponds to the metadata slice one by one.

[0071] Optionally, the specific process of determining the metadata encoding corresponding to the path information in the index shard may include:

[0072] 311. According to the path information of the directory metadata indicated by the processing request, determine the index information corresponding to the path information in the index shard.

[0073] In this embodiment, the path information corresponding to the directory metadata can be determined according to the second embodiment. According to the path information, the index information can be queried in the directory metadata. According to the index information.

[0074] 312. Determine the metadata encoding corresponding to the index information in the index shard.

[0075] In this embodiment, the index slice may store a correspondence between the index information and the metadata code. According to the correspondence, after the index information is determined, the metadata code corresponding to the index information may be determined.

[0076] Through the above steps 311 and 312, the query can be completed in the index shard according to the path information to obtain the index information and the data encoding corresponding to the index information, thereby realizing the determination of the metadata shard, improving the efficiency of determining the metadata shard, and improving the resolution efficiency of the path.

[0077] Optionally, when the processing request indicates to delete the directory metadata, or to create the directory metadata, the index code calculated in the second embodiment may include a first index code and a second index code. The first index code is the index code corresponding to the target path of the directory metadata indicated by the processing request. A first index fragment can be determined based on the first index code. In the first index fragment, a search can be performed based on the target path to obtain the index information of the target path and the metadata code corresponding to the index information. The metadata code can point to the metadata fragment of the directory metadata. The second index code is the index code corresponding to the target parent path of the parent directory of the directory metadata indicated by the processing request. A second first index fragment can be determined based on the second index code. In the first index fragment, a search can be performed based on the target parent path to obtain the index information of the target parent path and the metadata code corresponding to the index information. The metadata code can point to the metadata fragment of the parent directory of the directory metadata.

[0078] Optionally, when the processing request indicates to modify the path of the directory metadata, the index code calculated in the second embodiment may include a third index code and a fourth index code. The third index code is the index code corresponding to the first parent path of the parent directory of the directory metadata in the final path. A first index fragment can be determined based on the third index code. In the first index fragment, a search can be performed based on the first parent path to obtain the index information of the first parent path and the metadata code corresponding to the index information. The metadata code can point to the metadata fragment of the parent directory of the directory metadata in the final path. The fourth index code is the index code corresponding to the second parent path of the parent directory of the directory metadata in the initial path. A second first index fragment can be determined based on the fourth index code. In the first index fragment, a search can be performed based on the second parent path to obtain the index information of the second parent path and the metadata code corresponding to the index information. The metadata code can point to the metadata fragment of the second parent path of the parent directory of the directory metadata in the initial path.

[0079] 320. Determine, according to the metadata encoding, the metadata fragment corresponding to the directory metadata indicated by the processing request.

[0080] In this embodiment, after at least one metadata code is determined according to step 310, the corresponding metadata fragment can be obtained in the metadata unit according to the metadata code, thereby realizing the reading and writing of the metadata fragment.

[0081] In this embodiment, the metadata code is determined by querying the index information in the first index shard according to the path information, and the reading and writing of the metadata shard is implemented according to the metadata code, thereby improving the efficiency of determining the metadata shard and improving the efficiency of parsing the path in the distributed file system.

[0082] Figure 4 is a schematic diagram according to a fourth embodiment of the present disclosure, Figure 4 As shown, the present disclosure provides a method for processing directory metadata applied to a distributed file system. After determining the metadata shard, the method can adjust the information in the metadata shard. According to three different processing requests of deletion, creation and modification, the adjustment process of the metadata shard may include:

[0083] 410. When the processing request indicates to delete the directory metadata, delete the metadata slice corresponding to the directory metadata, and modify the information in the metadata slice corresponding to the parent directory of the directory metadata.

[0084] In this embodiment, when the processing request indicates to delete the directory metadata, according to the second embodiment and the third embodiment, two metadata slices can be obtained. Among them, one metadata slice is the metadata slice corresponding to the directory metadata, and the other metadata slice can be the metadata slice corresponding to the parent directory of the directory metadata. Since the directory metadata needs to be deleted in response to the processing request. When the directory metadata is deleted, the metadata allocation corresponding to the directory metadata will also lose its meaning. Therefore, the metadata slice corresponding to the directory metadata can be deleted accordingly. When the directory metadata is deleted, for the parent directory of the directory metadata, the subdirectory associated with it changes. Therefore, in order to ensure that the attribute information of the parent directory of the directory metadata matches the actual situation, it is necessary to update the information in the metadata slice of the parent directory of the directory metadata.

[0085] Optionally, the above processing of metadata fragments may include the following steps:

[0086] 411. Delete the first metadata fragment.

[0087] In this embodiment, the metadata fragment corresponding to the directory metadata is recorded as the first metadata fragment. The first metadata fragment can be deleted in the metadata unit.

[0088] 412. Subtract the amount of directory metadata from the information in the second metadata slice.

[0089] In this embodiment, the metadata slice corresponding to the parent directory of the directory metadata is recorded as the second metadata slice. The number of the directory metadata can be subtracted from the information of the second metadata slice. For example, the information of the second metadata slice can include the number of subdirectories under the parent directory. When the directory metadata is deleted, the number of subdirectories will be reduced by one.

[0090] Step 410 can be used to process the metadata fragments when the processing request is used to delete directory metadata, thereby ensuring the execution of the processing request, improving the response efficiency of the processing request, and improving the user experience.

[0091] 420. When the processing request indicates to create directory metadata, create a metadata slice corresponding to the directory metadata, and modify information in the metadata slice corresponding to the parent directory of the directory metadata.

[0092] In this embodiment, when the processing request indicates the creation of the directory metadata, according to the second embodiment and the third embodiment, two metadata slices can be obtained. Among them, one metadata slice is the metadata slice corresponding to the directory metadata, and the other metadata slice can be the metadata slice corresponding to the parent directory of the directory metadata. Since the directory metadata needs to be created in response to the processing request. Therefore, it is necessary to create the metadata slice corresponding to the directory metadata in the metadata unit. When the directory metadata is created, for the parent directory of the directory metadata, the subdirectory associated with it changes. Therefore, in order to ensure that the attribute information of the parent directory of the directory metadata matches the actual situation, it is necessary to update the information in the metadata slice of the parent directory of the directory metadata.

[0093] Optionally, the above processing of metadata fragments may include the following steps:

[0094] 421. Create a first metadata shard.

[0095] In this embodiment, the metadata slice corresponding to the directory metadata is recorded as the first metadata slice. The first metadata slice can be created in the metadata unit.

[0096] 422. Increase the amount of directory metadata in the information in the second metadata slice.

[0097] In this embodiment, the metadata slice corresponding to the parent directory of the directory metadata is recorded as the second metadata slice. The number of the directory metadata can be increased in the information of the second metadata slice. For example, the information of the second metadata slice can include the number of subdirectories under the parent directory. When the directory metadata is created, the number of subdirectories will be increased by one.

[0098] Step 420 can be used to process the metadata fragments when the processing request is used to create directory metadata, thereby ensuring the execution of the processing request, improving the response efficiency of the processing request, and improving the user experience.

[0099] 430. When the processing request indicates modifying the directory of the directory metadata, modify the metadata slice corresponding to the parent directory of the directory metadata indicated by the final path and the metadata slice corresponding to the parent directory of the directory metadata indicated by the initial path.

[0100] In this embodiment, when the processing request indicates to modify the directory of the directory metadata, according to the second embodiment and the third embodiment, two metadata slices can be obtained. Among them, one metadata slice is the metadata slice corresponding to the parent directory of the directory metadata indicated by the final path, and the other metadata slice can be the metadata slice corresponding to the parent directory of the directory metadata indicated by the initial path. The directory metadata responds to the processing request and changes the directory in which it is located, which is equivalent to changing the parent directory of the directory metadata. That is, it is deleted from the parent directory corresponding to the initial path and added to the parent directory corresponding to the final path. For the directory metadata itself, its subdirectory has not changed. Therefore, the metadata slice of the directory metadata itself has not changed.

[0101] Optionally, the above processing of metadata fragments may include the following steps:

[0102] 431. Increase the amount of directory metadata in the information in the third metadata slice.

[0103] In this embodiment, the metadata slice corresponding to the parent directory of the directory metadata indicated by the final path is recorded as the third metadata slice. The number of the directory metadata can be increased in the information of the third metadata slice. For example, the information of the third metadata slice can include the number of subdirectories under the parent directory. When the directory metadata is moved to the parent directory indicated by the final path, the number of subdirectories will increase by one.

[0104] 432. Subtract the amount of directory metadata from the information in the second metadata slice.

[0105] In this embodiment, the metadata slice corresponding to the parent directory of the directory metadata indicated by the initial path is recorded as the fourth metadata slice. The number of the directory metadata can be subtracted from the information of the fourth metadata slice. For example, the information of the fourth metadata slice may include the number of subdirectories under the parent directory. When the directory metadata is removed from the parent directory corresponding to the initial path, the number of subdirectories will be reduced by one.

[0106] Step 430 can be used to implement the processing of the metadata slice when the processing request is used to modify the directory metadata, thereby ensuring the execution of the processing request, improving the response efficiency of the processing request, and improving the user experience.

[0107] In this embodiment, by modifying the metadata slice, the response to the processing request in the metadata unit is realized, the response efficiency of the processing request is improved, and the user experience is improved.

[0108] Figure 5 is a schematic diagram according to a fifth embodiment of the present disclosure, Figure 5 As shown, the present disclosure provides a method for processing directory metadata applied to a distributed file system. The method can process the second index shard by determining the second index shard related to the directory metadata indicated by the processing request in the index shard. According to three different processing requests of deletion, creation and modification, the determination and processing adjustment process of the second index shard may include:

[0109] 510. When the processing request indicates deletion of the directory metadata, delete the index information corresponding to the target path in the index slice corresponding to the target path of the directory metadata.

[0110] In this embodiment, when the processing request indicates to delete the directory metadata, the path information of the processing request may include a target path. The target path is the path of the directory metadata to be deleted. According to the target path, the index code corresponding to the target path can be determined according to the method for determining the index fragments shown in the second embodiment, thereby determining the second index fragment. When the directory metadata is deleted, the index information of the directory metadata will lose its function. Therefore, the index information corresponding to the target path can be deleted in the index fragment.

[0111] Optionally, the above determination and processing of index fragments may specifically include the following steps:

[0112] 511. Determine a second index fragment according to a target path of the directory metadata.

[0113] In this embodiment, the MD5 value of the directory metadata can be determined according to the target path, and then the index code can be determined according to the modulus of the number of index fragments preset in the MD5 value domain, and the corresponding second index fragment can be determined according to the index code.

[0114] 512. Delete the index information corresponding to the target path of the directory metadata in the second index slice.

[0115] In this embodiment, the target path can be queried in the second index shard to obtain the index information corresponding to the target path. The index information is deleted in the second index shard. Optionally, deleting the index information includes deleting the metadata encoding stored in the second index shard corresponding to the index information. Optionally, deleting the index information includes deleting all information stored in the second index shard corresponding to the index information.

[0116] Step 510 can be used to process index fragments in the index unit when the processing request is used to delete directory metadata, thereby ensuring the execution of the processing request, improving the response efficiency of the processing request, and improving the user experience.

[0117] 520. When the processing request indicates the creation of directory metadata, create index information corresponding to the target path in the index slice corresponding to the target path of the directory metadata.

[0118] In this embodiment, when the processing request indicates the creation of directory metadata, the path information of the processing request may include a target path. The target path is the path of the directory metadata to be created. According to the target path, the index code corresponding to the target path can be determined according to the method for determining the index fragments shown in the second embodiment, thereby determining the second index fragment. When the directory metadata is created, it is necessary to synchronously create the index information of the target path of the directory metadata in the index fragment, thereby ensuring that the newly created directory metadata has the corresponding index information.

[0119] Optionally, the above determination and processing of index fragments may specifically include the following steps:

[0120] 521. Determine a second index fragment according to a target path of the directory metadata.

[0121] In this embodiment, the MD5 value of the directory metadata can be determined according to the target path, and then the index code can be determined according to the modulus of the number of index fragments preset in the MD5 value domain, and the corresponding second index fragment can be determined according to the index code.

[0122] 522. Create index information corresponding to the target path of the directory metadata in the index shard.

[0123] In this embodiment, the index information of the directory path can be created in the second index slice. Optionally, after the creation of the index information is completed, the metadata code of the metadata slice of the directory metadata that has been created can also be obtained from the metadata slice. The metadata code is stored in the fourth index slice, and the corresponding relationship between the metadata code and the index information is stored.

[0124] Step 520 can be used to process index fragments in the index unit when the processing request is used to create directory metadata, thereby ensuring the execution of the processing request, improving the response efficiency of the processing request, and improving the user experience.

[0125] 530. When the processing request indicates to modify the directory of the directory metadata, the index information containing the directory metadata is deleted from each index shard; and according to the modified path of the directory metadata and its subdirectories, the index information of the path is created in the index shard corresponding to the path.

[0126] In this embodiment, when the processing request indicates to modify the directory of the directory metadata, the path information of the processing request may include a final path and an initial path. The final path is the path after the directory metadata is modified, and the initial path is the path before the directory metadata is modified.

[0127] When the directory metadata is moved out of the initial path, all subdirectories of the directory metadata will be moved out together. All existing index information containing the directory metadata in the index unit will be invalid. Therefore, the index information containing the directory metadata can be deleted from each index shard.

[0128] After the directory metadata is moved to the final path, the path of the subdirectory in the directory metadata will be modified accordingly. The directory metadata and the index information of the subdirectory of the directory metadata need to be recreated. Therefore, the index information of the path can be created in the index shard corresponding to the path according to the modified path of the directory metadata and its subdirectory.

[0129] Optionally, the above determination and processing of index fragments may specifically include the following steps:

[0130] 531. If the index fragment contains directory metadata, determine that the index fragment is a second index fragment, and delete the index information containing the directory metadata in the second index fragment.

[0131] In this embodiment, the index information in each index shard may be traversed to determine whether the index information contains the directory metadata. When the index shard contains the directory metadata, the index shard may be determined to be the second index shard. When the index information contains the directory metadata, the index information may be deleted.

[0132] 532. Determine a modified path of the directory metadata and its subdirectories based on the final path and the initial path.

[0133] In this embodiment, the subdirectory path of the subdirectory of the directory metadata before modification can be obtained. The final path is used to replace the directory metadata and the part before it in the subdirectory path to obtain the modified path of the subdirectory. The modified path of the directory metadata is the final path.

[0134] 533. Create index information of the modified path in the corresponding second index fragment according to the modified path of the directory metadata and its subdirectories.

[0135] In this embodiment, the corresponding index slices can be determined one by one in the index unit according to the modified path obtained in step 532, and the index information of the modified path can be created in the index slices. Optionally, after the creation of the index information of the modified path is completed, the metadata of the directory and its subdirectories can also be encoded and written into the corresponding position of the index information of the modified path.

[0136] Step 530 can be used to process index fragments in the index unit when the processing request is used to delete directory metadata, thereby ensuring the execution of the processing request, improving the response efficiency of the processing request, and improving the user experience.

[0137] In this embodiment, the response to the processing request in the index unit is achieved by modifying the metadata slice, thereby improving the response efficiency of the processing request and improving the user experience.

[0138] Figure 6 is a schematic diagram according to a sixth embodiment of the present disclosure, Figure 6 As shown, the present disclosure provides a processing device 600 for directory metadata applied to a distributed file system, wherein the distributed file system includes multiple device nodes, and the device nodes correspond to metadata units and index units. The metadata unit includes metadata slices corresponding to the directory metadata in the device node, and the index unit includes multiple index slices, and the index slices are used to store index information of part of the directory metadata. It includes:

[0139] The determination unit 610 is used to determine and load the first index slice corresponding to the directory metadata indicated by the processing request in response to the processing request, and determine the metadata slice corresponding to the directory metadata indicated by the processing request according to the first index slice. The processing request represents the adjustment of the directory metadata in the device node.

[0140] The adjustment unit 620 is used to adjust the information in the determined metadata slice, and adjust the index information corresponding to the directory metadata in the second index slice related to the directory metadata indicated by the processing request.

[0141] The device of this embodiment can execute the technical solution in the above method. Its specific implementation process and technical principles are the same and will not be repeated here.

[0142] Figure 7 is a schematic diagram according to the seventh embodiment of the present disclosure, Figure 7 As shown, the present disclosure provides a processing device 700 for directory metadata applied to a distributed file system, wherein the distributed file system includes multiple device nodes, and the device nodes correspond to metadata units and index units, and the metadata units include metadata slices corresponding to the directory metadata in the device nodes, and the index units include multiple index slices, and the index slices are used to store index information of part of the directory metadata. The device includes:

[0143] The determination unit 710 is used to determine and load the first index slice corresponding to the directory metadata indicated by the processing request in response to the processing request, and determine the metadata slice corresponding to the directory metadata indicated by the processing request according to the first index slice. The processing request represents the adjustment of the directory metadata in the device node.

[0144] The adjustment unit 720 is used to adjust the information in the determined metadata slice, and adjust the index information corresponding to the directory metadata in the second index slice related to the directory metadata indicated by the processing request.

[0145] Optionally, the processing request includes path information of the directory metadata. The determining unit 710 includes:

[0146] The first calculation module 711 is used to calculate the index code according to the path information of the directory metadata. The index code value represents the code of the index fragment.

[0147] The first determination module 712 is used to determine and load the first index fragment corresponding to the directory metadata indicated by the processing request according to the index code.

[0148] Optionally, when the processing request indicates to delete the directory metadata, or to create the directory metadata, the path information of the directory metadata includes the target path. The calculation module 711 includes:

[0149] The first calculation submodule 7111 is used to calculate and obtain a first index code according to the target path.

[0150] The first determining submodule 7112 is used to determine the target parent path of the parent directory of the metadata in the target path according to the target path of the metadata.

[0151] The second calculation submodule 7113 is used to calculate and obtain a second index code according to the parent path.

[0152] Optionally, the first calculation submodule 7111 is specifically configured to:

[0153] Calculate the MD5 value of the target path.

[0154] The index code is determined according to the MD5 value and the number of index fragments preset in the index unit.

[0155] Optionally, when the processing request indicates to modify the path of the directory metadata, the path information of the directory metadata includes the initial path and the target path. The calculation module 711 includes:

[0156] The second determining submodule 7114 is used to determine the first parent path of the parent directory of the metadata in the target path according to the target path of the metadata.

[0157] The third calculation submodule 7115 is used to calculate and obtain a third index code according to the first parent path.

[0158] The third determining submodule 7116 is used to determine, according to the initial path of the metadata, a second parent path of the parent directory of the metadata in the initial path.

[0159] The fourth index calculation submodule 7117 is used to calculate the fourth index code according to the second parent path.

[0160] Optionally, the processing request includes path information of the directory metadata. The determining unit 710 includes:

[0161] The second determination module 713 is used to determine the metadata code corresponding to the directory metadata in the index slice according to the path information of the directory metadata indicated by the processing request. The metadata represents the code of the metadata slice.

[0162] The third determination module 714 is used to determine the metadata fragment corresponding to the directory metadata indicated by the processing request according to the metadata encoding.

[0163] Optionally, the second determining module 713 includes:

[0164] The first determination submodule 7131 is used to determine the index information corresponding to the path information in the index slice according to the path information of the directory metadata indicated by the processing request.

[0165] The second determination submodule 7132 is used to determine the metadata code corresponding to the index information in the index fragment.

[0166] Optionally, when the processing request indicates to delete the directory metadata, the metadata slice includes a first metadata slice corresponding to the directory metadata and a second metadata slice corresponding to the parent directory of the directory metadata. The adjustment unit 720 includes:

[0167] The first deletion module 7211 is used to delete the first metadata segment.

[0168] The first adjustment module 7212 is used to subtract the amount of directory metadata from the information in the second metadata slice.

[0169] Optionally, when the processing request indicates to create directory metadata, the metadata slice includes a first metadata slice corresponding to the directory metadata and a second metadata slice corresponding to the parent directory of the directory metadata. The adjustment unit 720 includes:

[0170] The first creation unit 7213 is used to create a first metadata segment.

[0171] The second adjustment module 7214 is used to increase the amount of directory metadata in the information in the second metadata slice.

[0172] Optionally, when the processing request indicates to modify the directory of the directory metadata, the metadata slices include a third metadata slice corresponding to the parent directory indicated by the final path of the directory metadata, and a fourth metadata slice corresponding to the parent directory indicated by the initial path of the directory metadata. The adjustment unit 720 includes:

[0173] The third adjustment module 7215 is used to increase the amount of directory metadata in the information in the third metadata slice.

[0174] The fourth adjustment module 7216 is used to subtract the amount of directory metadata from the information in the second metadata slice.

[0175] Optionally, when the processing request indicates to delete the directory metadata, the path information of the directory metadata includes the target path. The adjustment unit 720 includes:

[0176] The first determination module 7221 is used to determine the second index fragment according to the target path of the directory metadata.

[0177] The first deletion module 7222 is used to delete the index information corresponding to the target path of the directory metadata in the second index slice.

[0178] Optionally, when the processing request indicates to create directory metadata, the path information of the directory metadata includes the target path. The adjustment unit 720 includes:

[0179] The second determination module 7223 is used to determine the second index fragment according to the target path of the directory metadata.

[0180] The first creation module 7224 is used to create index information corresponding to the target path of the directory metadata in the index slice.

[0181] Optionally, when the processing request indicates to modify the directory of the directory metadata, the path information of the directory metadata includes the final path and the initial path. The adjustment unit 720 includes:

[0182] The second deleting module 7225 is used to determine that the index fragment is a second index fragment if the index fragment contains directory metadata, and delete the index information corresponding to the directory metadata in the second index fragment.

[0183] The third determining module 7226 is used to determine the subdirectory path of the subdirectory of the directory metadata according to the final path and the initial path.

[0184] The second creation module 7227 is used to determine the second index slice corresponding to the sub-directory path according to the sub-directory path of the directory metadata, and create index information of the sub-directory path in each index slice.

[0185] The device of this embodiment can execute the technical solution in the above method. Its specific implementation process and technical principles are the same and will not be repeated here.

[0186] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium and a computer program product.

[0187] According to an embodiment of the present disclosure, the present disclosure also provides a computer program product, which includes: a computer program, the computer program is stored in a readable storage medium, at least one processor of an electronic device can read the computer program from the readable storage medium, and at least one processor executes the computer program so that the electronic device executes the solution provided by any of the above embodiments.

[0188] Figure 8 A schematic block diagram of an example electronic device 800 that can be used to implement an embodiment of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or required herein.

[0189] like Figure 8As shown, the device 800 includes a computing unit 801, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 802 or a computer program loaded from a storage unit 808 to a random access memory (RAM) 803. In the RAM 803, various programs and data required for the operation of the device 800 can also be stored. The computing unit 801, the ROM 802, and the RAM 803 are connected to each other via a bus 804. An input / output (I / O) interface 805 is also connected to the bus 804.

[0190] A number of components in the device 800 are connected to the I / O interface 805, including: an input unit 806, such as a keyboard, a mouse, etc.; an output unit 807, such as various types of displays, speakers, etc.; a storage unit 808, such as a disk, an optical disk, etc.; and a communication unit 809, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 809 allows the device 800 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0191] The computing unit 801 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 801 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 801 performs the various methods and processes described above, such as a processing method for directory metadata applied to a distributed file system. For example, in some embodiments, the processing method for directory metadata applied to a distributed file system may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit 808. In some embodiments, part or all of the computer program may be loaded and / or installed on the device 800 via the ROM 802 and / or the communication unit 809. When the computer program is loaded into the RAM 803 and executed by the computing unit 801, one or more steps of the processing method for directory metadata applied to a distributed file system described above may be performed. Alternatively, in other embodiments, the computing unit 801 may be configured in any other appropriate manner (eg, by means of firmware) to execute the processing method for directory metadata applied to a distributed file system.

[0192] Fig. 9 1 is a schematic diagram of a distributed file system that can implement the embodiment of the present disclosure. The distributed file system may include multiple device nodes. Each device node may be set in an electronic device. Fig. 9As shown, a device node of the distributed file system may include a metadata unit and an index unit. The metadata unit may include multiple metadata slices. Each metadata slice may correspond to a directory metadata. The index unit may include multiple index slices. The number of index slices may be set when the device node is created. Each index slice may store index information of part of the directory metadata. In addition, the views of the directory metadata and file metadata in the device node may be as follows: Fig. 9 The tree structure in FIG. 1 shows a tree structure of all directory metadata and file metadata on the device node. The square box represents directory metadata and the circle represents file metadata. The view can include a tree structure of all directory metadata and file metadata on the device node.

[0193] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0194] The program code for implementing the method of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0195] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or device, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0196] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0197] The systems and techniques described herein may be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), and the Internet.

[0198] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship between the client and the server is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services ("Virtual Private Server", or "VPS" for short). The server may also be a server of a distributed system, or a server combined with a blockchain.

[0199] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this disclosure can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved, and this document does not limit this.

[0200] The above specific implementations do not constitute a limitation on the protection scope of the present disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present disclosure shall be included in the protection scope of the present disclosure.

Claims

1. A method for processing directory metadata applied to a distributed file system, applied to a distributed file system, wherein a metadata unit including a plurality of metadata fragments and an index unit including a plurality of index fragments are provided in a device node of the distributed file system, the method comprising: In response to the processing request, determining and loading a first index shard corresponding to the directory metadata indicated by the processing request; and determining, according to the first index slice, a metadata slice corresponding to the directory metadata indicated by the processing request; wherein the processing request represents an adjustment to the directory metadata in the device node; The information in the determined metadata slice is adjusted, and the index information corresponding to the directory metadata in the second index slice related to the directory metadata indicated by the processing request is adjusted.

2. The method according to claim 1, wherein: The processing request includes path information of the directory metadata; The determining and loading the first index shard corresponding to the directory metadata indicated by the processing request includes: An index code is calculated based on the path information of the directory metadata; the index code value represents the code of the index fragment; According to the index code, a first index fragment corresponding to the directory metadata indicated by the processing request is loaded.

3. The method according to claim 2, wherein: When the processing request indicates to delete the directory metadata, or to create the directory metadata, the path information of the directory metadata includes a target path; The target path indicates the path of the directory metadata; The step of calculating the index code according to the path information of the directory metadata includes: Calculate and obtain the first index code according to the target path; Determine, according to the target path of the metadata, a target parent path of a parent directory of the directory metadata in the target path; The second index code is calculated according to the parent path.

4. The method according to claim 3, wherein: The calculating, according to the target path, to obtain the first index code includes: Calculate the MD5 value of the target path; The index code is determined according to the MD5 value and the number of index fragments preset in the index unit.

5. The method according to claim 2, wherein: When the processing request indicates to modify the path of the directory metadata, the path information of the directory metadata includes an initial path and a final path; The initial path indicates the path when the directory metadata is not modified, and the final path indicates the path after the directory metadata is modified; The step of calculating the index code according to the path information of the directory metadata includes: Determine, according to the final path of the metadata, a first parent path of a parent directory of the directory metadata in the final path; The third index code is calculated according to the first parent path. Determine, according to the initial path of the metadata, a second parent path of the parent directory of the directory metadata in the initial path; The fourth index code is calculated according to the second parent path.

6. The method according to any one of claims 1 to 5, wherein: The processing request includes path information of the directory metadata; The determining, according to the first index slice, the metadata slice corresponding to the directory metadata indicated by the processing request includes: According to the path information of the directory metadata indicated by the processing request, determining the metadata code corresponding to the directory metadata in the first index slice; the metadata represents the code of the metadata slice; According to the metadata code, a metadata fragment corresponding to the directory metadata indicated by the processing request is determined.

7. The method according to claim 6, wherein: The determining, in the first index slice, the metadata code corresponding to the directory metadata according to the path information of the directory metadata indicated by the processing request, includes: According to the path information of the directory metadata indicated by the processing request, determining, in the index slice, index information corresponding to the path information; Determine the metadata code corresponding to the index information in the index slice.

8. The method according to any one of claims 1 to 7, wherein: When the processing request indicates to delete the directory metadata, the metadata slices include a first metadata slice corresponding to the directory metadata and a second metadata slice corresponding to a parent directory of the directory metadata; The adjusting of the information in the metadata slice determined includes: Deleting the first metadata shard; The amount of the directory metadata is subtracted from the information in the second metadata slice.

9. The method according to any one of claims 1 to 8, wherein: When the processing request indicates to create the directory metadata, the metadata slices include a first metadata slice corresponding to the directory metadata and a second metadata slice corresponding to a parent directory of the directory metadata; The adjusting of the information in the metadata slice determined includes: Creating the first metadata slice; The amount of the directory metadata is increased in the information in the second metadata slice.

10. The method according to any one of claims 1 to 9, wherein: When the processing request indicates to modify the directory of the directory metadata, the metadata slices include a third metadata slice corresponding to the parent directory indicated by the final path of the directory metadata, and a fourth metadata slice corresponding to the parent directory indicated by the initial path of the directory metadata; The adjusting of the information in the metadata slice determined includes: Increasing the amount of the directory metadata in the information in the third metadata slice; The amount of the directory metadata is subtracted from the information in the second metadata slice.

11. The method according to any one of claims 1 to 10, wherein: When the processing request indicates to delete the directory metadata, the path information of the directory metadata includes a target path; The adjusting the index information corresponding to the directory metadata in the second index slice related to the directory metadata indicated by the processing request includes: Determine a second index shard according to the target path of the directory metadata; Delete the index information corresponding to the target path of the directory metadata in the second index slice.

12. The method according to any one of claims 1 to 11, wherein: When the processing request indicates to create the directory metadata, the path information of the directory metadata includes a target path; The adjusting the index information corresponding to the directory metadata in the second index slice related to the directory metadata indicated by the processing request includes: Determine a second index shard according to the target path of the directory metadata; Create index information corresponding to the target path of the directory metadata in the index slice.

13. The method according to any one of claims 1 to 12, wherein: When the processing request indicates to modify the directory of the directory metadata, the path information of the directory metadata includes a final path and an initial path; The adjusting the index information corresponding to the directory metadata in the second index slice related to the directory metadata indicated by the processing request includes: If the index slice contains the directory metadata, determining that the index slice is a second index slice, and deleting the index information containing the directory metadata in the second index slice; Determine a modified path of the directory metadata and its subdirectories according to the final path and the initial path; According to the modified path of the directory metadata and its subdirectories, create index information of the modified path in the corresponding second index slice.

14. A processing device for directory metadata applied to a distributed file system, applied to a distributed file system, wherein a metadata unit including a plurality of metadata fragments and an index unit including a plurality of index fragments are provided in a device node of the distributed file system, the device comprising: A determination unit, configured to determine and load, in response to a processing request, a first index fragment corresponding to the directory metadata indicated by the processing request; and determining, according to the first index slice, a metadata slice corresponding to the directory metadata indicated by the processing request; wherein the processing request represents an adjustment to the directory metadata in the device node; An adjusting unit is used to adjust the information in the determined metadata slice, and to adjust the index information corresponding to the directory metadata in the second index slice related to the directory metadata indicated by the processing request.

15. An electronic device, comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 13.

16. A distributed file system, the distributed file system comprising a plurality of device nodes, wherein the device nodes are provided with a metadata unit and an index unit, wherein the metadata unit comprises a metadata slice corresponding to the directory metadata in the device node, and the index unit comprises a plurality of index slices, wherein the index slice is used to store index information of a part of the directory metadata; The device node is arranged in the electronic device as shown in claim 15.

17. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to cause the computer to execute the method according to any one of claims 1-13.

18. A computer program product, comprising a computer program, which implements the steps of the method according to any one of claims 1 to 13 when executed by a processor.

Images