Locking method and locking apparatus for locking directory tree

Abstract

The present disclosure provides a locking method for locking a directory tree and related products. The locking method includes: obtaining an operation request; and performing, according to the operation request, a multiple-granularity locking (MGL) operation on multiple MGL lock objects in an array of MGL lock objects, where the array of MGL lock objects includes a plurality of MGL lock objects used for recording locking status of nodes in the directory tree, and the multiple MGL lock objects correspond to multiple nodes including at least one target node and one or more parent nodes thereof. MGL of the directory tree can be implemented, thereby improving concurrency of filesystem metadata tree operations.

Description

LOCKING METHOD AND LOCKING APPARATUSFOR LOCKING DIRECTORY TREETECHNICAL FIELD

[0001] The present disclosure relates to the field of computer technologies, and in particular, to a locking method and a locking apparatus for locking a directory tree.BACKGROUND

[0002] A hierarchy of files and directories of a file system can be represented as a tree structure, i.e., a directory tree. The root node of the directory tree represents the root directoryeach intermediate node represents a subdirectory, and each leaf node represents a subdirectory or a file.

[0003] Read-write lock is the most common structure used in related art for synchronizing filesystem operations and avoiding a current operation not being affected by other operations. If an operation requires to read the contents of a node, a read lock needs to be obtained, and if an operation requires to change the contents of a node, a write lock needs to be obtained. Read-write lock can ensure non-interference during concurrent operations, but has poor concurrency.

[0004] This background information is provided to reveal information believed by the applicant to be of possible relevance to the present disclosure. No admission is necessarily intended, nor should be construed, that any of the preceding information constitutes prior art against the present disclosure.SUMMARY

[0005] In a first aspect, an embodiment of the present disclosure provides a lockingmethod for locking a directory tree, implemented by a locking apparatus, where the locking method includes: obtaining an operation request; and performing, according to the operation request, a multiple-granularity locking (MGL) operation on multiple MGL lock objects in an array of MGL lock objects, where the array of MGL lock objects includes a plurality of MGL lock objects used for recording locking status of nodes in the directory tree, and the multiple MGL lock objects correspond to multiple nodes in the directory tree, where the multiple nodes include at least one target node of the operation request and one or more parent nodes of the at least one target node.

[0006] By using an array of MGL lock objects to record locking status of nodes in the directory and performing an MGL operation on the MGL lock objects corresponding to the nodes in the directory tree which include at least one target node affected by the operation request and one or more parent nodes of the at least one target node, MGL locks can be set at various granularities for the operation request, and thus, MGL of the directory tree can be implemented, which allows for performing concurrent operations if they do not conflict with each other, thereby improving concurrency of filesystem metadata tree operations.

[0007] In a possible implementation of the first aspect, each node in the directory tree corresponds to a respective path in the file system, and each of the multiple nodes is mapped to a corresponding MGL lock object based on information of a path corresponding to the respective node. Since the nodes of the directory tree correspond to the paths in the file system respectively, each node can be mapped to a corresponding MGL lock object based on its corresponding path information. In other words, an index of an MGL lock object to which a node is mapped can be determined based on the path information of the node. In this way, the mapping between the nodes of the directory tree and the MGL lock objects can be implemented easily and effectively.

[0008] In a possible implementation of the first aspect, each of the multiple nodes is mapped to the corresponding MGL lock object based on a hash code value of the pathcorresponding to the respective node.

[0009] In a possible implementation of the first aspect, the hash code value is obtained by using a hashing algorithm on at least one string related to the path.

[0010] Since each node corresponds to a respective path, the hash code value of the respective path can be used to map the node to the corresponding MGL lock object. A path is generally represented by a string of characters, and the hash code value of the path may be determined by using a hashing algorithm on at least one string related to the path, which provides a possible and effective implementation for the mapping from the nodes to the MGL lock objects.

[0011] In a possible implementation of the first aspect, the array of MGL lock objects includes N MGL lock objects, where N is an integer greater than 1.

[0012] The size N of the array of MGL lock objects can be an integer greater than 1, which can be set according to actual conditions or demands, improving the adaptability and flexibility of the solution.

[0013] In a possible implementation of the first aspect, each of the N MGL lock objects in the array of MGL lock objects has a unique index.

[0014] Since each MGL lock object has a unique index, the nodes of the directory tree can be mapped to the MGL lock objects accurately.

[0015] In a possible implementation of the first aspect, indexes of the N MGL lock objects are in a range of [0, N-l]. In a possible implementation of the first aspect, each of the multiple nodes is mapped to the corresponding MGL lock object based on a reminder after dividing the hash code value of the path corresponding to the respective node by N.

[0016] When the size of the array of MGL lock objects is N, the indexes of the MGL lock objects in the array may range from 0 to N-l . A modulo-N operation can be performed on the hash code value of the path to map the hash code value to a number in the range of [0, N-l], thereby mapping each node to a corresponding MGL lock object in the array.

[0017] In a possible implementation of the first aspect, the performing, accordingto the operation request, the MGL operation on the multiple MGL lock objects in the array of MGL lock objects includes:

[0018] holding locks on the multiple MGL lock objects for the operation request, where lock modes of the locks to be held on the MGL lock objects are determined according to the operation request.

[0019] By holding locks on the MGL lock objects corresponding to the target node(s) and parent node(s) thereof according to the operation request, these nodes can be locked in appropriate modes for the operation request, thereby avoiding the operation request being interfered with by other operation request(s).

[0020] In a possible implementation of the first aspect, in a case that a lock has been held on a first MGL lock object of the multiple MGL lock objects, and a lock mode of a lock to be held on the first MGL lock object conflicts with a lock mode of the lock that has been held on the first MGL lock object, the locks are not held on the multiple MGL lock objects for the operation request until the lock that has been held on the first MGL is released.

[0021] When a lock to be held on an MGL lock object conflicts with a lock already held on the MGL lock object, locks will not be held on the multiple MLG lock objects for the operation request until the lock already held on the MGL lock object is released, thereby avoiding interference with other operation request(s) that are being processed.

[0022] In a possible implementation of the first aspect, in a case that the operation request indicates an operation affecting one target node, and the operation is a read operation, a share (S) lock is held on an MGL lock object corresponding to the affected target node, and an intention shared (IS) lock is held on each of one or more MGL lock objects corresponding to the one or more parent nodes of the affected target node.

[0023] When an operation indicated by the operation request affects one target node and the operation is a read operation, an S lock is held on the MGL lock object corresponding to the affected target node, and an IS lock is held on each of the MGL lock object(s) corresponding to the parent node(s). Since the locks are held on nodes at difference levels, and the IS lock is held on the MGL lock(s) corresponding to the parentnode(s), there will be more compatible operations that can be concurrently processed, thereby improving the concurrency of the file system.

[0024] In a possible implementation of the first aspect, in a case that the operation request indicates an operation affecting one target node, and the operation is a write operation, an exclusive (X) lock is held on an MGL lock object corresponding to the affected target node, and an intention exclusive (IX) lock is held on each of one or more MGL lock objects corresponding to the one or more parent nodes of the affected target node.

[0025] When an operation indicated by the operation request affects one target node and the operation is a write operation, an X lock is held on the MGL lock object corresponding to the affected target node, and an IX lock is held on each of the MGL lock object(s) corresponding to the parent node(s). Since the IX lock is held on the MGL lock(s) corresponding to the parent node(s), there will be more compatible operations that can be concurrently processed, thereby improving the concurrency of the file system.

[0026] In a possible implementation of the first aspect, in a case that the operation request indicates an operation affecting two or more target nodes, the multiple MGL lock objects include one or more second MGL lock objects for each of which one lock mode is required, and one or more third MGL lock objects for each of which two or more lock modes are required, and a lock mode of a lock to be held on each of the one or more second MGL lock objects is determined to be the lock mode required for the respective second MGL lock object, and a lock mode of a lock to be held on each of the one or more third MGL lock objects is determined based on a greatest lock mode of the two or more lock modes required for the respective third MGL lock object.

[0027] When an operation indicated by the operation request affects two or more target nodes, there may exist one or more MGL lock objects for each of which only one lock mode is required, and there may also exist one or more other MLG lock objects for each of which two or more lock modes are required. For a MLG lock object for which one lock mode is required, the lock mode required is taken as the lock mode ofthe lock to be held on the MGL lock object. For a MGL lock object for which two or more lock modes are required, the greatest lock mode of the two or more lock modes is taken as the lock mode of the lock to be held on the MGL lock object. In this way, MGL can also be implemented for a complex operation which affects two or more target nodes, and a complex operation may also be concurrently performed with other compatible operations, thereby further improving concurrency of the file system.

[0028] In a possible implementation of the first aspect, in a case that a target node and one or more parent nodes of the target node include two or more nodes that are mapped to a same MGL lock object, a lock mode of a lock to be held on the same MGL lock object is determined based on lock modes respectively required for the two or more nodes.

[0029] When two or more nodes on a path are mapped to a same MGL lock object, the lock mode of the lock to be held on that MGL lock object can be determined based on all the lock modes required for the two or more nodes, thereby avoiding the operation request being interfered with by other operation request(s).

[0030] In a second aspect, an embodiment of the present disclosure provides a locking apparatus including various modules configured to execute the locking method for locking a directory tree according to the first aspect or any possible implementation of the first aspect.

[0031] In a third aspect, an embodiment of the present disclosure provides a locking apparatus including processing circuitry for executing the locking method for locking a directory tree according to the first aspect or any possible implementation of the first aspect.

[0032] In a fourth aspect, an embodiment of the present disclosure provides a file system including the locking apparatus according to the second aspect, or the locking apparatus according to the third aspect.

[0033] In a fifth aspect, an embodiment of the present disclosure provides a file system including a metadata management component for executing the locking method for locking a directory tree according to the first aspect or any possible implementationof the first aspect.

[0034] In a sixth aspect, an embodiment of the present disclosure provides a computer-readable medium storing computer execution instructions which, when executed by a processor, causes the processor to execute the locking method for locking a directory tree according to the first aspect or any possible implementation of the first aspect.

[0035] In a seventh aspect, an embodiment of the present disclosure provides a computer program product including computer execution instructions which, when executed by a processor, causes the processor to execute the locking method for locking a directory tree according to the first aspect or any possible implementation of the first aspect.

[0036] The present disclosure provides a locking method for locking a directory tree and related products. An array of MGL lock objects is used for recording lock status of nodes in the directory tree. When an operation request is obtained, an MGL operation is performed on the MGL lock objects corresponding to the target node(s) affected by the operation request and the parent node(s) of the target node(s) according to the operation request. In this way, MGL locks can be set at various granularities for the operation request, and thus, MGL of the directory tree can be implemented, which allows for performing concurrent operations if they do not conflict with each other, thereby improving concurrency of filesystem metadata tree operations.BRIEF DESCRIPTION OF DRAWINGS

[0037] Reference will now be made, by way of example, to the accompanying drawings which show example embodiments of the present disclosure.

[0038] FIG. 1 is schematic diagram of HDFS architecture.

[0039] FIG. 2 is a schematic diagram showing structure of NameNode.

[0040] FIG. 3 is a schematic diagram showing a NameNode processing workflow.

[0041] FIG. 4 is a schematic flowchart of a locking method for locking a directorytree according to one or more example embodiments of the present disclosure.

[0042] FIG. 5 is a schematic diagram showing locks for a reading operation according one or more possible embodiments of the present disclosure.

[0043] FIG. 6 is a schematic diagram showing locks for a changing operation according one or more possible embodiments of the present disclosure.

[0044] FIG. 7 is a schematic diagram showing an example of concurrent operations according to one or more possible embodiments of the present disclosure.

[0045] FIG. 8 is a schematic diagram showing locks for a snapshot operation according one or more possible embodiments of the present disclosure.

[0046] FIG. 9 is a schematic diagram showing examples of collision scenarios according one or more possible embodiments of the present disclosure.

[0047] FIG. 10 is a schematic structural diagram of a locking apparatus according to one or more example embodiments of the resent disclosure.DESCRIPTION OF EMBODIMENTS

[0048] In the following description, reference is made to the accompanying figures, which form part of the present disclosure, and which show, by way of illustration, specific aspects of embodiments of the present disclosure or specific aspects in which embodiments of the present disclosure may be used. It is understood that embodiments of the present disclosure may be used in other aspects and include structural or logical changes not depicted in the figures. The following detailed description, therefore, is not to be taken in a limiting sense, and the scope of the present disclosure is defined by the appended claims.

[0049] A file system is a method of organizing files and directories on a storage medium. A distributed file system (DFS) is a file system distributed on multiple servers. DFS allows users to share files and storage over a network, and is widely used due to its high availability and scalability. A hierarchy of files and directories of a file system or DFS can be represented as a tree structure, i.e., a directory tree. The root node of thedirectory tree represents the root directory “ / ”, each intermediate node represents a subdirectory, and each leaf node represents a subdirectory or a file. Capability to process concurrent operations is important for the performance of the whole system, and better concurrency is especially desired for DFS which may be accessed by a large number of users or applications simultaneously

[0050] Read-write lock is the most common structure used in related art for synchronizing filesystem operations and avoiding an current operation not being affected by other operations. If an operation requires to read the contents of a node, a read lock needs to be obtained, and if an operation requires to change the contents of a node, a write lock needs to be obtained. Read-write lock can ensure non-interference during concurrent operations.

[0051] In an existing solution, a single read-write lock is used for the whole metadata tree. This solution has a simple implementation, but has poor concurrency since the whole metadata tree will be locked for each metadata tree operation and no other operation can be performed concurrently.

[0052] In another solution, read-write locks are added for every nodes of the metadata tree involved into the operation. Specifically, a fixed size array of read-write locks is applied per level of metadata tree. Canonical path of a destination object is split into directories and the lock on each level is held based on the directory name. Lock mode depends upon requested operation. For any read operation, a read lock is held for all nodes involved in the operation, while for any create / delete operation, a write lock is held for the parent node and a read lock is held for other higher nodes. This solution can provide better concurrency than the signal read-write lock solution. However, for operations that modify the content of the directory, a write lock is applied on the parent node and thus, any other parallel actions inside the directory will be blocked. In addition, this solution is not suitable for real life, because it significantly increases random access memory (RAM) usage by metadata tree.

[0053] The present disclosure provides a locking solution which can improve the performance of parallel operations with the file system metadata tree stored in memoryby using a set of multiple granularity locks. Hadoop Distributed File System (HDFS) is taken as an example in the following to describe the solution of the present disclosure. However, the solution is applicable for any file system, which stores the working set of data in memory in tree-like structure and performs concurrent modifications of different sub trees. Generally, this solution can be applied for synchronizing of concurrent metadata tree operations of any filesystem either distributed or local provided that metadata tree is in memory.

[0054] In order to facilitate the understanding of the solution of the present disclosure, some terms mentioned in the present disclosure are first introduced.

[0055] Multiple Granularity Locking (MGL) is a locking method where locks are set on objects that contain other objects. MGL exploits the hierarchical nature of the containing relationship. For example, a database may have files, which contain pages, which contain records. This can be thought of as a tree of objects, where each node contains its children. A lock on this structure (such as a shared or exclusive lock) locks the targeted node as well as all of its descendants.

[0056] Hadoop Distributed File System (HDFS) is a distributed file system designed to run on commodity hardware. It has many similarities with existing distributed file systems.

[0057] QHDFS is highly fault-tolerant and is designed to be deployed on low-cost hardware.

[0058] QHDFS provides high throughput access to application data and is suitable for applications that have large data sets.

[0059] QHDFS relaxes a few Portable Operating System Interface (POSIX) requirements to enable streaming access to file system data.

[0060] QHDFS was originally built as infrastructure for the Apache Nutch web search engine project.

[0061] QHDFS is part of the Apache Hadoop Core project.

[0062] HDFS NameNode is a master server that manages the file system namespace and regulates access to files by clients, and executes file system namespace operationslike opening, closing, and renaming files and directories. It also determines the mapping file data blocks to DataNodes. The simplest HDFS configuration includes one NameNode instance and several DataNodes.

[0063] HDFS DataNode manages storage attached to the nodes that they run on. HDFS exposes a file system namespace and allows user data to be stored in files. Internally, a file is split into one or more blocks and these blocks are stored in a set of DataNodes.

[0064] □The DataNodes are responsible for serving read and write requests from the file system’s clients.

[0065] □The DataNodes also perform block creation, deletion, and replication upon instruction from the NameNode.

[0066] MapReduce is a programming paradigm that enables massive scalability across hundreds or thousands of servers in a Hadoop cluster. As the processing component, MapReduce is the heart of Apache Hadoop. The term "MapReduce" refers to two separate and distinct tasks that Hadoop programs perform.

[0067] INode is a data structure in a Unix-style file system that describes a filesystem object such as a file or a directory.

[0068] INode ID is unique identifier of INode object.

[0069] Data block is a chunk with fixed size where HDFS stores file content.

[0070] Fslmage is a file stored on the operating system (OS) filesystem that contains the complete directory structure (namespace) of the HDFS with details about the location of the data on the data blocks and which blocks are stored on which DataNode.

[0071] EditLogs is a transaction log that records the changes in the HDFS file system or any action performed on the HDFS cluster such as addition of a new block, replication, deletion etc. It contains the changes since the last Fslmage was created.

[0072] BlockMap is a map that maps blocks of files to their physical locations on DataNodes.

[0073] INodeMap is a map that maps INode ID and INode objects.

[0074] Canonical path is a path to a file, or a directory from the root of the file system, which contain no dot (.), double-dot (..) components nor any repeated separators ( / ) or symbolic links or reserved segments like '.reserved'.

[0075] Apache Hadoop is an open source framework that is used to store and process large datasets ranging in size from gigabytes to petabytes of data. Hadoop is an Apache project; all components are available via the Apache open source license. The main Hadoop module is distributed file system (HDFS), that was originally designed to run on standard or low-end hardware. HDFS provides better data throughput than traditional file systems, in addition to high fault tolerance and native support of large datasets. HDFS stores file system metadata and application data separately. A dedicated server, called the NameNode, is responsible for metadata storing and processing. Application data are stored on other servers called DataNodes. All servers are fully connected and communicate with each other using TCP-based protocols. HDFS can be accessed from applications in many different ways. Natively, HDFS provides a Java API for applications to use. A C language wrapper for this Java API is also available. In addition, an HTTP protocol can also be used to browse the files of an HDFS instance.

[0076] FIG. 1 is a schematic diagram of HDFS architecture. As shown in FIG. 1, HDFS consists of the following components:NameNode, which• manages the file system namespace and regulates access to files by clients• executes file system namespace operations like opening, closing, and renaming files and directories• determines the mapping of file data blocks to DataNodes DataNode, which• manages the storage attached to the nodes that they run on• is responsible for serving read and write requests from the file system’s clients• performs block’s creation, deletion, and replication upon instruction fromthe NameNode

[0077] FIG. 2 is a schematic diagram showing structure of NameNode. HDFS namespace is a hierarchy of files and directories. Files and directories are represented on the NameNode by INodes. INodes record attributes like permissions, modification and access times, namespace and disk space quotas and links to the child INodes. The file content is split into large blocks (typically 128 megabytes, but user selectable file- by-file), and each block of the file is stored on the one or more DataNodes. The NameNode maintains the namespace tree and the mapping of blocks to DataNodes.

[0078] FIG. 3 is a schematic diagram showing a NameNode processing workflow, where multiple handlers use a single global lock to process request and write changes to the EditLog file. Currently, in open-source version of Hadoop, namespace is stored in thread-unsafe tree and all operations are performed under global read-write lock. It becomes a bottleneck when there are many concurrent requests from clients. The performance can be improved by replacing tandem of thread-unsafe tree and global read-write lock with solution which allows to perform concurrent operations with different nodes if they do not conflict with each other.

[0079] FIG. 4 is a schematic flowchart of a method for locking a directory tree according to one or more example embodiments of the present disclosure. The locking method can be implemented by a locking apparatus. Optionally, the locking apparatus can be a filesystem metadata management component that holds all metadata in memory, such as NameNode in HDFS, or other component that has similar function, which is not limited herein. As shown in FIG. 4, the locking method can include the following steps.

[0080] S410, obtain an operation request;

[0081] An operation request can be obtained, which may indicate an operation to be performed. Each node in the directory tree corresponds to a respective path in the file system. The operation request may affect one or more target nodes in the directory tree. For example, if an operation request indicates reading content of / dir 1 / dir2 / , where dirl and dir2 are names of directories, the target node affected by the operation requestis / dirl / dir2. For another example, if an operation request indicates creating a new file / dirl / dir2 / readme.txt, a new node is created, i.e. / dirl / dir2 / readme.txt, and the target node is the newly created node. For still another example, if an operation request indicates moving file / dirl / dir2 / readme.txt to a different folder / dirl / dir3, the node / dirl / dir2 / readme.txt will be removed and a new node / dirl / dir3 / readme.txt will be created. In this example, both the old node / dirl / dir2 / readme.txt and the new node / dirl / dir3 / readme.txt will be affected by the operation request, and there are two target nodes. Such an operation which affects two or more target nodes may also be called a “complex” operation in the present disclosure.

[0082] S420, perform, according to the operation request, a multiple-granularity locking (MGL) operation on multiple MGL lock objects in an array of MGL lock objects, where the array of MGL lock objects includes a plurality of MGL lock objects used for recording locking status of nodes in the directory tree, and the multiple MGL lock objects correspond to multiple nodes in the directory tree, where the multiple nodes include at least one target node of the operation request and one or more parent nodes of the at least one target node.

[0083] An array of MGL lock objects is used to locking status nodes in the directory tree. In other words, the array of MGL lock objects is used to hold locks on nodes of the directory tree, or paths of the file system. Since a file system may have almost unlimited height of hierarchy, made of folders, subfolders and files, the size of the array of MGL lock objects and the mapping between the nodes and MGL lock objects are important for the implementation of the solution.

[0084] The locking method for locking a directory tree according to embodiments of the present disclosure can improve the throughput of HDFS NameNode file concurrent operations in cases when these operations don’t conflict with each other by action-path with a fixed memory usage. For example: it is possible to concurrently perform operations of creating different files in a same directory, to concurrently perform operations of reading the contents of any directories, concurrently perform an operation of moving directory A and an operation of reading the content of directory Bwhich have different paths or some common parent directory, or perform snapshot of directory by copying the content of the directory into a subdirectory.

[0085] The solutions according to the embodiments of the present disclosure can be implemented as improvement of Hadoop NameNode by replacing one global readwrite lock with MGL lock object as described in the present disclosure. Also, the solutions may also be applied to other programs which performs concurrent read / modify operations on tree-like structures.

[0086] In a possible implementation, the array of MGL lock objects includes N MGL lock objects, where N is an integer greater than 1. In a possible implementation, the size N of the array of MGL lock objects is fixed. When a fixed size array of MGL lock objects is used, RAM usage for the array is also fixed and will not increase with the increases of nodes or height of the directory tree.

[0087] In a possible implementation, the size N of the array of MGL lock objects may be configured and initiated when the file system starts. The size N can be configured according to actual conditions or demands, improving the adaptability and flexibility of the solution.

[0088] If the value ofN is very small, the possibility of lock collision (which refers to a situation where two or more nodes are mapped to the same MGL lock object) will be high. Although MGL may still be used in case of lock collision as described in the following, frequent collisions may have some adverse impact on the performance of the system. In a possible implementation, the value of N is between a hundred and several thousands.

[0089] In a possible implementation, the value of N may be configured according to specification of the machine(s) on which the system is running. In a possible implementation, the size N may be configured according to the number of central processing unit (CPU) cores. With a small number of MGL lock objects compared to the number of cores, the throughput of the system is linearly limited by the number of MGL lock objects. On the other hand, with a very large number of MGL lock objects, the bandwidth is slowly moving up to its theoretical maximum, but there will be toomuch memory spent on lock array. For example, if N equals to the number of CPU cores, the concurrency may average about 63% of the maximum due to collisions; with N doubles the number of CPU cores, the concurrency may average about 79%; with N triples the number of CPU cores, the concurrency may average about 85%; and with the further increase of the lock number, the concurrency gain will be little noticeable. In addition, it is worth taking into account that if the critical section occupies a small amount of CPU workload, then the number of MGL lock objects is acceptable to reduce proportionally. In a possible implementation, N may be twice the number of CPU cores for good starting point. For example, on a server with 64 cores, it is possible to use N=128.

[0090] When using MGL, besides a shared (S) mode, which may also be called a read mode, and an exclusive(X) mode, which may also be called a write mode, there are three additional lock modes in MGL: an intention-shared (IS) mode, which indicates explicit locking at a lower level of the tree with one or more shared locks, an intentionexclusive (IX) mode, which indicates explicit locking at a lower level with one or more exclusive locks and / or one or more shared locks, and a shared and intention-exclusive (SIX) mode, which indicates the subtree rooted by that node is locked explicitly in shared mode and explicit locking is being done at a lower level with at least one exclusive locks.

[0091] The MGL operation is performed on multiple MGL lock objects corresponding to the multiple nodes which are involved in the operation request. The multiple nodes involved in the operation request include at least one target node of the operation request and one or more parent nodes of each of the at least one target node, where the one or more parent nodes of each target node include all the parent node(s) of the target node up to the root. The nodes can be mapped to the MGL lock objects based on some unique information of the nodes, which may be, for example, path information of the respective nodes.

[0092] By using an array of MGL lock objects to record locking status of nodes in the directory and performing an MGL operation on the MGL lock objects correspondingto the nodes in the directory tree which include at least one target node affected by the operation request and one or more parent nodes of the at least one target node, MGL locks can be set at various granularities for the operation request, and thus, MGL can be implemented for the directory tree of a file system, which allows for performing concurrent operations if they do not conflict with each other, thereby improving concurrency of filesystem metadata tree operations.

[0093] In a possible implementation, each node in the directory tree corresponds to a respective path in the file system, and each of the multiple nodes is mapped to a corresponding MGL lock object based on information of a path corresponding to the respective node. Since the nodes of the directory tree correspond to the paths in the file system respectively, each node can be mapped to a corresponding MGL lock object based on information of its corresponding path. In other words, an index of an MGL lock object to which a node is mapped can be determined based on the information of the path corresponding to the node. Since a filesystem metadata management component knows the path corresponding to each node, using information of paths for the mapping can make the mapping between the nodes of the directory tree and the MGL lock objects to be implemented more easily and effectively, and there’s no need to assign additional identification information or acquire physical address information for each node for the mapping which may need more computing and storage resources.

[0094] In a possible implementation, each of the multiple nodes is mapped to a corresponding MGL lock object based on a hash code value of the path corresponding to the respective node. Each directory or file may be named by a string of characters based on some naming rules, where the characters may include letters, numbers or even some special characters such asA path to a target directory may be represented by a string including names of a series of directories from the root directory to the target directory, and a path to a target file may be represented by a string including names of a series of directories from the root directory to the parent directory of the file as well as the name of the file. Since a path is generally represented by a string of characters while the index of an MGL lock object is an integer, hashing may be employed toconvert the string or some other information obtained from the string to an integer value, based on which the index of the MGL lock object can be obtained. Therefore, a hash code value may be derived from the string and then be used for mapping a node or a path to a corresponding MGL lock object. Since the hash code value of a same path will be the same, the mapping between the nodes and MGL lock objects will not change (unless some new nodes are created or some old nodes are removed), a same node can be mapped to a same MGL lock object each time, and thus the locking status of a same node can be accurately recorded by a same MGL lock object. Then, when an operation request is obtained, the current locking status of relevant nodes can be obtained accurately, facilitating the implementation of MGL.

[0095] In a possible implementation, the hash code value may be obtained by using a hashing algorithm on at least one string related to the path, which provides a possible and effective implementation for the mapping from the nodes to the MGL lock objects. In a possible implementation, standard Java hashCode computation may be used to get the hash code value of the path. The input for the hashCode computation may be the whole string representing the path. It is also possible to represent a path as a string array of segments, starting from root (empty string), and then calculate the hash code value of the string array, for example, by using Java static method Arrays.hashCode(String[]). As an example, for a path / foo / bar, a hash code value of an array of three elements {“”, “foo”, “bar”} can be obtained. Using the segment array to obtain the hash code may reduce calculations or make the calculations more efficient. Specifically, in the solution according to the present disclosure, it is needed to calculate the hash code not only for the target node, but for all its parent nodes, starting from the root, and in the process of calculating the hash code for the segment array, the calculation is performed along this array from left to right, and as intermediate results, the hash code values of paths for all the parent nodes, starting at the root, are also obtained, and there will be no need to perform two unnecessary and heavy operations: getting the paths to all the parent nodes from the path to the target node and calculating the hash code value for each of them separately.

[0096] In a possible implementation, each of the N MGL lock objects in the array of MGL lock objects has a unique index. Since each MGL lock object has a unique index, each node of the directory tree can be mapped to the corresponding MGL lock object accurately.

[0097] In a possible implementation, indexes of the N MGL lock objects are in a range of [0, N-l]. In a possible implementation, each of the multiple nodes may be mapped to the corresponding MGL lock object based on a reminder after dividing the hash code value of the path corresponding to the respective node by N. That is, when the size of the array of MGL lock objects is N, the indexes of the MGL lock objects in the array may range from 0 to N-l, and a modulo-N operation can be performed on the hash code value of the path to map the hash code value to an integer in the range of [0, N-l], thereby mapping each node to a corresponding MGL lock object in the array. In this way, the array of MGL lock objects with a size of N can be used to locking the directory tree. The indexes of MGL lock objects may have other values, and the nodes of the directory tree may also be mapped to the MGL lock objects in other manners.

[0098] Taking the path / foo / bar as an example again, the hash code value for the array of three elements“foo”, “bar”} is 3275884, and if N is set to 50, the index of MGL lock object can be calculated as 3275884 mod 50 = 34, and thus the node corresponds to the path / foo / bar can be mapped to the MGL lock with an index of 34.

[0099] For another example, for a target node corresponding to a canonical path / din / dir2 / ... / dirM-i / dirM, the parent nodes of the target node correspond to subpaths from / to / din / din / ... / dirM-i, and the mapping of the target node and parent nodes to the MGL lock objects may be performed the following way:Ho = hashCodep'P') mod NHi - hashCodep'ld\Y\ '‘) mod NH2 = hashCodep' / dirddiri ') mod NHM-I = hashCode^ldvcddxxil ... / dirM-i”) mod NHM - hashCodet," I dir 1 / dir2 / .. , / dirM-i / dirM”) mod Nwhere mod represents modulo operation, and hashCode(s) represents a hash function, which returns a hash code value for a given input string s. The input string s may be the whole string representing the path, or an array of segments of the string represent the path as described above. For example, when using the whole string as the input string, the hash code is calculated for the target node / dir i / dirz / ... / dirM-i / dirM and each of parent nodes from / to Zdin / dirs / ... / dirM-i separately by taking each string as the input string. For another example, when an array of segments is used, the array {"", "din", "din", ... , "dirM"} is the input string for calculating the hash code value of the target node, and then as intermediate results, the hash code values of paths for all the parent nodes, starting at the root / to / din / din / . -. / dirM-i, can be obtained, and the final result of the hash code calculation is the hash code value of the target node / din / din / .- . / dirM-i / dirM. The values HO...HM are sorted ascending and denotes MGL lock objects in the array of MGL lock objects to which the MGL operation will be applied. Sorting operation is needed to avoid deadlocks in concurrent operations.

[0100] In a possible implementation, the the MGL operation is performed, according to the operation request, on the multiple MGL lock objects in the array of MGL lock objects by holding locks on the multiple MGL lock objects for the operation request, where lock modes of the locks to be held on the MGL lock objects are determined according to the operation request. By holding locks on the MGL lock objects corresponding to the target node(s) and parent node(s) thereof according to the operation request, these nodes can be locked in appropriate modes for the operation request, thereby avoiding the operation request being interfered with by other operation request(s).

[0101] An operation may affect one or more target nodes. Operations affecting one target node may be divided into read operations and write operations. A read operation may refer to an operation that does not change the local state of the affected target node, while a write operation may refer to an operation that changes the local state of the affected target node, where the local state of the affected target node may include presence of the affected target node, contents of the affected target node, configurableattributes of the affected target node.

[0102] Presence of a node may refer to a state of whether the node exists or not. For example, an operation of creating a file / din / din / readme.txt will lead to the creation of a new node / dir i / din / readme.txt, then the node Zdin / din / readme.txt is changed from not existing to existing, and the presence of the node is changed by the creation operation, and thus the creation operation is a write operation. For another example, an operation of deleting a directory / dir i / dirs will lead to the deletion of the node / dir i / dirs, then the node / dir i / dirs will be changed from existing to not existing, and the presence of this node is changed by the deletion operation, and thus the deletion operation is also a write operation. In yet another example, for an operation of reading the file / din / din / readme.txt, the node / din / din / readme.txt exists before and after the operation, and the presence of the node / diri / din / readme.txt will not be changed by this operation.

[0103] Configurable attributes may refer to those attributes that can be separately configured by an application or a user. Generally, these attributes will not automatically change along with the changes in the contents of the file or directory, and will not be changed by a read operation. For a file, the configurable attributes may include location of the file, permissions for the file, replication factor (i.e., the number of copies) of the file, etc. For a directory, the configurable attributes may include location of the directory, permissions for the directory, quotas for the directory which may include name quota for the number of names used and space quota for the amount of space used for the directory, etc.

[0104] If an operation does not make any change in the local state of the affected target node, for example, the operation does not create or delete the node and does not change the contents and configurable attributes of the affected target node, the operation will be a read operation. The read operation may be an operation of reading contents of a directory, an operation of reading contents of a file, an operation of opening a file, etc.

[0105] If an operation changes the local state of the affected target node, for example, the operation creates or deletes the target node, or changes the contents or a configurable attribute of the affected target node, the operation will be a write operation.The write operation may be an operation of creating a file or a directory, an operation of deleting a file or a directory, an operation of updating a file, an operation of changing permissions of a file or a directory, an operation of changing the replication factor of a file, an operation of changing quotas of a directory, etc.

[0106] In a possible implementation, the operation request may indicate an operation affecting one target node, and the operation is a read operation. In such a case, an S lock is held on an MGL lock object corresponding to the target node, and an IS lock is held on each of one or more MGL lock objects corresponding to the one or more parent nodes of the target node.

[0107] When an operation indicated by the operation request affects one target node and the operation is a read operation, an S lock can be held on the MGL lock object corresponding to the affected target node, and an IS lock can be held on each of the MGL lock object(s) corresponding to the affected parent node(s). For example, for an operation of reading content of / dirl / dir2 / ... / dirM-l / dirM, IS locks are held on all the MGL lock objects corresponding to subpaths from / to / dir 1 / dir 2 / .. , / dirM-l / dirM-l , and an S lock is held on the MGL lock object corresponding to / dirl / dir2 / ... / dirM-l / dirM. For another example, when the operation is reading / dirl / dir3 / readme.txt, an S lock is held on the MGL lock object corresponding to the target node / dirl / dir3 / readme.txt, and IS locks are held on the MGL lock objects corresponding to the three parent nodes I , / dirl, and / dirl / dir3. Since the locks are held on nodes at difference levels, and the IS lock is held on the MGL lock(s) corresponding to the parent node(s), there will be more compatible operations that can be concurrently processed, thereby improving the concurrency of the file system.

[0108] FIG. 5 is a schematic diagram showing locks for a reading operation according one or more possible embodiments of the present disclosure. FIG. 5 shows how the locks are held on MGL lock objects when the operation request indicates reading content of / din / din. As indicated by 530 in FIG. 5, the target node / din / diri is mapped to the MGL lock object LockN. As indicated by 520, the parent node / din is mapped to the MGL lock object LockN-2. And as indicated by 510, another parent node,the root node / , is mapped to the MGL lock object Locki. Since the operation is a reading operation, an S lock is held on the MGL lock object LockN corresponding to the target node / din / din, and IS locks are held on the MGL lock objects Locki and LockN-2 corresponding to the two parent nodes I and / din, respectively. In such a condition, since / diri / din is locked in S mode, and / and / diri are locked in IS mode, then, in addition to reading operations in these three directories or other subdirectories (not shown in FIG. 5) under the three directories, some write operations, such as creating a file or a directory, deleting a file, may also be concurrently performed in the directory / din, or in other directories such as / diri / dirs (not shown in FIG. 5) under / diri and other directories such as / din, / dirs (not shown in FIG. 5) under the root path / . Therefore, there will be more compatible operations that can be concurrently processed, thereby improving the concurrency of the file system. Such processing method also applied to other reading operations.

[0109] In a possible implementation, the operation request may indicate an operation affecting one target node, and the operation is a write operation. In such a case, an X lock is held on an MGL lock object corresponding to the target node, and an IX lock is held on each of one or more MGL lock objects corresponding to the one or more parent nodes of the target node.

[0110] When an operation indicated by the operation request affects one target node and the operation is a write operation, an X lock is held on the MGL lock object corresponding to the target node, and an IX lock is held on each of the MGL lock object(s) corresponding to the parent node(s). For example, for an operation of creating / din / dir2 / ... / dirM-i / dirM, IX locks are held on all the MGL lock objects corresponding to subpaths from / to / din / dir2 / ... / dirM-i / dirM-i, and an X lock is held on the MGL lock object corresponding to / diri / din / . - . / dirM-i / dirM. For another example, when the operation is deleting / diri / din / readme.txt, an X lock is held on the MGL lock object corresponding to the target node / diri / din / readme.txt, and IX locks are held on the MGL lock objects corresponding to the three parent nodes / , / diri, and / din / din. Since the IX lock is held on the MGL lock corresponding to the parent node, there will be morecompatible operations that can be concurrently processed, thereby improving the concurrency of the file system.

[0111] FIG. 6 is a schematic diagram showing locks for a changing operation according one or more possible embodiments of the present disclosure. FIG. 6 shows how the locks are held on MGL lock objects when the operation request indicates creating / din / dir2. In such an operation, a new node / din / din is created and taken as the target node. As indicated by 630, the target node / din / din is mapped to the MGL lock object LockN. As indicated by 620, the parent node / din is mapped to the MGL lock object LockN-2. And as indicated by 610, the root node / is mapped to the MGL lock object Locki. Since the operation is a creating operation, which changes the local state of the target node, an X lock is held on the MGL lock object LockN corresponding to the target node / din / dir2, and IX locks are held on the MGL lock objects Locki and LockN-2 corresponding to the two parent nodes I and / din, respectively. In such a condition, since / and / din are locked in IX mode, then, reading operations such as reading a directory or a file, or changing operations such as creating a file or a directory, deleting a file, may be concurrently performed in other directories such as / dir i / din (not shown in FIG. 6) under the directory / dir i or in other directories such as / din, / dirs (not shown in FIG. 6) under the root directory / . Therefore, there will be more compatible operations that can be concurrently processed, thereby improving the concurrency of the file system.

[0112] In a possible implementation, a lock may have been held on a first MGL lock object of the multiple MGL lock objects, and a lock mode of a lock to be held on the first MGL lock object may conflict with a lock mode of the lock that has been held on the first MGL lock object. In such a case, the locks are not held on the multiple MGL lock objects for the operation request until the lock that has been held on the first MGL is released.

[0113] When a lock to be held on an MGL lock object conflicts with a lock already held on the MGL lock object, the operation request needs to wait for other operation(s) to release the lock already held on the MGL lock object, and the locks will not be heldon the multiple MLG lock objects for the operation request until the lock already held on the MGL lock object is released, thereby avoiding interference with other operation request(s) that are being processed.

[0114] The MGL operation should follow the compatibility matrix as shown in table 1 , where “V” indicates the two lock modes are compatible with each other, while “x” indicates that the two lock mode conflict with each other.Table 1

[0115] FIG. 7 is a schematic diagram showing an example of concurrent operations according to one or more possible embodiments of the present disclosure. FIG. 7 shows an example of MGL lock modes for concurrent operations inside directory / diri. A first operation request by Client 1 indicates an operation of reading the content of / din / din, and a second operation request by Client 2 indicates an operation of creating / din / din. In such a condition, the following locks will be held: for the first operation request of Client 1 , IS lock for I and / dir i , and S lock for / din / din; for the second operation request of Client 2, IX lock for / and / diri, and X lock for / din / din. Since IS and IX are compatible with each other, the two operation request can be processed concurrently.

[0116] In a possible implementation, the operation request may indicate an operation affecting two or more target nodes, and a lock mode of a lock to be held on each of the multiple MGL lock object can be determined based on one or more lock modes required for the respective MGL lock object. In this case, the multiple MGL lock objects may include one or more second MGL lock objects for each of which one lock mode is required, and one or more third MGL lock objects for each of which two ormore lock modes are required, and a lock mode of a lock to be held on each of the one or more second MGL lock objects may be determined to be the lock mode required for the respective second MGL lock object, and a lock mode of a lock to be held on each of the one or more third MGL lock objects is determined based on the greatest lock mode of the two or more lock modes required for the respective third MGL lock object. In this way, MGL can also be implemented for a complex operation which affects two or more target nodes, and a complex operation may also be concurrently performed with other compatible operations, thereby further improving concurrency of the file system.Table 2

[0117] When the operation request indicates an operation affecting two or more target nodes, which for example, may be a move operation, a rename operation, a snapshot operation, etc., more than one lock modes may be required for a single node. In the multiple MGL lock objects, there may exist one or more MGL lock objects for each of which only one lock mode is required, and there may also exist one or more other MLG lock objects for each of which two or more lock modes are required. For a MLG lock object for which one lock mode is required, the lock mode required can be taken as the lock mode of the lock to be held on the MGL lock object. For a MGL lock object for which two or more lock modes are required, the lock mode of the lock to be held on the MGL lock object can be determined by summarizing all the lock modes required for the MGL lock object. Summarizing or adding of the lock modes follows the lock mode arithmetic as shown in table 2.

[0118] In a possible implementation, for an operation request affecting two or more target nodes, lock modes of the locks to be held on respective MGL lock objects may be determined as follows: obtaining, according to the operation request, a first list of nodes for which an X lock is required, a second list of nodes for which an S lock is required, a third list of nodes for which an IX lock is required, and a fourth list of nodes for which an IS lock is required; mapping the four lists of nodes to corresponding MGL lock objects; and summarizing the lock mode(s) required for each of the multiple MGL lock object to obtain a single resulting lock mode for each of the multiple MGL lock objects. The first list of nodes and the second list of nodes are all target nodes. The third list of nodes includes the parent node(s) of the first list of nodes, while the fourth list of nodes includes the parent node(s) of the second list of nodes. The four lists of nodes may share one or more common nodes. Depending on the specific complex operation, there may be no nodes requiring S or IS mode, that is, the second list and the fourth list may be empty.

[0119] Some examples of complex operations will be described to facilitate understanding of the solutions of the locking method for locking a directory tree according to the present disclosure.

[0120] In an example, the operation request may indicate HDFS snapshot which performs simultaneous read and write operations in the same directory. For example, for taking a snapshot of / din / din / .- . / dirM-i / dirM, a special destination directory / diri / din / . -. / dirM-i / dirM / .snapshot is created inside the source one / diri / diri / ... / dirM- i / ditM / . In this case, both / dir i / din / ... / dirM-i / dirM and / din / dir2 / ... / dirM-i / dirM / .snapshot are target nodes. An S lock is required for the target node / diri / din / ... / dirM-i / dirM of which the content is read, and IS locks are required for its parent nodes from / to / din / dir2 / ... / dirM-i. An X lock is required for the target node / din / din / .. , / dirM- i / diW-snapshot which is the destination node for the operation, and IX locks are required for its parent nodes from / to / diri / din / ... / dirM. Then, locks will be held as follows:• X lock for destination path / din / din / ... / dirM-i / dirM / .snapshot• S lock + IX lock = SIX lock for path / din / dir? / ... / dirM-i / dirM• IS lock + IX lock = IX lock for all sub-paths from / to / diri / dir2 / ... / dirM-i.

[0121] FIG. 8 is a schematic diagram showing locks for a snapshot operation according one or more possible embodiments of the present disclosure. FIG. 8 shows how the locks are held on MGL lock objects when the operation request indicates taking an HDFS snapshot of / din. This operation will perform simultaneous read and write operations in / din . As indicated by 810 in FIG. 8, the root node / is mapped to the MGL lock object Locki. As indicated by 820, the node / din is mapped to the MGL lock object LockN-2. And as indicated by 830, the node / dir 1 / .snapshot is mapped the MGL lock object LockN-i. The node / din is the target node of the read operation involved, so an S lock is required for / din, and an IS lock is required for its parent node / . The node / din / .snapshot is the target node of the write operation involved, so an X lock is required for the node / din / .snapshot, and IX locks are required for its parent nodes / and / din. In this case, both an S lock and an IX lock are required for the node / din, then an SIX lock is to be held for the node / din based on S+IX=SIX. Both an IS lock and an IX lock are required for the root node / , then an IX lock is to be held for the root node / based on IS+IX=IX.

[0122] In another example, the operation request may indicate moving a file / din / din / readme .txt to a different folder / din / din. For this operation, the node / din / din / readme.txt will be removed and a new node / din / din / readme.txt will be created. Both nodes / din / din / readme.txt and / din / dira / readme.txt are affected by the operation, and thus are target nodes. X locks are required for the two target nodes since both of them are changed, where one is removed while the other is created. The target node / diri / dir2 / readme.txt has three parents nodes, i.e. / , / din. and / din / din, for which IX locks are required. The target node / din / din / readme.txt has three parents node, i.e. I , / din, and / din / dirs, for which IX locks are required. For each of the parent nodes / din / din and / din / din, since each of them is the parent node of one target node, only one lock is required, which is the IX lock, then the lock to be held on the correspondingMGL lock object is an IX lock. For the parent nodes I and / diri, they are parents of both target nodes, then as the parent of one target node, an IX lock is required, and as the parent of the other target node, also an IX lock is required, and therefore, IX locks are to be held on the MGL lock objects corresponding to these two parent node / and / diri since IX+IX=IX.

[0123] In another example, the operation request may indicate renaming a file / din / diri / readme.txt to / din / din / drink.txt. For this operation, the node / diri / dir2 / readme.txt will be removed and a new node / diri / diri / drink.txt will be created. Both nodes / din / din / readme.txt and / diri / din / drink.txt are affected by the operation, and thus are target nodes. X locks are required for the two target node since both of them are changed, one is removed while the other is created. The two target nodes have three common parent nodes I , / din and / dir i / din. Along with X locks for both target nodes, IX locks are required for all parent nodes of the two target nodes based on IX+IX=IX.

[0124] In some cases, some values of Ho...HM may match, which can cause a lock collision inside a single path. Such collision may occur, especially when the array size N is of a relatively small value. Here, lock collision, which may also be called hashcode collision or mapping collision, refers to a situation where two or more nodes on a single path are mapped to a same MGL lock object due to the same hash code value, in other words, among the target node and the parent node(s) of the target node, there are two or more nodes that have the same hash code value and thus are mapped to the same MGL lock object. In this case, the lock mode of the lock to be held on the MGL lock object may be determined based on the lock modes required for the two or more nodes mapped to the MGL lock object, for example, by summarizing the lock modes required for the two or more nodes mapped to the MGL lock object. Summarizing the lock modes also follows the lock mode arithmetic as shown in table 2.

[0125] In an example, if the nodes having the same hash code value correspond to locks with the same lock mode (for example IS or IX), the structure of locks will remain the same as in the case when there is no collision. When the nodes correspond to lockswith different lock modes (for example IX and X, or IS and S), the stricter level will be applied for this lock (X and S respectively) and it will correspond to the lock mode required for the node on the higher level in the directory tree. In this case another concurrent operation can’t hold incompatible lock on this level until it will be released.

[0126] When two or more nodes on a path are mapped to a same MGL lock object, and more than one locks modes are required for the MGL lock object, a lock of the stricter-level lock mode can be held on the MGL lock object, thereby avoiding the operation request being interfered with by other operation request(s).

[0127] FIG. 9 is a schematic diagram showing examples of collision scenarios according one or more possible embodiments of the present disclosure. Each circle in FIG. 9 represents a node, and the number in parenthesis inside the circle indicates the index of the MGL lock object corresponds to the node. Lock information is also marked in square brackets besides each node. The lock information includes a list of MGL lock objects for locking the node in a specific lock mode. For each element in the list, the letter(s) indicates the lock mode, while the number after the letter(s) indicates the index of MGL lock object.

[0128] For example, for the node / var / run, “(4)” in the circle indicates that the node / var / run is mapped to MGL lock object 4, and the lock information [x4, ix6, ixl5] indicates that, for locking / var / run in X mode, an X lock is held on MGL lock object 4, and IX locks are held on MGL lock objects 6 and 15 (corresponding to its parent nodes / var and / respectively). Similarly, the node / var / log corresponds to MGL lock object 1, and for locking / var / log in X mode, an X lock is held on MGL lock object 1, and IX locks are held on MGL lock objects 6 and 15 (corresponding to its parent nodes / var and / respectively). Concurrent operations can hold X lock [x4, ix6, ixl 5] on / var / run and [xl, ix6, ixl 5] on / var / log. On the path / var / run and the path / var / log, each node is mapped to a different MGL lock object, and there’s no collision.

[0129] Further, as shown in FIG. 9, for the path / usr / bin, both the root node / and the node / usr / bin are mapped to the same MGL lock object 15. The lock information [xl5, ixl 7] for / usr / bin indicates that for locking / usr / bin in X mode, an X lock is heldon MGL lock object 15 and an IX lock is held on MGL lock object 17 (corresponding to it parent node / usr). In this case, since both the root node I and the node / usr / bin are mapped to MGL lock object 15, an X lock is required for / usr / bin, and an IX lock is required for / , then a stricter lock, i.e. the X lock (which may be obtained by IX+X=X according to table 2), is held on MGL lock object 15, and then the root node / is also locked in X mode. Similarly, for the path / usr / lib, both / usr and / usr / lib are mapped to MGL lock object 17. When an X lock is required for / usr / lib, and an IX lock is required for / usr, a stricter lock, i.e. the X lock (which may be obtained by IX+X=X according to table 2) is held on MGL lock object 17, and then / usr is also locked in X mode. Please be noted that an operation that holds an X lock on / usr / bin will not be concurrently processed with an operation which holds an X lock on / var / run or / var / log or an operation which holds an X lock on / usr / lib since locking / usr / bin in X mode will cause the root node to be locked in X mode too, and X lock on the root / is not compatible with any lock. The lock information [xl 5] beside the root node I is intended to explain that the root node I will be locked in X mode when an X lock is held on MGL lock object 15, but not to indicate that the root I may be locked in X mode while / var / run or / var / log is locked in X mode.

[0130] Some examples have been described to facilitate understanding of the solutions of the locking method for locking a directory tree according to the present disclosure. It should be noted that the examples are for illustration purpose only, which should not be construed as limitations to the solution of the present disclosure.

[0131] Next, embodiments of products related to the locking method for locking a directory tree will be described.

[0132] FIG. 10 shows a schematic structural diagram of a locking apparatus according to one or more example embodiments of the present disclosure. The locking apparatus may be applied to a filesystem metadata management component that holds all metadata in memory, such as NameNode in HDFS, or other component that has similar function, which is not limited herein. As shown in FIG. 10, the locking apparatus 1000 includesa obtaining module 1010, configured to obtain an operation request; and a processing module 1020, configured to perform, according to the operation request, a multiple-granularity locking (MGL) operation on multiple MGL lock objects in an array of MGL lock objects, where the array of MGL lock objects includes a plurality of MGL lock objects used for recording locking status of nodes in the directory tree, and the multiple MGL lock objects correspond to multiple nodes in the directory tree, where the multiple nodes include at least one target node of the operation request and one or more parent nodes of the at least one target node.

[0133] In a possible implementation, each node in the directory tree corresponds to a respective path in the file system, and each of the multiple nodes is mapped to a corresponding MGL lock object based on information of a path corresponding to the respective node

[0134] In a possible implementation, each of the multiple nodes is mapped to the corresponding MGL lock object based on a hash code value of the path corresponding to the respective node.

[0135] In a possible implementation, the hash code value is obtained by using a hashing algorithm on at least one string related to the path.

[0136] In a possible implementation, the array of MGL lock objects includes N MGL lock objects, where N is an integer greater than 1.

[0137] In a possible implementation, each of the N MGL lock objects in the array of MGL lock objects has a unique index.

[0138] In a possible implementation, indexes of the N MGL lock objects are in a range of [0, N-l].

[0139] In a possible implementation, each of the multiple nodes is mapped to the corresponding MGL lock object based on a reminder after dividing the hash code value of the path corresponding to the respective node by N.

[0140] In a possible implementation, the processing module is specifically configured to hold locks on the multiple MGL lock objects for the operation request, where lock modes of the locks to be held on the MGL lock objects are determinedaccording to the operation request.

[0141] In a possible implementation, in a case that a lock has been held on a first MGL lock object of the multiple MGL lock objects, and a lock mode of a lock to be held on the first MGL lock object conflicts with a lock mode of the lock that has been held on the first MGL lock object, the locks are not held on the multiple MGL lock objects for the operation request until the lock that has been held on the first MGL is released.

[0142] In a possible implementation, in a case that the operation request indicates an operation affecting one target node, and the operation is a read operation, an S lock is held on an MGL lock object corresponding to the affected target node, and an IS lock is held on each of one or more MGL lock objects corresponding to the one or more parent nodes of the affected target node.

[0143] In a possible implementation, in a case that the operation request indicates an operation affecting one target node, and the operation is a write operation; an X lock is held on an MGL lock object corresponding to the affected target node, and an IX lock is held on each of one or more MGL lock objects corresponding to the one or more parent nodes of the affected target node.

[0144] In a possible implementation in a case that the operation request indicates an operation affecting two or more target nodes, the multiple MGL lock objects include one or more second MGL lock objects for each of which one lock mode is required, and one or more third MGL lock objects for each of which two or more lock modes are required, and a lock mode of a lock to be held on each of the one or more second MGL lock objects is determined to be the lock mode required for the respective second MGL lock object, and a lock mode of a lock to be held on each of the one or more third MGL lock objects is determined based on a greatest lock mode of the two or more lock modes required for the respective third MGL lock object.

[0145] In a possible implementation, in a case that a target node and one or more parent nodes of the target node include two or more nodes that are mapped to a same MGL lock object, a lock mode of a lock to be held on the same MGL lock object isdetermined based on lock modes required for the two or more nodes.

[0146] An embodiment of the present disclosure provides a locking apparatus including processing circuitry for executing the locking method for locking a directory tree described above.

[0147] An embodiment of the present disclosure provides a file system including the locking apparatus as described with reference to FIG. 10 or the locking apparatus including processing circuitry for executing the locking method for locking a directory tree described above.

[0148] An embodiment of the present disclosure provides a file system including a metadata management component for executing the locking method for locking a directory tree described above.

[0149] An embodiment of the present disclosure provides a computer-readable storage medium storing computer execution instructions which, when executed by a processor, cause the processor to execute the locking method for lock a directory tree described above.

[0150] An embodiment of the present disclosure provides a computer program product including computer execution instructions which, when executed by a processor, cause the processor to execute the locking method for locking a directory tree described above.

[0151] The embodiments set forth herein represent information sufficient to practice and illustrate ways of practicing such subject matter. Upon reading the description in light of the accompanying figures, those of skill in the art will understand the concepts of the claimed subject matter and will recognize applications of these concepts not particularly addressed herein. It should be understood that these concepts and applications fall within the scope of the disclosure and the accompanying claims.

[0152] Moreover, it will be appreciated that any module, component, or device disclosed herein that executes instructions may include, or otherwise have access to, a non-transitory computer / processor readable storage medium or media for storage of information, such as computer / processor readable instructions, data structures, programmodules and / or other data. A non-exhaustive list of examples of non-transitory computer / processor readable storage media includes magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, optical disks such as compact disc read-only memory (CD-ROM), digital video discs or digital versatile discs (i.e., DVDs), Blu-ray Disc™, or other optical storage, volatile and non-volatile, removable and non-removable media implemented in any method or technology, random-access memory (RAM), read-only memory (ROM), electrically erasable programmable readonly memory (EEPROM), flash memory or other memory technology. Any such non- transitory computer / processor storage media may be part of a device / apparatus or accessible or connectable thereto. Computer / processor readable / executable instructions to implement a method, an application or a module described herein may be stored or otherwise held by such non-transitory computer / processor readable storage media.

[0153] In some aspects of the present disclosure, there is provided a non-transitory computer-readable medium storing instructions, the instructions, when executed by a processor, may cause the processor to implement the method of the present disclosure.

[0154] In some aspects of the present disclosure, there is provided an apparatus / chipset system including at least one processor executing instructions stored in a computer-readable medium to implement the method of the present disclosure.

[0155] The present disclosure encompasses various embodiments, including not only method embodiments, but also other embodiments such as apparatus embodiments and embodiments related to non-transitory computer readable storage media. Embodiments may incorporate, individually or in combinations, the features disclosed herein.

[0156] Although this disclosure refers to illustrative embodiments, this is not intended to be construed in a limiting sense. Various modifications and combinations of the illustrative embodiments, as well as other embodiments of the disclosure, will be apparent to persons skilled in the art upon reference to the description.

[0157] Features disclosed herein in the context of any particular embodiments may also or instead be implemented in other embodiments. Method embodiments, forexample, may also or instead be implemented in apparatus, system, and / or computer program product embodiments. In addition, although embodiments are described primarily in the context of methods and apparatus, other implementations are also contemplated, as instructions stored on one or more non-transitory computer-readable media, for example. Such media could store programming or instructions to perform any of various methods consistent with the present disclosure.

[0158] Further, in the disclosure, the word “a” or “an” when used in conjunction with the term “including” or “including” in the claims and / or the specification may mean “one”, but it is also consistent with the meaning of “one or more”, “at least one”, and “one or more than one” unless the content clearly dictates otherwise. Similarly, the word “another” may mean at least a second or more unless the content clearly dictates otherwise. The term “and / or” herein when used in association with a list of items means any one or more of the items including that list.

Claims

CLAIMS1. A locking method for locking a directory tree of a file system, implemented by a locking apparatus, comprising: obtaining an operation request; and performing, according to the operation request, a multiple-granularity locking (MGL) operation on multiple MGL lock objects in an array of MGL lock objects, wherein the array of MGL lock objects comprises a plurality of MGL lock objects used for recording locking status of nodes in the directory tree, and the multiple MGL lock objects correspond to multiple nodes in the directory tree, wherein the multiple nodes comprise at least one target node of the operation request and one or more parent nodes of the at least one target node.

2. The locking method according to claim 1, wherein each node in the directory tree corresponds to a respective path in the file system, and each of the multiple nodes is mapped to a corresponding MGL lock object based on information of a path corresponding to the respective node.

3. The locking method according to claim 2, wherein each of the multiple nodes is mapped to the corresponding MGL lock object based on a hash code value of the path corresponding to the respective node.

4. The locking method according to claim 3, wherein the hash code value is obtained by using a hashing algorithm on at least one string related to the path.

5. The locking method according to claim 3 or 4, wherein the array of MGL lock objects comprises N MGL lock objects, wherein N is an integer greater than 1.

6. The locking method according to claim 5, wherein each of the N MGL lock objects in the array of MGL lock objects has a unique index.

7. The locking method according to any one of claims 1 to 6, wherein the performing, according to the operation request, the MGL operation on the multiple MGL lock objects in the array of MGL lock objects comprises:holding locks on the multiple MGL lock objects for the operation request, wherein lock modes of the locks to be held on the MGL lock objects are determined according to the operation request.

8. The locking method according to claim 7, wherein in a case that a lock has been held on a first MGL lock object of the multiple MGL lock objects, and a lock mode of a lock to be held on the first MGL lock object conflicts with a lock mode of the lock that has been held on the first MGL lock object, the locks are not held on the multiple MGL lock objects for the operation request until the lock that has been held on the first MGL is released.

9. The locking method according to any of claims 1 to 8, wherein in a case that the operation request indicates an operation affecting one target node, and the operation is a read operation, a share (S) lock is held on an MGL lock object corresponding to the affected target node, and an intention shared (IS) lock is held on each of one or more MGL lock objects corresponding to the one or more parent nodes of the affected target node.

10. The locking method according to any of claims 1 to 9, wherein in a case that the operation request indicates an operation affecting one target node, and the operation is a write operation, an exclusive (X) lock is held on an MGL lock object corresponding to the affected target node, and an intention exclusive (IX) lock is held on each of one or more MGL lock objects corresponding to the one or more parent nodes of the target node.

11. The locking method according to any one of claims 1 to 10, wherein in a case that the operation request indicates an operation affecting two or more target nodes, the multiple MGL lock objects comprise one or more second MGL lock objects for each of which one lock mode is required, and one or more third MGL lock objects for each of which two or more lock modes are required, and a lock mode of a lock to be held on each of the one or more second MGL lock objects is determined to be the lock mode required for the respective second MGL lock object, and a lock mode of a lock to be held on each of the one or more third MGL lock objects is determined based on agreatest lock mode of the two or more lock modes required for the respective third MGL lock object.

12. The locking method according to any one of claims 1 to 11, wherein in a case that a target node and one or more parent nodes of the target node comprise two or more nodes that are mapped to a same MGL lock object, a lock mode of a lock to be held on the same MGL lock object is determined based on lock modes required for the two or more nodes.

13. A locking apparatus comprising: a obtaining module, configured to obtain an operation request; and a processing module, configured to perform, according to the operation request, a multiple-granularity locking (MGL) operation on multiple MGL lock objects in an array of MGL lock objects, wherein the array of MGL lock objects comprises a plurality of MGL lock objects used for recording locking status of nodes in the directory tree, and the multiple MGL lock objects correspond to multiple nodes in the directory tree, wherein the multiple nodes comprise at least one target node of the operation request and one or more parent nodes of the at least one target node.

14. The locking apparatus according to claim 13, wherein each node in the directory tree corresponds to a respective path in the file system, and each of the multiple nodes is mapped to a corresponding MGL lock object based on information of a path corresponding to the respective node15. The locking apparatus according to claim 14, wherein each of the multiple nodes is mapped to the corresponding MGL lock object based on a hash code value of the path corresponding to the respective node.

16. The locking apparatus according to claim 15, wherein the hash code value is obtained by using a hashing algorithm on at least one string related to the path.

17. The locking apparatus according to claim 15 or 16, wherein the array of MGL lock objects comprises N MGL lock objects, wherein N is an integer greater than 1.

18. The locking apparatus according to claim 17, wherein each of the N MGL lock objects in the array of MGL lock objects has a unique index.

19. The locking apparatus according to any one of claims 13 to 18, wherein the processing module is configured to: hold locks on the multiple MGL lock objects for the operation request, wherein lock modes of the locks to be held on the MGL lock objects are determined according to the operation request.

20. The locking apparatus according to claim 19, wherein in a case that a lock has been held on a first MGL lock object of the multiple MGL lock objects, and a lock mode of a lock to be held on the first MGL lock object conflicts with a lock mode of the lock that has been held on the first MGL lock object, the locks are not held on the multiple MGL lock objects for the operation request until the lock that has been held on the first MGL is released.

21. The locking apparatus according to any of claims 13 to 20, wherein in a case that the operation request indicates an operation affecting one target node, and the operation is a read operation, a share (S) lock is held on an MGL lock object corresponding to the target node, and an intention shared (IS) lock is held on each of one or more MGL lock objects corresponding to the one or more parent nodes of the target node.

22. The locking apparatus according to any of claims 13 to 21, wherein in a case that the operation request indicates an operation affecting one target node, and the operation is a write operation, an exclusive (X) lock is held on an MGL lock object corresponding to the target node, and an intention exclusive (IX) lock is held on each of one or more MGL lock objects corresponding to the one or more parent nodes of the target node.

23. The locking apparatus according to any one of claims 13 to 22, wherein in a case that the operation request indicates an operation affecting two or more target nodes, the multiple MGL lock objects comprise one or more second MGL lock objects for each of which one lock mode is required, and one or more third MGL lock objects for each of which two or more lock modes are required, and a lock mode of a lock to be held on each of the one or more second MGL lock objects is determined to be the lock moderequired for the respective second MGL lock object, and a lock mode of a lock to be held on each of the one or more third MGL lock objects is determined based on a greatest lock mode of the two or more lock modes required for the respective third MGL lock object.

24. The locking apparatus according to any one of claims 13 to 23, wherein in a case that a target node and one or more parent nodes of the target node comprise two or more nodes that are mapped to a same MGL lock object, a lock mode of a lock to be held on the same MGL lock object is determined based on lock modes required for the two or more nodes.

25. A locking apparatus, comprising a processing circuitry for executing the locking method for locking a directory tree according to any one of claims 1 to 12.

26. A file system comprising a metadata management component for executing the locking method for locking a directory tree according to any one of claims 1 to 12.

27. A computer-readable storage medium storing computer execution instructions which, when executed by a processor, cause the processor to execute the locking method for lock a directory tree according to any one of claims 1 to 12.

28. A computer program product including computer execution instructions which, when executed by a processor, cause the processor to execute the locking method for locking a directory tree according to any one of claims 1 to 12.

Images