Distributed file system renaming operation method based on read-write lock ring detection mechanism

Abstract

The present invention discloses a distributed file system renaming operation method based on a read-write lock ring detection mechanism. A central node uniformly processes all ring detection tasks of renaming operations. After receiving a renaming operation request, the central node performs ring detection in a read-write lock manner. The present invention uses a lightweight distributed file system renaming operation ring detection mechanism, which can effectively avoid the occurrence of orphan ring data loss, and effectively improves the concurrency and throughput of the renaming operation through a more fine-grained locking scheme mechanism.

Description

Technical Field

[0001] The present invention relates to the technical field of computer software, and in particular to a distributed file system renaming operation method based on a read-write lock ring detection mechanism. Background Art

[0002] With the rapid development of the Internet, the industry has generated business needs with huge amounts of data, such as big data and training large models. The amount of business data involved is rising rapidly. Improving the storage capacity on a single machine is far from meeting the rapid growth of business data today. Therefore, the industry has changed its goal from increasing the storage capacity of a single machine to increasing the number of machines to increase the storage capacity. Multiple machines are used to store the amount of data required for the business. This is distributed storage, such as distributed file systems, distributed databases, etc. In a single-machine storage system, all functional operations are not complicated, because all read and write operations are performed on a single machine, and do not involve read and write consistency issues of other nodes. In a distributed operating environment, due to the fault tolerance mechanism, multiple copies need to be stored on multiple machines, which will lead to data consistency issues for multiple copies in the production environment; at the same time, with the increase in the number of users, the storage system not only needs to expand the storage capacity, but also needs to meet the parallel read and write requests of multiple clients to achieve high throughput and low latency performance requirements.

[0003] In a distributed file system, the upper layer provides developers with an interface for reading and writing files, and encapsulates the processing of multiple copies and transaction consistency at the bottom layer, which can facilitate developers' maintenance costs. Among the read and write interfaces provided by the file system to the upper layer, the rename operation is relatively complex. It includes path resolution of files or directories, locking of modified data, and changes to data. Therefore, the rename operation needs to be encapsulated as a transaction at the bottom layer to ensure the atomicity of the rename operation; at the same time, under the demand for concurrent reading and writing by multiple clients, the concurrency of the rename operation needs to be guaranteed as much as possible. Therefore, one challenge in implementing the rename operation is that when designing the locking scheme for the rename operation, it is necessary to meet the atomicity of the rename operation during the operation, and to meet the concurrency of multiple threads as much as possible to meet the performance requirements of high throughput and low latency.

[0004] Another requirement is that the rename operation must not only meet the atomicity of the transaction during the concurrent process, but also prevent the orphan loop problem. To put it simply, when two or more clients operate on the directory tree of the file directory at the same time, a loop may occur between the directories, that is, starting from a directory, continuously searching for the parent directory upwards, and finally finding the directory itself. If this happens, the path resolution operation of the distributed file system cannot find all the data of the directory and its subdirectories, which will cause data loss and seriously affect the user's data security.

[0005] In previous industry papers, it is generally believed that rename operations account for a small proportion of the operations in the entire distributed file system. Therefore, when considering the performance of distributed file systems, the performance requirements of rename operations are often ignored. However, in the industry's latest big data workloads, rename operation bursts may occur in distributed file systems. For a period of time, there may be a high peak of rename operation requests and long duration.

[0006] There are four main processing methods in the rename operation locking mechanism currently used in the industry:

[0007] (1) The traditional approach of the earliest distributed file systems is to add a global shared lock to all rename operations in the distributed file system. For example, the Linux kernel, HDFS, Lustre and other distributed file systems. In this case, the rename operation in the distributed file system cannot be concurrent at all.

[0008] (2) The source directory of the rename operation and all directories and files in the subtree under the directory are locked. This method can effectively prevent the formation of orphan loops. Rename operations without parent-child directory relationships can be concurrent. However, since this locking method locks all subdirectories, the locking overhead increases with the size of the subtree. Taking HopsFS data as an example, this locking method requires sub-second overhead. LocoFS also uses this locking method to ensure data security during concurrent reading and writing of rename operations.

[0009] (3) Currently, the industry can also add a central node to complete the task of ring detection to complete the rename operation ring detection. All rename operation requests must pass through the same central node to detect the feasibility of concurrent rename operations. However, due to the limited throughput of the central node and the locking, the concurrency of the rename operation is still very low. For example, the central node of Baidu's distributed file system CFS will obtain the least common ancestor node of the source node and the destination node, and then lock both the least common ancestor node-source node and the least common ancestor node-destination node. This coarse-grained locking method will make most of the rename operations under the least common ancestor node and its descendant nodes unable to be concurrent when locked. CephFS uses the metadata server MDS to detect rings, which will also increase performance overhead. Some distributed file systems may introduce broadcasts, further increasing performance overhead, such as InfiniFS.

[0010] (4) Some distributed file systems completely abandon the central check of rename operations, such as JuiceFS. Although the concurrency is greatly improved, data loss is likely to occur when dangerous concurrent rename operations occur.

[0011] In view of this, the present invention is proposed. Summary of the invention

[0012] The purpose of the present invention is to provide a distributed file system renaming operation method based on a read-write lock ring detection mechanism, which uses a lightweight locking method to improve the concurrency of the renaming operation and improve the overall cluster throughput while ensuring data security and data correctness.

[0013] The objective of the present invention is achieved through the following technical solutions:

[0014] A distributed file system renaming operation method based on a read-write lock ring detection mechanism, comprising:

[0015] The distributed file system sends all received rename operation requests to the central node;

[0016] After receiving the rename operation request, the central node performs loop detection in the form of a read-write lock. The central node uses the access metadata stored in itself to determine the least common ancestor node of the rename operation source node and the destination node through path resolution, and creates a write lock on the least common ancestor node; then, from the least common ancestor node to the destination node, in a series of nodes, in order from top to bottom, read locks are created for other nodes except the least common ancestor node; then a write lock is created for the source node, and the write lock of the least common ancestor node is released;

[0017] A rename transaction request is sent to the node storing the directory entry of the metadata storage engine of the lower layer, and the rename transaction process is executed. After the rename transaction is executed, all read-write locks added in the previous locking process are released.

[0018] It can be seen from the technical solution provided by the present invention that the use of a lightweight distributed file system renaming operation ring detection mechanism can effectively avoid the occurrence of orphan ring data loss, and through a finer-grained locking scheme mechanism, the concurrency and throughput of the renaming operation are effectively improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative work.

[0020] Figure 1 A flowchart of a distributed file system renaming operation method based on a read-write lock ring detection mechanism provided by an embodiment of the present invention;

[0021] Figure 2 A schematic diagram of orphan ring formation provided by an embodiment of the present invention;

[0022] Figure 3 A schematic diagram of the overall storage architecture of a distributed file system provided by an embodiment of the present invention;

[0023] Figure 4 A schematic diagram of decoupling metadata provided by an embodiment of the present invention into content metadata and access metadata;

[0024] Figure 5 A schematic diagram of a HashMap-based memory lock structure provided in an embodiment of the present invention;

[0025] Figure 6 A schematic diagram of the correctness of the ring detection of the minimum common ancestor node provided by an embodiment of the present invention;

[0026] Figure 7 A schematic diagram of concurrent execution of rename transactions provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0027] The following is a clear and complete description of the technical solutions in the embodiments of the present invention in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the protection scope of the present invention.

[0028] First, the terms that may be used in this article are explained as follows:

[0029] The terms "include", "comprises", "contains", "has" or other descriptions with similar semantics should be interpreted as non-exclusive inclusion. For example, including certain technical feature elements (such as raw materials, components, ingredients, carriers, dosage forms, materials, dimensions, parts, components, mechanisms, devices, steps, procedures, methods, reaction conditions, processing conditions, parameters, algorithms, signals, data, products or products, etc.) should be interpreted as including not only certain technical feature elements explicitly listed, but also other technical feature elements known in the art that are not explicitly listed.

[0030] The term "consisting of..." means excluding any technical feature elements not explicitly listed. If this term is used in a claim, it will make the claim closed, so that it does not contain technical feature elements other than the technical feature elements explicitly listed, except for the conventional impurities related to them. If this term only appears in a clause of a claim, it only limits the elements explicitly listed in the clause, and the elements recorded in other clauses are not excluded from the overall claim.

[0031] The following is a detailed description of a distributed file system renaming operation method based on a read-write lock ring detection mechanism provided by the present invention. The contents not described in detail in the embodiments of the present invention belong to the prior art known to professional and technical personnel in the field. If no specific conditions are specified in the embodiments of the present invention, they shall be carried out according to the conventional conditions in the field or the conditions recommended by the manufacturer.

[0032] like Figure 1 As shown, a distributed file system renaming operation method based on a read-write lock ring detection mechanism mainly includes the following steps:

[0033] Step 1: The distributed file system sends all received rename operation requests to the central node.

[0034] In the embodiment of the present invention, the distributed file system parses all rename operation requests received from the client and sends them to the central node of the metadata storage engine, so that all rename operations will be subjected to ring detection on the same central node.

[0035] Step 2: After receiving the rename operation request, the central node performs a ring detection in a read-write lock manner.

[0036] In the embodiment of the present invention, the central node of the metadata storage engine uses a lightweight lock mechanism to perform ring detection on the rename operation request and avoid the generation of orphan rings.

[0037] (1) The central node uses the access metadata stored in itself to determine the lowest common ancestor node of the renaming operation source node and destination node through path resolution.

[0038] In the embodiment of the present invention, the memory of the central node stores the access metadata of the entire distributed file system, including the tree structure relationship and read-write permissions between directories. During the renaming process, the distributed file system needs to perform path resolution on the parameters of the renaming. During the path resolution process, it is necessary to perform permission checks on the user who performs the renaming operation. If a directory does not have the read-write permissions of the current user, the renaming operation will be terminated. Therefore, the read-write permissions of the directory need to be stored in the central node, and the distributed file system's requests for directory access and modification need to be executed through the central node. The central node splits the parameters in the renaming operation request into a list of directory names according to the level, and then matches the directory names of each level to find the minimum common ancestor node; if the directory names of the first-level nodes are not the same, the minimum common ancestor node is set as the root node.

[0039] (2) Parse the least common ancestor node according to the parameters in the rename operation request, and write a lock on the least common ancestor node in memory to prevent deadlock that may occur during the locking process.

[0040] In the embodiment of the present invention, the parameter in the rename operation request is the path of the file. For example, in the rename operation request rename( / A / B, / A / C), / A / B and / A / C are the file paths and also the parameters of the rename operation.

[0041] In an embodiment of the present invention, the memory of the central node stores access metadata of the distributed file system, that is, the inode number of the parent directory (that is, the number of the node used to index the directory and file metadata in the distributed file system), the directory name of the current directory and the access rights. The central node accesses its own memory without using network communication, and continuously accesses the least common ancestor node from the root node downward in a recursive order. If it is found that the least common ancestor node does not exist or is locked, an error is reported and the renaming operation is abandoned; if the least common ancestor node is successfully parsed, a hash table is created in the memory to record the mapping between the directory node and the read-write lock, and a new write lock (that is, an exclusive lock) is created and stored in the hash table.

[0042] (3) In a series of nodes from the least common ancestor node to the destination node, read locks are created for other nodes except the least common ancestor node in order from top to bottom. That is, the least common ancestor node will not be locked in this process. This process is to detect whether the ancestor node of the destination node has a source node involved in other renaming operations.

[0043] In an embodiment of the present invention, the central node starts from the least common ancestor node, and sequentially queries the inode number (metadata index node number) and read / write permissions of each node in the node sequence of the destination path (i.e., the path from the least common ancestor node to the destination node), and establishes a read lock (i.e., a shared lock) for the corresponding node according to the hash table of the inode number (metadata index node number) in the memory. If the corresponding node does not exist, it means that the corresponding node has been changed. At this time, the node on the least common ancestor is released, and all added read locks are released, and the renaming operation is abandoned, and the information that the renaming operation failed is replied to the upper layer; if a write lock (i.e., an exclusive lock) already exists on the corresponding node in the hash table, the write lock of the corresponding node is blocked and waits for the release of the write lock; in this case, the write lock state is maintained. Since the write lock already exists, it proves that other renaming operations have locked it here, so it is necessary to wait for other operations to release the lock before performing the renaming operation.

[0044] (4) Create a write lock for the source node.

[0045] In an embodiment of the present invention, the central node recursively parses the path of the source node (the path from the lowest common ancestor node to the source node) from the memory, and when the last node of the source node path, i.e., the source node, is found, a write lock is created for the source node in the hash table in the memory.

[0046] (5) Release the write lock of the least common ancestor node.

[0047] The main purpose of the write lock on the least common ancestor node is to avoid deadlock caused by two r rename operations occurring simultaneously, which may produce an orphan ring.

[0048] Step 3: The central node sends a rename transaction request to the persistent storage dentry node (i.e., the node that stores directory entries) of the metadata storage engine of the lower layer, executes the rename transaction process, and after completing the rename transaction, releases all read-write locks added in the previous locking process.

[0049] In an embodiment of the present invention, metadata is decoupled into access metadata and content metadata. The central node mainly stores access metadata in memory, while the persistent storage dentry node (i.e., the node storing directory entries) mainly stores access metadata and content metadata persistently in SSD (SolidState Disk or Solid State Drive). To execute a rename transaction, it is necessary to atomically modify the access metadata of the source node and the destination node and the content metadata of their respective parent directories. Since such a modification involves the reading and writing of multiple shards and the central node and other persistent nodes store access metadata at the same time, the entire rename operation needs to be executed with a transaction interface to ensure the atomicity of the operation.

[0050] In the embodiment of the present invention, during the process of performing ring detection in a read-write lock manner, all nodes created with read locks and write locks are stored in a sequence, and after the rename transaction is completed, the read locks and write locks of all nodes in the sequence are released.

[0051] The above scheme of the embodiment of the present invention adds a write lock on the least common ancestor node, uses a read lock on the destination node path, adds a write lock on the source node, and finally releases the write lock on the least common node. Compared with the ring detection of the rename operation in the traditional distributed file system, the concurrency of the rename operation is improved while ensuring the data security of the rename operation, and the related performance of the entire cluster is improved.

[0052] In order to more clearly demonstrate the technical solution and technical effects provided by the present invention, a distributed file system renaming operation method based on a read-write lock ring detection mechanism provided by an embodiment of the present invention is described in detail.

[0053] 1. Analysis of the causes of distributed file system ring detection.

[0054] Distributed file systems use files and directories to manage data storage, and adopt a hierarchical namespace, which is conducive to operating smaller files. However, since its metadata management method is more complex than block storage and object storage, it is necessary to maintain the data security of the distributed file system when performing distributed file system operations.

[0055] Figure 2 The reason for the formation of orphan loops is given. The renaming operation includes moving a file or directory and the subtree under the directory to another directory. When two or more renaming operations are executed concurrently in a hierarchical namespace, if the source node of each renaming operation is moved to the descendant node under the source node of other renaming operations.

[0056] like Figure 2As shown, node G is the child node of node F, node D is the child node of node B, and node E is the child node of node C. When the three renaming operations of rename( / A / F, / A / B / D / F), rename( / A / B, / A / C / E / B), and rename( / A / C, / A / F / G / C) occur simultaneously, that is, node F is moved to node D as the child node of D, node B is moved to node E as the child node of E, and node C is moved to node G as the child node of g, these six nodes will be connected front and back to form a ring structure. This will cause the search starting from the root node to fail to find the metadata in these six nodes. From the client's perspective, the directories represented by these six nodes no longer exist. Although the user has not performed any deletion operations, the directory data still exists in the data storage engine of the distributed file system. However, due to the orphan ring phenomenon of metadata in the tree structure, all data of these six nodes and their child nodes will be lost.

[0057] The key to preventing orphan loops is to prevent the source directory nodes of other renaming operations from appearing on the path from the least common ancestor node of the source node and the destination node to the destination node during the renaming operation.

[0058] 2. Distributed file system metadata storage solution.

[0059] The overall storage architecture of the distributed file system is as follows: Figure 3 As shown in the figure, it is divided into the following parts. The client of the distributed file system provides POSIX, HDFS and Object interfaces to developers to meet the needs of developers for file storage and object storage. At the bottom of the distributed file system client, it is divided into metadata storage engine and data storage engine. According to the needs of the distributed file system, the data engine connects to the bottom file storage or object storage. Figure 3 In the example, ns (namespace) refers to the access metadata of different namespaces stored in the central node, and shard refers to the metadata shard stored in the persistent storage node.

[0060] When the client is ready to read or write a file, it must first initiate a metadata request to the metadata server to obtain metadata read and write permissions and location information, and then initiate a data request to the data server to read and write data.

[0061] The metadata storage engine used in the distributed file system adopts a central node approach. The distributed file system decouples metadata into content metadata and access metadata. Figure 4The access metadata includes the inode number of the parent directory of the directory, the directory name of the directory, the inode of the directory itself, and the read and write permissions of the directory itself. The content metadata includes the dentry list (i.e., the list of directory entries) of its subdirectories, the timestamp of the directory, and the number of links to the directory. All metadata operations must be resolved through path resolution before the node that the interface needs to operate can be resolved.

[0062] In the embodiment of the present invention, the central node stores the access metadata part, which speeds up the path resolution of files and directories, so that the path resolution does not need to go through multiple RPCs (remote procedure calls), and improves the performance of distributed file system operations. The central node also undertakes the task of renaming operation ring detection, and uses a lightweight locking method to complete the ring detection of the renaming operation.

[0063] The distributed file system metadata storage engine also deploys other nodes to persistently store all metadata, including access metadata and content metadata. Therefore, when performing a rename operation in the distributed file system, it is necessary to modify the access metadata of the central node and the related metadata in the persistent storage at the same time. Such operations need to provide POSIX atomicity upward, so transactions must be used to ensure the atomicity of the metadata operation.

[0064] 3. Read-write lock based on HashMap storage memory.

[0065] The method for detecting a rename ring in a hierarchical namespace of a distributed file system in an embodiment of the present invention is implemented based on a read-write lock in a memory. Figure 5 As shown, the memory lock is based on the mapping between the HashMap (hash map) storage directory inode and the address of the read-write lock on the directory. When performing a rename operation in a distributed file system, in order to ensure thread safety, the read-write lock in the HashMap is used to perform the ring detection process. The inode number (metadata index node number) is a unique identifier used to identify files in a distributed file system.

[0066] The C++ read-write lock of the embodiment of the present invention is a multi-thread synchronization mechanism that allows multiple threads to obtain read access at the same time, but only allows one thread to have exclusive write access. The design of this lock helps to improve concurrency performance because multiple threads can read shared data at the same time, and only when writing is needed, an exclusive lock is required.

[0067] When the read-write lock is in the write-locked state, all threads trying to lock it will be blocked before it is unlocked; when the read-write lock is in the read-locked state, all threads trying to lock it in read mode can get access, but if you want to lock it in write mode, the thread will be blocked.

[0068] In the example of the present invention, when exclusive locking is required, std::unique_lock is used to manage the exclusive lock, and std::shared_lock is used to obtain the shared lock.

[0069] like Figure 5 After the central node receives rename( / A / B / C, / A / D / E / C), it will store A, C, D, and E in the HashMap in memory to maintain an independent read-write lock for each directory. The rename operation will also retain the sequence of all locked nodes in the operation, with each node inode as an element in the sequence, that is, [A_inode, C_inode, D_inode, E_inode], so as to find the previous lock sequence in the HashMap when releasing. Among them, A and C will perform exclusive locks, and D and E will perform shared locks. Before executing the rename transaction, the exclusive lock on the least common ancestor node, that is, node A, will be released. After the rename operation transaction is completed, the added locks will be released in sequence according to the previously saved lock sequence.

[0070] 4. Correctness of ring detection based on the least common ancestor node.

[0071] In the distributed file system central node renaming ring detection algorithm, the least common ancestor node can ensure the correctness of the ring detection algorithm. When two or more renaming operations that may form a ring may occur, adding an exclusive lock on the least common ancestor node can prevent these renaming operations that may generate orphan rings from causing data security issues. Specifically, the exclusive lock added to the least common ancestor node can prevent multiple renaming operations that may generate orphan rings from deadlocking.

[0072] From the analysis of the causes of orphan rings, it can be seen that adding a shared lock on the path to the destination node can prevent multiple renaming operations that may cause orphan rings from occurring concurrently. However, when multiple renaming operations occur at the same time, locking the destination node path and the source node may cause deadlock. Adding an exclusive lock on the least common ancestor node of each renaming operation can effectively prevent deadlock.

[0073] like Figure 6 As shown, assume that n rename operations that may cause orphan loops occur simultaneously (n=5 in the example). According to the design of the rename loop detection algorithm, n rename operations will generate n least common ancestor nodes. It will be proved below that two of these n least common ancestor nodes must be the same.

[0074] First, since the n renaming operations each generate a destination node and a source node, there will be a total of n source nodes and n destination nodes. And since it is known that the n renaming operations may form an orphan loop, each of the n destination nodes will have a corresponding node in the n source nodes as its ancestor node, such as Figure 6 The G in the destination node corresponds to the B in the source node.

[0075] The n rename operations generate a total of n least common ancestor nodes. According to the properties of the least common ancestor node, each least common ancestor node has at least one source node and one destination node on both sides, and each source node has a destination node as its descendant node, so each least common ancestor node has at least one destination node on both sides. However, according to the drawer principle, if there is at least one destination node on both sides of n two different least common ancestor nodes, there will be at least n+1 destination nodes as their leaf nodes. Therefore, the original hypothesis does not hold. Among the n least common ancestor nodes corresponding to the n rename operations that may concurrently generate orphan rings, there must be two identical nodes.

[0076] Therefore, it can be proved that in the locking link, when n renaming operations enter the ring detection phase concurrently, they will compete for the exclusive locks on the n-1 least common ancestor nodes. There must be one renaming operation that cannot enter the ring detection phase at the same time. Therefore, it can be guaranteed that the n renaming operations will not be deadlocked, and the data correctness of the renaming operation ring detection concurrent operations is guaranteed.

[0077] 5. Concurrency of lightweight ring detection method.

[0078] The lightweight ring detection method uses a lightweight lock method in the ring detection phase to avoid concurrent execution of renaming operations that may cause orphan rings. In the transaction execution phase of the renaming operation, the exclusive lock on the least common ancestor node is released to improve the concurrency of the entire distributed file system.

[0079] In the embodiment of the present invention, Figure 7 As shown in the figure, if rename( / A / B / G, / A / C / E / J / G) and rename( / A / C / D, / A / B / F / I / D) occur at the same time, serial execution is required only when adding memory lock in the ring detection phase (because both renaming operations need to compete for the write lock of the least common ancestor node A). However, because the overhead of memory lock is very small, the serial execution of this phase is not the bottleneck of the performance of the entire renaming operation.

[0080] After the ring detection phase is over, when the transaction execution layer performs the rename operation, the two can be executed concurrently, which greatly improves the concurrency and performance of the rename operation ring detection.

[0081] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above embodiments can be implemented by software, or by means of software plus necessary general hardware platforms. Based on such understanding, the technical solutions of the above embodiments can be embodied in the form of software products, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.), including several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in the various embodiments of the present invention.

[0082] The above is only a preferred specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed in the present invention should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. A distributed file system renaming operation method based on a read-write lock ring detection mechanism, characterized in that: include: The distributed file system sends all received rename operation requests to the central node; After receiving the rename operation request, the central node performs loop detection in the form of a read-write lock, including: the central node uses the access metadata stored in itself to determine the least common ancestor node of the rename operation source node and the destination node through path resolution, and creates a write lock on the least common ancestor node; then, from the least common ancestor node to the destination node, in a series of nodes, in order from top to bottom, read locks are created for other nodes except the least common ancestor node; then, a write lock is created for the source node, and the write lock of the least common ancestor node is released; Send a rename transaction request to the node storing the directory entry of the metadata storage engine at the lower layer, execute the rename transaction process, and after the rename transaction is completed, release all read-write locks added in the previous locking process; The central node determines the least common ancestor node of the source node and the destination node of the rename operation by path resolution using the access metadata stored in the central node, including: the access metadata of the distributed file system is stored in the memory of the central node, including the tree structure relationship and read and write permissions between directories; the central node splits the parameters in the rename operation request into a directory name list according to the level, and then matches the directory names of each level to find the least common ancestor node; if the directory names of the first-level nodes are different, the least common ancestor node is set as the root node; The step of creating read locks for other nodes except the least common ancestor node in a series of nodes from the least common ancestor node to the destination node in order from top to bottom includes: starting from the least common ancestor node, the central node sequentially queries the inode number and read / write permissions of each node in the node sequence of the destination path, and establishes a read lock for the corresponding node according to the hash table of the inode number in the memory, wherein the inode number is the metadata index node number, and if the corresponding node does not exist, it means that the corresponding node has been changed, and at this time, the node on the least common ancestor is released, and all read locks added at this time are released, and the renaming operation is abandoned, and the information that the renaming operation failed is replied to the upper layer; if a write lock already exists on the corresponding node in the hash table, the corresponding node is blocked and waits for the release of the write lock; wherein the destination path is the path from the least common ancestor node to the destination node; The process of executing a rename transaction includes: decoupling metadata into access metadata and content metadata, the central node saves the access metadata, and persistently stores the access metadata and content metadata in the node storing the directory entry; executing a rename transaction requires atomically modifying the access metadata of the source node and the destination node and the content metadata of their respective parent directories, and executing the rename operation with a transaction interface.

2. According to claim 1, a distributed file system renaming operation method based on a read-write lock ring detection mechanism is characterized in that: The step of creating a write lock on the least common ancestor node includes: The central node's memory stores the access metadata of the distributed file system. The central node accesses its own memory without using network communication, and continuously accesses the least common ancestor node from the root node in a recursive order. If the least common ancestor node is found to not exist or is locked, an error is reported and the renaming operation is abandoned. If the least common ancestor node is successfully parsed, a hash table is created in the memory to record the mapping between the directory node and the read-write lock, and a new write lock is created and stored in the hash table.

3. According to the distributed file system renaming operation method based on the read-write lock ring detection mechanism of claim 1, it is characterized in that: The method of creating a write lock for a source node includes: the central node recursively resolves the path from the least common ancestor node to the source node from the memory, and when the source node is found, creates a write lock for the source node in the hash table in the memory; wherein the path of the source node is the path from the least common ancestor node to the source node.

4. According to claim 1, a distributed file system renaming operation method based on a read-write lock ring detection mechanism is characterized in that: After the rename transaction is executed, all read-write locks added in the previous locking process are released, including: In the process of performing ring detection in the form of read-write locks, all nodes that have been created with read locks and write locks are stored in a sequence. After the rename transaction is completed, the read locks and write locks of all nodes in the sequence are released.

Images