Distributed metadata ordered synchronization method based on inode semantics

Abstract

This invention discloses a distributed metadata ordered synchronization method based on inode semantics, belonging to the field of distributed file system technology. This method parses the semantic dependencies of inode metadata operations, constructs a directed acyclic graph of metadata operation dependencies, and groups operations with semantic dependencies into a serial synchronization queue, while operations without dependencies and belonging to different directory tree branches into a parallel synchronization queue. A queue-level lock is allocated to the serial synchronization queue, and ordered synchronization commits are executed through a distributed consensus algorithm. When a cluster anomaly causes synchronization interruption, the operation logs of the master and slave nodes are compared, and the metadata state is verified and repaired according to the dependency order. This invention can ensure the semantic consistency of metadata in a distributed cluster, reduce lock contention during synchronization, improve the concurrent processing capability of metadata operations, and reduce the difficulty of metadata repair in cluster anomaly scenarios.

Description

Technical Field

[0001] This invention relates to the field of distributed file system technology, and in particular to a distributed metadata ordered synchronization method based on inode semantics. Background Technology

[0002] Distributed file systems (DFS), as a core infrastructure for big data storage and computing, are widely used in cloud computing, artificial intelligence, internet services, and other fields, undertaking the tasks of storing, managing, and accessing massive amounts of data. Metadata is a core component of DFS, used to describe the attributes, location, permissions, and other information of files and directories. The consistency and synchronization efficiency of metadata directly determine the overall performance and reliability of the DFS. With the continuous growth of data scale and the increasing concurrency of business operations, DFS generally adopts a multi-replica metadata cluster architecture, ensuring data availability and fault tolerance by synchronizing metadata operations across multiple nodes. Currently, mainstream DFSs implement multi-replica metadata synchronization based on distributed consensus algorithms, using log replication mechanisms to synchronize metadata operations from the master node to the slave nodes, ensuring that the metadata state of all nodes in the cluster remains consistent.

[0003] Most existing distributed file system metadata synchronization methods only synchronize based on the chronological order of operations, failing to fully consider the inherent inode semantic dependencies between metadata operations. This easily leads to metadata semantic inconsistencies such as disordered parent-child relationships between directories and files, abnormal hard link reference counts, and invalid permission inheritance. Furthermore, to ensure data consistency during synchronization, existing methods typically employ a global lock mechanism to serialize all metadata operations, resulting in severe lock contention and significantly reducing the concurrent processing capability of metadata operations. In addition, when abnormal situations such as node failures, network partitions, or master-slave switching cause metadata synchronization interruptions, existing recovery methods mostly require scanning and comparing the entire set of metadata, a time-consuming process that severely impacts the availability and stability of the distributed file system. Summary of the Invention

[0004] The purpose of this invention is to overcome the shortcomings of the prior art and provide a distributed metadata ordered synchronization method based on inode semantics.

[0005] The objective of this invention is achieved through the following technical solution: A distributed metadata ordered synchronization method based on inode semantics is provided, which includes the following steps: S1. Perform semantic dependency parsing on inode metadata operations, identify metadata operation types, including directory creation, directory deletion, file creation, file deletion, hard link creation, hard link deletion, renaming, permission modification, and directory entry update, parse the inode semantic dependency relationship corresponding to each type of metadata operation, including directory parent-child relationship, hard link reference relationship, and permission inheritance relationship, and generate dependency information containing operation ID and preceding dependent operation ID; S2. Use dependency information to construct a directed acyclic graph of metadata operation dependencies, verify that there are no circular dependencies, group metadata operations with inode semantic dependencies into the same serial synchronization queue, and group metadata operations without inode semantic dependencies and belonging to different directory tree branches into different parallel synchronization queues, generate queue IDs and bind operation lists and dependencies. S3. Allocate queue-level locks to the serial synchronization queue, and perform synchronous commits through a distributed consensus algorithm. The serial synchronization queue commits metadata operations in dependency order, while the parallel synchronization queue commits metadata operations in batches. S4. When a node crashes, the network partitions, or the master-slave switch causes a interruption in metadata synchronization, compare the metadata operation logs of the master and slave nodes, verify the metadata operation commit status in the order of dependency, roll back the subsequent operations that were not committed before the preceding operations, and resubmit the incomplete metadata operations.

[0006] Furthermore, step S1 includes the following sub-steps: S1.1. Receive metadata operation requests and extract the operation type identifier and target inode identifier contained in the metadata operation request; S1.2. Identify the metadata operation type based on the operation type identifier, parse the parent directory creation completion dependency corresponding to the directory creation operation and file creation operation, parse the target inode creation completion dependency and the synchronization completion dependency of the directory where the hard link is located corresponding to the hard link creation operation, parse the source file metadata synchronization completion dependency and the target directory metadata synchronization completion dependency corresponding to the rename operation, and parse the target inode creation completion dependency corresponding to the permission modification operation. S1.3. Generate operation IDs for metadata operations, mark all preceding dependent operation IDs corresponding to the metadata operations, and integrate the operation IDs and preceding dependent operation IDs to generate dependency information.

[0007] Furthermore, step S2 includes the following sub-steps: S2.1. Construct a directed acyclic graph of dependencies, using metadata operations as nodes and prior dependencies as directed edges. S2.2. Traverse all nodes and directed edges of the directed acyclic graph of dependencies, detect whether there is a circular dependency in the directed acyclic graph of dependencies, intercept metadata operations with circular dependencies and return an exception response; S2.3. Sort the metadata operations in the serial synchronization queue according to the dependency order. If the preceding metadata operation has not been committed, the subsequent metadata operation will not enter the commit process. S2.4. Generate a queue ID for each serial synchronization queue and each parallel synchronization queue, and persist the queue ID, operation list, and dependencies to the metadata operation log.

[0008] Furthermore, step S3 includes the following sub-steps: S3.1. Allocate a queue-level lock to each serial synchronization queue, and do not allocate locks to parallel synchronization queues; S3.2. The serial synchronization queue initiates synchronization commit requests to the metadata slave nodes in the order of dependency. Only after more than half of the slave nodes have completed the current metadata operation will the synchronization commit request for the subsequent metadata operation be initiated. S3.3. The parallel synchronization queue simultaneously sends batch synchronization commit requests to all metadata slave nodes without waiting for the metadata operations of other parallel synchronization queues to complete. S3.4. After confirming that the metadata operation has been committed by more than half of the slave nodes, return a successful metadata operation response to the client.

[0009] Furthermore, step S4 includes the following sub-steps: S4.1. Detect the cluster running status and trigger the metadata consistency repair process when the cluster experiences node failure, network partition, or master-slave switch causing metadata synchronization interruption. S4.2. Extract the metadata operation logs of the master node and the slave node, and compare the contents of the metadata operation logs of the master and slave nodes; S4.3. Starting from the root node of the directed acyclic graph of dependencies, verify the commit status of metadata operations on each slave node. S4.4. Rollback of preceding metadata operations that did not complete all subsequent metadata operations on the slave node, and re-initiate the synchronous commit of the incomplete metadata operations in the dependency order.

[0010] Furthermore, in step S2, the source file metadata operations, source directory metadata operations, source directory parent directory metadata operations, target directory metadata operations, and target directory parent directory metadata operations corresponding to the cross-directory renaming operation are grouped into the same serial synchronization queue. The metadata operations in the serial synchronization queue are sorted according to the semantic dependency order of the renaming operations. First, the metadata operation for deleting directory entries in the source directory is submitted, and then the metadata operation for creating directory entries in the target directory is submitted. The metadata operations corresponding to the changes in hard link reference relationships involved in the cross-directory renaming operation are grouped into the same serial synchronization queue as the cross-directory renaming operation and are sorted and submitted according to the semantic dependency order of the hard link reference relationships.

[0011] Furthermore, in step S2, in both the batch directory creation scenario and the batch file import scenario, the metadata operation dependencies within the same directory tree are identified by the time window of the metadata operation request and the directory path prefix of the target inode. Metadata operations with a common prefix in the parent directory path of the target inode are grouped into the same serial synchronization queue, while metadata operations with a non-common prefix in the parent directory path of the target inode are grouped into different parallel synchronization queues. The batch metadata operations within the serial synchronization queue are sorted according to the directory hierarchy from root to leaf, and there are no commit order constraints for metadata operations between different parallel synchronization queues.

[0012] Furthermore, in step S2, a queue merging threshold and a time window length are set, and the number of metadata operations without inode semantic dependencies in the same directory within the time window is counted. When the number of metadata operations without inode semantic dependencies reaches the queue merging threshold, the metadata operations without inode semantic dependencies are merged into batch metadata operations, and the batch metadata operations are assigned to the parallel synchronization queue. The metadata operations within the batch metadata operations have no commit order constraints. The batch metadata operations initiate a synchronization commit request to the metadata slave nodes as a synchronization commit unit. When all metadata operations within the batch metadata operations have been committed by more than half of the slave nodes, the batch metadata operations are confirmed to be synchronized.

[0013] Furthermore, in step S3, the queue-level lock is a mutex lock, allowing only one serial synchronization queue to hold the queue-level lock at a time. The queue-level lock is applied only when metadata operations within the serial synchronization queue enter the commit process. The application scope of the queue-level lock only covers the metadata operation commit process within the serial synchronization queue. The queue-level lock does not affect the execution of metadata operations in other serial synchronization queues and parallel synchronization queues. When the last metadata operation in the serial synchronization queue is committed by more than half of the slave nodes, the queue-level lock is immediately released. No lock is applied during the synchronous commit process of the parallel synchronization queue.

[0014] Furthermore, in step S4, the metadata consistency repair process does not perform a full metadata scan. It only verifies the commit status of metadata operations on the master and slave nodes according to the dependency order. The metadata operation logs of the master node and the slave node are compared one by one by the operation ID. The root node of the directed acyclic graph of dependency relationship is the metadata operation corresponding to the root directory of the file system. If a metadata operation that is not recorded by the master node is found on the slave node during the verification process, the metadata operation on the slave node is deleted. The verification result is used to perform rollback and recommit operations. The recommit of incomplete metadata operations is carried out in the topological order.

[0015] The beneficial effects of this invention are: (1) By parsing the native semantic dependencies of inodes, a directed acyclic graph of dependencies is constructed. Metadata operations are grouped into serial synchronization queues and parallel synchronization queues according to dependencies to achieve ordered synchronization of metadata and ensure semantic consistency of metadata in the distributed cluster. (2) Apply queue-level fine-grained locks to metadata operation groups with semantic dependencies, and adopt batch submission for metadata operations without dependencies to reduce lock contention during synchronization and improve the concurrent processing capability of metadata operations. (3) When the cluster anomaly causes the metadata synchronization to be interrupted, the metadata status is checked and repaired according to the dependency order of the directed acyclic graph of the dependency relationship, without the need to perform a full metadata scan, thus reducing the difficulty of repair in the case of cluster anomaly. Attached Figure Description

[0016] Figure 1 A flowchart illustrating the steps of a distributed metadata ordered synchronization method based on inode semantics; Figure 2 The flowchart illustrates the specific steps of a distributed metadata ordered synchronization method based on inode semantics, as provided in this embodiment. Detailed Implementation

[0017] The technical solution of the present invention will be clearly and completely described below with reference to the embodiments. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0018] Example 1 See Figure 1 This embodiment provides a distributed metadata ordered synchronization method based on inode semantics, which includes the following steps: S1. Perform semantic dependency parsing on inode metadata operations, identify metadata operation types, including directory creation, directory deletion, file creation, file deletion, hard link creation, hard link deletion, renaming, permission modification, and directory entry update, parse the inode semantic dependency relationship corresponding to each type of metadata operation, including directory parent-child relationship, hard link reference relationship, and permission inheritance relationship, and generate dependency information containing operation ID and preceding dependent operation ID; S2. Use dependency information to construct a directed acyclic graph of metadata operation dependencies, verify that there are no circular dependencies, group metadata operations with inode semantic dependencies into the same serial synchronization queue, and group metadata operations without inode semantic dependencies and belonging to different directory tree branches into different parallel synchronization queues, generate queue IDs and bind operation lists and dependencies. S3. Allocate queue-level locks to the serial synchronization queue, and perform synchronous commits through a distributed consensus algorithm. The serial synchronization queue commits metadata operations in dependency order, while the parallel synchronization queue commits metadata operations in batches. S4. When a node crashes, the network partitions, or the master-slave switch causes a interruption in metadata synchronization, compare the metadata operation logs of the master and slave nodes, verify the metadata operation commit status in the order of dependency, roll back the subsequent operations that were not committed before the preceding operations, and resubmit the incomplete metadata operations.

[0019] In some embodiments, step S1 includes the following sub-steps: S1.1. Receive metadata operation requests and extract the operation type identifier and target inode identifier contained in the metadata operation request; S1.2. Identify the metadata operation type based on the operation type identifier, parse the parent directory creation completion dependency corresponding to the directory creation operation and file creation operation, parse the target inode creation completion dependency and the synchronization completion dependency of the directory where the hard link is located corresponding to the hard link creation operation, parse the source file metadata synchronization completion dependency and the target directory metadata synchronization completion dependency corresponding to the rename operation, and parse the target inode creation completion dependency corresponding to the permission modification operation. S1.3. Generate operation IDs for metadata operations, mark all preceding dependent operation IDs corresponding to the metadata operations, and integrate the operation IDs and preceding dependent operation IDs to generate dependency information.

[0020] In some embodiments, step S2 includes the following sub-steps: S2.1. Construct a directed acyclic graph of dependencies, using metadata operations as nodes and prior dependencies as directed edges. S2.2. Traverse all nodes and directed edges of the directed acyclic graph of dependencies, detect whether there is a circular dependency in the directed acyclic graph of dependencies, intercept metadata operations with circular dependencies and return an exception response; S2.3. Sort the metadata operations in the serial synchronization queue according to the dependency order. If the preceding metadata operation has not been committed, the subsequent metadata operation will not enter the commit process. S2.4. Generate a queue ID for each serial synchronization queue and each parallel synchronization queue, and persist the queue ID, operation list, and dependencies to the metadata operation log.

[0021] In some embodiments, step S3 includes the following sub-steps: S3.1. Allocate a queue-level lock to each serial synchronization queue, and do not allocate locks to parallel synchronization queues; S3.2. The serial synchronization queue initiates synchronization commit requests to the metadata slave nodes in the order of dependency. Only after more than half of the slave nodes have completed the current metadata operation will the synchronization commit request for the subsequent metadata operation be initiated. S3.3. The parallel synchronization queue simultaneously sends batch synchronization commit requests to all metadata slave nodes without waiting for the metadata operations of other parallel synchronization queues to complete. S3.4. After confirming that the metadata operation has been committed by more than half of the slave nodes, return a successful metadata operation response to the client.

[0022] In some embodiments, step S4 includes the following sub-steps: S4.1. Detect the cluster running status and trigger the metadata consistency repair process when the cluster experiences node failure, network partition, or master-slave switch causing metadata synchronization interruption. S4.2. Extract the metadata operation logs of the master node and the slave node, and compare the contents of the metadata operation logs of the master and slave nodes; S4.3. Starting from the root node of the directed acyclic graph of dependencies, verify the commit status of metadata operations on each slave node. S4.4. Rollback of preceding metadata operations that did not complete all subsequent metadata operations on the slave node, and re-initiate the synchronous commit of the incomplete metadata operations in the dependency order.

[0023] In some embodiments, in step S2, the source file metadata operations, source directory metadata operations, source directory parent directory metadata operations, target directory metadata operations, and target directory parent directory metadata operations corresponding to the cross-directory renaming operation are grouped into the same serial synchronization queue. The metadata operations in the serial synchronization queue are sorted according to the semantic dependency order of the renaming operations. First, the metadata operation for deleting directory entries in the source directory is submitted, and then the metadata operation for creating directory entries in the target directory is submitted. The metadata operations corresponding to the changes in hard link reference relationships involved in the cross-directory renaming operation are grouped into the same serial synchronization queue as the cross-directory renaming operation and are sorted and submitted according to the semantic dependency order of the hard link reference relationships.

[0024] In some embodiments, in step S2, under the scenarios of batch directory creation and batch file import, the metadata operation dependencies within the same directory tree are identified by the time window of the metadata operation request and the directory path prefix of the target inode. Metadata operations with a common prefix in the parent directory path of the target inode are grouped into the same serial synchronization queue, while metadata operations with a common prefix in the parent directory path of the target inode are grouped into different parallel synchronization queues. The batch metadata operations in the serial synchronization queue are sorted according to the directory hierarchy from root to leaf, and there are no commit order constraints on the metadata operations between different parallel synchronization queues.

[0025] In some embodiments, in step S2, a queue merging threshold and a time window length are set, and the number of metadata operations without inode semantic dependencies in the same directory within the time window is counted. When the number of metadata operations without inode semantic dependencies reaches the queue merging threshold, the metadata operations without inode semantic dependencies are merged into batch metadata operations, and the batch metadata operations are assigned to the parallel synchronization queue. The metadata operations within the batch metadata operations have no commit order constraints. The batch metadata operations initiate a synchronization commit request to the metadata slave nodes as a synchronization commit unit. When all metadata operations within the batch metadata operations have been committed by more than half of the slave nodes, the synchronization of the batch metadata operations is confirmed.

[0026] In some embodiments, in step S3, the queue-level lock is a mutex lock, allowing only one serial synchronization queue to hold the queue-level lock at a time. The queue-level lock is applied only when metadata operations within the serial synchronization queue enter the commit process. The application scope of the queue-level lock only covers the metadata operation commit process within the serial synchronization queue. The queue-level lock does not affect the execution of metadata operations in other serial synchronization queues and parallel synchronization queues. When the last metadata operation in the serial synchronization queue is committed by more than half of the slave nodes, the queue-level lock is immediately released. No lock is applied during the synchronous commit process of the parallel synchronization queue.

[0027] In some embodiments, in step S4, the metadata consistency repair process does not perform a full metadata scan, but only verifies the commit status of metadata operations on the master node and slave node according to the dependency order. The metadata operation logs of the master node and the slave node are compared one by one by the operation ID. The root node of the directed acyclic graph of dependency relationship is the metadata operation corresponding to the root directory of the file system. If a metadata operation that is not recorded by the master node is found on the slave node during the verification process, the metadata operation on the slave node is deleted. The verification result is used to perform a rollback operation and a recommit operation. The recommit of incomplete metadata operations is performed in the topological order.

[0028] Example 2 This embodiment provides a specific implementation process for a distributed metadata ordered synchronization method based on inode semantics. This method is applied to a multi-replica metadata cluster architecture of a distributed file system. It achieves ordered metadata synchronization by parsing the native semantic dependencies of file system inodes, ensuring metadata consistency and concurrent processing capabilities in the distributed cluster. Simultaneously, it simplifies the metadata consistency repair process under cluster anomaly scenarios. Figure 2 As shown, the specific implementation process is as follows: Step 1. Semantic dependency resolution for inode metadata operations: Step 1.1. Receiving and Extracting Metadata Operation Requests: A metadata operation request is a set of instructions initiated by a client to the distributed file system metadata cluster to modify the file system metadata state. This set of instructions contains all the basic information required for the operation to be executed. In this embodiment, the master node of the metadata cluster receives the metadata operation request initiated by the client and extracts the operation type identifier and the target inode identifier from the request. An inode is a data structure in the file system used to store file or directory metadata information. Each file or directory corresponds to a unique inode identifier. The inode stores core metadata information such as file size, creation time, permissions, and data block pointers. In this embodiment, the inode identifier is used to locate the metadata object to be operated on, ensuring the accuracy of the operation execution.

[0029] Step 1.2. Metadata operation type identification and semantic dependency resolution: The metadata operation type is identified based on the extracted operation type identifier. Metadata operation types include directory creation, directory deletion, file creation, file deletion, hard link creation, hard link deletion, renaming, permission modification, and directory entry update. Semantic dependencies refer to the execution order constraints between metadata operations; violating these constraints will result in invalid metadata states or semantic inconsistencies. In this embodiment, the corresponding inode semantic dependencies are parsed for each type of metadata operation. Inode semantic dependencies include directory parent-child relationships, hard link reference relationships, and permission inheritance relationships. Specifically, a directory parent-child relationship means that a subdirectory or file must exist under a parent directory; the existence of the parent directory is a prerequisite for the existence of the subdirectory or file. A hard link reference relationship means that a hard link is a directory entry pointing to the same inode; the existence of a hard link depends on the existence of the target inode. Permission inheritance means that the default permissions of a subdirectory or file are inherited from the permission settings of the parent directory.

[0030] The specific parsing process is as follows: Parsing the parent directory creation completion dependency for directory creation and file creation operations, meaning that directory creation and file creation operations must be executed after their parent directory creation is complete and metadata synchronization is complete; parsing the target inode creation completion dependency and the directory containing the hard link creation dependency for hard link creation operations, meaning that hard link creation operations must be executed after the target inode creation is complete and metadata synchronization is complete, and the metadata of the directory containing the hard link is also synchronized; parsing the source file metadata synchronization completion dependency and the target directory metadata synchronization completion dependency for renaming operations, meaning that renaming operations must be executed after the metadata of the source file is synchronized and the metadata of the target directory is synchronized; parsing the target inode creation completion dependency for permission modification operations, meaning that permission modification operations must be executed after the target inode creation is complete.

[0031] Step 1.3. Operation Identifier Generation and Dependency Information Integration: An operation ID is generated for each metadata operation. The operation ID is a string used to uniquely identify the metadata operation. In this embodiment, operation IDs are used to distinguish different metadata operations, enabling operation tracking and management. All preceding dependent operation IDs corresponding to the metadata operation are marked. Preceding dependent operation IDs refer to the operation IDs of other metadata operations that the current metadata operation must wait for to complete. The operation IDs and preceding dependent operation IDs are integrated to generate dependency information. Dependency information is a data set describing the dependencies between metadata operations. This data set provides the foundation for constructing a directed acyclic graph of dependencies.

[0032] In some embodiments, the inode semantic dependencies corresponding to file attribute modification operations and file timestamp update operations can also be resolved. The dependencies corresponding to file attribute modification operations and file timestamp update operations are dependencies on the completion of target inode creation, that is, file attribute modification operations and file timestamp update operations must be executed after the target inode creation operation is completed.

[0033] In some specific implementations, the semantic dependencies between metadata operation types and their corresponding inodes are shown in Table 1. The dependency type for directory creation operations is a parent-child directory relationship, with the precondition that the parent directory's metadata synchronization is complete. The execution priority is high. During parsing, when a directory creation operation is detected, the commit status of the parent directory's metadata is first queried. If the parent directory has not completed synchronization, the directory creation operation is added to the waiting queue until the parent directory completes synchronization before further processing. The dependency type for file creation operations is a parent-child directory relationship, with the precondition that the parent directory's metadata synchronization is complete. The execution priority is high. During parsing, the same parent directory status query process is performed as for directory creation operations. The dependency type for hard link creation operations is a hard link reference relationship, with the precondition that the target inode and its directory are synchronized. The execution priority is medium. During parsing, the commit status of both the target inode and the directory containing the hard link are queried simultaneously. If both are complete... Synchronization is performed before subsequent processing. The renaming operation depends on a directory parent-child relationship, with the precondition that the source file and target directory are synchronized. Its execution priority is medium, and during parsing, the commit status of both the source file and target directory is queried. The permission modification operation depends on a permission inheritance relationship, with the precondition that the target inode has been created. Its execution priority is low, and during parsing, only the existence status of the target inode is queried. The directory entry update operation depends on a directory parent-child relationship, with the precondition that the metadata of the directory is synchronized. Its execution priority is low, and during parsing, the commit status of the directory is queried. The file deletion operation depends on a hard link reference relationship, with the precondition that the target file has no incomplete hard links. Its execution priority is medium, and during parsing, all hard link directory entries pointing to the target inode are traversed to confirm that all hard link creation operations are completed before executing the file deletion operation's dependency resolution.

[0034] Table 1. Metadata Operation Semantic Dependency Mapping Table Metadata operation types Dependency type Prerequisites Execution order priority Directory creation Parent-child relationship in the directory Parent directory metadata synchronization complete high File creation Parent-child relationship in the directory Parent directory metadata synchronization complete high Hard link creation Hard link references The target inode and its directory are synchronized. middle Rename Parent-child relationship in the directory Source files and target directories are synchronized. middle Permission Modification Access Control Inheritance Target inode creation complete Low Directory item update Parent-child relationship in the directory Metadata synchronization of the current directory is complete. Low File deletion Hard link references The target file has no incomplete hard link operations. middle Step 2. Construction of the Directed Acyclic Graph of Dependencies and Queue Grouping: Step 2.1. Constructing a Directed Acyclic Graph of Dependencies: A directed acyclic graph (DAG) is a data structure composed of nodes and directed edges. There is no path that starts from a node, traverses several directed edges, and returns to the same node. This embodiment uses a DAG to represent the dependencies between metadata operations, clearly and intuitively showing the order in which operations are executed. A dependency-based DAG is constructed using metadata operations as nodes and preceding dependencies as directed edges. The specific construction process is as follows: Each metadata operation is added to the graph as an independent node. For each metadata operation, based on the preceding dependency operation ID recorded in its dependency information, the corresponding preceding metadata operation node is found. A directed edge is drawn from the preceding metadata operation node to the current metadata operation node, indicating that the current metadata operation depends on the completion of the preceding metadata operation.

[0035] Step 2.2. Circular Dependency Detection and Exception Handling: This method traverses all nodes and directed edges of the dependent directed acyclic graph (DAG) to detect cyclic dependencies. A cyclic dependency occurs when two or more metadata operations depend on each other, forming a closed loop that makes it impossible to determine the execution order. This embodiment uses a depth-first search algorithm to traverse the DAG, recording the nodes on the current access path during the traversal. If a node is found to already exist on the current access path when accessing it, a cyclic dependency is determined. Metadata operations with cyclic dependencies are intercepted, and an exception response is returned to notify the client that the metadata operation cannot be executed.

[0036] Step 2.3. Grouping and sorting the metadata operation queue: Metadata operations with inode semantic dependencies are grouped into the same serial synchronization queue. A serial synchronization queue is a queue in which metadata operations must be executed sequentially according to a specified order; operations within the queue cannot be executed in parallel. Metadata operations without inode semantic dependencies but belonging to different directory tree branches are grouped into different parallel synchronization queues. Parallel synchronization queues are queues in which metadata operations can be executed simultaneously, and operations between different parallel synchronization queues can also be executed in parallel. Metadata operations within the serial synchronization queues are ordered according to the dependency order of the directed acyclic graph (DAG). Subsequent metadata operations are not committed until a preceding metadata operation has been completed, ensuring that the operation execution order conforms to the requirements of inode semantic dependencies.

[0037] Step 2.4. Queue Identifier Generation and Information Persistence: A queue ID is generated for each serial synchronization queue and each parallel synchronization queue. The queue ID is a string used to uniquely identify the synchronization queue. In this embodiment, the queue ID is used to distinguish different synchronization queues, enabling queue management and scheduling. The queue ID, operation list, and dependencies are persisted to the metadata operation log. The metadata operation log is a persistent storage file used to record all metadata operation information in the metadata cluster. This file records information such as the operation ID, operation type, operation time, dependencies, and commit status of each metadata operation. By persisting queue information, this embodiment ensures that the queue state can be restored and unfinished metadata operations can continue to be executed when the cluster experiences an anomaly.

[0038] In some embodiments, the source file metadata operations, source directory metadata operations, source directory parent directory metadata operations, target directory metadata operations, and target directory parent directory metadata operations corresponding to the cross-directory renaming operation are grouped into the same serial synchronization queue. The metadata operations in the serial synchronization queue are ordered according to the semantic dependency order of the renaming operations. First, the metadata operation for deleting directory entries in the source directory is submitted, and then the metadata operation for creating directory entries in the target directory is submitted. The metadata operations corresponding to the changes in hard link reference relationships involved in the cross-directory renaming operation are grouped into the same serial synchronization queue as the cross-directory renaming operation, and are ordered and submitted according to the semantic dependency order of the hard link reference relationships.

[0039] In some embodiments, in batch directory creation and batch file import scenarios, the metadata operation dependencies within the same directory tree are identified by the time window of the metadata operation request and the directory path prefix of the target inode. Metadata operations with a common prefix in the parent directory path of the target inode are grouped into the same serial synchronization queue, while metadata operations with a non-common prefix in the parent directory path of the target inode are grouped into different parallel synchronization queues. Batch metadata operations within the serial synchronization queue are sorted according to the directory hierarchy from root to leaf, and there are no commit order constraints on metadata operations between different parallel synchronization queues.

[0040] In some specific implementations, the queue merging parameters are configured as shown in Table 2. A time window length of 100 milliseconds, a merging threshold of 8, and a maximum batch operation count of 16 are used. This is suitable for high-concurrency small file operations. During queue grouping, the number of metadata operations without inode semantic dependencies within the same directory within the time window is counted. When the number reaches the merging threshold, these metadata operations are merged into a batch metadata operation, where the number of metadata operations contained within the batch metadata operation does not exceed the maximum batch operation count. Alternatively, a time window length of 200 milliseconds, a merging threshold of 16, and a maximum batch operation count of 32 are used for batch file imports. In this scenario, appropriately increasing the time window length and merging threshold can improve the merging efficiency of batch operations and reduce the number of synchronous submission requests.

[0041] Table 2 Queue Merging Parameter Configuration Table Parameter name Parameter meaning default value Applicable Scenarios Time window length Time range of statistical metadata operations 100 milliseconds High-concurrency small file operations Merging threshold Number of operations that trigger batch merging 8 High-concurrency small file operations Maximum number of batch operations Maximum number of operations in a single batch operation 16 High-concurrency small file operations Time window length Time range of statistical metadata operations 200 milliseconds Batch file import Merging threshold Number of operations that trigger batch merging 16 Batch file import Maximum number of batch operations Maximum number of operations in a single batch operation 32 Batch file import In some embodiments, a queue merging threshold and a time window length are set, and the number of metadata operations without inode semantic dependencies in the same directory within the time window is counted. When the number of metadata operations without inode semantic dependencies reaches the queue merging threshold, the metadata operations without inode semantic dependencies are merged into a batch metadata operation, and the batch metadata operation is assigned to a parallel synchronization queue. The metadata operations within the batch metadata operation have no commit order constraints. The batch metadata operation initiates a synchronization commit request to the metadata slave node as a synchronization commit unit. When all metadata operations within the batch metadata operation have been committed by more than half of the slave nodes, the batch metadata operation is confirmed to be synchronized.

[0042] In some specific implementations, the synchronization queue types and corresponding lock control strategies are shown in Table 3. Serial synchronization queues use mutex locks with queue-level granularity. Locks are held when an operation enters the submission process. After queue grouping, a corresponding mutex lock is initialized for each serial synchronization queue. The lock's lifecycle is consistent with the queue's lifecycle. When all metadata operations within the queue have been submitted and the queue is destroyed, the corresponding mutex lock resource is released. Parallel synchronization queues do not use any locking mechanism; lock granularity is zero, and lock holding is not specified. In the submission process of parallel synchronization queues, multiple metadata operations can simultaneously submit to the slave queue. Nodes initiate synchronous commit requests without waiting for other operations to complete. The batch operation queue does not use any locking mechanism; the lock granularity is batch-level, and the lock is held when the batch operation enters the commit process. In the batch operation commit process, multiple metadata operations are committed as a whole, and no locks are needed between metadata operations within the batch operation. The repair operation queue uses mutex locks; the lock granularity is repair unit-level, and the lock is held when the repair unit is executed. In the repair process, the metadata operations that need to be repaired are divided into multiple independent repair units. Each repair unit is locked separately during execution, and the lock is released immediately after the repair is completed.

[0043] Table 3. Correspondence Table of Synchronization Queue Lock Control Strategies Synchronization queue type Lock type Lock granularity Lock-up holding time Serial Synchronization Queue mutex queue level When the operation enters the submission process Parallel Synchronization Queue none none none Batch operation queue none Batch level When batch operations enter the submission process Repair operation queue mutex Repair unit level When the repair unit is executed Step 3. Queue-level lock control and ordered synchronous submission: Step 3.1. Queue-level lock allocation and control: A queue-level lock is assigned to each serial synchronization queue, while no lock is assigned to the parallel synchronization queue. A queue-level lock is a synchronization mechanism used to control the metadata operation submission process within a serial synchronization queue. In this embodiment, the queue-level lock is a mutex lock. A mutex lock is a locking mechanism that allows only one execution unit to hold the lock at a time, ensuring that only one serial synchronization queue can hold the queue-level lock simultaneously, thus avoiding metadata state conflicts caused by multiple serial synchronization queues submitting operations at the same time. The application scope of the queue-level lock only covers the metadata operation submission process within the serial synchronization queue; the queue-level lock does not affect the metadata operation execution of other serial synchronization queues and parallel synchronization queues.

[0044] Step 3.2. Ordered submission of serial synchronization queue: The serial synchronization queue initiates synchronization commit requests to the metadata slave nodes in dependency order. Only after more than half of the slave nodes have committed the current metadata operation will a synchronization commit request for the subsequent metadata operation be initiated. This embodiment uses a distributed consensus algorithm to achieve synchronous commit of metadata operations. A distributed consensus algorithm is an algorithm used to reach a consistent state among multiple nodes in a distributed system. Common distributed consensus algorithms include Raft and Paxos. This embodiment uses the Raft algorithm, which achieves consistency in a distributed system through three core phases: master node election, log replication, and security assurance. In this embodiment, the metadata master node replicates the metadata operation log to the metadata slave nodes. When more than half of the slave nodes have completed log replication and committed the operation, the master node confirms the completion of the operation.

[0045] In some specific implementations, the Raft algorithm log replication parameters are configured as shown in Table 4. The batch log entry count is 32, the heartbeat interval is 100 milliseconds, the election timeout is 500 milliseconds, and the log commit delay is 50 milliseconds. The applicable scenario is a general distributed file system. During the log replication process, the master node packages 32 consecutive metadata operation log entries into a log replication request and sends it to the slave node. The master node sends a heartbeat message to the slave node every 100 milliseconds to maintain the connection between the master and slave nodes. When the slave node does not receive a heartbeat message from the master node within 500 milliseconds, the election process is triggered. After the slave node writes the log entry to local storage, it delays for 50 milliseconds before committing the log entry to improve the efficiency of batch commit.

[0046] Table 4 Raft Algorithm Log Replication Parameter Configuration Table Parameter name Parameter meaning default value Applicable Scenarios Batch log entry count Number of log entries contained in a single replication request 32 General Distributed File System Heart rate interval Interval of the master node sending heartbeat messages 100 milliseconds General Distributed File System Election overtime Timeout period for node-triggered election 500 milliseconds General Distributed File System Log commit delay Delay time for node to submit logs 50 milliseconds General Distributed File System Step 3.3. Batch submission to the parallel synchronization queue: The parallel synchronization queue simultaneously sends batch synchronization commit requests to all metadata slave nodes, without waiting for metadata operations in other parallel synchronization queues to complete. Batch synchronization commit refers to packaging multiple metadata operations into a single commit request and sending it to the slave nodes. The slave nodes process multiple metadata operations at once, reducing network transmissions and processing overhead, and improving the efficiency of metadata synchronization. Since there are no inode semantic dependencies between metadata operations within the parallel synchronization queue, batch commit does not lead to metadata semantic inconsistencies.

[0047] Step 3.4. Synchronously complete confirmation and response return: Once it is confirmed that the metadata operation has been committed by more than half of the slave nodes, a successful metadata operation response is returned to the client. This embodiment adopts the majority commit principle, which means that as long as more than half of the slave nodes have completed the commit of the metadata operation, the operation is considered to have been committed in the entire cluster. This principle can ensure cluster consistency while tolerating the failure of a minority of nodes, thereby improving the availability of the cluster.

[0048] In some embodiments, the queue-level lock is applied only when metadata operations in the serial synchronization queue enter the commit process. The queue-level lock is released immediately after all metadata operations in the serial synchronization queue have been committed, and no lock is applied during the synchronous commit process of the parallel synchronization queue. This lock application and release mechanism can minimize the lock holding time, reduce the performance overhead caused by lock contention, and improve the concurrent processing capability of metadata operations.

[0049] In some specific implementations, the Raft algorithm's log replication process uses a batch replication method. Multiple consecutive metadata operation log entries are packaged into a single log replication request and sent to the slave nodes. Upon receiving the log replication request, the slave nodes write multiple log entries to their local storage at once and then return a replication success response to the master node. After receiving replication success responses from more than half of the slave nodes, the master node marks the metadata operations corresponding to these log entries as committed and updates the commit index to the latest position. Subsequently, it returns an operation success response to the client. This batch replication method can effectively reduce the number of network transmissions and improve the throughput of metadata synchronization.

[0050] Step 4. Metadata consistency repair for abnormal scenarios: Step 4.1. Cluster anomaly detection and repair process triggered: The system monitors the cluster's operational status, including node online status, network connectivity, and master / slave node role status. When a node crashes, a network partition occurs, or a master-slave failover disrupts metadata synchronization, a metadata consistency repair process is triggered. A node crash refers to a metadata node ceasing operation due to hardware or software failures; a network partition occurs when some nodes in the cluster cannot communicate due to network failures, forming multiple independent sub-clusters; a master-slave failover is the process of electing a new master node to take over metadata services when the original master node fails. All of these abnormal scenarios can lead to metadata synchronization interruptions, resulting in inconsistent metadata states between master and slave nodes.

[0051] Step 4.2. Compare the metadata operation logs of the master and slave nodes: Extract the metadata operation logs of the master node and the slave node, and compare their contents. The metadata operation log is a persistent record of metadata operations, containing complete information about all metadata operations. By comparing the metadata operation logs of the master and slave nodes, discrepancies in metadata operations between them can be identified, providing a basis for subsequent commit status verification and repair operations.

[0052] Step 4.3. Metadata operation commit status verification: Starting from the root node of the directed acyclic graph (DAG), the commit status of metadata operations on each slave node is verified. The root node of the DAG refers to a metadata operation node with no preceding dependencies, typically corresponding to metadata operations in the root directory of a file system. Verifying according to the dependency order of the DAG ensures that the commit status of all preceding metadata operations has been confirmed before verifying subsequent metadata operations, avoiding verification errors caused by unknown preceding operation statuses.

[0053] Step 4.4. Rollback of abnormal operations and resubmission of incomplete operations: If a rollback operation fails to complete all subsequent metadata operations corresponding to the preceding metadata operations on the slave node, the incomplete metadata operations will be re-initiated for synchronous submission according to the dependency order of the directed acyclic graph (DAG). A rollback operation restores the metadata state of the slave node to its state before the abnormal operation occurred. In this embodiment, the rollback operation is implemented using the pre-operation state information recorded in the metadata operation log. When resubmitting incomplete metadata operations, the dependency order of the DAG is still followed to ensure that the repaired metadata state conforms to the requirements of inode semantic dependencies.

[0054] In some embodiments, the metadata consistency repair process does not perform a full metadata scan. Instead, it verifies the commit status of metadata operations on the master and slave nodes according to the dependency order of the directed acyclic graph of dependencies. The metadata operation logs of the master and slave nodes are compared one by one by operation ID. The root node of the directed acyclic graph of dependencies corresponds to the metadata operation of the file system root directory. If a metadata operation that is not recorded by the master node is found on the slave node during the verification process, the metadata operation on the slave node is deleted. The verification results are used to perform rollback and recommit operations. The recommit of incomplete metadata operations is performed in the topological order.

[0055] In some specific implementations, the cluster anomaly types and corresponding metadata consistency repair processes are shown in Table 5. The triggering condition for node crash anomaly is metadata node heartbeat timeout. The repair steps are to elect a new master node from the surviving nodes, compare the master and slave node operation logs, roll back uncommitted post-operations, and recommit incomplete operations. The impact scope is the metadata operations that the crashed node did not synchronize. The triggering condition for network partition anomaly is the interruption of network connectivity between nodes. The repair steps are to wait for the network to recover, merge the operation logs of multiple sub-clusters, and verify and repair inconsistent metadata operations according to the dependency order. The impact scope is the metadata operations generated during the partitioning period. The trigger condition for master-slave switchover anomaly is failure of the original master node. The repair steps are to have the new master node take over the service, compare the operation logs of the master and slave nodes, and repair inconsistent metadata operations. The affected scope is metadata operations that were not completed during the switchover. The trigger condition for log corruption anomaly is failure of metadata operation log verification. The repair steps are to copy the complete log from other nodes and rebuild the local metadata. The affected scope is metadata operations corresponding to the corrupted logs. The trigger condition for data inconsistency anomaly is different metadata states of the master and slave nodes. The repair steps are to verify in the order of dependency, roll back the erroneous operations, and resynchronize the metadata. The affected scope is inconsistent metadata objects and their child nodes.

[0056] Table 5. Cluster Anomaly Repair Process Correspondence Table Exception types Triggering conditions Repair steps Scope of influence Node crash Metadata node heartbeat timeout Electing a new master node, comparing logs, rolling back, and recommitting. Metadata operations not synchronized by the failed node Network partition Inter-node network connectivity interruption Waiting for network recovery, log merging, and verification repair. Metadata operations generated during partitioning Master-slave switch Original master node failure The new master node takes over the service, compares logs, and fixes inconsistencies. Metadata operations not completed during the switchover Log corruption Metadata operation log verification failed Copy the complete log from other nodes and rebuild the local metadata. Metadata operations corresponding to corrupted logs Data inconsistency The metadata status of master and slave nodes is different Perform validation, rollback of erroneous operations, and resynchronization in dependency order. Inconsistent metadata objects and their child nodes In some specific implementations, rollback operations are implemented using an undo log mechanism. Each metadata operation generates a corresponding undo log entry before execution. The undo log entry records the state information before the metadata operation is executed. When a rollback operation is required, the metadata state is restored to the state before the operation is executed based on the undo log entry. The rollback operation is executed in the reverse order of the original operation, that is, the last metadata operation executed is rolled back first, and then the metadata operations executed before are rolled back, ensuring that no new metadata state inconsistencies are generated during the rollback process.

[0057] This embodiment constructs a directed acyclic graph (DAG) of dependencies by parsing the native semantic dependencies of inodes. Based on this graph, metadata operations are grouped and synchronously committed in an ordered manner, ensuring semantic consistency of metadata in the distributed file system. This avoids problems such as disordered parent-child relationships between directories and files and abnormal hard link references, which are common in traditional synchronization schemes based solely on update order. Furthermore, this embodiment uses queue-level fine-grained locks instead of traditional global locks, applying locks only to semantically dependent operation groups, reducing lock contention during metadata synchronization and improving the concurrent processing capability of metadata operations. In addition, this embodiment uses the DAG of dependencies to automatically repair metadata consistency in abnormal scenarios without requiring a full scan of metadata. It can locate and repair inconsistent metadata operations, reducing the difficulty of repairing cluster anomalies and improving the availability and stability of the distributed file system. The technical solution of this embodiment is compatible with existing multi-replica metadata cluster architectures and general distributed consensus algorithms of distributed file systems, possessing versatility and scalability, and can be applied to distributed file system deployment scenarios of different scales.

[0058] The above description is merely a preferred embodiment of the present invention. It should be understood that the present invention is not limited to the forms disclosed herein and should not be construed as excluding other embodiments. It can be used in various other combinations, modifications, and environments, and can be altered within the scope of the concept described herein through the above teachings or related technologies or knowledge. Modifications and variations made by those skilled in the art that do not depart from the spirit and scope of the present invention should be within the protection scope of the appended claims.

Claims

1. A distributed metadata ordered synchronization method based on inode semantics, characterized in that, Includes the following steps: S1. Perform semantic dependency parsing on inode metadata operations, identify metadata operation types, including directory creation, directory deletion, file creation, file deletion, hard link creation, hard link deletion, renaming, permission modification, and directory entry update, parse the inode semantic dependency relationship corresponding to each type of metadata operation, including directory parent-child relationship, hard link reference relationship, and permission inheritance relationship, and generate dependency information containing operation ID and preceding dependent operation ID; S2. Use dependency information to construct a directed acyclic graph of metadata operation dependencies, verify that there are no circular dependencies, group metadata operations with inode semantic dependencies into the same serial synchronization queue, and group metadata operations without inode semantic dependencies and belonging to different directory tree branches into different parallel synchronization queues, generate queue IDs and bind operation lists and dependencies. S3. Allocate queue-level locks to the serial synchronization queue, and perform synchronous commits through a distributed consensus algorithm. The serial synchronization queue commits metadata operations in dependency order, while the parallel synchronization queue commits metadata operations in batches. S4. When a node crashes, the network partitions, or the master-slave switch causes a interruption in metadata synchronization, compare the metadata operation logs of the master and slave nodes, verify the metadata operation commit status in the order of dependency, roll back the subsequent operations that were not committed before the preceding operations, and resubmit the incomplete metadata operations.

2. The method according to claim 1, characterized in that, Step S1 includes the following sub-steps: S1.

1. Receive metadata operation requests and extract the operation type identifier and target inode identifier contained in the metadata operation request; S1.

2. Identify the metadata operation type based on the operation type identifier, parse the parent directory creation completion dependency corresponding to the directory creation operation and file creation operation, parse the target inode creation completion dependency and the synchronization completion dependency of the directory where the hard link is located corresponding to the hard link creation operation, parse the source file metadata synchronization completion dependency and the target directory metadata synchronization completion dependency corresponding to the rename operation, and parse the target inode creation completion dependency corresponding to the permission modification operation. S1.

3. Generate operation IDs for metadata operations, mark all preceding dependent operation IDs corresponding to the metadata operations, and integrate the operation IDs and preceding dependent operation IDs to generate dependency information.

3. The method according to claim 1, characterized in that, Step S2 includes the following sub-steps: S2.

1. Construct a directed acyclic graph of dependencies, using metadata operations as nodes and prior dependencies as directed edges. S2.

2. Traverse all nodes and directed edges of the directed acyclic graph of dependencies, detect whether there is a circular dependency in the directed acyclic graph of dependencies, intercept metadata operations with circular dependencies and return an exception response; S2.

3. Sort the metadata operations in the serial synchronization queue according to the dependency order. If the preceding metadata operation has not been committed, the subsequent metadata operation will not enter the commit process. S2.

4. Generate a queue ID for each serial synchronization queue and each parallel synchronization queue, and persist the queue ID, operation list, and dependencies to the metadata operation log.

4. The method according to claim 1, characterized in that, Step S3 includes the following sub-steps: S3.

1. Allocate a queue-level lock to each serial synchronization queue, and do not allocate locks to parallel synchronization queues; S3.

2. The serial synchronization queue initiates synchronization commit requests to the metadata slave nodes in the order of dependency. Only after more than half of the slave nodes have completed the current metadata operation will the synchronization commit request for the subsequent metadata operation be initiated. S3.

3. The parallel synchronization queue simultaneously sends batch synchronization commit requests to all metadata slave nodes without waiting for the metadata operations of other parallel synchronization queues to complete. S3.

4. After confirming that the metadata operation has been committed by more than half of the slave nodes, return a successful metadata operation response to the client.

5. The method according to claim 1, characterized in that, Step S4 includes the following sub-steps: S4.

1. Detect the cluster running status and trigger the metadata consistency repair process when the cluster experiences node failure, network partition, or master-slave switch causing metadata synchronization interruption. S4.

2. Extract the metadata operation logs of the master node and the slave node, and compare the contents of the metadata operation logs of the master and slave nodes; S4.

3. Starting from the root node of the directed acyclic graph of dependencies, verify the commit status of metadata operations on each slave node. S4.

4. Rollback of preceding metadata operations that did not complete all subsequent metadata operations on the slave node, and re-initiate the synchronous commit of the incomplete metadata operations in the dependency order.

6. The method according to claim 1, characterized in that, In step S2, the source file metadata operations, source directory metadata operations, source directory parent directory metadata operations, target directory metadata operations, and target directory parent directory metadata operations corresponding to the cross-directory renaming operation are grouped into the same serial synchronization queue. The metadata operations in the serial synchronization queue are sorted according to the semantic dependency order of the renaming operations. First, the metadata operation for deleting directory entries in the source directory is submitted, and then the metadata operation for creating directory entries in the target directory is submitted. The metadata operations corresponding to the changes in hard link reference relationships involved in the cross-directory renaming operation are grouped into the same serial synchronization queue as the cross-directory renaming operation and are sorted and submitted according to the semantic dependency order of the hard link reference relationships.

7. The method according to claim 1, characterized in that, In step S2, under the scenarios of batch directory creation and batch file import, the metadata operation dependencies within the same directory tree are identified by the time window of the metadata operation request and the directory path prefix of the target inode. Metadata operations with a common prefix in the parent directory path of the target inode are grouped into the same serial synchronization queue, while metadata operations with a non-common prefix in the parent directory path of the target inode are grouped into different parallel synchronization queues. The batch metadata operations in the serial synchronization queue are sorted according to the directory hierarchy from root to leaf, and there are no commit order constraints on the metadata operations between different parallel synchronization queues.

8. The method according to claim 1, characterized in that, In step S2, a queue merging threshold and a time window length are set. The number of metadata operations without inode semantic dependencies in the same directory within the time window is counted. When the number of metadata operations without inode semantic dependencies reaches the queue merging threshold, the metadata operations without inode semantic dependencies are merged into a batch metadata operation. The batch metadata operation is then assigned to a parallel synchronization queue. The metadata operations within the batch metadata operation have no commit order constraints. The batch metadata operation initiates a synchronization commit request to the metadata slave node as a synchronization commit unit. When all metadata operations within the batch metadata operation have been committed by more than half of the slave nodes, the synchronization of the batch metadata operation is confirmed.

9. The method according to claim 1, characterized in that, In step S3, the queue-level lock is a mutex lock, allowing only one serial synchronization queue to hold the queue-level lock at a time. The queue-level lock is applied only when metadata operations within the serial synchronization queue enter the commit process. The application scope of the queue-level lock only covers the metadata operation commit process within the serial synchronization queue. The queue-level lock does not affect the execution of metadata operations in other serial synchronization queues and parallel synchronization queues. When the last metadata operation in the serial synchronization queue is committed by more than half of the slave nodes, the queue-level lock is immediately released. No lock is applied during the synchronous commit process of the parallel synchronization queue.

10. The method according to claim 1, characterized in that, In step S4, the metadata consistency repair process does not perform a full metadata scan. It only verifies the commit status of metadata operations on the master and slave nodes according to the dependency order. The metadata operation logs of the master node and the slave node are compared one by one by operation ID. The root node of the directed acyclic graph of dependency relationship is the metadata operation corresponding to the root directory of the file system. If a metadata operation that is not recorded by the master node is found on the slave node during the verification process, the metadata operation on the slave node is deleted. The verification result is used to perform rollback and recommit operations. The recommit of incomplete metadata operations is carried out in the topological order.

Images