Metadata synchronization method, apparatus, and system

Abstract

A metadata synchronization method, apparatus, and system. The method is applied to a distributed file system. The system comprises a plurality of nodes, each node storing file system metadata, wherein a first node operates on a file and updates first metadata. The first node sends a first metadata update request to other nodes by means of a collision-free data synchronization protocol. The first node receives a second metadata update request sent by a second node on the basis of the collision-free data synchronization protocol, and the first node updates second metadata on the basis of the request. In the method, a distributed metadata management mode is used, thereby avoiding the problem in centralized management that a metadata management node is highly loaded and easily becomes a system bottleneck. The distributed management mode reduces the risk of a single point of failure, and facilitates system scalability and reliability. Metadata synchronization is performed on the basis of the collision-free data synchronization protocol, thereby ensuring data consistency, avoiding the problem of high delay caused by a complex election mechanism and a communication process, and improving synchronization efficiency.

Description

A method, apparatus and system for synchronizing metadata

[0001] Cross-reference to related applications

[0002] This application claims priority to Chinese Patent Application No. 202411496101.2, filed on October 23, 2024, entitled "A Metadata Synchronization Method, Apparatus and System", the entire contents of which are incorporated herein by reference. Technical Field

[0003] This application relates to the field of computer technology, and in particular to a method, apparatus and system for synchronizing metadata. Background Technology

[0004] With the rapid development of internet applications, distributed file systems have become an important component of modern data centers and large-scale computing environments. Distributed file systems enable efficient data access and management by distributing data resources across multiple nodes.

[0005] Traditional distributed file systems typically employ a centralized metadata management approach, as shown in Figure 1. Such systems manage the entire file system's metadata information through one or a few centralized nodes (such as a master node). While centralized metadata management simplifies metadata management and retrieval, it also introduces a significant single point of failure risk. The centralized architecture overloads the metadata management node, easily becoming a system bottleneck, especially in large-scale distributed environments where excessive load on the metadata node can lead to performance degradation. As data volume and access demands increase, these centralized management nodes significantly impact system scalability and reliability. Furthermore, when distributed across geographical regions, centralized management results in higher network transmission latency, further affecting system response speed and data consistency.

[0006] Therefore, there is an urgent need for a distributed file system that can avoid the risk of single point of failure and enhance the scalability and reliability of the file system. Summary of the Invention

[0007] This application provides a metadata synchronization method, apparatus, and system to improve the efficiency of metadata synchronization while achieving metadata consistency.

[0008] In a first aspect, embodiments of this application provide a metadata synchronization method. The method is applied to a distributed file system, the system including multiple file system nodes, each of which stores metadata of the file system. The method includes: a first file system node in the system operating on a file and updating the first metadata of the file; the first file system node sending a first metadata update request to other file system nodes in the system through a conflict-free data synchronization protocol, the first metadata request including updated first metadata information; the first file system node receiving a second metadata update request and updating the second metadata according to the second metadata update request, the second metadata update request being sent by a second file system node in the system based on the conflict-free data synchronization protocol, the second metadata request including updated second metadata information.

[0009] In the embodiments described above, a distributed metadata management approach is adopted. This approach does not rely on a central server or a small number of centralized nodes to manage the metadata of the entire system. This avoids the problem of excessive load on metadata management nodes and the risk of them becoming system bottlenecks when dealing with large-scale data. Furthermore, the distributed management approach reduces the risk of single points of failure, which is beneficial to the system's scalability and reliability. In this distributed metadata management approach, each file system node synchronizes metadata based on a conflict-free data synchronization protocol. While ensuring data consistency, this avoids the high latency issues caused by complex election mechanisms and node communication processes. Especially in cross-regional or high-latency network environments, it significantly improves the efficiency of data synchronization and reduces communication overhead, thus making the system more suitable for large-scale, cross-regional deployments.

[0010] In one possible implementation, the conflict-free data synchronization protocol includes any one of the following protocols: a data synchronization protocol based on the conflict-free replication type CRDT, a data synchronization protocol based on state machine replication SMR, and a time-based data synchronization protocol.

[0011] Metadata synchronization based on a conflict-free data synchronization protocol eliminates the dependence on a central metadata server, enabling metadata synchronization between nodes and automatic conflict resolution, thus ensuring data consistency across nodes. Compared to traditional data synchronization methods based on Paxos or Raft algorithms, it avoids the overhead of complex election mechanisms and lock contention. It does not require confirmation that each file system node has completed metadata updates; a successful metadata update request is sufficient. Even in cases of network instability or disconnection, effective metadata update operations can still be performed, avoiding the risk of data loss. Furthermore, the process is simple, reducing data synchronization latency and communication overhead, thereby improving file system performance.

[0012] In one possible implementation, the first metadata update request includes timestamp and / or version number information, enabling other file system nodes to update their metadata based on the timestamp and / or version number information. Adding timestamp and / or version number information to the metadata update request can avoid metadata update conflicts; it also allows some nodes to update metadata even when encountering network instability or outages. After the network is restored, a metadata update request carrying timestamp and / or version number information is sent, enabling other nodes to update their metadata based on the timestamp and / or version number information. Alternatively, if multiple metadata update requests are received after network restoration, the metadata can be updated based on the timestamp and / or version number information in the multiple metadata update requests to ensure data consistency.

[0013] In one possible implementation, the first file system node sends a first metadata update request to other file system nodes in the system via a conflict-free data synchronization protocol. This includes: if the first file system node determines that the third file system node is offline, after the third file system node comes online, the first file system node sends a first metadata update request to the third file system node via the conflict-free data synchronization protocol. In this implementation, if the third file system node is offline, the first file system node waits for the third file system node to come online before sending the metadata update request to the third file system node; however, while waiting for the third file system node to come online, the first file system node can still update its metadata, and can also update its metadata based on metadata update requests sent by other file system nodes.

[0014] In one possible implementation, the first file system node sends a first metadata update request to other file system nodes in the system via a conflict-free data synchronization protocol. This includes: if the first file system node is currently offline, after the first file system node comes online, it sends the first metadata update request to other file system nodes in the system via the conflict-free data synchronization protocol. In this implementation, the first file system node can still update its metadata even when it is offline, and achieves metadata consistency by sending metadata update requests after coming online.

[0015] In one possible implementation, the second metadata update request includes timestamp and / or version number information; updating the second metadata according to the second metadata update request includes: determining whether to update the second metadata according to the second metadata update request based on the timestamp and / or version number information; if it is determined that the second metadata should be updated, updating the second metadata according to the updated second metadata information. Updating metadata based on timestamp and / or version number information can avoid metadata update conflicts, thereby ensuring data consistency.

[0016] In one possible implementation, the method further includes: the first file system node receiving a third metadata update request, requesting an update to the second metadata, the third metadata update request including timestamp and / or version number information; determining whether to update the second metadata according to the second metadata update request based on the timestamp and / or version number information includes: based on the timestamp and / or version number information in the third metadata update request and the second metadata update request, if it is determined that the second metadata update time corresponding to the third metadata update request is earlier than the second metadata update time corresponding to the second metadata update request, then determining to update the second metadata according to the second metadata update request; if it is determined that the second metadata update time corresponding to the third metadata update request is later than the second metadata update time corresponding to the second metadata update request, then determining not to update the second metadata according to the second metadata update request. Updating metadata based on timestamp and / or version number information can avoid metadata update conflicts, thereby ensuring data consistency.

[0017] In one possible implementation, the file system node uses RocksDB as its metadata storage engine. RocksDB is a high-performance key-value store designed for scenarios requiring high throughput and low latency, and is particularly suitable for applications storing large amounts of data and requiring fast access. Whether used as a database storage engine, a caching system, or a log system, RocksDB provides a reliable and efficient solution.

[0018] In one possible implementation, the first file system node sends a first metadata update request to other file system nodes in the system via a conflict-free data synchronization protocol. This includes: the first file system node using a Remote Procedure Call (RPC) or gRPC communication mechanism to send the first metadata update request to other file system nodes in the system via the conflict-free data synchronization protocol. The RPC communication mechanism can reduce message transmission volume, improve transmission efficiency, and support multiple languages. Different programming languages ​​can easily call the far-field interface, and communication can be based on standardized data cores, enabling correct data transfer between different languages. It is characterized by high efficiency, flexibility, and high versatility. The gRPC communication mechanism can operate in various environments and can efficiently connect file system nodes, making it particularly suitable for scenarios requiring higher communication reliability.

[0019] In one possible implementation, updating the first metadata of the file includes: the first file system node writing the updated first metadata into a cache. By employing distributed caching technology, the updated metadata can be written into the cache first, thereby reducing the time spent reading metadata and improving the read performance of the file system. Subsequently, depending on the actual application, the cached metadata can be written to disk to avoid data loss.

[0020] Secondly, embodiments of this application provide a distributed file system, the system including multiple file system nodes, each file system node storing file system metadata; a first file system node in the system operates on a file and updates the first metadata of the file; the first file system node sends a first metadata update request to other file system nodes in the system through a conflict-free data synchronization protocol, the first metadata request including the updated first metadata information; the first file system node receives a second metadata update request and updates the second metadata according to the second metadata update request, the second metadata update request being sent by a second file system node in the system based on the conflict-free data synchronization protocol, the second metadata request including the updated second metadata information.

[0021] In one possible implementation, the conflict-free data synchronization protocol includes any one of the following protocols: a data synchronization protocol based on the conflict-free replication type CRDT, a data synchronization protocol based on state machine replication SMR, and a time-based data synchronization protocol.

[0022] In one possible implementation, the first metadata update request includes timestamp and / or version number information, so that the other file system nodes update the metadata based on the timestamp and / or version number information.

[0023] In one possible implementation, the first file system node sends a first metadata update request to other file system nodes in the system through a conflict-free data synchronization protocol, including: if the first file system node determines that the third file system node is offline, after the third file system node comes online, sending a first metadata update request to the third file system node through a conflict-free data synchronization protocol.

[0024] In one possible implementation, the first file system node sends a first metadata update request to other file system nodes in the system through a conflict-free data synchronization protocol, including: if the first file system node is currently offline, after the first file system node comes online, sending the first metadata update request to other file system nodes in the system through a conflict-free data synchronization protocol.

[0025] In one possible implementation, the second metadata update request includes timestamp and / or version number information; the first file system node updates the second metadata according to the second metadata update request, including: the first file system node determines whether to update the second metadata according to the second metadata update request based on the timestamp and / or version number information; if it is determined to update the second metadata, the first file system node updates the second metadata according to the updated second metadata information.

[0026] In one possible implementation, the first file system node is further configured to: receive a third metadata update request, requesting an update to the second metadata, the third metadata update request including timestamp and / or version number information; the first file system node determines, based on the timestamp and / or version number information, whether to update the second metadata according to the second metadata update request, including: the first file system node, based on the timestamp and / or version number information in the third metadata update request and the second metadata update request, if it is determined that the second metadata update time corresponding to the third metadata update request is earlier than the second metadata update time corresponding to the second metadata update request, determines to update the second metadata according to the second metadata update request; if it is determined that the second metadata update time corresponding to the third metadata update request is later than the second metadata update time corresponding to the second metadata update request, determines not to update the second metadata according to the second metadata update request.

[0027] In one possible implementation, the file system node uses RocksDB as its metadata storage engine.

[0028] In one possible implementation, the first file system node sends a first metadata update request to other file system nodes in the system via a conflict-free data synchronization protocol, including: the first file system node uses a remote procedure call (RPC) communication mechanism or a gRPC communication mechanism to send the first metadata update request to other file system nodes in the system via a conflict-free data synchronization protocol.

[0029] In one possible implementation, the first file system node updates the first metadata of the file by writing the updated first metadata into a cache.

[0030] Thirdly, embodiments of this application provide a metadata synchronization device, which is applied to a distributed file system. The system includes multiple devices, each of which stores metadata of the file system. The device includes a unit for performing the method as described in the first aspect and any implementation thereof.

[0031] For example, the device may include a processing unit and a communication unit. The processing unit is configured to: operate on a file and update the first metadata of the file; send a first metadata update request to other file system nodes in the system via the communication unit through a conflict-free data synchronization protocol, wherein the first metadata request includes updated first metadata information; receive a second metadata update request via the communication unit and update the second metadata according to the second metadata update request, wherein the second metadata update request is sent by a second file system node in the system based on the conflict-free data synchronization protocol, and the second metadata request includes updated second metadata information.

[0032] Fourthly, embodiments of this application provide a metadata synchronization device, the device including the processor, the processor being coupled to a memory, the memory being used to store programs or instructions, and when the program or instructions are executed by the processor, causing the device to perform the method as described in the first aspect and any implementation thereof.

[0033] Fifthly, embodiments of this application provide a computer-readable storage medium storing instructions that, when executed on a computer, cause the computer to perform the method as described in the first aspect and any implementation thereof.

[0034] In a sixth aspect, embodiments of this application provide a computer program product containing instructions that, when run on a computer, cause the methods described in the first aspect and any implementation thereof to be executed.

[0035] In a seventh aspect, embodiments of this application provide a chip, including: a processor coupled to a memory for storing instructions, wherein when the instructions are executed by the processor, the chip causes the chip to implement the method described in the first aspect and any of its implementations.

[0036] For the technical effects that can be achieved by any possible implementation of any of the second to seventh aspects mentioned above, please refer to the description of the technical effects that can be achieved by the corresponding implementation scheme in the first aspect mentioned above. Repeated points will not be discussed. Attached Figure Description

[0037] Figure 1 is a schematic diagram of the centralized metadata management method provided in an embodiment of this application;

[0038] Figure 2 is a schematic diagram of the distributed metadata management method provided in an embodiment of this application;

[0039] Figure 3 is a schematic diagram of the user space file system provided in an embodiment of this application;

[0040] Figure 4 is a schematic diagram of the Ceph file system provided in an embodiment of this application;

[0041] Figures 5(a) and 5(b) are schematic diagrams of the architecture of the distributed file system provided in the embodiments of this application;

[0042] Figure 6 is a flowchart illustrating a metadata synchronization method provided in an embodiment of this application;

[0043] Figure 7 is a flowchart illustrating another metadata synchronization method provided in an embodiment of this application;

[0044] Figure 8 is a flowchart illustrating another metadata synchronization method provided in an embodiment of this application;

[0045] Figure 9 is one of the schematic diagrams of test results provided in an embodiment of this application;

[0046] Figure 10 is a second schematic diagram of the test results provided in the embodiments of this application;

[0047] Figure 11 is a third schematic diagram of the test results provided in the embodiments of this application;

[0048] Figure 12 is a fourth schematic diagram of the test results provided in the embodiments of this application;

[0049] Figure 13 is a schematic diagram of the test results provided in the embodiments of this application (the fifth one).

[0050] Figure 14 is a schematic diagram of the metadata synchronization device provided in an embodiment of this application;

[0051] Figure 15 is a schematic diagram of the metadata synchronization device provided in an embodiment of this application. Detailed Implementation

[0052] File systems employing centralized metadata management are susceptible to single points of failure; if the metadata management node fails, the file system may malfunction. Furthermore, excessive load on the metadata management node can easily become a system bottleneck, impacting system scalability and reliability.

[0053] To address the limitations of centralized architectures, distributed metadata management solutions have been proposed, as shown in Figure 2. Distributed metadata management systems use distributed consensus algorithms (such as Paxos or Raft) to ensure data consistency across nodes. However, these algorithms often rely on complex communication and election mechanisms, leading to inefficiency in high-latency and cross-regional environments, resulting in high synchronization latency and significant management overhead. As data scale and geographical distribution increase, file systems using these consensus algorithms struggle to balance data consistency and efficiency. Furthermore, these consensus algorithms also exhibit limitations and complexity when handling large numbers of small files or high-frequency access scenarios.

[0054] At the local file system level, filesystem in userspace (FUSE) is a common design approach. It simplifies file system development and testing by implementing a file system (such as EXT4, Btrfs, or XFS) in user space, as shown in Figure 3. However, in high-concurrency access scenarios, FUSE incurs context switching overhead between user space and kernel space, which significantly impacts file operation performance. Especially in data-intensive applications, the performance bottleneck caused by context switching limits the scalability and efficiency of FUSE.

[0055] Ceph is a unified, distributed file system designed for excellent performance, reliability, and scalability, widely used in data center and cloud storage environments. Ceph's core design goal is to provide a highly scalable storage system while ensuring data consistency and high availability. Ceph achieves these goals through a distributed object storage architecture and a distributed metadata management mechanism. As shown in Figure 4, Ceph's architecture includes the following key components:

[0056] Object storage device (OSD) nodes: The basic storage units of the system, used to store the actual data and metadata of files. Each OSD node is responsible for maintaining its own data state and health, and for synchronizing and replicating data with other OSD nodes in the system.

[0057] Monitor (MON) nodes: These nodes manage the cluster's global state information, which can include cluster members, data distribution, and metadata layout. MON nodes employ distributed consensus algorithms (such as Paxos) to ensure data consistency and fault recovery capabilities.

[0058] Metadata storage (MDS): This is responsible for managing the metadata information of the file system. The system stores metadata on one or more MDS nodes and uses a distributed consensus algorithm (such as Raft) to manage the updates and synchronization of metadata, ensuring metadata consistency.

[0059] Client: The system's client is responsible for interacting with OSD nodes, MON nodes, and MDS nodes, handling file operation requests (such as file creation, deletion, reading, and writing), as well as obtaining and updating metadata.

[0060] The Ceph file system achieves high availability and consistency of data through distributed object storage and metadata management mechanisms. Its design advantages lie in supporting high-concurrency access, large-scale data distribution, and dynamic expansion. However, the Ceph file system faces several technical challenges in achieving data consistency and high availability:

[0061] 1. High latency and communication overhead: The Ceph file system relies on distributed consensus algorithms (such as Paxos or Raft) to manage metadata and data consistency. Although distributed consensus algorithms perform well in ensuring data consistency, their complex election and communication mechanisms are inefficient in high-latency and cross-regional environments, resulting in high synchronization latency and high management overhead.

[0062] 2. Limited scalability: MDS nodes in the Ceph file system need to manage all metadata information in the system. Although metadata management can be performed by multiple MDS nodes, the load on the MDS nodes will increase rapidly when the data scale and access frequency increase, which can easily become a system bottleneck and limit the scalability of the system.

[0063] 3. Single point of failure risk: Although the Ceph file system is designed with multiple MON nodes to improve reliability, the management and synchronization process of MDS nodes is complex and may still lead to single points of failure or performance bottlenecks under certain circumstances, especially when deployed on a large scale, where the single point of failure problem is more prominent.

[0064] 4. Data consistency issues under unstable network conditions: In the event of network instability or interruption, the distributed consensus algorithm on which the Ceph file system relies may experience synchronization delays and data inconsistencies, reducing the availability and reliability of the system.

[0065] GlusterFS is a scalable distributed file system designed to provide a storage solution capable of large-scale scaling, particularly well-suited for handling unstructured data (such as documents, videos, and audio). GlusterFS achieves distributed storage and access management of data by distributing storage resources across multiple nodes. GlusterFS can include the following key components:

[0066] Volume: A storage unit in GlusterFS that aggregates disk space from multiple storage nodes into a single logical storage volume. Each volume can be configured as a different storage type (such as a replicated volume, striped volume, distributed volume, etc.) to suit different application scenarios.

[0067] Distributed Hash Table (DHT): GlusterFS uses DHT to manage the distribution and location of files across storage nodes. DHT determines the storage location of a file by hashing its filename, enabling fast file lookup and access.

[0068] The GlusterFS daemon manages file system storage operations and data transfer, such as file creation, deletion, reading, and writing. It also handles data synchronization and replication between nodes, ensuring data consistency and high availability.

[0069] Client: The client accesses the storage volume and initiates file operation requests through GlusterFS client tools or mount points.

[0070] GlusterFS's architecture is designed to scale linearly as the cluster size increases, supporting high-throughput applications and various storage types. However, GlusterFS presents certain challenges in terms of data consistency and management complexity, especially in cross-regional deployments and high-concurrency environments.

[0071] In view of this, embodiments of this application provide a distributed file system that can reduce system overhead while meeting the file system's requirements for data synchronization and consistency, thereby improving the overall performance of the distributed file system.

[0072] Figures 5(a) and 5(b) provide an exemplary schematic diagram of a distributed file system architecture. As shown, the distributed file system may include the following components:

[0073] User client (also known as user application): Users can initiate file operation requests through the user client, such as requesting file creation, reading, updating, and deletion. The user client can communicate with file system nodes; optionally, the user client can communicate with the nearest file system node and perform file operations through that node.

[0074] File system node (SFS for short in the diagram): A file system instance is deployed at each physical location. It is responsible for handling local file system operations (such as creating, reading, updating, and deleting files), maintaining a copy of local metadata, and synchronizing metadata with other nodes. A distributed file system can include multiple file system nodes. Different file system nodes can be deployed on different hardware devices, or they can be deployed on the same hardware device. This application embodiment does not limit this.

[0075] Optionally, each file system node includes a metadata server to manage the file system's metadata. The metadata server can receive and process metadata update requests and resolve potential metadata conflicts. It utilizes a conflict-free data synchronization strategy to ensure metadata consistency across all nodes. The metadata server maintains a local copy of the metadata while synchronizing metadata with other nodes.

[0076] Furthermore, the distributed file system may also include a virtual file system (VFS): a kernel software layer that is an abstraction layer above the specific file system. It is used to handle all calls related to the file system, and it provides a common interface for various file systems, enabling upper-layer applications to access different file systems using the common interface. It also provides a medium for communication between different file systems.

[0077] This distributed file system can also include a local file system: this means file storage is controlled locally, i.e., directly accessed by device drivers. Local file storage can be directly managed by the computer system without requiring another computer to act as a file server. However, for a clustered file system, the file system is local and therefore follows the cluster protocol.

[0078] Furthermore, the distributed file system may also include a FUSE driver: after receiving a request from the VFS, it can save the request to a queue, and then submit the request to the FUSE daemon responsible for handling the FUSE file system through a specific connection instance in the / dev / fuse file.

[0079] The aforementioned distributed file system can be deployed on general-purpose computer hardware and is applicable to general-purpose computing nodes in existing data centers and distributed computing environments.

[0080] The core components of a distributed file system (such as file system nodes and metadata servers within file system nodes) can all run on general-purpose server hardware, without requiring dedicated network elements. Optionally, the server acting as a file system node can be configured with multi-core processors, large-capacity memory, and high-speed network interfaces to support high-performance data processing and transmission requirements.

[0081] The functionality of a distributed file system can be achieved by installing and running dedicated software programs on a server. The software program mainly includes: 1. A file system software module, responsible for local file system operations, metadata management, and metadata synchronization; 2. A metadata service module: handling metadata update requests and achieving metadata synchronization through a conflict-free data synchronization mechanism; 3. A communication module: supporting communication between nodes for the synchronization of metadata and file operations.

[0082] Existing distributed file systems can be modified and extended to possess the functionalities of the distributed file system provided in this application embodiment. On the software side, existing distributed file systems can be upgraded by installing or enabling new file system software modules to achieve the functionalities of the distributed file system provided in this application embodiment, thereby enhancing the system's scalability and data consistency management capabilities. On the hardware side, network accelerators or storage accelerators can be added to some key nodes to further optimize data transmission and storage performance.

[0083] This application provides a metadata synchronization method applicable to the aforementioned distributed file system, which can be implemented by performing the following steps as shown in Figure 6:

[0084] Step 601: The first file system node in the file system operates on the first file and modifies the first file's first metadata.

[0085] The first file system node can be any file system node in the file system; the first file can be any file stored in the file system or a newly created file.

[0086] The first file system node can perform corresponding operations on the first file upon receiving an operation request from a user client, thereby modifying the metadata of the first file. For example, if a user sends a request to the first file system node to read or modify the first file, the first file system node will read the first file and modify its metadata upon receiving the request; or, if a user sends a request to the first file system node to create the first file, the first file system will create the first file in the file system and add new metadata for the first file upon receiving the request; or, if a user sends a request to the first file system node to delete the first file, the first file system will delete the first file and its metadata accordingly.

[0087] Operations such as reading, creating, modifying, and deleting the first file will all involve changes to the first file's metadata. Therefore, the first file system node will modify the first file's metadata accordingly, such as modifying the first file's size, storage address, last access time, and last modification author.

[0088] For example, if the first file system node reads the first file, it will modify information such as the last read time in the file's metadata and needs to synchronize this information with other file system nodes. Alternatively, if the first file system node changes the storage location of the first file, it will modify information such as the storage address in the file's metadata and needs to synchronize this changed information with other file system nodes. Or, if the first file system node modifies the content of the first file, its size, last modification time, and last modification author information may change; that is, the first file system node modifies the size, last modification time, and last modification author information in the file's metadata and needs to synchronize this information with other file system nodes.

[0089] Step 602: The first file system node sends a first metadata update request to other file system nodes in the file system through a conflict-free data synchronization protocol. The first metadata update request includes information about the modified first metadata.

[0090] The first file system node sends a first metadata update request to every file system node in the file system except itself, so that each file system node can update the metadata it maintains according to the first metadata update request, thereby making the metadata maintained by each node in the file system consistent.

[0091] Metadata synchronization based on a conflict-free data synchronization protocol eliminates the dependence on a central metadata server, enabling metadata synchronization and conflict resolution among nodes and ensuring data consistency across nodes. Compared to traditional data synchronization methods based on Paxos or Raft algorithms, it avoids the overhead of complex election mechanisms and lock contention. It does not require confirmation that each file system node has completed metadata updates; a successful metadata update request is sufficient. Even in cases of network instability or disconnection, effective metadata update operations can still be performed, avoiding the risk of data loss. Furthermore, the process is simple, reducing data synchronization latency and communication overhead, thereby improving file system performance.

[0092] Optionally, the conflict-free data synchronization protocol may include any of the following protocols: a data synchronization protocol based on conflict-free replicated data type (CRDT), a data synchronization protocol based on state machine replication (SMR), or a data synchronization protocol based on checkpoint-based synchronization.

[0093] Among them, the CRDT-based data synchronization protocol can ensure consistency through data structure, supporting simultaneous modification of metadata by multiple parties without conflicts caused by concurrent modifications, thus preventing inconsistent operation results or even data loss. In distributed metadata systems, data replication between different nodes suffers from consistency issues. Each node maintains a copy of metadata; if the nodes are not coordinated, inconsistencies in metadata copies will occur. When update conflicts arise, restoring consistency and data integrity may require partial or complete deletion of the updates. However, using the CRDT-based data synchronization protocol for metadata synchronization allows metadata to be automatically merged and conflicts resolved according to preset rules when multiple parties modify metadata, thereby achieving strong consistency.

[0094] SMR is a technology that achieves fault tolerance through replication servers. Each node has a state machine whose input comes from a log containing command sequences. For the same command input, the output of each node's state machine is deterministic and identical, thus ensuring the consistency of node states. The data synchronization protocol based on SMR can also guarantee the consistency of metadata across file system nodes, and compared to traditional algorithms such as Paxos or Raft, it is simpler to implement and has lower overhead.

[0095] A checkpoint-based data synchronization protocol periodically saves synchronization progress to checkpoints, enabling rapid recovery of synchronization progress after unexpected task crashes. When the task restarts, it first resumes synchronization from the most recent checkpoint and then continues data synchronization from that point. This checkpoint-based protocol also ensures consistency of metadata across file system nodes and is simpler to implement and has lower overhead compared to traditional Paxos or Raft algorithms.

[0096] As mentioned above, a file system node may include a metadata server for performing the function of managing the metadata information of the file system. In this case, step 602 above can be performed by the first metadata server in the first file system node. That is, when the first metadata server determines that the first file system node has modified the metadata of the first file, it sends a first metadata update request to the metadata servers in the file system nodes other than the first file system node, so that the metadata servers in other file system nodes can update the metadata information of the first file they maintain according to the first metadata update request, or create or delete the metadata of the first file.

[0097] Step 603: The first file system node receives the second metadata update request sent by the second file system node based on the conflict-free data synchronization protocol, and updates the second metadata according to the second metadata update request.

[0098] In this context, the second file system node is any file system node other than the first file system node in the file system. Each file system node in the file system can update its own maintained metadata. When other file system nodes update their metadata, they should also send metadata update requests to all file system nodes in the file system. The second metadata can be the same as the first metadata, or it can be different metadata.

[0099] After receiving the second metadata update request, the first file system node updates its own maintained metadata according to the second metadata update request, so that the metadata maintained by the first file system node and the second file system node are consistent.

[0100] Secondary metadata update requests based on conflict-free data synchronization protocols also help reduce communication overhead and data synchronization latency, thereby improving file system performance.

[0101] As mentioned above, a file system node may include a metadata server for managing the metadata information of the file system node. In the above steps, the first metadata server of the first file system node may receive a second metadata update request sent by the second metadata server of the second file system node, and the first metadata server may update the metadata according to the second metadata update request.

[0102] In one possible implementation, if the first file system node discovers that the third file system node is offline (due to network latency, network failure, or other reasons) when sending a first metadata update request to the third file system node in the file system, the first file system node can wait until the third file system node comes online before sending the first metadata update request to the third file system node. The third file system node can be any file system node in the file system other than the first file system node. In the embodiment shown in Figure 6, the third file system node and the second file system node are different nodes; in other embodiments, the third file system node and the second file system node can also be the same node.

[0103] Furthermore, when the first file system node detects that the third file system node is offline, it can monitor the status of the third file system node. When it detects that the third file system node is online, it will send the first metadata update request that was not successfully sent to the third file system node.

[0104] Alternatively, the first file system node can periodically send the first metadata update request to the third file system node until the first metadata update request is successfully sent.

[0105] Once the first file system node determines that it has sent the first metadata update request to all file system nodes in the file system except itself, it can consider the metadata update for the first file system file system complete, without needing to confirm whether other file system nodes have completed their updates. If there are any file system nodes that are offline, the first metadata update request will be sent to them after they come back online.

[0106] Furthermore, even if the metadata update for the first file is incomplete, the first file system node can continue to update the metadata again and send the metadata update request to each file system node again. For example, if the first metadata update request fails to be sent to the third file system node, the first file system node can also modify other metadata or modify the metadata of the first file again and send the third metadata update request to each file system node. The first file system node can send both the first and third metadata update requests to the third file system node after detecting that the third file system node is online; alternatively, the first file system node can periodically send both the first and third metadata update requests to the third file system node until both requests are successfully sent.

[0107] In one possible implementation, each metadata update request may include timestamp and / or version number information to indicate the time when the metadata was modified or the time or version number when the metadata update request was generated. This allows the file system node to update the metadata based on the timestamp and / or version number information in the metadata update request when it receives multiple metadata update requests, thus avoiding data conflicts.

[0108] For example, if the first file system node experiences a network outage at time t1, but needs to update metadata A at time t2, the first file system node can perform the metadata update operation without waiting for the network to be restored. However, the first file system node records the time t2 when updating the metadata. Then, if the first file system node restores the network at time t3, it sends a metadata update request 1 containing the time t2 indication information to all file system nodes. Upon receiving metadata update request 1, other file system nodes determine whether to update the metadata based on time t2. If other file system nodes have updated metadata A after time t2, then they will not update metadata A again; if other file system nodes have not updated metadata A after time t2, then they will update metadata A according to metadata update request 1.

[0109] For example, a first file system node sends metadata update request 1 to all nodes at time t1, which includes version number 001. At time t2, it sends metadata update request 2, which includes version number 002. A second file system node goes offline at both times t1 and t2, and then comes back online at time t3. Upon coming back online, the second file system node will receive both metadata update requests 1 and 2. It can determine the order of metadata updates based on the version number information: updating the metadata first according to metadata update request 1, and then updating it according to metadata update request 2. However, if the metadata update requests do not include timestamp and version number information, the second file system node, upon receiving both requests, may not be able to determine the order of metadata modifications and might update the metadata first according to metadata update request 2, potentially leading to inconsistencies between the updated metadata and that of other file system nodes.

[0110] For example, at time t1, the first file system node sends metadata update request 1 to all nodes, requesting updates to metadata A and metadata B; at time t2, the second file system node sends metadata update request 2 to all nodes, requesting updates to metadata B and metadata C; the third file system node goes offline at times t1 and t2, and comes back online at time t3. After coming back online, the third file system node will receive metadata update request 1 and metadata update request 2. The third file system node can then determine, based on the timestamp, to update metadata A according to metadata update request 1, and update metadata B and metadata C according to metadata update request 2.

[0111] In one possible design, the file system nodes can communicate with each other using a remote procedure call (RPC) mechanism. That is, the first file system node can send a first metadata update request to each other in the file system via RPC, and receive metadata update requests (such as the second metadata update request mentioned above) from other file system nodes via RPC. The RPC communication mechanism uses a custom communication protocol, which can reduce the amount of message transmission and improve transmission efficiency; it can also support multiple languages, allowing different programming languages ​​to easily call the far-field interface; and it can communicate based on a standardized data core, enabling correct data transfer between different languages, thus possessing the characteristics of high efficiency, flexibility, and high versatility.

[0112] Furthermore, the gRPC communication mechanism can be used between file system nodes to further improve the efficiency and security of data transmission between them. The gRPC communication mechanism can operate in various environments, efficiently connecting file system nodes, and is particularly suitable for scenarios requiring higher communication reliability.

[0113] In one possible implementation, the file system nodes could use LevelDB as the metadata storage engine. LevelDB is an embedded key-value store engine, a lightweight, high-performance database, particularly suitable for embedding into applications requiring efficient read and write operations.

[0114] Alternatively, file system nodes can use RocksDB as their metadata storage engine. RocksDB is a high-performance key-value store designed for scenarios requiring high throughput and low latency, and is particularly suitable for applications storing large amounts of data and requiring fast access. Whether used as a database storage engine, a caching system, or a log system, RocksDB provides a reliable and efficient solution.

[0115] To further improve the local read performance of the file system, distributed caching or data sharding caching technologies can be used to cache metadata. For example, during CRDT-based metadata synchronization, updated metadata is typically written to disk; however, with distributed caching, updated metadata can be written to the cache first, reducing the time spent reading metadata and improving file system read performance. Subsequently, depending on the actual application, the cached metadata can be written to disk to avoid data loss.

[0116] Optionally, when some file system nodes go offline, different breakpoint resumption techniques or transaction log-based rollback and recovery mechanisms can be used to restore metadata. These mechanisms can periodically generate checkpoints, and after network recovery, the file system restores the data state from the checkpoints, maintaining data consistency without relying on the decentralized merging characteristics of CRDT.

[0117] In the embodiments described above, a distributed metadata management approach is adopted. This approach does not rely on a central server or a small number of centralized nodes to manage the metadata of the entire system. This avoids the problem of excessive load on metadata management nodes and the potential for them to become system bottlenecks when dealing with large-scale data. Furthermore, the distributed management approach reduces the risk of single points of failure, which is beneficial to the scalability and reliability of the system. In this distributed metadata management approach, each file system node synchronizes metadata based on a conflict-free data synchronization protocol. While ensuring data consistency, this avoids the high latency issues caused by complex election mechanisms and node communication processes. Especially in cross-regional or high-latency network environments, it significantly improves the efficiency of data synchronization and reduces communication overhead, thus making the system more suitable for large-scale, cross-regional deployments. Moreover, the conflict-free data synchronization mechanism allows multiple file system nodes to update metadata simultaneously, and offline file system nodes can automatically merge and update after network recovery, avoiding the communication latency caused by traditional consistency protocols. In addition, adding timestamps and / or version numbers to metadata update requests can avoid metadata update conflicts. It can also support some nodes updating metadata even when encountering network instability or outages. After the network is restored, a metadata update request carrying timestamps and / or version numbers can be sent so that other nodes can update their metadata based on the timestamps and / or version numbers. Alternatively, if multiple metadata update requests are received after the network is restored, the metadata can be updated based on the timestamps and / or version numbers in the multiple metadata update requests to ensure data consistency.

[0118] To better understand the above embodiments of this application, examples are provided below with reference to Figures 7 and 8. In the embodiments shown in Figures 7 and 8, the distributed file system may include multiple file system nodes such as file system node 1, file system node 2, and file system node 3 (other file system nodes are not shown in Figures 7 and 8).

[0119] The specific embodiment shown in Figure 7 may include the following steps:

[0120] Step 701a: System initialization of file system node 1.

[0121] Step 701b: File system node 2 system initialization.

[0122] Step 701c: File system node 3 system initialization.

[0123] Each file system node runs the installed file system software program, loads the local file system instance, starts the metadata service, and completes registration via RPC communication to establish an initial synchronization state. The software program can include a file system software module, a metadata server module, and a communication module. The file system software module is responsible for local file system operations, metadata management, and metadata synchronization; the metadata service module handles metadata update requests and achieves metadata synchronization through a conflict-free data synchronization mechanism; and the communication module supports communication between nodes, thereby enabling the synchronization of metadata and file operations.

[0124] Step 702: User A sends a file operation request 1 to file system node 1 through the user client, requesting that data be written to file 1.

[0125] Optionally, the user client can send file operation request 1 to the nearest file system node. For example, if user A requests to write data to file 1 through a user client installed on server 1, and file system node 1 is also deployed on server 1, then the user client can send file operation request 1 to file system node 1. Alternatively, if no file system node is deployed on server 1, but file system node 1 is deployed on server 2, which is the closest server to server 1, then the user client on server 1 can send file operation request 1 to file system node 1 on server 2.

[0126] Step 703: File system node 1 writes data to file 1 and updates the metadata of file 1.

[0127] After receiving file operation request 1, file system node 1 writes data to file 1. Then, the metadata of file 1 (such as size, last access time, last modification author, etc.) also needs to be updated accordingly.

[0128] Step 704: File system node 1 sends a CRDT-based metadata update request 1 to file system node 2 and file system node 3.

[0129] This metadata update request 1 can be used to request an update to the metadata of file 1.

[0130] Optionally, the metadata update request 1 may include a timestamp (i.e., an indication of time t1) of the file system node 1 updating the metadata of file 1.

[0131] Optionally, the metadata update request 1 can also be a metadata update request based on SMR or checkpoint synchronization.

[0132] Optionally, file system node 1 can call the communication module to send metadata update request 1 to file system node 2 and file system node 3 based on the RPC communication mechanism or gRPC communication mechanism.

[0133] Optionally, file system node 1 includes metadata server 1, file system node 2 includes metadata server 2, and file system node 3 includes metadata server 3; then in step 704 above, metadata server 1 sends metadata update request 1 to metadata server 2 and metadata server 3.

[0134] At this time, file system node 2 is online, while file system node 3 is temporarily offline due to poor network conditions. Therefore, file system node 2 successfully receives metadata update request 1, but file system node 3 does not receive metadata update request 1.

[0135] Step 705: File system node 2 updates the metadata of file 1 according to CRDT-based metadata update request 1.

[0136] Optionally, the above steps can be performed by the metadata server 2 in the file system node 2, that is, the metadata server 2 updates the metadata of file 1 according to the metadata update request 1.

[0137] Step 706: File system node 1 monitors the status of file system node 3.

[0138] Step 707: User B sends a file operation request 2 to file system node 2 through the user client, requesting to read file 1.

[0139] Optionally, the user client can send file operation request 2 to the nearest file system node. For example, if user B requests to read file 1 through a user client installed on server 2, and file system node 2 is also deployed on server 2, then the user client can send file operation request 2 to file system node 2. Alternatively, if no file system node is deployed on server 2, but file system node 2 is deployed on server 3, which is the closest server to server 2, then the user client on server 2 can send file operation request 2 to file system node 2 on server 3.

[0140] Step 708: File system node 2 reads file 1 and updates file 1's metadata.

[0141] When file system node 2 reads file 1, the metadata of file 1 (such as the last access time) also needs to be updated accordingly.

[0142] Step 709: File system node 2 sends a CRDT-based metadata update request 2 to file system node 1 and file system node 3.

[0143] Metadata update request 2 can be used to request an update to the metadata of file 1.

[0144] Optionally, the metadata update request 2 may include a timestamp (i.e., an indication of time t2) for updating the metadata of file 1 by file system node 2.

[0145] Optionally, the metadata update request 2 can also be a metadata update request based on SMR or checkpoint synchronization.

[0146] Optionally, file system node 2 can send metadata update request 2 to file system node 1 and file system node 3 based on RPC communication mechanism or gRPC communication mechanism.

[0147] Optionally, file system node 1 includes metadata server 1, file system node 2 includes metadata server 2, and file system node 3 includes metadata server 3; then in step 709 above, metadata server 2 sends metadata update request 2 to metadata server 1 and metadata server 3.

[0148] At this time, file system node 1 is online, while file system node 3 is still offline due to poor network conditions. Therefore, file system node 1 successfully received metadata update request 2, but file system node 3 did not receive metadata update request 2.

[0149] Step 710: File system node 1 updates the metadata of file 1 according to the CRDT-based metadata update request 2.

[0150] Optionally, the above steps can be performed by metadata server 1 in file system node 1, that is, metadata server 1 updates the metadata of file 1 according to metadata update request 2.

[0151] Step 711: File system node 2 monitors the status of file system node 3.

[0152] Step 712a: File system node 1 detects that file system node 3 is online and sends a CRDT-based metadata update request 1 to file system node 3.

[0153] Step 712b: File system node 2 detects that file system node 3 is online and sends a CRDT-based metadata update request 2 to file system node 3.

[0154] Step 713: File system node 3 updates metadata based on CRDT-based metadata update request 1 and CRDT-based metadata update request 2.

[0155] Optionally, file system node 3 performs the corresponding metadata update operation according to the timestamp information in metadata update request 1 and metadata update request 2, in the order of the timestamps. That is, it first updates the metadata of file 1 according to metadata update request 1, and then updates the metadata of file 1 according to metadata update request 2, so as to keep the metadata of file 1 consistent in each node.

[0156] Optionally, the above steps can be performed by the metadata server 3 in the file system node 3, that is, the metadata server 3 updates the metadata of file 1 according to metadata update request 1 and metadata update request 2.

[0157] The specific embodiment shown in Figure 8 may include the following steps:

[0158] Step 801a: System initialization of file system node 1.

[0159] Step 801b: File system node 2 system initialization.

[0160] Step 801c: File system node 3 system initialization.

[0161] The steps 801a, 801b, and 801c described above are similar to steps 701a, 701b, and 701c in the aforementioned embodiments, and will not be repeated here.

[0162] Step 802: User A sends a file operation request 1 to file system node 1 through the user client, requesting the creation of file 1.

[0163] Optionally, the user client can send file operation request 1 to the nearest file system node. In this embodiment, file system node 1 is the file system node closest to the user client used by user A.

[0164] Step 803: File system node 1 creates file 1 and its metadata.

[0165] Step 804: File system node 1 sends a CRDT-based metadata update request 1 to file system node 2 and file system node 3.

[0166] This metadata update request 1 can be used to request an update to the metadata of file 1.

[0167] Optionally, the metadata update request 1 may include a timestamp (i.e., an indication of time t1) of the file system node 1 updating the metadata of file 1.

[0168] Optionally, the metadata update request 1 can also be a metadata update request based on SMR or checkpoint synchronization.

[0169] Optionally, file system node 1 can call the communication module to send metadata update request 1 to file system node 2 and file system node 3 based on the RPC communication mechanism or gRPC communication mechanism.

[0170] Optionally, file system node 1 includes metadata server 1, file system node 2 includes metadata server 2, and file system node 3 includes metadata server 3; then in step 804 above, metadata server 1 sends metadata update request 1 to metadata server 2 and metadata server 3.

[0171] At this time, both file system node 2 and file system node 3 are online, so both file system node 2 and file system node 3 successfully received the metadata update request 1 based on CRDT.

[0172] Step 805a: File system node 2 creates metadata for file 1 based on CRDT-based metadata update request 1.

[0173] Optionally, the above steps can be performed by the metadata server 2 in the file system node 2, that is, the metadata server 2 creates the metadata of file 1 according to the metadata update request 1.

[0174] Step 805b: File system node 3 creates metadata for file 1 based on CRDT-based metadata update request 1.

[0175] Optionally, the above steps can be performed by the metadata server 3 in the file system node 3, that is, the metadata server 3 creates the metadata of file 1 according to the metadata update request 1.

[0176] Step 806: User A sends a file operation request 2 to file system node 1 through the user client, requesting that data be written to file 1.

[0177] Step 807: File system node 1 writes data to file 1 and updates the metadata of file 1.

[0178] After receiving file operation request 1, file system node 1 writes data to file 1. Then, the metadata of file 1 (such as size, last access time, last modification author, etc.) also needs to be updated accordingly.

[0179] At this time, due to a network failure, file system node 1 is temporarily unable to send metadata update requests to file system nodes 2 and 3, but file system node 1 can record the time when the metadata of file 1 is updated, i.e., time t2.

[0180] Step 808: User B sends a file operation request 3 to file system node 2 through the client, requesting that data be written to file 1.

[0181] Optionally, the user client can send file operation request 2 to the nearest file system node. In this embodiment, file system node 2 is the file system node closest to the user client used by user B.

[0182] Step 809: File system node 2 writes data to file 1 and updates the metadata of file 1.

[0183] After receiving file operation request 2, file system node 2 writes data to file 1. Then the metadata of file 1 (such as size, last access time, last author of modification, etc.) also needs to be updated accordingly.

[0184] Step 810: File system node 2 sends a CRDT-based metadata update request 2 to file system node 1 and file system node 3.

[0185] Metadata update request 2 can be used to request an update to the metadata of file 1.

[0186] Optionally, the metadata update request 2 may include a timestamp (i.e., an indication of time t3) for the file system node 2 to update the metadata of file 1.

[0187] Optionally, the metadata update request 2 can also be a metadata update request based on SMR or checkpoint synchronization.

[0188] Optionally, file system node 2 can send metadata update request 2 to file system node 1 and file system node 3 based on RPC communication mechanism or gRPC communication mechanism.

[0189] Optionally, file system node 1 includes metadata server 1, file system node 2 includes metadata server 2, and file system node 3 includes metadata server 3; then in step 810 above, metadata server 2 sends metadata update request 2 to metadata server 1 and metadata server 3.

[0190] At this time, file system node 3 is online, while file system node 1 is still offline due to network failure. Therefore, file system node 3 successfully received metadata update request 2, but file system node 1 did not receive metadata update request 2.

[0191] Step 811: File system node 2 monitors the status of file system node 1.

[0192] Step 812a: After the network is restored, file system node 1 sends a CRDT-based metadata update request 3 to file system node 2 and file system node 3.

[0193] This metadata update request 3 can be used to request an update to the metadata of file 1.

[0194] Optionally, the metadata update request 3 may include a timestamp (i.e., an indication of time t2) of the file system node 1 updating the metadata of file 1.

[0195] Optionally, file system node 1 can send metadata update request 3 to file system node 2 and file system node 3 based on RPC communication mechanism or gRPC communication mechanism.

[0196] Optionally, in step 812a above, metadata server 1 in file system node 1 can send metadata update request 3 to metadata server 2 in file system node 2 and metadata server 3 in file system node 3.

[0197] Step 812b: After detecting that file system node 1 is online, file system node 2 sends a metadata update request 2 based on CRDT to file node 1.

[0198] Optionally, in step 812b above, the metadata server 2 in file system node 2 may send a metadata update request 2 to the metadata server 1 in file system node 1.

[0199] Step 813a: File system node 1 updates the metadata of file 1 according to the CRDT-based metadata update request 2.

[0200] Optionally, file system node 1 can update the metadata of file 1 based on the timestamp information in metadata update request 2. Since the timestamp in metadata update request 2 (i.e., time t3) is after time t2 when file system node 1 updates the metadata of file 1, file system node 1 can directly update the metadata of file 1 based on metadata update request 2.

[0201] Optionally, in the above steps, the metadata of file 1 can be updated by the metadata server 1 in file system node 1 according to the metadata update request 2.

[0202] Step 813b: File system node 2 and file system node 3 update the metadata of file 1 according to the CRDT-based metadata update request 3.

[0203] Optionally, file system nodes 2 and 3 can update the metadata of file 1 based on the timestamp information in metadata update request 3. Since the timestamp in metadata update request 3 (i.e., time t2) is before time t3 when file system node 2 updates the metadata of file 1, if metadata update request 3 requests an update to metadata A of file 1, and file system node 2 does not update metadata A at time t3, then file system nodes 2 and 3 will update metadata A according to metadata update request 3; if metadata update request 3 requests an update to metadata B of file 1, and file system node 2 updates metadata B at time t3, then file system nodes 2 and 3 should maintain the updated metadata B of file 1 and cannot update metadata B according to metadata update request 3.

[0204] As can be seen from the embodiments shown in Figures 7 and 8, even if there are problems such as node network failure or network latency, it does not affect other nodes from updating and synchronizing metadata during this period, and metadata consistency can still be maintained after the failed node recovers.

[0205] Furthermore, the metadata synchronization method provided in this application embodiment also improves the read and write performance of the file system. If a centralized metadata management approach is used, file system nodes need to access the metadata management node to read metadata information before operating on files. However, in this application embodiment, each file system node maintains its own metadata. Therefore, when operating on files, the file's metadata can be obtained locally without communicating with the metadata management node, thereby improving the file system's read and write performance, enabling the file system to quickly expand to new geographical locations, and significantly optimizing cross-node data transmission efficiency. Sequential read and write tests were conducted on the file system applying the metadata synchronization method provided in this application embodiment. The test results are shown in Figure 9. The file system provided in this application embodiment has advantages in terms of input / output operations per second (IOPS), bandwidth, and latency.

[0206] Metadata synchronization based on a conflict-free data synchronization protocol can solve the problems of low efficiency, reliance on central node coordination, and easy system bottlenecks of traditional distributed consensus protocols (such as Paxos and Raft) in high-latency environments, achieving real-time synchronization and automatic conflict resolution. Taking metadata synchronization based on the CRDT data synchronization protocol of the file system as an example, random read and sequential read tests of the file system are shown in Figure 10. In terms of IOPS, bandwidth, and latency, the file system provided in this application embodiment has advantages. Random write and sequential write tests of the file system are shown in Figure 11. In terms of IOPS, bandwidth, and latency, the file system provided in this application embodiment also has advantages. Load tests of the file system using the applications fileserver, webserver, and varmail are shown in Figures 12(a), (b), and (c). The file system provided in this application embodiment has advantages in throughput.

[0207] The file system provided in this application embodiment also supports an asynchronous synchronization strategy, meaning that file operations can still be performed even when the network is unstable or disconnected. Once the network is restored, it will automatically synchronize with each node to maintain metadata consistency. Metadata transmission and synchronization tests were performed on the file system, and the test results are shown in Figure 13. The file system provided in this application embodiment has a operations per second (TPS) of approximately 1200 ops / s, with a single operation latency of approximately 80 milliseconds, significantly enhancing the reliability and availability of the file system in complex network environments.

[0208] Based on the same technical concept, embodiments of this application also provide a metadata synchronization device, including a module / unit that performs the functions of the first file system node in the above method embodiments. This device can be the first file system node, a component of the first file system node (e.g., a processor, chip, or chip system), or a logical node, logical module, or software capable of implementing all or part of the functions of the first file system node. These modules / units can be implemented in hardware or by hardware executing corresponding software.

[0209] For example, the intent recognition device may include a processing module 1401 and a communication module 1402, as shown in FIG14. The communication module 1402 is used for message sending and receiving processing, and the processing module 1401 is used to implement the device's processing of messages. It should be understood that the processing module 1401 in this embodiment may be implemented by a processor or processor-related circuit components (or, referred to as processing circuitry), and the communication module 1402 may be implemented by a communication bus, a transceiver, or transceiver-related circuit components.

[0210] Specifically, the processing module 1401 is used to operate on the file and update the first metadata of the file; through the communication module 1402, it sends a first metadata update request to other file system nodes in the system through a conflict-free data synchronization protocol, the first metadata request including the updated first metadata information; through the communication module 1402, it receives a second metadata update request and updates the second metadata according to the second metadata update request, the second metadata update request being sent by a second file system node in the system based on the conflict-free data synchronization protocol, the second metadata request including the updated second metadata information.

[0211] In one possible implementation, the conflict-free data synchronization protocol includes any one of the following protocols: a data synchronization protocol based on the conflict-free replication type CRDT, a data synchronization protocol based on state machine replication SMR, and a time-based data synchronization protocol.

[0212] In one possible implementation, the first metadata update request includes timestamp and / or version number information, so that the other file system nodes update the metadata based on the timestamp and / or version number information.

[0213] In one possible implementation, the processing module 1401 sends a first metadata update request to other file system nodes in the system via a conflict-free data synchronization protocol through the communication module 1402, including: if it is determined that a third file system node is offline, after the third file system node comes online, sending a first metadata update request to the third file system node via the communication module 1402 through a conflict-free data synchronization protocol.

[0214] In one possible implementation, the processing module 1401 sends a first metadata update request to other file system nodes in the system via the communication module 1402 through a conflict-free data synchronization protocol, including: if the device is currently offline, after the device comes online, sending the first metadata update request to other file system nodes in the system via the communication module 1402 through a conflict-free data synchronization protocol.

[0215] In one possible implementation, the second metadata update request includes timestamp and / or version number information; when the processing module 1401 updates the second metadata according to the second metadata update request, it is specifically used to: determine whether to update the second metadata according to the second metadata update request based on the timestamp and / or version number information; if it is determined to update the second metadata, update the second metadata according to the updated second metadata information.

[0216] In one possible implementation, the processing module 1401 is further configured to: receive a third metadata update request via the communication module 1402, requesting an update to the second metadata, the third metadata update request including timestamp and / or version number information; when determining whether to update the second metadata according to the second metadata update request based on the timestamp and / or version number information, the processing module 1401 is specifically configured to: based on the timestamp and / or version number information in the third metadata update request and the second metadata update request, if it is determined that the second metadata update time corresponding to the third metadata update request is earlier than the second metadata update time corresponding to the second metadata update request, determine to update the second metadata according to the second metadata update request; if it is determined that the second metadata update time corresponding to the third metadata update request is later than the second metadata update time corresponding to the second metadata update request, determine not to update the second metadata according to the second metadata update request.

[0217] In one possible implementation, the processing module 1401 uses RocksDB as its metadata storage engine.

[0218] In one possible implementation, the processing module 1401 sends a first metadata update request to other file system nodes in the system via the communication module 1402 through a conflict-free data synchronization protocol, including: the processing module 1401 sends the first metadata update request to other file system nodes in the system via the communication module 1402 using a remote procedure call (RPC) communication mechanism or a gRPC communication mechanism through a conflict-free data synchronization protocol.

[0219] In one possible implementation, when updating the first metadata of the file, the processing module 1401 is specifically used to: write the updated first metadata into a cache.

[0220] Based on the same technical concept, embodiments of this application also provide a metadata sharing device, including a module / unit that performs the functions of the first file system node in the above method embodiments. This device may be the first file system node, a component of the first file system node (e.g., a processor, chip, or chip system), or a logical node, logical module, or software capable of implementing all or part of the functions of the first file system node. These modules / units can be implemented in hardware or by hardware executing corresponding software.

[0221] For example, the metadata synchronization device, as shown in Figure 15, includes a processor 1501 and a communication interface 1502. Further, it may also include a memory 1503 and a bus 1504. The processor 1501, communication interface 1502, and memory 1503 can be interconnected via the bus 1504. The bus 1504 can be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus, etc. The bus 1504 can be divided into an address bus, a data bus, and a control bus, etc. For ease of illustration, only one line is used in Figure 15, but this does not mean that there is only one bus or one type of bus.

[0222] Processor 1501 may be a central processing unit (CPU), a network processor (NP), or a combination of a CPU and an NP. The processor may further include hardware chips. These hardware chips may be application-specific integrated circuits (ASICs), programmable logic devices (PLDs), or combinations thereof. The PLD may be a complex programmable logic device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL), or any combination thereof. Memory 1503 may be volatile memory or non-volatile memory, or may include both. The non-volatile memory may be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. Volatile memory can be random access memory (RAM), which is used as an external cache.

[0223] The processor 1501 is used to implement the data processing operation of the communication device, and the communication interface 1502 is used to implement the receiving and sending operations of the communication device.

[0224] Specifically, the processor 1501 is used to operate on the file and update the first metadata of the file; through the communication interface 1502, it sends a first metadata update request to other file system nodes in the system through a conflict-free data synchronization protocol, the first metadata request including the updated first metadata information; through the communication interface 1502, it receives a second metadata update request and updates the second metadata according to the second metadata update request, the second metadata update request being sent by a second file system node in the system based on the conflict-free data synchronization protocol, the second metadata request including the updated second metadata information.

[0225] Furthermore, the aforementioned components can also be used to support other processes executed by the first file system node in the embodiments shown in Figures 6 to 8. The beneficial effects are described above and will not be repeated here.

[0226] Based on the same technical concept, embodiments of this application also provide a communication system, which may include multiple metadata processing devices or multiple metadata processing equipment.

[0227] Based on the same technical concept, embodiments of this application also provide a computer-readable storage medium storing computer-readable instructions, which, when executed on a computer, cause the method steps executed by the aforementioned first file system node to be performed.

[0228] Based on the same technical concept, this application also provides a computer program product containing instructions that, when run on a computer, cause the method steps executed by the first file system node to be executed.

[0229] Based on the same technical concept, this application embodiment also provides a chip, including: a processor coupled to a memory for storing instructions, which, when executed by the processor, cause the chip to implement the method steps executed by the first file system node described above.

[0230] It should be understood that in the description of this application, terms such as "first" and "second" are used only for distinguishing purposes and should not be construed as indicating or implying relative importance or order. References to "one embodiment" or "some embodiments" in this specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, phrases such as "in one embodiment," "in some embodiments," "in other embodiments," and "in still other embodiments" appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0231] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0232] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in one or more flowchart illustrations and / or one or more block diagrams.

[0233] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means that implement the functions specified in one or more flowcharts and / or one or more block diagrams.

[0234] These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process, such that the instructions, which execute on the computer or other programmable apparatus, provide steps for implementing the functions specified in one or more flowcharts and / or one or more block diagrams.

[0235] Although preferred embodiments of this application have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of this application.

[0236] Obviously, those skilled in the art can make various modifications and variations to the embodiments of this application without departing from the spirit and scope of the embodiments of this application. Therefore, if these modifications and variations to the embodiments of this application fall within the scope of the claims of this application and their equivalents, this application also intends to include these modifications and variations.

Claims

1. A metadata synchronization method, characterized in that, The method is applied to a distributed file system, the system comprising multiple file system nodes, each file system node storing file system metadata, and the method comprising: The first file system node in the system operates on the file and updates the file's first metadata; The first file system node sends a first metadata update request to other file system nodes in the system through a conflict-free data synchronization protocol. The first metadata request includes the updated first metadata information. The first file system node receives the second metadata update request and updates the second metadata according to the second metadata update request. The second metadata update request is sent by the second file system node in the system based on a conflict-free data synchronization protocol. The second metadata request includes the updated second metadata information.

2. The method according to claim 1, characterized in that, The conflict-free data synchronization protocol includes any one of the following protocols: Data synchronization protocols based on CRDT (Conflict-Free Replication), SMR (State Machine Replication), and punctuality-based data synchronization protocols.

3. The method according to claim 1 or 2, characterized in that, The first metadata update request includes timestamp and / or version number information, so that the other file system nodes update the metadata based on the timestamp and / or version number information.

4. The method according to any one of claims 1-3, characterized in that, The first file system node sends a first metadata update request to other file system nodes in the system via a conflict-free data synchronization protocol, including: If the first file system node determines that the third file system node is offline, after the third file system node comes online, it sends a first metadata update request to the third file system node through a conflict-free data synchronization protocol.

5. The method according to any one of claims 1-4, characterized in that, The first file system node sends a first metadata update request to other file system nodes in the system via a conflict-free data synchronization protocol, including: If the first file system node is currently offline, after the first file system node comes online, a first metadata update request is sent to other file system nodes in the system through a conflict-free data synchronization protocol.

6. The method according to any one of claims 1-5, characterized in that, The second metadata update request includes timestamp and / or version number information; The step of updating the second metadata according to the second metadata update request includes: Based on the timestamp and / or version number information, determine whether to update the second metadata according to the second metadata update request; If it is determined that the second metadata needs to be updated, the second metadata is updated according to the updated second metadata information.

7. The method according to claim 6, characterized in that, The method further includes: The first file system node receives a third metadata update request, requesting an update to the second metadata. The third metadata update request includes timestamp and / or version number information. The step of determining whether to update the second metadata according to the second metadata update request based on the timestamp and / or version number information includes: Based on the timestamp and / or version number information in the third metadata update request and the second metadata update request, if it is determined that the second metadata update time corresponding to the third metadata update request is earlier than the second metadata update time corresponding to the second metadata update request, it is determined that the second metadata should be updated according to the second metadata update request. If it is determined that the update time of the second metadata corresponding to the third metadata update request is later than the update time of the second metadata corresponding to the second metadata update request, it is determined that the second metadata will not be updated according to the second metadata update request.

8. The method according to any one of claims 1-7, characterized in that, The metadata storage engine used by the file system node is RocksDB.

9. The method according to any one of claims 1-8, characterized in that, The first file system node sends a first metadata update request to other file system nodes in the system via a conflict-free data synchronization protocol, including: The first file system node uses a remote procedure call (RPC) communication mechanism or a gRPC communication mechanism to send a first metadata update request to other file system nodes in the system through a conflict-free data synchronization protocol.

10. The method according to any one of claims 1-9, characterized in that, The update of the first metadata of the file includes: The first file system node writes the updated first metadata into the cache.

11. A distributed file system, characterized in that, The system includes multiple file system nodes, each of which stores file system metadata; The first file system node in the system operates on the file and updates the file's first metadata; The first file system node sends a first metadata update request to other file system nodes in the system through a conflict-free data synchronization protocol. The first metadata request includes the updated first metadata information. The first file system node receives the second metadata update request and updates the second metadata according to the second metadata update request. The second metadata update request is sent by the second file system node in the system based on a conflict-free data synchronization protocol. The second metadata request includes the updated second metadata information.

12. The system according to claim 11, characterized in that, The conflict-free data synchronization protocol includes any one of the following protocols: Data synchronization protocols based on CRDT (Conflict-Free Replication), SMR (State Machine Replication), and punctuality-based data synchronization protocols.

13. The system according to claim 11 or 12, characterized in that, The first metadata update request includes timestamp and / or version number information, so that the other file system nodes update the metadata based on the timestamp and / or version number information.

14. The system according to any one of claims 11-13, characterized in that, The first file system node sends a first metadata update request to other file system nodes in the system via a conflict-free data synchronization protocol, including: If the first file system node determines that the third file system node is offline, after the third file system node comes online, it sends a first metadata update request to the third file system node through a conflict-free data synchronization protocol.

15. The system according to any one of claims 11-14, characterized in that, The first file system node sends a first metadata update request to other file system nodes in the system via a conflict-free data synchronization protocol, including: If the first file system node is currently offline, after the first file system node comes online, a first metadata update request is sent to other file system nodes in the system through a conflict-free data synchronization protocol.

16. The system according to any one of claims 11-15, characterized in that, The second metadata update request includes timestamp and / or version number information; The first file system node updates the second metadata according to the second metadata update request, including: The first file system node determines whether to update the second metadata according to the second metadata update request based on the timestamp and / or version number information; If it is determined that the second metadata needs to be updated, the first file system node updates the second metadata according to the updated second metadata information.

17. The system according to claim 16, characterized in that, The first file system node is also used for: Receive a third metadata update request, requesting an update to the second metadata, wherein the third metadata update request includes timestamp and / or version number information; The first file system node determines, based on the timestamp and / or version number information, whether to update the second metadata according to the second metadata update request, including: Based on the timestamp and / or version number information in the third metadata update request and the second metadata update request, if the first file system node determines that the second metadata update time corresponding to the third metadata update request is earlier than the second metadata update time corresponding to the second metadata update request, it determines to update the second metadata according to the second metadata update request. If it is determined that the update time of the second metadata corresponding to the third metadata update request is later than the update time of the second metadata corresponding to the second metadata update request, it is determined that the second metadata will not be updated according to the second metadata update request.

18. The system according to any one of claims 11-17, characterized in that, The metadata storage engine used by the file system node is RocksDB.

19. The system according to any one of claims 11-18, characterized in that, The first file system node sends a first metadata update request to other file system nodes in the system via a conflict-free data synchronization protocol, including: The first file system node uses a remote procedure call (RPC) communication mechanism or a gRPC communication mechanism to send a first metadata update request to other file system nodes in the system through a conflict-free data synchronization protocol.

20. The system according to any one of claims 11-19, characterized in that, The first file system node updates the first metadata of the file, including: The first file system node writes the updated first metadata into the cache.

21. A metadata synchronization device, characterized in that, The device is used in a distributed file system, the system including multiple devices, each of which stores metadata of the file system; The apparatus includes a unit for performing the method as described in any one of claims 1-10.

22. A metadata synchronization device, characterized in that, include: A processor coupled to a memory for storing programs or instructions that, when executed by the processor, cause the apparatus to perform the method as described in any one of claims 1-10.

23. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores instructions that, when executed on a computer, cause the computer to perform the method as described in any one of claims 1-10.

24. A computer program product containing instructions, characterized in that, When the instructions are executed on a computer, the computer causes the computer to perform the method as described in any one of claims 1-10.

Images